Alex Balk

Structure a Vue.js App from Containers and Components

Recently we’ve begun using Vue.js as a frontend framework for one of our infrastructure projects. We’ve contracted Dr. Yoram Kornatzky to join our Delivery team and dive together, headlong, into this brave new world.

In this blog post by Yoram, and others to come, we’ll share snippets from this journey. We hope you find these beneficial, and invite you to share your own experiences and feedback.


Vue.js using Vuex for state management does not have a clear distinction between containers and components. This is in clear contrast to React using Redux for state management.

We argue that such a distinction between containers components in beneficial for Vue.js as well.

Dyploma

Dyploma is a system for managing containerized applications and services on top of Kubernetes in Outbrain. Dyploma includes the concepts of:

  • artifacts
  • builds
  • deployments
  • services

Dyploma is made out of a Java Spring backend and a Python command-line tool (CLI). The command-line tool operates through API calls to the backend.

The Dyploma Web Application

To facilitate broader adoption of containers within Outbrain, we set up to develop a web application that will have the capabilities of the Dyploma CLI.

The web application will operate by fetching data from the backend and sending operations for execution in the backend. This will be done through the same REST API used by the CLI.

A Vue.js Web Application

We chose Vue.js for constructing the web application. The app was constructed using vue-cli with the webpack template.

The application has three generic screens:

  • list
  • detail
  • form

All concepts have screens from each of these types with similar structure and look and feel, but with different actions and different data.

Vuex

Vuex is the standard state management approach for Vue.js.

Containers vs Components in React

Let us first recap what are containers and components in React.

A container interacts with the Redux and contains a component. The container supplies data to the component through selectors on the store and provides the actions on the store to the component.

Components are given data and render HTML. They use the actions provided from their container to interact with the state. Such actions modify the state, resulting in the selectors fetching new data, and causing the component to be rendered again.

Vue.js with Vuex

Vue.js standard practice does not have the containers vs components distinction. While constructing the Dyploma web application we found it useful to make such a distinction for the benefits of better code structure and reusability.

Let us first describe how the structure of the Dyploma web application.

Generic Components

We constructed three generic components:

  1. list
  2. detail
  3. form

Which can be composed of a component tree that can have more than 3 levels.

Each of these generic screens was used with some variations by multiple types of data. But the look and feel could be configured through a common JSON describing for each type of data, the different fields.

Type Specific Actions and Getters

The getters and actions to be used for each type of data were different. We constructed our Vuex store with modules and needed to use a separate module for each type.

Distinguish Components and Containers

So we had to think how to resolve two opposite requirements. For the benefits of reusability, we need unified generic components. But for the type specific actions and data, we need to use separate modules. We decided up front that the whole app will be constructed as a set of single file components (SFC).

To resolve these two opposite directions, we found it useful to think of our app as consisting of two things:

  • containers – type-specific that interact with store
  • components – generic

Components

We defined each component to a data props for the data it should render, and a description of the structure of data. For any changes and actions required, it will emit an event.

Data is passed from a component to its constituents with v-bind, like v-bind:list=”deployments”.

Events are hooked up with v-on like v-on:search=”search”.

Components are composed of smaller components. Events are propagated up the tree of components. This bottom-up propagation may be disturbing to some, but it is the best approach for Vue.js. In this respect, Vue.js is definitely different from React.

The component is a single file component (SFC).

Such a component is not necessarily functional.

A Container for Each Type of Data

A container knows which module of the store it deals with, and knows its actions and getters. It fetches data from the store using getters. Such data is passed to the components as props.

It listens to events coming from the components using v-on like v-on:search=”search”. In response to such events, it dispatches actions.

The container does not render anything itself, this is done by the component it contains.

The container is a single file component (SFC)s.

A Clean Separation Facilitates Reusability

This clean separation of components and containers make it simpler to see opportunities for reusability. Come to think of it, in most web apps, the real effort in reusability is reusability of the component. The mixing of components and containers causes many components to be coupled with the store. This makes it harder to identify reusability. By distinguishing components and containers, we isolate the components from the store and see more clearly opportunities for reusability.

Easier Testing

Writing unit tests becomes easier with this separation. One can write three classes of tests:

  1. components
  2. containers
  3. store

Each becoming simpler.

We will discuss this further in a separate article.

Conclusions

Split your Vue.js web app into containers and components.

Goodbye static CNAMEs, hello Consul

Nearly every large scale system becomes distributed at some point: a collection of many instances and services that compose the solution you provide. And as you scale horizontally to provide high availability, better load distribution, etc…, you find yourself spinning up multiple instances of services, or using systems that function in a clustered architecture. That’s all cool in theory, but soon after you ask yourself, “how do I manage all of this? How should these services communicate with each other? And how do they even know what instances (or machines) exist?”

Those are excellent questions!

What methods are in use today?

The naive approach, which we’d followed in Outbrain for many years, is to route all inter-service traffic via load balancers (HAProxy in our case). Every call to another system, such as a MySql slave, is done to the load balancer (one in a pool of many), via an agreed upon name, such as a DNS CNAME. The load balancer, which holds a static configuration of all the different services and their instances, directs the call to one of those instances, based on the predefined policy.

backend be_onering_es   ## backend name
  balance leastconn     ## how to distribute load
  option httpchk GET /  ## service health check method
  option httpclose      ## add “Connection: close” header if missing
  option forwardfor     ## send client IP through XFF header
  server ringdb-20001 ringdb-20001:9200 check slowstart 10s weight 100   ## backend node 1
  server ringdb-20002 ringdb-20002:9200 check slowstart 10s weight 100   ## backend node 2

The load balancer is also responsible for checking service health, to make sure requests are routed only to live services, as dead ones are “kicked out of the pool”, and revived ones are brought back in.

An alternative to the load balancer method, used in high throughput systems such as Cassandra, is configuring CNAMEs that point to specific nodes in the cluster. We then use those CNAMES in the consuming applications’s configuration. The client is then responsible to activate a policy of balancing between those nodes, both for load and availability.

OK, so what’s the problem here?

There’s a few actually:

  1. The mediator (Load balancer), as quick as it may be in processing requests (and HAProxy is really fast!), is another hop on the network. With many services talking to each other, this could prove a choke point in some network topologies. It’s also a shared resource between multiple services and if one service misbehaves, everyone pays the price. This is especially painful with big payloads.
  2. The world becomes very static! Moving services between hosts, scaling them out/in, adding new services – it all involves changing the mediator’s config, and in many cases done manually. Manual work requires expertise and is error prone. When the changes becomes frequent… it simply does not scale.
  3. When moving ahead to infrastructure that is based on containers and resource management, where instances of services and resources are allocated dynamically, the whole notion of HOSTNAME goes away and you cannot count on it in ANY configuration.

What this all adds up to is “the end of the static configuration era”. Goodbye static configs, hello Dynamic Service Discovery! And cue Consul.

What is Consul?

In a nutshell, Consul is a Service Discovery System, with a few interesting features:

  1. It’s a distributed system, made out of an agent in each node. Nodes talk to each other via a gossip protocol, making node discovery simple, robust, and dynamic. There’s no configuration file describing all members of a Consul cluster.
  2. It’s fault tolerant by design, and using concepts such as Anti Entropy, gracefully handles nodes disappearing and reappearing – a common scenario in VM/container based infrastructure.
  3. It has first-class treatment of datacenters, as self-contained, interconnected entities. This means that DC failure / disconnection would be self-contained. It also means that a node in one DC can query for information in another DC with as little knowledge as the remote DC’s name.
  4. It holds the location (URI) and health of every service on every host, and makes this data available via multiple channels, such as a REST API and GUI. The API also lets you make complex queries and get the service data segment you’re interested in. For example: Get me all the addresses of all instances of service ‘X’ from Datacenter ‘Y’ in ‘staging env’ (tag).
  5. There is a very simple way to get access to “Healthy” service instances by leveraging the Consul DNS interface. Perfect for those pesky 3rd party services whose code you can’t or don’t want to modify, or just to get up and running quickly without modifying any client code (disclaimer: doesn’t fit all scenarios).

How does Consul work?

You can read all about it here, but let me take you through a quick tour of the architecture:

click to enlarge

As you can see, Consul has multi datacenter awareness built right in (you can read more about it here). But for our case, let’s keep it simple, and look at the case of a single datacenter (Datacenter 1 in the diagram).

What the diagram tags as “Clients” are actually “Consul agents”, running locally on every participating host. Those talk to each other, as well as the Consul servers (which are “agents” configured as Servers), through a “Gossip protocol”. If you’re familiar with Cassandra, and that rings a bell, then you’re right, it’s the same concept used by Cassandra nodes to find out which ones are up or down in a cluster. A Gossip protocol essentially makes sure “Everybody knows Everything about Everyone”. So within reasonable delay, all agents know (and propagate) state information about other agents. And you just so happen to have an agent running locally on your node, ready to share everything it knows via API, DNS or whatnot. How convenient!

Agents are also the ones performing health checks to the services on the hosts they run on, and gossiping any health state changes. To make this work, every service must expose a means to query its health status, and when registered with its local Consul agent, also register its health check information. At Outbrain we use an HTTP based “SelfTest” endpoint that every one of our homegrown services exposes (through our OB1K container, practically for free!).

Consul servers are also part of the gossip pool and thus propagate state in the cluster. However, they also maintain a quorum and elect a leader, who receives all updates (via RPC calls forwarded from the other servers) and registers them in it’s database. From here on, the data is replicated to the other servers and propagated to all the agents via Gossip. This method is a bit different from other Gossip based systems that have no servers and leaders, but it allows the system to support stronger consistency models.

There’s also a distributed key-value store we haven’t mentioned, rich ACLs, and a whole ecosystem of supporting and derived tools… but we said we’d keep it simple for now.

Where does that help with service discovery?

First, what we’ve done is taken all of our systems already organized in clusters and registered them with Consul. Systems such as Kafka, Zookeeper, Cassandra and others. This allows us to select a live service node from a cluster, simply by calling a hostname through the Consul DNS interface. For example, take Graphite: Outbrain’s systems are currently generating ~4M metrics per minute. Getting all of these metrics through a load balancer, or even a cluster of LBs, would be suboptimal, to say the least. Consul allows us to have each host send metrics to a hostname, such as “graphite.service.consul”, which returns a random IP of a live graphite relay node. Want to add a couple more nodes to share the load? no problem, just register them with Consul and they automagically appear in the list the next time a client resolves that hostname. Which, as we mentioned, happens quite a few times a minute. No load balancers in the way to serve as choke points, no editing of static config files. Just simple, fast, out-of-band communication.

How do these 3rd party services register?

We’re heavy users of Chef, and have thus created a chef cookbook to help us get the job done. Here’s a (simplified) code sample we use to register Graphite servers:

ob_consul 'graphite' do
  owner 'ops-vis'         ## add ‘owner’ tag to identify owning group
  port 1231               ## port the service is running on
  check_cmd "echo '' | nc localhost 1231 || exit 2"    ## health check shell command
  check_interval '60s'    ## health check execution interval
  template false          ## whether the health check command is a Chef template (for scripts)
  tags [‘prod’]           ## more tags
end

How to do clients consume services?

Clients simply resolve the DNS record they’re interested in… and that’s it. Consul takes care of all the rest, including randomizing the results.

$ host graphite
graphite.dc_name.outbrain.com is an alias for relayng.service.consul.
relayng.service.consul has address 10.10.10.11
relayng.service.consul has address 10.10.10.12

How does this data reach the DNS?

We’ve chosen to place Consul “behind” our internal DNS servers, and forward all requests for the “consul” domain name to a consul agent running on the DNS servers.

zone "consul" IN {
    type forward;
    forward only;
    forwarders { 127.0.0.1 port 8600; };
};

Note that there’s other ways to go about this, such as routing all DNS requests to the local Consul agent running on each node, and having it forward everything “non-Consul” to your DNS servers. There’s advantages and disadvantages to each approach. For our current needs, having an agent sit behind the DNS servers works quite well.

Where does the Consul implementation at Outbrain stand now?

At Outbrain we’re already using Consul for:

  • Graphite servers.
  • Hive Thrift servers that are Hive interfaces to the Hadoop cluster they’re running on. Here the Consul CNAME represents the actual Hadoop cluster you want your query to run on. We’ve also added a layer that enables accessing these clusters from different datacenters using Consul’s multi-DC support.
  • Kafka servers.
  • Elasticsearch servers.

And our roadmap for the near future:

  • MySql Slaves – so we can eliminate the use of HAProxy in that path.
  • Cassandra servers where maintaining a list of active nodes in the app configuration becomes stale over time.
  • Prometheus – our new monitoring and alerting system.
  • Zookeeper clusters.

 

But that’s not all! stay tuned for more on Consul, client-side load balancing, and making your environment more dynamic.