Nitsan Friedrich

How to take innovation into production

old-new-computers

Outbrain’s Hackathon

The Outbrain Hackathon which is held twice a year, is a 24-hours event in which employees and friends are invited to build and present an original product or innovation.

The Hackathon is a mini festival held at all Outbrain’s offices around the globe where the offices are open for 24 hours and meals and beers are served all day long.

The winning team is rewarded with a worthy present and the opportunity to turn the idea into a working feature/product.

Few weeks before the event, people start to raise ideas and to team up.

In the beginning of the event, a representative from each team has 5 minutes to present his/her idea in front of the whole company.

In the end of the event, each representative presents a demo of the software his/her team developed.

Immediately afterwards, a vote is conducted and the winners are declared.

In this post I will share my experience from the Hackathon which my team and I won.

 

About Our hack: Let’s do some innovation

One of our legacy services is called the “Editorial Reviewer”.

It is a user interface for approving/rejecting newly created promoted content, based on Outbrain’s content guidelines (See: http://www.outbrain.com/amplify/guidelines).

This was an old service from Outbrain’s early days. It was slow and was using old technologies and frameworks. We decided to rewrite it with a fresh breeze of technologies and make it function and look awesome.

 

Let’s get to work…

Prior to the Hackathon we did some research about what we could remove or improve about the current service and if there are new features or demands from the users we could add during the makeover.

We then divided the work among the team members and chose the technologies and frameworks based on our needs and desires.

One of our main goal was to improve the performance of the old tool.

Switching from multi page web application architecture to a single page web application made a real change but wasn’t enough.

The real challenge was to speed up the database access calls that the service makes.

We analyze the current queries, and found out they can be dramatically improved.

After a few hours, we already had a working demo. It looked a bit childish but it was already performing much better!!

We decided to get some advice from our masters and took it up with our UX designer who came up with a really cool sketch which we were excited to implement.

After a long night of hacking and tons of coffee we finally had an impressive working demo that we could present to the company.

The feedbacks we got were awesome! The teams from Outbrain that were using the old tool were super excited and couldn’t wait to start using our new hack.

As part of the prize, we got the time to develop it into a full-blown product.

 

2 months later, we got the chance to invest more time in our idea.

We added more tests, monitors and dashboards, did some fine-tuning and at the end, came up with a really cool and sexy single page application that was much faster, comfortable and reliable than the old tool.

 

Summary

The atmosphere was great. All participants worked around the clock and did their best to kick ass!

The challenge of working on a project that we chose and the fact we were striving to make it happen regardless of the tight time frame was amazing and so was the final outcome.

How to substitute dynamic URL parameters: a template engine use case

Background

Outbrain is one of the world’s largest content discovery platforms, serving more than 200B recommendations monthly and reaching over 561 million unique visitors from across the globe.

As Outbrain becomes an important source of traffic and revenues for publishers, partners are looking for more tracking capabilities that will help them determine the amount of traffic generated by Outbrain, and how this traffic is distributed throughout their site. In addition, publishers want to see this information on their analytic tool of choice (Omniture, GA, Webtrends etc.).

Outbrain now provides them with tracking parameters that can be appended to the URL of recommendations.

Motivation

A url is constructed of three parts: prefix, actual url and suffix, where the prefix and suffix serve as means to let our partners add tracking capabilities in the form of dynamic parameters (for instance: the url’s title, publish date and id). Eventually, the dynamic parameters are added as a query string (field1=value1&field2=value2&field3=value3) to the recommendation url.

The legacy code would contract the three url parts and to use String replace in order to add the dynamic parameters. This implementation was hard to maintain (in situations where we wanted to support more dynamic parameters) as well as difficult to use since there was a need to import a lot of code to a project, and to depend on many modules.

When revisiting the problem, we understood that it would be appropriate to use a Separation of Concerns approach, separating the template from the model, and from the transformation logic itself – a template engine sounded like the right choice! In addition, since more and more dynamic parameters were being added, we used a builder pattern in order to achieve an easier and cleaner usage for clients, and easier maintenance in the future.

The problem – using a template engine doesn’t guarantee better performance

We decided to use StringTemplate as our template engine since we were familiar with it and had some good experience using it. The result of the refactor was a cleaner, shorter and maintainable API.

Unfortunately, when we deployed the new changes to production, we noticed a significant increase in serving time that was unacceptable in terms of user experience. After investigating the root cause, we found out that the usage of StringTemplate was pretty expensive. Even though the templates could be reused, we couldn’t reuse them all. For instance, we created a new template for each request, since it was constructed with different dynamic parameters. (The form of the url though, was the same: prefix, actual url and suffix).

So at that moment we had a clean and elegant solution that wasn’t performing well. We then looked for some alternative solutions for StringTemplate that could save us the expensive cost of constructing a new template for each request.

The solution – right tool for the job

Eventually, we found a light-weight template engine, that allowed us to keep using the Separation of Concerns approach and still achieve good performance. We ended up using Apache Commons Lang 3.0 StrSubstitutor – a simpler alternative to StringTemplate. This time, we made sure that it outperformed our last implementation by doing some micro benchmarking, and indeed the results were much better. The new implementation executed more than 4 times operation per second.

MicroBenchmarking Results

We used Java Microbenchmark Harness in order to perform our performance measurements.

microbenchmark

Raw data:

raw_data

For more information see: http://openjdk.java.net/projects/code-tools/jmh/