“I'm not afraid of failing; that's the only way to reach success.” - Nuno Peralta

Scalability Methods

« Introduction
Content Delivery Network »


Clustering is a popular method of scaling a web application. A cluster is a set of connected computers that work together, and they can be viewed as a single system. This method consists in continuously adding servers to the cluster, in order to distribute the traffic between resources.

Facebook runs thousands of MySQL servers in multiple data centers, and it doesn’t really matter if a single server disappears, because other servers will back it up [2].

When using Clustering, a master/slave configuration should be used. The Master server has the most updated files of the application and other configurations, while all the Slaves will just synchronize from the Master when convenient.

When a user types the domain on the address bar of the browser, it will look up the IP address for that domain name, if not in cache yet. To mitigate the bottleneck, big websites use Round-robin DNS.

Using Round-robin, DNS lookup returns multiple IP addresses, rather than just one. For example, Facebook maps to four IP addresses. A root domain with A records point directly to the servers of that domain. Multiple IP addresses are returned to provide failover in case one or more of the servers are down. It’s possible that different IP addresses are returned based on the load of the servers, but, random distribution of IP addresses should give enough load distribution [3].

In addition, in order to make Clustering to work, expensive high-performance Load-balancers should be installed. This is a piece of software or hardware that listens on a particular IP address and forwards the requests to other servers [4]. This can be done with a multilayer switch, called Layer 4 Load Balancer, more correctly a NAT with port and transaction awareness, that sends incoming packets to machines that are hidden behind a single IP address [5].

When we ping static.ak.fbcdn.net, we get a response from an akamai.net server. Also, if we ping a couple of times, we may get responses from different servers, which demonstrates the load-balancing that the servers use to support the big traffic incoming [4].

Figure 1. Cluster of machines running Hadoop at Yahoo! (Source: Yahoo!, 2007).
[2] Rodney Gedda (Feb 24, 2010). Open source helps Facebook achieve massive app scalability. CIO. http://www.cio.com.au/article/337284/open_source_helps_facebook_achieve_massive_app_scalability/
[3] Tom Robinson (May 11, 2011). Scalability: How does Heroku work?. Quora. http://www.quora.com/Scalability/How-does-Heroku-work
[4] Igor Ostrovsky (Feb 9, 2011). What really happens when you navigate to a URL. Igor Ostrovsky Blogging. http://igoro.com/archive/what-really-happens-when-you-navigate-to-a-url/
[5] Malcolm Turnbull. Layer-4 Load Balancing. Load Balancing FAQ. http://www.loadbalancing.org/#Layer-4_Load_Balancing_

Article written by Nuno Peralta, 2012