How fast can you scale?

The first & foremost thing that any popular web application on Internet today tries to achieve is to provide its user a memorable & seamless UI experience. And do we realize what this great popularity on Internet means in terms of traffic or usage or pure numbers.

Just to bring your attention to some usage statistics on few popular sites

-  Twitter hits 1 billion queries per day & see 50 million tweets per day which means 600 tweets per second

-  Facebook gets 60 million status updates per day & 2.5 billion photos are uploaded on it each month

-  Google is processing 2 billion search queries per day

-  YouTube get 5 billion video streams every month & gets 15 hours of video uploaded every minute

-  Flickr now hosts more than 4 billion images

But do you think it's so easy for all these websites to handle these millions & billions of user requests each day & still give a seamless & uninterrupted browsing experience. 

This thought made me really curious to study the web architecture of some of these websites that see massive traffic every day. And what I found was that all of these have almost similar kind of architecture with minor differences that has resulted in so robust & efficient systems.

The very backbone of all these website is this very popular LAMP architecture. LAMP is an acronym for a solution stack of free, open source software, originally coined from the first letters of Linux (operating system),Apache HTTP Server, MySQL (database software) and PHP/Perl/Python scripting languages. The software combination has become popular because it is free of cost, open-source, and therefore easily adaptable. Moreover almost every distribution of Linux includes Apache, MySQL, PHP, and Perl, so installing the LAMP software is almost as easy as saying it.

Now to appreciate the most sophisticated web architecture one must should also be aware of simple & less efficient solutions as well. So let me talk about different solutions around this basic LAMP architecture.


1. One “Box” Solution

- Basic Web application

- Low Traffic

- Apache/MySQL/PHP on one machine

- Bottlenecks are Disk I/O and Context     switching 

2. Two "Box" Solution

- Higher traffic application

- Apache/PHP on box A and MySQL on box B 

- Bottlenecks like Disk I/O, Network I/O 

3. Many "Boxes" Solution with Replication

- Yet even Higher traffic

- Apache/PHP on box A & MySQL on many boxes

- Writes are separated from reads. Web Servers have read/write ratio of somewhere between 80/20 and 90/10.

- Master gets Insert/Update/Delete & slaves get Select

- Load Balancing is used

Bottleneck
- Slaves can’t keep up with replication as they are too busy Reading (production traffic) and   Writing (replication)

- So this manifests as Comments/photos/any user entered data doesn’t show up on the site right away. So users will repeat the action thinking that it didn’t happen the first time, making situation worse.

4. Many Boxes Solution with Hardware Load Balancing MySL

- Standard MySQL master/slave replication

- All writes (inserts/updates/deletes) from application go to Master 

- All reads (selects) from application go to a load balanced VIP (virtual IP) spreading out load across all slaves

Benefits of Load balancing
 - Add/remove slaves without affecting applications

- Additional monitoring point & some automatic failure handling 

- Capacity planning lot easier if the ceiling of each slave is known.

Some more add-ons or tweaks
- Web Server (machine with APACHE/PHP) can also be scaled with up with multi-threading.
- Increase App Servers too with a load balancing mechanism along with DB Clusters. (Each cluster has Master-Slave DB’s of their own).
- Add caches to your App Server. Cache your static content. SQUID is good.
- Also have Memcached which is distributed memory caching solution.
- Use RAID10 instead of RAID5 as it has more read capacity & less write penalty
- Use interleaving memory
- Choose SCSI hard drive over SATA
- Use MySQL with SAN (Storage Area Network). SAN is better than RAID.
- Use CDNs (Hello Akamai)


Hence if we look at an overall picture of any good & robust architecture behind any website, we can easily generalize into something I've shown below.