Facebook Stream Changes through Worm Hole

Facebook engineering commented on its in-house implementation of Web Scale change propagation messaging infrastructure called Worm Hole. Essentially it Tails on a DB / FS updates and publisher propagates them over streams across data center.

It seems to support some basic messaging Properties:

1. Reliable In-Order Delivery

2. Automicity

3. Low latency

with some web scale abilities:

1. Data Partitioning

2. Rewind in Time

Facebook engineering Listed Advantages as:

1. Volume: Wormhole processes over 1 trillion messages every day (significantly more than 10 million messages every second).

2. Ability to deal with failures

3. Ability to scale up / down

4. INtegration with monitoring systems

5. Cache synchronization across data centers in a shorter span

6. Lowe consumption of resources (Quoate: Compared to the previous system, Wormhole reduced CPU utilization on UDBs by 40% and I/O utilization by 60%)

This is not yet open source.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s