Your “network" is a community that helps
keep you united with others and offers many benefits.
Networking
via social media sites has revolutionized how we use the Internet and is at the forefront of what we now call Web 2.0.
Facebook
is social networking. People have been “facebooking” each other for about 6 years now, making Facebook the most
used social network with over 350 million users worldwide.
But how does Facebook work?
In this article, we will discuss
Facebook’s inner workings, covering its architecture and frontend/backend infrastructure””the nuts and bolts
that hold Facebook together.
How Does
Facebook Work? The Front End
Facebook uses a variety of services, tools, and programming languages to make up its core
infrastructure.
At the front end, their servers run a LAMP (Linux, Apache, MySQL, and PHP) stack with Memcache. Not a computer science
expert? Let’s take a look at exactly what that means.
Linux & Apache
This part
is pretty self-explanatory. Linux is a Unix-like computer operating system kernel. It’s open source, very customizable,
and good for security. Facebook runs the Linux operating system on Apache HTTP Servers. Apache is also free and is the most
popular open source web server in use.
MySQL
For the database, Facebook utilizes MySQL because of its speed and reliability. MySQL is
used primarily as a key-mly distributed amongst rge set of logical instances. These logical instances are spread out across
physical nodes and load balancing is done at the physical node level.
As far as customizations are concerned, Facebook has developed a custom partitioning scheme
in which a global ID is assigned to all data.
They also have a custom archiving scheme that is based on how frequent and recent data is
on a per-user basis. Most data is distributed randomly.
PHP
Facebook uses PHP because it is a good web programming language with extensive support and
an active developer community and it is good for rapid iteration. PHP is a dynamically typed/interpreted scripting language.
Memcache
Memcache
is a memory caching system that is used to speed up dynamic database-driven websites (like Facebook) by caching data and objects
in RAM to reduce reading time. Memcache is Facebook’s primary form of caching and helps alleviate the database load.
Having
a caching system allows Facebook to be as fast as it is at recalling your data.
If it doesn’t have to go to the database it
will just fetch your data from the cache based on your user ID.
Downsides to Using LAMP
Facebook
has realized that there are downsides to using the LAMP stack. Notably, PHP is not necessarily optimized for large websites
and therefore hard to scale. Also, it is not the fastest executing language and the extension framework is difficult to use.
Mike Schroepfer,
Facebook’s Vice President of Engineering, recently did an interview at EmTech@MIT concerning this. “Scaling any
website is a challenge,” Schroepfer said, “but scaling a social network has unique challenges.”
He went
on to say that unlike other websites, you can’t just add more servers to solve the problem because of Facebook’s
“huge interconnected dataset.” New connections are created all the time due to user activity.
Facebook
has grown so quickly that they are often faced with issues regarding database queries, caching, and storage of data. Their
database is huge and largely complex. To account for this, Facebook has started a lot of open source projects and backend
services.
How Does Facebook Work? The Back End
Facebook’s
backend services are written in a variety of different programming languages including C++, Java, Python, and Erlang. Their
philosophy for the creation of services is as follows:
1. Create a service if needed
2. Create
a framework/toolset for easier creation of services
3. Use the right programming language for the task
A list
of all of Facebook’s open source developments can be found here. I will discuss a few of the essential tools that Facebook
has developed.
Thrift (protocol)
Thrift is a lightweight remote procedure call framework
for scalable cross-language services development. Thrift supports C++, PHP, Python, Perl, Java, Ruby, Erlang, and others.
It’s quick, saves development time, and provides a division of labor of work on high-performance servers and applications.
Scribe
(log server)
Scribe is a server for aggregating log data streamed in real-time from many other servers.
It is a scalable framework useful for logging a wide array of data. It is built on top of Thrift.
Cassandra (database)
Cassandra is a database management system designed
to handle large amounts of data spread out across many servers. It powers Facebook’s Inbox Search feature and provides
a structured key-value store with eventual consistency.
HipHop
for PHP
HipHop
for PHP is a source code transformer for PHP script code and was created to save server resources.
HipHop transforms PHP source code into optimized C++. After doing this, it uses g++ to compile
it to machine code.
Conclusion
In a nutshell, that’s Facebook.
This article could easily be 37 pages longer if I
were to go into more detail, but to answer the question “How does Facebook work?” we think this will suffice.
If you
look past all of the features and innovations the main idea behind Facebook is really very basic keeping people connected.
Facebook realizes the power of social networking and is constantly innovating to keep their service the best in the business.
From various readings and
conversations, our understanding of Facebook's current (and secret!) architecture is:
Web front-end written in PHP. Facebook's
HipHop Compiler [1] then converts it to C++ and compiles it using g++, thus providing a high performance templating and Web
logic execution layer.
Because of the limitations of relying
entirely on static compilation, Facebook's started to work on a HipHop Interpreter [2] as well as a HipHop Virtual Machine
which translate PHP code to HipHop ByteCode [3].
Business
logic is exposed as services using Thrift [4]. Some of these services are implemented in PHP, C++ or Java depending on service
requirements (some other languages are probably used...)
Services
implemented in Java don't use any usual enterprise application server but rather use Facebook's custom application server.
At first this can look as wheel reinvented but as these services are exposed and consumed only (or mostly) using Thrift, the
overhead of Tomcat, or even Jetty, was probably too high with no significant added value for their need.
Persistence is done using MySQL, Memcached
[5], Hadoop's HBase [6].
Memcached
is used as a cache for MySQL as well as a general purpose cache.
Offline processing is done using Hadoop and Hive.
Data such as logging, clicks and feeds transit using Scribe [7] and are aggregating and stored in HDFS using Scribe-HDFS
[8], thus allowing extended analysis using MapReduce
BigPipe [9] is their custom technology to accelerate page rendering using a pipelining
logic
Varnish Cache [10] is used for HTTP proxying. They've prefered
it for its high performance and efficiency [11].
The storage of the billions of photos posted by the users is handled by Haystack, an ad-hoc
storage solution developed by Facebook which brings low level optimizations and append-only writes [12].
Facebook Messages is using its own architecture
which is notably based on infrastructure sharding and dynamic cluster management. Business logic and persistence is encapsulated
in so-called 'Cell'. Each Cell handles a part of users ; new Cells can be added as popularity grows [13].
Persistence is achieved using HBase [14].
Facebook Messages' search engine is
built with an inverted index stored in HBase [15]
Facebook
Search Engine's implementation details are unknown as far as I know
The typeahead search uses a custom storage and retrieval logic [16]
Chat is based on an Epoll server developed in Erlang and accessed using Thrift
[17]
They've
built an automated system that respond to monitoring alert by launching the appropriated repairing workflow, or escalating
to humans if the outage couldn't be overcome [18].
About the resources provisioned for each of these components, some information and numbers
are known:
Facebook
is estimated to own more than 60,000 servers [18]. Their recent datacenter in Prineville, Oregon is based on entirely self-designed
hardware [19] that was recently unveiled as Open Compute Project [20].
300 TB of data is stored in Memcached processes [21]
Their
Hadoop and Hive cluster is made of 3000 servers with 8 cores, 32 GB RAM, 12 TB disks that is a total of 24k cores, 96 TB RAM
and 36 PB disks [22]
100 billion hits per day, 50 billion photos, 3 trillion objects cached, 130 TB of logs per day as of july 2010 [22]
NOTE: GOOGLE
ARCHITECTURE IS LIKE FACEBOOK ARCHITECTURE....BUT...FACEBOOK ARCHITECTURE IS A SECRET....DON'T BELIEVE ME? TRY A GOOGLE SEARCH
YOURSELF OF "FACEBOOK ARCHITECURE" = ZERO IS BASICALLY WHAT YOU'LL GET!
Social Networking has become very
popular during the past few years, but it can still be very difficult to understand for someone new to social networking.
The open-ended nature of social networks add to this. Once signed onto a social network,
having answered a few basic profile questions, it is easy to sit back and wonder what you are supposed to do next.
The easiest way to understand social
networking is to think of it like high school. You had friends in school, and you knew quite a few people even if you weren't
friends with all of them, but it's likely that you didn't know everyone.
If you ever moved to a new school -- or if you can
imagine moving to a new school -- you start out with no friends.
After attending classes, you start meeting people, and as you meet them, you begin associating with those that have
similar interests.
Getting started with social networking
is much the same as starting at a new school.
At first, you don't have any friends.
But as you join groups, you begin
to meet people, and you build a friends list of those with similar interests.
Social networking is based on a certain structure
that allow people to both express their individuality and meet people with similar interests.
This structure includes having profiles, friends, blog posts, widgets, and usually something
unique to that particular social networking website -- such as the ability to 'poke' people on Facebook or high-five someone
on Hi5.
Profile.
This is where you tell the world about yourself. Profiles contain basic information, like
where you live and how old you are, and personality questions, like who's your favorite actor and what's your favorite book.
Social networks dedicated to a
special theme like music or movies might ask questions related to that theme.
Friends.
Friends are trusted members of the site that are allowed to post comments on your profile
or send you private messages.
You can
also keep tabs on how your friends are using social networking, such as when they post a new picture or update their profile.
Friends are the heart and soul of social networking.
It should be noted that not all social networks refer
to them as 'friends' -- LinkedIn refers to them as 'connections -- but all social networks have a way to designate members
as trusted.
Groups.
Most social networks use groups
to help you find people with similar interests or engage in discussions on certain topics.
A group can be anything from "Johnson High Class of '98" to "People Who Like
Books" to "Doors Fans".
They are both a way to connect with like-minded people and a way to identify your interests.
Sometimes,
groups are called by other names, such as the 'networks' on Facebook.
Discussions.
A primary focus of groups is to create interaction between users in the form of discussions.
Most social networking websites
support discussion boards for the groups, and many also allow members of the group to post pictures, music, video clips, and
other tidbits related to the group.
Blogs.
Another feature of some social networks is the ability to create your own blog entries.
While not as feature-rich as blog hosts like Wordpress or Blogger, blogging through a social network is perfect for keeping
people informed on what you are up to.
Widgets.
A popular way of letting your personality shine through is by gracing your social networking
profile with web widgets.
Many social networks allow a variety of widgets, and you can usually find interesting widgets located on widget galleries.
Why Start Social Networking?
Social Networking is a nice form of entertainment, great for meeting people with similar
interests, and can be a very effective business technique for entrepreneurs, writers, actors, musicians or artists.
Most of us have hobbies, or things
that we are keenly interested in such as books, television, video games or movies.
Social networks allow us to reach out to others that
have the same interests.
For example, if you like movies, Flixster can help you decide if you might like a new movie just out at the theaters.
If you like music, Last.FM can help you find new artists that are similar to your favorite bands.
Am I Too Old To Start Social Networking?
You are never too old to get involved
in social networking!
There are plenty of social networks to choose from including niche social networks that focus on a specific theme
like TakePart, a network for social activism, and 43 Things, a social network dedicated to goal setting.
Facebook is the world’s largest social network, boasting
more than 900 million users and dominating the online world in several countries around the world. On May 18, the company
is expected to go public and raise several billion dollars that it can use to fund acquisitions and promote future grow.
It’s a lofty goal, for sure, but it’s one that, given the history of major Web
IPOs, doesn’t seem out of the question.
Yet another thing that’s not out of the question as it relates to Facebook is its
ability to become the next Google. Since its IPO, Google has become a Web and technology industry juggernaut, establishing
itself as a leader in search, advertising, and even mobile phones.
In addition, the company’s innovative exploits are second to none.
But for the first time in Google
relatively brief and storied history, a new company has come along with a record of rapid growth and global presence that
could challenge even Google’s unparalleled record.
If Facebook makes the right management decisions and continues to innovate in the social
networking space it might use its IPO riches to become an even greater technology industry powerhouse.
Read on to find out how Facebook
could become the next Google:
1. Consider the IPOs
The similarities between Google and Facebook perhaps become most striking when one considers
their initial public offerings. As of this writing, Google’s IPO is the largest of any Web firm. But Facebook is on
its way to shattering that record. First things first: in order for Facebook to become the next Google, it needs to follow
in its footsteps. It appears to be doing that with its IPO.
2. Advertising
Although Facebook started without a clear path to monetization, the company is now doing
a fine job of generating cash off of its user base.
In fact, last quarter, it made about $1 billion in revenue by advertising alone. And over
time, the company is expected to continue to increase its ad revenue as more marketers warm to the idea of promoting content
to people based on many more factors—like when they were born and where they work—than Google’s offering.
3. The user base is expanding
According to the latest S-1 Registration
Statement Facebook filed with the SEC, the company has 901 million monthly active users.
That’s a monstrous figure that is anticipated to hit 1 billion sooner rather than
later. The most important reason why Google became so rich and powerful is because of its ever-increasing worldwide user base.
So far Facebook has the same thing going for itself.
4. Facebook has eyes on other markets
Although Google started in search and monetized its
service with advertising, the company quickly expanded into other markets, including mobile, maps, applications many others.
Facebook, meanwhile, is following that same path with talk of establishing a greater presence in mobile and even offering
an email service through its platform. Facebook is no longer just a social network.
5. The founders remain in control
One of
the interesting things about Google is that its co-founders Sergey Brin and Larry Page stated their intention to retain some
control over their company’s direction. In fact, they both had a controlling stake in decision-making at Google even
after its IPO.
After Facebook’s IPO, Zuckerberg will have a 57 percent ownership of the company’s shares giving him
total control over Facebook’s decision-making.
Make no mistake, Zuckerberg, Brin, and Page all love control.
6.
Regulatory concerns
Despite Google’s continued success, the company has faced a host of regulatory issues with governing bodies
in the U.S. and Europe. Looking ahead, it’s expected to face even more trouble. Even before its IPO, Facebook has come
under regulatory scrutiny over concerns about how its users data privacy. It even acknowledges in its own “risk factors”
in its S-1 Registration Statement that it could very well face more scrutiny in the coming years. See the similarities?
7. Facebook is indeed eyeing search
Although Google might not like
it, Facebook has been slowly but surely starting to creep its way in the search game. When users type in keywords into Facebook’s
search bar, they’re not only presented with access to pages related to that, but also Web results. With so many users,
why shouldn’t Facebook try its luck at search?
8. The app battle
Whereas Google has its own application stores, including Google Play and the Chrome Web
Store, Facebook allows users to access applications from its own service. With help from Facebook Credits, the social network
shares revenue with developers who earn cash from the applications they produce. It sounds awfully Google-like, doesn’t
it?
9.
Facebook’s Open Graph tentacles
One of the issues many stakeholders have with Google is that it attempts to bring its innumerable
services to as many Web sites as possible. Although Facebook started as a walled garden, the company’s Open Graph initiative
has allowed it to extend its tentacles across the Web. Look for that to continue in the coming years.
10. Power and influence
Finally, it’s difficult to
overestimate the sheer power and influence both Google and Facebook have in the online world. With so many users and so many
Web sites relying on their services for one thing or another, the companies have far more influence in the online world than
most others.
Chances are that kind of power will only be augmented as both companies pick up would-be competitors in blockbuster
acquisitions.
Clear-Cloud.com
Clear-Cloud.com is a Mobile Device Broker
reseller and Trainer Channel. Mobile professional buyers can source commercial surplus inventory (i.e, Enterprise-Corporate
Off-Loads) and government surplus assets in an online environment. Bulk lots are sold by the truckload, pallet, or small package,
and conditions range from new in a box to customer returns and used. Our wide variety of product categories includes
Smartphones, Laptops, Tablet PCs, Netbooks, eReaders, mobile Hotspot devices, and more.
Grade explanations:
Excellent
(A)
These units are "Like New" - Fully tested and functional, with no cosmetic or display defects. Excellent
condition all around. This Grade applies to approximately 95% of our Liquidation offerings.
Light Use (B)
These units have very light scratches and minor
cosmetic blemishes. These are fully tested and functional, with no display defects.
Moderate
Use (C)
These units have some very small scratches or scuffs. These are fully tested and functional units with no
display defects.
4GB 7” capacitive multi-touch touchscreen Tablet Android 4.0 camera,
3G New Min order 44 Price per unit $110.00 Off-loaded to
Enterprise-Corporate liquidation channels. Original acquisition was 334 units. Now 77 left. If interested email us at clearcloud101@gmail.com or call 1-661-670-6092. Min order
is 44 Tablets. Prior Enterprise owner was Allstate Insurance, Chicago, Il....USA only, NO foreign orders accepted.
Like New - 8GB 7" capacitive
touchscreen Tablet Android 4.0 webcam, 3G Min order 97 units, $46.75 per Tablet Prior owner of lot was Farmers Insurance San Francisco, Ca. A very
popular Tablet...an easy sell on your website, or, locally via swap meets. Original acquisition was 884 units. Now 122
left. If interested email us at clearcloud101@gmail.com or call 1-661-670-6092. Min order
is 97 Tablets. Prior Enterprise owner was Farmers Insurance, Austin, Texas....USA only, NO foreign orders accepted.
Verizon Mobile hotspot; Corporate-Enterprise
Off-load, Frys Computers Inc (prior lot owner was Frys.com, USA) 368 units left; min order 411 units, $9.22 each. An excellent
choice....sell locally or on your blog or website...make an easy $35 per to $55 per sale....very popular with businesses and
consumers. Original acquisition was 1,274 units. Now 868 left. If interested email us at clearcloud101@gmail.com or call 1-661-670-6092. Min order
is 411 Verizon Mobile Hotspots. Prior Enterprise owner was Frys.com Woodland Hills, Ca....USA only, NO foreign orders accepted. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Panasonic 3D Blu-ray player
Semi New - Min order 397 units, $23.75 per Blu-ray player Prior owner of lot was Pacific Premier, LA Ca. A very popular Blu-ray player...an easy sell on your website, blog, or, locally via swap meets. Original acquisition was 884 units. Now
977 left. If interested email us at clearcloud101@gmail.com or call 1-661-670-6092. Min order is 397 Blu-ray players. USA only, NO foreign orders accepted. +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ C Grade Dell Laptops original MSRP $2106
SEE IMAGE BELOW
Original Enterprise owner of this lot was 3M Corporation, Dallas Texas Lot acquisition was 3,345 units....min order is 419 Laptops.....$113.55 each. C Grade Dell Laptops original MSRP $2106 Corporate-Enterprise Off-load, 3M Corporation,
Dallas Texas (prior lot owner
was 3M Corporation, Dallas Texas USA) 211 units left; min order 419 units, $113.55 each. An excellent choice....sell locally
or on your blog or website...make an easy $255 per sale....very popular with businesses and consumers. Original acquisition was 3345 units. Now 211 left. If interested email
us at clearcloud101@gmail.com or call 1-661-670-6092. Min order is 419 C Grade Dell Laptops. Prior Enterprise owner was 3M corporation Dallas Texas....USA only, NO
foreign orders accepted. ++++++++++++++++++++++++++++++++++++++++++++++
T-Mobile Samsung T679 Exhibit II 4G Prepaid Android Smartphone (UNLOCKED) (SEE IMAGE BELOW) Min 193 units $36.75
each Original Enterprise owner
of this lot was SEARS CORP. NEW YORK, Lot acquisition was 1323 units....min order is 193 Units.....$36.75 each. (An excellent choice....sell locally or on your blog or website...make
an easy $98 to $133 per sale....very popular with businesses and consumers) If interested email us at clearcloud101@gmail.com or call 1-661-670-6092. USA only, NO foreign orders accepted.
With the huge proliferation of Mobile
Devices and Mobile Cloud Computing...the Enterprises-Corporates we work with are now putting thousands of VALUABLE Mobile
Devices onto the IT Asset Disposal pipeline.
For example, Hertz Intl in New York just released 1,183 business Laptops
from HP...onto the IT Asset Disposal Off-Load channel.
These are like new devices - used by their employees only for 122 days. They have embedded 4G Broadband modem, 4GB
Ram, quad core CPU, optical drive; 7500 rpm 2TB hard drive...Windows Professional 7.....wifi
802.11 n.....17.6 wide screen...and Bluetooth.
These devices are available to our Brokers for $38 each...on Amazon and eBay the same models sell for
$198 to $337 every day. So there is lots of room for profits....but the new Broker has to realize that the units can only
be acquired by lot-batches of (usually) minimum 110 to 127.
The units have their hard drives scrubbed...then a new version of the op sys is installed....and
usually the basic set of business applications.
We have found - without a doubt - that our most successful Brokers make the most money by performing
FACE-TO-FACE sales...that is, at swap meets, flea markets, and by the Broker placing small cheap ads in the local newspaper....this
is much easier and BETTER than selling on the web....because...we show how, once you make a face-to-face sale, you gain a
long term repeat client.
Dell Laptops Dell Laptops Latitude business laptop original MSRP $1106
SEE IMAGE ABOVE Original Enterprise owner of this lot was Sears Corp Boston Ma
Lot acquisition was 5,895 units....min order is 106 Laptops.....$55.99 each. Corporate-Enterprise Off-load, Sears Corp Boston
Ma
(If interested email us at clearcloud101@gmail.com or call 1-661-670-6092. USA only, NO foreign orders accepted. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Dell Laptops business
SEE IMAGE BELOW Original Enterprise owner of this lot was Sears USA, New York, NYC
Lot acquisition was 4335 units....min order is 77 Laptops.....$66.99 each. Dell Latitude Laptops
original MSRP $499.00 Corporate-Enterprise
Off-load, Boston Ma
(An excellent choice....sell locally or on your blog or website...make an easy $114 to $255 per sale....very popular
with businesses and consumers) If
interested email us at clearcloud101@gmail.com or call 1-661-670-6092. USA only, NO foreign orders accepted.
Dell Laptops business
RIM Blackberry Curve 3G 9330 Smartphone (SEE
IMAGE BELOW) Min 186 units $38.65 each Original Enterprise owner of this lot was Sears Corp El Paso Tx. Lot acquisition was 1266 units....min order is 186 units.....$38.65
each. (An excellent choice....sell
locally or on your blog or website...make an easy $177 to $229 per sale....very popular with businesses and consumers) If interested email us at clearcloud101@gmail.com or call 1-661-670-6092. USA only, NO foreign orders accepted.
SEE IMAGE BELOW
rim-smartphone
Get with it....Now! Realize that today many, many,
many of “yesterday’s jobs” are going bye..bye....... Why? Because of super smart cloud based algorithms, advanced cloud computing concepts, and
AUTOMATION. You can join www.clear-cloud.com in the Mobile Cloud Computing Revolution (email us at clearcloud101@gmail.com) …..you can easily make over $2K weekly
by acquiring the Enterprise-Corporate semi-new Mobile Devices at pennies on the dollar, and then selling these devices at
your regional-local swap meets....clearing (without breaking a sweat) an average of $100 per sale.....and remember....the
HUGE American “underground economy” is $7.7 Billion dollars, (i.e., swap meets!) and is growing....weekly.....
Dell Business Laptop MSRP $588
Min lot buy is 51 units...price per laptop is $71.22 This ultramobile 13” business laptop offers vital security
features and 3rd Gen Intel® Core™ processors. Stay protected, boot up and load apps in a flash. Dell Laptops Vostro 3360 (SEE IMAGE BELOW) businesslaptops SEE IMAGE BELOW Original Enterprise owner of this lot was Sears USA, Chicago, Lot acquisition was 4935 units....min order is 51 Laptops.....$71.22 each. (An
excellent choice....sell locally or on your blog or website...make an easy $214 to $275 per sale....very popular with businesses
and consumers) If interested
email us at clearcloud101@gmail.com or
call 1-661-670-6092. USA only, NO foreign orders accepted. ++++++++++++++++++++++++++++++++++++