In the cloud networking course, we will see what the network needs to do to enable cloud computing. We will explore current practice by talking to leading industry experts, as well as looking into interesting new research that might shape the cloud network’s future. This course will allow us to explore in-depth the challenges for cloud networking—how do we build a network infrastructure that provides the agility to deploy virtual networks on a shared infrastructure, that enables both efficient transfer of big data and low latency communication, and that enables applications to be federated across countries and continents? Examining how these objectives are met will set the stage for the rest of the course. This course places an emphasis on both operations and design rationale—i.e., how things work and why they were designed this way. We're excited to start the course with you and take a look inside what has become the critical communications infrastructure for many applications today.
1.1.1 Application and Traffic Patterns
Loading...
來自 University of Illinois at Urbana-Champaign 的課程
Cloud Networking
148 個評分
從本節課中
Week 1
We will start by looking at the applications running in data centers and the traffic patterns resulting from them. This will help us differentiate the networking needs within data centers from computer networking in general and set the stage for other material in the course. We’ll then dive into the first of these topics, physical network structure, where we will look at how the increasing demand for bandwidth within data centers is influencing the network’s design.
與講師見面
P. Brighten GodfreyAssociate Professor
Department of Computer Science
Ankit SinglaAssistant Professor
Department of Computer Science, ETH Zürich
In this lesson we'll examine data center applications
and network traffic,
and see what is implies for the network's design.
Pretty much every popular web application
we use today runs on data center networks.
Not only do data centers run these applications,
but they also run support tasks,
which make the information available
for these applications.
For example, if you're thinking of FIB seach,
you have to create search indices
by collecting data from the web, and processing it.
So this is something that these data centers, servers,
work together on.
Such infrastructure also supports
big science applications, such as climate modeling.
All of this creates massive amounts of data,
being moved around inside data centers.
Let's take a slightly closer look at how something,
like a web search query, might work.
So I make a search query,
it hits a server in a data center.
This server might query several other servers
which further might communicate with other servers, and so on.
These responses are then collated,
and the final search response is sent to me.
This kind of a traffic pattern is referred to as,
scatter-gather, or partition aggregate.
So you're scattering the request,
and then you're gathering the results.
For one query, there might be a large number
of server to server interactions within the data center.
If you think this picture is complicated,
take a look at this.
This is what Bing's query workflow
for producing the first page of results
from a search query looks like.
So the query comes in here at the top.
It's split into multiple requests,
which further might be split into sub-requests and so on.
So you can see the scatter pattern in play here.
A server makes several different requests,
to fulfill the search query.
And then you can also see the gather pattern
closer to the end of this graph.
Where are the results
from multiple, different subqueries,
are gathered and processed together.
And in the end, a response is produced,
which is sent to the user.
The processing of a query can be a multi-stage process,
and each step can involve as many as
40 different sub-requests.
And this kind of scatter-gather pattern,
is not exclusive to search.
Facebook, for example, also sees several,
internal requests in processing a user request for a page.
Loading one of Facebook's popular pages causes an average
of 521 distinct requests on the backend.
For the same page in the 91st percentile,
you require 1740 items to be fetched.
So really, inside these web applications,
you can make a large number of requests
involving one user request.
Further these data centers also run,
big data processing tools, such as Hadoop, Spark,
Dryad, Databases,
which all work to process information
and make it available to these web applications.
These data processing frameworks
can move massive amounts of data around.
So what does the actual measured traffic
and data centers look like
when they're running these applications?
The short answer unfortunately,
even though it seems like a cop-out is, it depends.
On the applications that you're running,
on the scale you're running them at,
and the network's design as well as the design
of the applications.
But nevertheless, let's take a look
at some of the published data.
One thing that's unambiguously true
is traffic volume inside data centers is growing rapidly,
and is the majority of traffic that these servers see.
The majority of the traffic is not,
to and from the Internet but, is rather,
inside the data center.
Here what you're looking at is data from Google
showing the traffic generated by their servers,
and their datacenters over a period of time,
a bit more than six years which is on the X-axis.
And on the Y-axis is the aggregate traffic volume.
The absolute numbers are not available,
but over a six-year period
the data volume grows by 50 times.
Google has noted that this e-traffic doubling every year.
Google's paper is not quite clear about
whether this is data center internal traffic only
but Facebook has mentioned that machine to machine traffic
is several orders of magnitude larger
than what goes out to the Internet.
So really, most of the traffic in these facilities is going
to be within the facility, as opposed to
to and from the Internet,
and it's growing quite rapidly.
So what does this traffic look like?
One question we might want to have answered
is about locality.
Do machines communicate with neighboring machines,
or is traffic uniformly spread
throughout the data center or such?
So here we have some data from Facebook
that goes some way towards addressing this question.
Let's focus on this part of the table,
where all of Facebook's data center traffic
is partitioned by locality.
Within rack, within a cluster, within a data center,
or across data centers.
As you can see roughly 13% of the traffic is within a rack.
So these are just machines within the rack
talking to each other.
A rack might host some tens of machines, for example,
40 machines seems to be quite common.
Then we see that there is 58% of traffic,
which stays within a cluster but not within the rack.
So this traffic is across racks within a cluster.
Further, 12% of the traffic stays within the data center,
but it not cluster local.
So this is traffic,
between multiple clusters in the data center.
Also interesting is that 18% of the traffic
is between data centers.
This is actually larger than the rack local traffic.
So locality in this work load is not really high.
Also worth noting, is that Hadoop
is the single largest driver of traffic
in Facebook data centers.
More data at rack locality comes from Google.
What you're looking at here
is data from 12 blocks of servers.
Blocks are groups of racks,
so a block might have a few hundred servers.
This is a smaller granularity than a cluster,
but a larger granularity than a rack.
Here, what you're looking at, in the figure on the right
is traffic, or a block, that is leaving for other blocks,
so that is non-local traffic.
For each of these 12 blocks in the figure,
you see that most of the traffic is non-local,
so most of the traffic goes to other blocks.
Now there are 12 blocks here.
If traffic was uniformly distributed,
you would see 1/12 of the traffic being local,
and 11/12, that is roughly 90%, being non-local,
which is exactly what this graph shows.
Part of this definitely stems
from how Google organizes storage.
The paper notes, that for great availability,
they spread data around different fall domains.
So for example, a block might have all its power supply
from one source.
If that power supply fails,
you lose everything in that block.
So if data is spread around well, over multiple blocks,
you might still have the service be available.
So this kind of organization is good for availability,
but is bad for locality.
Another set of measurements comes from Benson et al.,
who evaluated three university clusters,
two private enterprise networks,
and five commercial cloud networks.
The paper obscures who the cloud provider is,
but each of these datacenters host 10,000 or more servers
running applications including web search,
and one of the authors works at Microsoft.
The university and private data centers
have a few hundred to 2,000 servers each.
While the cloud data centers here, have 10 to 15,000 servers.
The cloud data centers one through three,
run many applications including web, mail, etcetera,
but cloud four and five run more MapReduce style workloads.
One thing that's worth noticing is,
the amount of rack-locality here is much larger.
70% or so for the cloud data centers,
which is very different from what we saw earlier
for Google and Facebook measurements.
There're many possible reasons for these differences.
For one, the workloads might be different.
Not even all MapReduce jobs are the same.
It's entirely possible that Google and Facebook
run large MapReduce tasks which do not fit in a rack.
While this mystery data center runs smaller tasks
that do fit in a rack, and, hence,
traffic is mostly rack-local.
There might also just be different ways
of organizing storage and compute.
There's also a five-year gap
between the publishing of these measurements.
Perhaps, things are just different now,
hap sizes might have grown substantially,
or people have changed how they do these things.
Having looked at locality, let's turn our attention
to flow level characteristics.
How many flows does a server see concurrently?
Facebook's measurements, and I quote here:
And Hadoop nodes have, approximately,
25 concurrent connections on average.
In contrast, for a 1500 server cluster,
running Hadoop style workloads,
the numbers are much smaller.
Two servers within a rack and four servers outside a rack
is the number of correspondents for a server.
So a server talks to six other servers in the median.
Contrast that with the 25 in Facebook's workloads.
The only things we can conclude from this, perhaps,
are that: A) there are differences across applications,
so web servers, caches, and Hadoop,
all have different numbers of concurrent flows.
And also, that even all Hadoop workloads
are not created the same.
So Facebook's Hadoop workloads seem to be different from this,
mystery data center workload.
Another thing we are interested in finding out,
is, what is the arrival rate for flows?
How soon do new flows arrive in the system?
Facebook's measurements put the median
flow inter-arrival time at a server
at approximately 2 milliseconds.
The mystery cluster has inter-arrival times
in 10s of milliseconds,
so a-less than a 10th of Facebook's rate.
Regardless of the difference in these two measurements,
one higher order bit that's worth mentioning here
is that at the cluster scale, the flow of inter-arrival times,
would be in the microseconds,
if you had a 1000 server cluster, for example.
So, really, new flows enter the system
at a very high frequency, as a whole.
What about flow sizes?
Most Hadoop flows, it seems,
are quite small in Facebook's measurement.
The median flow is smaller than a kilobyte,
and less than 5% of flows, exceed a megabyte,
or last longer than 100 seconds.
For caching workloads, most flows are long lived,
but they're burst internally.
So you can think of these as, perhaps,
long lived TCP connections,
where you do not want to incur the cost of a DCP handshake,
and the rate increase, due to slow start every time,
so you keep the DCP connection alive,
but data is only exchanged sporadically.
Also of interest are heavy hitters.
So, are there a small fraction of flows
that move most of the bytes?
For this set of measurements, the answer seems to be no.
Heavy-hitters are roughly the same size as the median flow,
and are not persistent.
So some flows which might be moving a large fraction
of the bytes, now, do not continue to be
the heavy-hitters in the next time interval.
These measurements do a great interval
from the measurements from the 1500 server cluster
also running Hadoop.
There are more than 80% of these servers,
last less than 10 seconds, but,
more than 50% of bytes are in flows
that are also small or short, less than 25 seconds.
Some part of this, in particular the heavy-hitters
being roughly similar in size to the median flow,
stands surely from application level load balancing.
For example, for caching, if you're spreading the load
across these different caches,
you will see roughly similar flow sizes.
So where do we go from here?
What does data center traffic look like?
We don't have a definitive answer.
And right now not a whole lot of data is available.
There are, however, some conclusions we can draw
from the nature of data center applications,
as well as the points of agreement
of these different data sets.
We look at these in detail, but briefly:
One, data center traffic, internally,
is quite big and increasing.
Two, we have very tight deadlines for network I/O.
That is, we are targeting small latencies
for applications, like web search in particular.
Three, because we have a large number of flows,
we'll see congestion in the network,
and problems like TCP incast.
Four, we need isolation across multiple applications
that are running in these data centers.
Five, centralized control at the flow level
might be difficult
because of the large flow rates we are seeing.
So let's talk about each of these in more detail.
One thing that's amply clear from these measurements
and from the growth of these applications,
is that need for bandwidth into the centers
is rapidly growing.
As the application scale, we need to provide
higher and higher capacity networks,
and in a cheap and scalable and fault-tolerant manner.
We want efficient network topologies
and efficient routing to achieve high capacity.
Apart form moving large volumes of data around the data center,
we also want to do this at very small latencies.
Particularly for applications like search,
where the deadlines for responses can be very small,
10's of milliseconds.
Just looking at this Bing query workflow tires me out.
Let's take a small candy break.
Here, I have a bunch of candy I've collected.
[Rattling]
And I'll drop in this, washed and cleaned, stone into it.
It's roughly the same size and weight
as the individual pieces of candy.
Now, if I were to pick a piece from this box randomly,
I'm highly likely to end up with candy.
No teeth broken.
So that worked.
But if I grab a handful of candy now,
let's see what happens.
Oh, here it is.
I have the rock now.
[Rattling]
This makes an important point.
It's a simple point about probability.
But when you are making a handful of selections,
you have a higher chance of coming up
with one of the undesirable objects.
If there's even just one undesirable
in a lot of objects you are choosing from.
This is a lesson you're sure to use
in terms of how web requests work.
So as we discussed,
the deadlines for network I/O in the data center
can be quite small.
Now suppose that all of your web servers are really nice,
and their response times are quite small.
For most requests, in fact, for 99% of the requests,
the issue response is in 10 milliseconds or less.
But for 1% of the requests,
they take an entire second to respond.
This is an example I'm stealing from Google's Jeff Dean.
Now if you're making just one request,
the odds that your response will take 1 second,
or be slower than that, are just 1%.
But, if you were to make hundred requests,
and wait for them all to finish,
the odds are 63% that you'll have to wait a second or more.
Just like choosing a handful of candy from that box.
You'll likely to end up with one of those servers
that responds slowly.
Now given of what you've seen
about the nature of the applications,
they make lots of small requests.
This kind of thing can happen quite often,
63% is very poor odds.
You do not want your service time to be such large numbers.
And as these services expand, the problem only gets worse.
Thus, what we need is to reduce variability in latency,
and also to tolerate some latency, perhaps,
at higher layers.
For example, the application might make redundant requests
in just takes the one that returns faster.
Because there will be some variation latency inherent
to anything that's networked.
With the large number of requests,
we'll also see a large number of flows that share
the network's capacity,
leading perhaps to congestion, and a problem called TCP incast.
TCP incast is something we'll look at in more detail
later in the course, but essentially,
when a large number of hosts try to share the same buffer,
when a large number of flows, try to share
the same network buffer,
they overwhelm that buffer, and network throughput drops,
and latency increases.
There are various application layer fixes to this,
but ultimately, all of these complicate
the application's logic.
This is a problem that we should attempt to solve in the network,
and we look at solutions to do that.
Further, with the variety of applications
that might be running together in these data centers,
particularly if you're a cloud provider, like Amazon,
you also need to look at isolating these applications
from each other.
So if the applications that are latency sensitive
are stuck in buffers behind applications
that are just moving bulk data,
we will see very poor quality of service.
So we need some way of isolating
these applications from each other.
Applications with different objectives
are, after all, sharing the network.
Lastly, given that data centers see very high flow arrival rates
and most of the flows are short, that is not persistent,
centralized control at the flow level
could be very difficult to scale.
Particularly to large deployments
of tens of thousands of servers.
So perhaps, we'll need distributed control,
possibly with some centralized tinkering.
The implications we have discussed here
will factor into our discussion of all the topics
we see in the course,
from routing, congestion control,
to software-defined networking.
