Welcome to the Cloud Computing Applications course, the first part of a two-course series designed to give you a comprehensive view on the world of Cloud Computing and Big Data! In this first course we cover a multitude of technologies that comprise the modern concept of cloud computing. Cloud computing is an information technology revolution that has just started to impact many enterprise computing systems in major ways, and it will change the face of computing in the years to come. We start the first week by introducing some major concepts in cloud computing, the economics foundations of it and we introduce the concept of big data. We also cover the concept of software defined architectures, and how virtualization results in cloud infrastructure and how cloud service providers organize their offerings. In week two, we cover virtualization and containers with deeper focus, including lectures on Docker, JVM and Kubernates. We finish up week two by comparing the infrastructure as a service offering by the big three: Amazon, Google and Microsoft. Week three moves to higher level of cloud offering, including platform as a service, mobile backend as a service and even serverless architectures. We also talk about some of the cloud middleware technologies that are fundamental to cloud based applications such as RPC and REST, JSON and load balancing. Week three also covers metal as a service (MaaS), where physical machines are provisioned in a cloud environment. Week four introduces higher level cloud services with special focus on cloud storage services. We introduce Hive, HDFS and Ceph as pure Big Data Storage and file systems, and move on to cloud object storage systems, virtual hard drives and virtual archival storage options. As discussion on Dropbox cloud solution wraps up week 4 and the course.
3.3.6 Load Balancer Introduction
Loading...
來自 University of Illinois at Urbana-Champaign 的課程
Cloud Computing Applications, Part 1: Cloud Systems and Infrastructure
286 個評分
從本節課中
Module 3: MAAS, PAAS, Web Services
Welcome to the third module, where we introduce Metal as a Service (provision real hardware in the cloud), Platform as a Service (provide a platform to run user code on) and Web Middleware as the glue technology that empowers cloud computing.
與講師見面
Roy H. CampbellProfessor of Computer Science
Department of Computer Science
Reza FarivarData Engineering Manager at Capital One, Adjunct Research Assistant Professor of Computer Science
Department of Computer Science
[NOISE] So,
in this set of slides what we're going to be talking about
is how lots of clients can talk to lots of services and
scale the services according to how many clients we've got.
We're going to use a load balance to do that.
So a request enters a router,
it’s forwarded around until it gets to the Cloud.
The cloud router is going to transmit the request to an appropriate server.
The load balancer is the mechanism in the cloud that actually determines which
service, which node in the cloud,
which server is actually going to be used to provide the web service.
So, maybe we've got 10,000 nodes in the cloud and we make a web request,
it will go to one of those 10,000, farmed out by the load balancer.
There are all sorts of interesting questions about how load balancers
operate, and we'll try to give you a quick overview of how that works,
but in your NP,
you will actually be designing a simple one just to experiment the whole idea.
So, we got a request coming from the cloud,
enters the router and the load balancing server
is going to determine which web service should process the request.
Now when you're sending it from the internet it's going to go to a particular
IP address.
The router is going to take that IP address, deliver it to load balancer,
the load balance will change the IP request.
It'll change the actual IP address to
the particular server in the cloud that's responsible for responding to you.
Now if you know from our discussions before,
we're using a simple sort of rest protocol.
In interacting with the cloud, and that protocol implies very,
very simple relationship between the request and what's serviced.
As we are doing this, what we're going to do is to make sure that
any particular server we choose has all the information that it needs to be
able to respond to that particular rest request.
So, if I ask for a particular employee and
their data, then what I would expect to happen is that the load balancer
would send it to one of the servers that has that information.
And there is presumably a large number so
it can load balance over all those servers so that you get good response time and
you get a fair, quick reply to your web service.
In this diagram, you will see that flow of how this operates.
In particular, what you notice is that for every request,
there has to be a reply, a response, which is going to go back to the client,
which will reveal to the client the data that was fetched.
And the client probably will display that on their screen.
So there are dotted lines back through this mechanism.
The load balancer has to keep account of those dotted lines.
If it gets a request, it's going to find a server but then that server
has to be able to reply back to the correct client.
So it's going to keep, if you like,
a trail of how to return the packets from that request.
And funnel,
the particular request from the particular server back to the right client.
So Google, for example, provides web search for the whole of the Earth.
Lots and lots of people do web searches.
How is that organized?
Well the request, the web search request, goes in as an HTTP request.
It will go forward it to Google web server, and
it doesn't really matter where you are on the planet, you're going to say Google.
But it will get served by a particular Google part of the cloud.
And in that cloud, there will be a unique web address for data requests.
Now, that is going to talk in turn, to all sorts of
load-balancers, to actually answer that request.
One of the things that it will do,
just to indicate some of the additional add-ons to the load balancer,
is it will determine out of all of the different data sources it has,
it will determine which of those data sources is the most interesting,
from the point of view of your search question.
So it'll take your search query and it will look in various index servers
for a response from that request.
And the front end will actually sort of forward the packets,
your requests to those document service,
corresponding to the correct indices of the query that you've made.
So you're coming through, make your web request, it's going to take your query,
it's going to index it, it's going to find from the index servers
which document servers are the interesting ones.
In all of those circumstances, you could have multiple index servers.
You could have multiple document servers.
And the load balancer is going to split up those packets
to each of the appropriate index servers it's found,
and to each of the document servers, it's going to choose one of those
in order to reply to you and it's going to keep track of all that complexity.
So, we're going to stick to a very, very simple example but
this slide demonstrates as you're making requests,
you can get into really sort of complex implementations within the cloud.
Your load balancer is an integral part of providing this service, but
there may be other parts to that service that the cloud will have to worry about.
For example, if you're looking for something Google-wide and
the whole universe, whole worldwide, there may be so many different things that you
have to find the right set of web servers to actually respond to your request.
But it's all the same mechanism.
And what we've described is really sort of fundamental to doing
any of these types of requests.
And here's just the sort of diagram processing elements.
You're going to be talking to the front end here with your web requests,
it's going to do the processing and, actually sort of figure out where
it needs to find the index and find the documents.
And, down here,
you've got lots of worker processes that are going to be doing all the processing.
And your load balancing is going to make this all sort of seamless and very fast.
So here's a if you like, block diagram of what's happened,
you got a query coming in, there's the Google Web Server.
It may do various things like check your spelling, it may give you an ad,
it may provide an update with news, and books, and so on.
That would all be processed in the Google web server responding to your query.
Then the query itself is going to go out to index servers, and
some of these will be replicated.
And these index servers will find the right item that you're looking for.
But that item will be a document, but it won't reside on the index servers,
it will reside on the document servers over here.
So it's actually then going to to respond to the web server.
The web server's going to go back and find the document server,
it will get the document, return the document to here.
And then what's going to happen is it's going to integrate all of these
advertised and miscellaneous services and
return a webpage to you with all of this information on there.
Now, where is the load balancers?
The load balance is making sure if you've got, say the Olympics,
or you have a really important news event, when there's queries for
it, there are enough index servers, there's enough document servers and
the loads are spread over those index servers and the document servers to
actually be able to process all of those flood of requests.
So we have this idea of providing web service,
but it can be a huge amount of information.
That information is distributed over lots and lots of different nodes in the cloud.
How do we do that?
And this is just the sort of high level view how we're doing that, and
we'll get to details later on.
But, if you have a lot of servers and
you have a lot of data, enourmous amount of data,
there are ways obviously to devise serving this data.
The data itself can be split up, so we could take this red one and
sort of pop it over onto a particular server.
But then we don't have any replication and so we can't do load balancing.
So we might take one of these and put it on several different servers.
That way there's two servers that can actually service the same request.
Now in general what we would like to do is to take all this data and
map it, we could either split it up.
So that each element here goes on to a different server.
Or, we can do this type of duplication, so we can do load balancing and
so if one of the server farms dies say, we have a spare copy.
So, a picture of what we're talking about here is, we need replication.
What we can do is to decide how much replication there is.
In this case we've sort of got six sets of replication.
We can talk about where we're going to put that onto the web service and
we can partition that data so we can sort of say well okay,
there's going to be two yellows over here on this set of machines.
There's two blues over in this machines.
There's a yellow and a white over this machines.
We can have a very flexible way of allocating the date to the servers, and
so in the load balancer itself, we need to accommodate all of that information.
How things have been replicated,
how they've been partitioned across the network, internally, in the cloud.
So, it's invisible to the cloud user.
And it gives you the flexibility, first of all,
of being able to respond to lots more users of the cloud, clients.
And it also responds to crashes of systems within the cloud and updates.
Perhaps, this is being updated, and so it's busy.
Is there a spare copy of blue you could use somewhere else?
So this can get very complicated.
The load balancing approaches, generally,
are the mechanism to overcome this difficulty.
There are various different load balancers,
we won't be talking at all about all of them this time.
But just to give you awareness,
there are load balances that deal with content, and locality.
An example that would use DNS servers to actually decide how to find the content.
So DNS servers is relating machines to names, and what you would do is to
associate the content and locality with the name of the server and
then use your DNS service to do it.
Another way of approaching all of this is your load balance might be size aware,
could be centralized, what it does is to distribute over
all the different nodes that have got all the data, and
the data might be split up between multiple nodes.
So that the load balancer has to be aware of where that data is,
than a centralized type of operation may be very practical and in doing this.
Another type of load balancer's workload aware, where you have a lot of servers but
they're all serving different things, and you have a different
response to clients to trying to get the information off those servers.
So let's say all the clients want to see the 515 race at a horse track,
then what you need to do is have the workload aware, some type of system,
and duplicate the result of that race on multiple different servers and
having a distributed dispatcher that will actually go and
spread the load over all of those different servers.
So what we've seen is to process
events efficiently, we need to do optimizations with the load balancing.
We're going to send and
process requests to a web server that has files in the cache.
We're going to make requests to that web server with the least amount of requests
because we can get a really fast response.
And then we're going to send and
process those requests to a web server determined by the size of the requests.
So those are the types of things that we're going to be doing
with a load balancers.
[MUSIC]
