Welcome to the Cloud Computing Applications course, the first part of a two-course series designed to give you a comprehensive view on the world of Cloud Computing and Big Data! In this first course we cover a multitude of technologies that comprise the modern concept of cloud computing. Cloud computing is an information technology revolution that has just started to impact many enterprise computing systems in major ways, and it will change the face of computing in the years to come. We start the first week by introducing some major concepts in cloud computing, the economics foundations of it and we introduce the concept of big data. We also cover the concept of software defined architectures, and how virtualization results in cloud infrastructure and how cloud service providers organize their offerings. In week two, we cover virtualization and containers with deeper focus, including lectures on Docker, JVM and Kubernates. We finish up week two by comparing the infrastructure as a service offering by the big three: Amazon, Google and Microsoft. Week three moves to higher level of cloud offering, including platform as a service, mobile backend as a service and even serverless architectures. We also talk about some of the cloud middleware technologies that are fundamental to cloud based applications such as RPC and REST, JSON and load balancing. Week three also covers metal as a service (MaaS), where physical machines are provisioned in a cloud environment. Week four introduces higher level cloud services with special focus on cloud storage services. We introduce Hive, HDFS and Ceph as pure Big Data Storage and file systems, and move on to cloud object storage systems, virtual hard drives and virtual archival storage options. As discussion on Dropbox cloud solution wraps up week 4 and the course.
3.3.7 Load Balancer Schemes
Loading...
來自 University of Illinois at Urbana-Champaign 的課程
Cloud Computing Applications, Part 1: Cloud Systems and Infrastructure
284 個評分
從本節課中
Module 3: MAAS, PAAS, Web Services
Welcome to the third module, where we introduce Metal as a Service (provision real hardware in the cloud), Platform as a Service (provide a platform to run user code on) and Web Middleware as the glue technology that empowers cloud computing.
與講師見面
Roy H. CampbellProfessor of Computer Science
Department of Computer Science
Reza FarivarData Engineering Manager at Capital One, Adjunct Research Assistant Professor of Computer Science
Department of Computer Science
[SOUND] Okay, this video ties into
your homework assignment.
It's about load balancing, it built on the first load balancing video.
And what I'm going to explain is all the different ways that
load balances can be organized.
We're not going to get you to do them all for the homework, but this will give you
an appreciation of just how complicated those load balancers can become.
So what you need to do is to get your HTTP
request to the server from the user to the server,
to get the needed resources to actually provide the data from the HTTP and
what you're going to do is get that data and send it back to the client.
And remember, you're going to go from multiple clients to multiple
servers and what we want to do is load balancing of all those servers.
So when the load balance receives the packet, and
it's going to receive the packet the way we set up all the IP addresses,
what it then needs to do is to know that there is a server available.
And it mustn't break the user's impression that they're working with the session.
If you're clicking through a set of HTTP
web slides, web presence,
whatever, you don't really want to have the knowledge that you're actually
talking to a back end to interfere with user's view of what's going on.
Okay, so the user must always get the same resource.
The SLB must always ensure that it happens.
And in order for that to work, well, the SLB must know the servers,
know the IP/ports, know the availability,
understand the details of some protocols, FTP, SIP and so on.
And it may need to play with the network addresses.
So there may be a NAT, a Network Address Translation, associated with it,
which would take the incoming address on the packets and
translate it into an address that is not inside the cloud.
Pass that through to the routers, get the server.
On the way back it would need to fix that so that the packet actually looks like it
came from the server the client requested, and so
then that will also, perhaps, modify the packet
coming back to make sure that it actually looks it's coming off the web server.
So why do you load-balance where it's scaling?
It's so that I can have a web service, say 20 servers serving 10,000 people.
And then I can go to 20,000 people and double the number of web service.
It's the ease of administration maintenance.
I don't have to know where everything is.
I can lay it out in my cloud and when I get it right then I can turn everything
on, and it's all managed for me.
It's transparent.
It removes the notion of physical service from what's there.
If I need to repair something, if I need to do maintenance,
if some piece of the system fails I can still provide my web service.
And it gives you resource sharing.
We can run multiple instances of the client's service on the server.
Can be running on a different port for each instance,
can low balance the different ports, whatever you want,
whatever would fancy and make your job easier as a cloud manager.
There's a lot of load balancing algorithms.
The ones that we need to deal with,
most predominate, first would be the least connections.
When I get a connection coming in from a client I find the server that has
the least connections going to it and
I just send that request onto that particular server.
You can modify that if you want to include the weight of the service.
Some HTTP servers, like a search, may have more effort involved In that case,
what I can use is a weighted least connections and
I can send it to a server that has the least load and I can adjust the weights so
that that reflects how much load is required or is imposed on the server.
I can do round robin if I want to just distribute it fairly.
I can do weighted round robin, which also adds the weight but they're very, very
simple algorithms, they need to be because this is very, very fast at that front end.
It's very transparent.
So main goal is you try to predict how best to take all those client requests and
distribute them to the service so the server can give really good responses.
So how do SLB devices make decisions?
There's several factors, one is it can obtain
information from the packet headers, the IP addresses, the port numbers.
It can also look beyond that, it can look at the HTTP cookies,
it can look at the HTTP URL that's being requested.
It can also look at the client certificate, the SSL certificate and
use that.
The decision is strictly based on flow counts.
They can be based on what the application is, like search.
It may take a little bit longer.
The decisions can be based on essentially any parameter you want.
For some protocols like FTP, as you're sending the data across,
and because FTP sorts HTTP, it's the main protocol you would use,
you would want to know how that FTP IP is set up and so on.
So that the connections always connect to the right place.
FTP has this association once you build it between the client and the server,
the next HTTP request going down the FTP would also sort of go to that same server.
So you need to sort of keep those in line.
So if you're downloading a large file, say, from a server then what you need to
do is have the load balancer keep track of that FTP request,
and make sure it always looks like a continuous stream.
Otherwise your TCP/IP will get all upset.
When a new flow arrives, you determine if the server exists, or
the load balancer does I determine the level of service required.
Is this moving fast, is this an FTP, is this, do I want multiple FTPs,
and so I'm going to average or do some other type of algorithm.
You pick a real server and
then you do all the things that needed to actually get the communication across.
The packet can be breached, if you want, using layer two, or it can be at
layer three, or layer four, and even layer five, and we'll describe those in a sec.
There's some very different architectures, traditional
SLB device sits between the clients and the servers being load-balanced,
however as you scale up, you may need a distributed SLB.
And what it does is to sit off to the side, and
it has forwarding mechanisms to actually do the load-balancer And
the SLB just provides those forwarding mechanisms the information they
need to route traffic and we'll show you that in a second.
So this is the traditional NAT view.
We've got a NAT.
Incoming client goes to the cloud.
The SLB will do the NAT translation and
send the request off to one of the servers.
And what you notice is it could be two way.
The acknowledgements coming back from the server will go back to SLB,
will go back from the client.
Then you would say oh, for that means the SLB is doing a lot of work, and
of course it is.
So another obvious optimization is just to send it straight back to the server from
the client.
But if you do that, then you need that NAT box in the way.
You need to make sure that addresses aren't messed up by doing that shortcut.
Next as we move on, you can do this at layer three and four and
that gives you some more semantics.
Now you can look at the destination IP address, the port,
you can make load balancing decisions on some of the content on the packet.
And so you need to do all that before you actually send the packet onwards.
So here's an example of that.
The client sends his packet.
The SLB now sort of does its decision for the server.
It picks a server and is a request coming in for process, sends it back,
sends it back to the client and the rest of the flow continues through HTTP GET and
server responses channeled through the SLB.
And everything is nice.
One of the issues though with all of this is just what
road does the SLB, the device play in all of the communications.
Sometimes, it can just be transparent, it just forward the packets forward.
With three and four, it may need to look at the packets,
look at seeing what's the internals to the packet going back with some forwards as
we saw with the previous slide.
In layer five it has to terminate the TCP flow,
now we're dealing with high levels of protocol, it has to
terminate the TCP flow if the amount of time before the SLB decision can be made,
then it's going to send that TCP connection onwards.
And so, there is, sort of handshake going on.
And that's illustrated here, the client makes his request.
The SLB thinks about it.
It says okay, let's go ahead and do this.
And then there's acknowledgement going and some communication and we do a get,
and that synced to the server.
The server then will reply and that gets
acknowledged, you have a further acknowledgement, and
back comes the data from the get and everything's happy,
but you've got multiple steps in this handshaking business.
What you see is the SLBs really acting as an intermediary.
The client actually makes the connection to the TCP connection to the SLB
the SLB then says okay that's right I've got it I'll go open the port everything's
happy and then the client makes its get and the SLB handles that get it goes off
finds the server it needed It would then pass that request onto the server.
So it's a sort of two phase to get your packet across to start the flow.
And if things like file transfer, that's what you're going to do because you're
going to negotiate bandwidths and other things.
You want to know a lot more about what's going on and this allows you to do it.
So lots of work for the SLB.
How do you make that fast?
Well you can do it through distributive architecture.
Here's a picture of the distributive architecture,
you've got the packets coming into the cloud, the cloud will forward them instead
of sending them to the SLB, it now sends them to a forwarding engine,
which is a slave, if you like, of the SLB.
So the SLB tells the forwarding engine what
to do under all the different circumstances.
If the FE doesn't know, it asks the SLB.
And so in this way, we can have many more connections being forwarded and
responded to in a distributive manner the SLB
is not in the direct line of all that data being transmitted.
So here's an example.
You set up a request from a client, there's a TCP sync.
It then goes to the four out of the four.
It talks to the SLB.
The SLB recognizes what it needs to do, sends back instructions,
and it maps it to a service, Server2.
Now, what you do is to send that flow to the Server2, the Server2 responds.
And now you've got a request going backwards and
forwards between the client and Server 2, avoiding the SLB but keeping the FE busy.
So you've distributed the load.
So what you've seen here is how complicated the SLB can be.
How load balancing can be a really vital part in distributing the load,
and it can cope with distributing the load at different protocol levels and
you can achieve different things whether it's for file transfer, to distributing
search requests, to distributing cloud requests as we see with RPCs.
What's missing from this is, just how did that flow get to the server,
how'd it go from the client to the server in a nice easy manner?
What we've seen is the distribution mechanism,
now we're going to turn our attention to, how does the protocol actually work?
We're going to bring all this knowledge about RPC's and low balances and
everything else together into sort of finale for the lecture one,
which is actually transporting that data, what's used to transport that data.
So join me in the next video.
[MUSIC]
