We live in a complex world with diverse people, firms, and governments whose behaviors aggregate to produce novel, unexpected phenomena. We see political uprisings, market crashes, and a never ending array of social trends. How do we make sense of it? Models. Evidence shows that people who think with models consistently outperform those who don't. And, moreover people who think with lots of models outperform people who use only one. Why do models make us better thinkers? Models help us to better organize information - to make sense of that fire hose or hairball of data (choose your metaphor) available on the Internet. Models improve our abilities to make accurate forecasts. They help us make better decisions and adopt more effective strategies. They even can improve our ability to design institutions and procedures. In this class, I present a starter kit of models: I start with models of tipping points. I move on to cover models explain the wisdom of crowds, models that show why some countries are rich and some are poor, and models that help unpack the strategic decisions of firm and politicians. The models covered in this class provide a foundation for future social science classes, whether they be in economics, political science, business, or sociology. Mastering this material will give you a huge leg up in advanced courses. They also help you in life. Here's how the course will work. For each model, I present a short, easily digestible overview lecture. Then, I'll dig deeper. I'll go into the technical details of the model. Those technical lectures won't require calculus but be prepared for some algebra. For all the lectures, I'll offer some questions and we'll have quizzes and even a final exam. If you decide to do the deep dive, and take all the quizzes and the exam, you'll receive a Course Certificate. If you just decide to follow along for the introductory lectures to gain some exposure that's fine too. It's all free. And it's all here to help make you a better thinker!
Diversity Prediction Theorem
Loading...
來自 University of Michigan 的課程
模型思维
1193 個評分
從本節課中
Learning Models: Replicator Dynamics & Prediction and the Many Model Thinker
In this section, we cover replicator dynamics and Fisher's fundamental theorem. Replicator dynamics have been used to explain learning as well as evolution. Fisher's theorem demonstrates how the rate of adaptation increases with the amount of variation. We conclude by describing how to make sense of both Fisher's theorem and our results on six sigma and variation reduction. The readings for this section are very short. The second reading on Fisher's theorem is rather technical. Both are excerpts from Diversity and Complexity.
與講師見面
Scott E. PageProfessor of Complex Systems, Political Science, and Economics
Center for the Study of Complex Systems
Hi, In the Sun Lectures we're talking about prediction, And here's the idea, we
want to think of individual people make prediction based on models. Those models
can be based on categories or linear models or Markov models, any of the models
you've learned in this class you could use to make some sort of prediction. And what
we want to talk about is where collective wisdom can come from. So if we have a
whole bunch of people using a whole bunch of different models, how does that enable
the crowd of models to do better at making sense of the world, making an accurate
prediction? Now, the essence of our argument is going to be something called
The Diversity Prediction Theorem. And, the Diversity Prediction Theorem is gonna
relate the wisdom of the crowd to the wisdom of the individual. So, in other
words the accuracy of the crowd, in relationship to the accuracy of the
individuals. Now one logic that should come to you right away is if I had more
accurate individuals, I should also get a more accurate crowd. But a logic that
might not come to you right away is that if I had a more diverse crowd, I should
also get more accuracy. So, we can think of things like the crowds accuracy is
going to depend on the individual accuracy Plus The diversity. Now the question is
how much do these things matter. How much does it depend on, individual accuracy and
how much does it depend on diversity. That's why we wanna use a model, to figure
out. So let's first do an example just to get some bearings, some inner bearings in
terms of what it means for a crowd to make a mistake versus an individual to make a
mistake and what diversity is. So we'll do an extremely simple example. We have three
people, Amy, Bell and Carlos. And let's suppose that they're picking the number of
people that come to our diner on a particular day for lunch. And so Amy
predicts it's gonna be ten. Val predicts it's gonna be sixteen and Carla predicts
it's gonna be 25. Now if I add these up, I'm gonna get 51 and divide by three, I'm
gonna get an aver age value of seventeen. So the crowd predicts seventeen. Now let's
suppose the actual value is eighteen. Now again, if, let's suppose the crowd is
pretty accurate, it's not gonna matter but this is just for the purpose of the
example, I'm gonna make it so the crowd actually does pretty well, I just wanna
work to the logic. So the first thing I'm gonna do is I wanna figure out how
accurate are these people. Well what I can do is I can compute the error of each
person. So remember the, the true value was eighteen, that's the number that
showed up. And I can ask what's the error of each individual? And remember we
computed errors by looking at variations, squared error. So INU predicted ten, the
truth was eighteen, so her squared error is 64. Belle predicted sixteen, The true
value is eighteen, So her score there is four. Carlos predicts 25, The true value
is eighteen, So his score there is 49. And if we add all those up, I get 117, And if
I divide that by three, I get the average there that's 39. So on average, these
people are off by 39. Some people, Belle, are really accurate, She's only off by
four. Other people, Amy, is off by quite a bit. She's off by, her error is 64, But
the average is 39. So this sort of gives us a sense of how accurate the individuals
are. The individuals off, are off by an average of 64, four and 49, for an average
of 39. Now we can ask how accurate was the crowd? Remember the crowd predicted
seventeen, because that was The average prediction of the three people. The
[inaudible] is eighteen, so we get the crowd was only off by one. So the crowd
here is, if you notice, better than anybody in it. So, we get "the wisdom of
crowds." Well, let's try and think about. Why that makes sense and to do that we're
going to look at diversity. So diversity [inaudible] is the variation in the
predictions. So how do we do the variation in the predictions? We look at each
person's prediction and its distance from the mean prediction not from the true
[inaudible], the mean prediction. So the mean predicti on was seventeen so. Amy's
contribution to this sort of total variation of predictions is ten minus
seventeen squared, which is 49. Belle's is sixteen. It should be seventeen minus
seventeen squared, which is one, and Carlos's is 25 minus seventeen square
root, which is gonna be 64. Now if I add all these up, I get 114, And if I divide
by three, I get 38. So the diversity of these predictions is 38. Well notice this.
The crowd's error was one. The average error was 39 and the diversity was 38. So
I look at that, I get one equals 39, minus 38. The crowd's error in this case equals
the average error minus the diversity. But I just Set this up, What turns out That's
always true. This is what the diversity prediction theorem says; That the crowd's
error equals the average error minus the diversity. Now, this isn't some, you know,
feel good setting, This is a mathematical fact, This is an identity. So no
assumptions have to be made here. This, there's no opposite [inaudible]. This is
just true. If I have a set of predictions, it will always be the case that the error
of the crowd to the average errors. Squared error, the average prediction
squared error is going to equal the average squared error of the people in
that crowd, minus the diversity of their predictions. Now the way to write that
formally is like this, Now this looks pretty scary, but let's just walk through
it. So let's let C be the crowd's prediction. Data be the truth, so data is
equal to the true value, And so this [inaudible] thing. This is the crowd
square [inaudible]. So it's the distance from the crowd to the truth. Let's let SI
here equal individual I's prediction So individual I's prediction. And so we're
gonna get... This is I's prediction minus the truth squared. And then we sum that
all up over all the individuals, and we divide by the number of individuals. So
that's just gonna be the average error. So crowd [inaudible] equals average error
minus... Now we take each person's prediction minus the crowd's prediction,
which is C. Remem ber, because C is the crowd. So this tells us how far people are
from the crowd on average. We sum those up and rate divide by N, So we get the crowds
there equals the average airlines diversity. Now if you take this equation
and expand all the terms and cancel everything out you'll see that it's an
identity. It's a mathematical identify. So it's always true. Crowds there equals
average year minus diversity. Let me give a famous example to sort of drive this on.
So in a book called The Wisdom of Crowds by Jim Surowiecki, he talks about the 1906
West of England Fat, Stock, and Poultry Exhibition. At this exhibition, 787
people, Guess the weight of a steer. Their average guess was I think, 197 pounds; the
actual weight of the steer was 198 pounds. So they're only off by a pound. So you're
looking at it and say, oh my gosh that's amazing, that's the wisdom of crowds. But
let's think about it, what's going on? We've got a bunch of predictions, there's
a true value, there's an average value, our theorem, this thing, this [inaudible]
theorem must hold. And, in fact, if you take Galton's data. And you plug it all
in, here's what you get. The crowd's error is actually a little bit less than a
pound, it's.6. The average error is 2956. Now, wait, that seems crazy, 'cause
remember, the steer only weighs 1100 pounds. So if this thing weighs 1100
pounds, how could they be up by 2956? Whenever these are squared errors, so if I
square 50 I get 2500 and if I square 60 I get 3600. So this is probably 55, 56
squared. Something like that. Well, that makes sense because people could probably
guess the weight of a steer within about 55, 56 pounds. Well, why is that? Well
think about it. A steer's five times the size of a person. If you can guess the
weight of a person within about ten pounds, you can probably guess the weight
of a steer to about 50 pounds. So what you've got is you've got some sort of, you
know people are reasonably good at guessing the weight of steers. They're not
geniuses, but they're also not crazy. They're not guessing 15,000 pounds. So
these are reasonably knowledgeable people who for whatever reason are you know
making these errors of about 55, 56 pounds. Not it's interesting is that there
diversity is 29 55. So, what you get is. The crowd is wise because they're
moderately accurate. I drop by 55, 56 pounds and they're are also diverse and
it's that accuracy plus diversity that makes the crowd do so well. Now if you
think about this book, the wisdom of crowds, see we can get a bunch of
examples, well that's the case. Let's think about in the context of our theorem,
so we've got crowd air, equals average air, minus diversity. Now in this book,
Sir Wiki says, here's what matters, diversity matters a lot. Well why does
diversity matter a lot f we're looking at the wisdom of crowds, let's see, this
actually, the math will tell us why. If you make it into the book, the wisdom of
crowds, what has to be true? This has to be small. The collective area has to be
small. So if the collective area isn't small it doesn't make the book it's not
the wisdom of crowds, it's the madness of crowds. So for the wisdom of crowds to
exist this has to be small, collective area has to be small. Let's think what
else has to be true, the average air has to be fairly large, why does it have to be
fairly large? If the average air is small, that means it was a easy thing to predict,
everybody can pretty much get it right. So if it's interesting enough to make a book
called the wisdom of crowds, where the crowd is smart and the people aren't, if
the people are not smart that means the average area has to be large. Well if you
got something small equal something large minus something else. This other thing has
to be large, Which means diversity has to be large. So when Surowiecki walks through
all these examples and he looks at what's going on, he says, look there's a lot of
diversity. And diversity seems to be a key component in the wisdom of crowds. You
want to encourage people to think about the world in different ways if you wanna
get the wisdom of crowds. And our model explains why that's the case. It's
collective error equals average error minus diversity. If people aren't that
smart, average error is gonna be big. If you want the crowd to be smart, the only
way to get it is by having that crowd be diverse. So if we look at the first
example of [inaudible]. Where we get 0.6, 29 56 and 29 55. We see that's exactly the
case, small crowd error, you know fairly large average error because it's not an
easy thing to do and then high diversity. And if you take examples of wisdom of
crowds from all over the place, you'll see they all look like this, they look exactly
like this, Small crowd error, large individual error, large diversity.
Question is, how do you get how do you get that diversity? Well you get that
diversity by people using Different categorizations. Different that your
models. Maybe people using entirely different models. Maybe, one person's
using a mark off model and one person's using a diffusion model. Maybe one
person's using a linear model. Maybe one person's got a non linear terminal model.
There is a lot of different variables. So, what you get is this how originating what
we see the world. In the boxes we use and the variables we use, and the models that
we construct. I would give Diversity to these collective predictions. In that
collective prediction, those collective predictions, then, lead to accurate
crowds, provided you've got reasonably accurate people who are reasonably
diverse. And what we've learned is by constructing a very simple model of that
predictive task, where the wisdom of crowds come from. And we've learned that
individual ability. And collective diversity matter equally, they're equal
partners. So if someone were to say to you, where does the wisdom of crowds come
from? You could say, well it comes from. You know, reasonably smart people who are
diverse, And you could also ask, where does the madness of crowds come from? How
could it be that a crowd could get something totally wrong? Well, that's not
har d either; cuz crowd error equals average error times diversity. Well, if I
want this to be large. I want large collective air, then I need large average
air, cuz I need people to, on average, be getting things wrong, and I need diversity
to be small. So the [inaudible] of crowds come from like-minded people who are all
wrong, and again, the equation gives us that result. Alright, thank you.
