Hi, in this last lecture about Markov processes, what I want to talk about is exaptation. I want to talk about how we can use the Markov model in context, and for problems we never would have thought of. And we do this in two ways. First I'm gonna talk about taking it whole hog, taking the entire process and modeling other things, just like we modeled the process of states becoming free or dictatorial. Second, what I wanna do is, I wanna just take part of the Markov model, its transition probability matrix, and use that to understand some things that are really kinda surprising and interesting. So let's just remind ourselves of what a Markov process is. Fixed set of states, and then there's fixed transition probabilities between those states. Now if it's possible to get from any state to any other through a sequence of transitions, then remember, we have this Markov convergence theorem that says the process is going to go to a unique equilibrium. So history doesn't matter. Initial conditions don't matter. All that sort of stuff. Let's think about where else we might apply that. So we, one place right away is voter turnout. So you can think about, there's a set of voters at time t, and there's a set a of non-voters at time t, and what we can do is draw a little matrix and say, how many of those are going to vote at time t+1, and how many of these are not going to vote at time t+1? And it could be that 80 percent of voters at time t vote in t+1, twenty percent don't. It could be of nonvoters that 40 percent vote and 60 percent don't vote. This would be our Markov transition matrix and if you apply this, what you'd get is a unique equilibrium that should tell you the number of people you'd expect to vote in any election. Now it's not going to be the same people, right, because the process is going to churn. It's a statistical equilibrium, not a fixed point. But this model, if it's right, if these transition probabilities stay fixed, this would tell us what turnouts should be. Even though it won't tell us who votes. Where else would I use it? Well, we can use it for school enrollment. Same sort of thing, right? [laugh] You can imagine, here's kids who go to school, and here's the ones that don't go to school. And then we could ask, okay at time t+1, how many go, and how many don't go? It could be that of those that go, 90 percent go the next day, and ten percent don't. Of those that don't go, it could be that there's only a 50/50 chance that they come the next day. Well again, if you work through the logic here, you'll get some sort of percentage of people, students, who show up each day, and some percentage that don't, there'll still be a churn, but this model will give you an estimate of what total enrollment should be on a given day. What percentage of students show up. These two applications are very standard applications. They're not unlike the one we looked at, in terms of alert and bored students, and they're not unlike the one with free countries versus dictatorial countries. What I wanna do next is sort of go way outside the box. I wanna just take part of the Markov model. The Markov transition matrix. And I want to think about what that tells us. What it tells us is, if this the state of time t, what are the likelihoods beyond these other states of time t+1? So think about this for a second. There's all sorts of things you could use this framework for, where something happening at time t, and then it transitions into something at time t+1. I'm gonna just talk about three uses of this matrix, three very surprising uses. First one, to identify writers. I mean, [inaudible] use this idea, this transition matrix again, to figure out who wrote a book. So suppose some anonymous person writes a book, and, you're trying to figure out, did this person write it? Did Bob write it? Or you trying to figure out, did you know, Carlos write it? And you can't tell. Well, what you can do is the following: You can figure out transition probabilities. What do I mean? Well, take this book. Take the book written by an anonymous author. And then say, okay, every time this book uses the word "for"-- You're loading the whole book in the computer. What percentage of the time does it follow the word "for", with "the record"? What percentage of the time does it follow the word "for" with "example"? And what percentage of the time does it follow the word "for" with "the sake of". And what I'm doing is I'm creating, just, a giant matrix. So if it, in some sense, like, at time t, I'm using "for". And then I'm saying, what's the probability that I follow it with "the record", "example" or "sake"? And I'm putting a .17, .9, .11, so I'm just trying to get a big transition matrix. What you can do is you can take some key words, create these giant transition matrices, and then figure out what does it look like. Does this transition matrix look like Bob's transition matrix, or does it look like Carlos' transition matrix? Now how do I know what Bob's transition matrix is and what Carlos's transition matrix is? Well, that's easy. I just take one of their other books, load it into the computer and figure out what their transition matrix is for the other book. Once I got that in there I can figure out, what, does this look more like Carlos or does this look more like Bob? Now, this actually gets used. And let me tell an interesting story. So this is Arlene Saxenhouse. She's one a of my colleagues at the University of Michigan. And when she was a young graduate student she found, in the library at Yale, this book of, it included four essays by someone who she thought was a young Thomas Hobbes. Book was published in 1620. She thought, 1620, and she thought, oh my gosh, these are essays by Hobbes. And the thing is, she's a young grad student. Who's necessarily going to believe her? How does she prove it? So she couldn't prove it. She had a strong instinct that it was true. Well eventually she found someone who knew how to do this stuff, knew how to do these transition probabilities, and they took the essays and they put in some of Hobbes' other writings, and what they showed was that it seems fairly clear that three of the four essays were actually written by Hobbes, and now those three essays are actually considered part of Hobbes' work. Now, even though appeal, you know, it just sort of felt to her like they were written by Hobbes. That comes down to a matter of opinion. They were having a model, they were having these transition probabilities models. Right? And by able to take that, and take other Hobbes work and this work, you can show statistically that it seems very, very likely that Hobbes wrote the work. So this combination of things. The model alone doesn't do it. The combination of the model, plus her intuition, plus the intuition of others, gives us a common understanding now, at least we think, that Hobbes wrote those particular volumes. It's really cool. Let me give you another example, medical diagnosis. So, if you think about giving someone a treatment for some disease, typically, there's a sequence of reactions to that treatment. Whether it's a drug protocol, or it's an exercise or diet regimen, what you can do is, you can write down transition probabilities. Now, these can be multi-stage. So it could be, for example, that if it's going to be successful, if this treatment is going to be successful, that you go through the following transition: you first feel some pain, then you're slightly depressed, then more pain, but then you get better. Alternatively, if it's not successful, it could be that, initially, you're depressed. Then there's mild pain. Then there's no pain. And then the system fails. Not only the system fails, but you fail, the drug fails, the regimen fails. So what does this mean? This means that, if I give someone the treatment, and then I see this sequence of pain and depression, I can say to them, you know you're feeling pain. You're feeling depression, but guess what, that's consistent with a, a regimen that's gonna be successful. Whereas alternatively someone else could say, well you know, I feel depressed but now I'm definitely not feeling much pain. You can say to them, okay, even though you're not feeling much pain, this probably isn't a good sign. It doesn't look like the treatment is gonna work. So by gathering all sorts of data and past experiences you can use that transition probability, to figure out early on in a treatment protocol, whether it looks like it's working or not working. Another example: Lead up to war. Suppose you got two countries and there's a little bit of tension. So what you get is the following: you get, let's say first you get some political statements on each side. Then that leads to trade embargoes. Then that leads to military buildup. So now you got this sequence of three things. You have these transitions between these three things. You can ask, historically, when I've had those three transitions, what's the likelihood that I've had war, and the likelihood that I haven't had war. And it could be that there's a twenty percent chance of war, and an 80 percent chance of not having war. So, if you're just sort of on the ground watching what going on, you say, oh boy, look at this! First, it was political statements, now there's a trade embargo, now we're seeing military buildups. Looks like things are gonna go to war. If we actually gather a lot of data and, and calculate these transition probability matrices, you could figure out, you know, this actually happens a lot, and only twenty percent of the time does actually lead up to war. So again, we're not using the full power of the Markov model. We're not saying these transition probabilities necessarily stay fixed. We're not worried about solving for the equilibrium. All we're trying to do is just use this matrix to organize the data in such a way that we can think more clearly about what's likely to happen. So that's Markov processes. Markov process is a fixed set of states, fixed transition probabilities. You can get from any one state to any other, and then you get an equilbrium. So that equilibrium doesn't depend on where you start, it doesn't depend on interventions. And it doesn't depend on history in any way. The model is really powerful. And so if you wanna argue history matters. Or if you wanna argue interventions matter. If someone gonna argue that this isn't a transition, that this isn't a Markov process. Or that you've gotta argue that you're changing the transition probabilities. Now that isn't impossible. And in fact, policies that really make a difference, interventions that really make a difference, do change transition probabilities. So what's really cool about this Markov process is, is that it's given us this model. Is it's given us a new lens to look at the world. When we think about it, I want to take this action. It's gonna make things better. We have to be saying, it's changing the transition probabilities, not just that it's changing the state. So if I make my students a little more alert for three seconds by screaming or something, that's not going to change the long running equilibrium of alert and bored students. If I change my teaching style, or if I add more interesting examples in class or something, then it's possible I can change those condition probabilities and end up with more alert students. We've also seen in this last lecture that we don't even need to use the full Markov process model, just the transition probabilities. Just that idea, the matrix of transition probabilities, and we can find out all sorts of interesting things, like who wrote a book? Is there likely to be more? Or is this medical treatment working? So that framework, the transition probability framework and that matrix of transition probabilities is a really powerful tool to keep in your pocket when you confront some sort of dynamic process and you're trying to figure out, what do I think is likely to happen. Thank you.