Now, I want to continue with the Fokker-Planck Equation in log space that we derived in the last video, and which I'll show you here once again, along with the typical form of the potential that can arise for such problem. Such potentials with the metal stable state and decaying potential behind the barrier are in fact, well studied in physics, where they describe tons of differences in distance phenomena. In particular they describe the famous problem of a decay of a metastable state in statistical physics, which was solved by Kramer in 1940s. It is for this reason that this problem is sometimes called a Kramer Escape problem. We will talk about it in a sec but first let's discuss something more conventional with the Fokker-Planck equation, namely the existence of a steady state and time independent solution. This problem is easy to solve, at least formerly. So for a steady state the left-hand side of the Fokker-Plank equation should vanish. But the right-hand side of this equation is a total derivative of a quantity called the probability current. So if the left-hand side vanishes, we can express the Fokker-Plank equation as a conservation of the probability current. And the in first stationary state, also called the ground state if the state exists then the current should vanish. And this simple argument leads to a suggestive status state solution in the form shown in equation 51 here. This is what is known as Boltzmann or exponential distribution with energy U. Exponential distribution survey popular tools in both physics and machine learning. And we mentioned exponential models a few times in our courses. Here such a distribution is derived from a physics motivated model. Now, this distribution has a normalization factor Z in front of it. And this part is a bit tricky but also the most interesting one. The point is that a steady state density only exists if it's normalizeable which requires that the normalization constant Z should be finite and positive. But if there is escape from a metastable state through tunnel link, the system cannot have a genuine steady state. What can be obtained instead in such a system would be a Quasi-stationary state that might be very long lived in fact, but still will decay in a very long time. And the time it takes for such state to decay is driven by the height of the pillar and the noise level. The existence of such process shows up in divergence or zeros of the normalization factor Z. That is called a partition function in physics. And it turns out that such a scape and tunneling problems can also be described using the language of quantum mechanics. This can be done starting with the Fokker-Planck equation as follows. First, we change the dependent variable and try the unknown density pure white as shown in equation 52 as a product of the exponential factor and a new unknown function K(y). And please note that the exponential factor here is a square root of the steady-state distribution. Now if we plug such answers into the Fokker-Planck Equation, we get another equation for the newer non-function K(y), and this is shown in equation 53. This equation is actually the same as the Schrodinger equation of quantum mechanics, where sigma squared plays the role of the Planck constant. In quantum mechanics, the Schrodinger equation is the main equation of dynamics, and it replaces the Newtonian, Lagrangian, and Hamiltonian methods for the classical mechanics to the setting of quantum domains. But in the original Schrodinger equation, there is an additional imaginary unit, i, in the left-hand side in front of the time derivative. And here there is none. So this equation can be interpreted as a Schrodinger equation in imaginary time. So as a result, we find that the classical stochastic system is mathematically equivalent to quantum mechanics in imaginary time, and a variance sigma square playing the role of the Planck constant. The Hamiltonian for the result in Schrodinger equation for this case is shown in equation 54. And it turns out that the Schrodinger equation has many, many interesting properties. And one of them is called supersymmetry or SUSY as physicists call it. Not only Schrodinger equation has supersymmetry, but this one obtained from the classical stochastic dynamics does have it. An the supersymmetry of the Schrodinger equation is based on the observation that are Hamiltonian H can be factorized as a product of two operators, A and A+, as shown in equations 55, and 56 here. These operators are sometimes called the supercharge generators. And the function, U prime is called the superpotential. Now if we have these generators we can swap the order and get a new Hamiltonian H+ shown in equation 57. And the most interesting thing about this pair of Hamiltonians H and H+ is that they have the generate spectra for all states excluding the lowest energy state with zero energy, if such state exists. And these can be seen using the simple chain of transformations shown in equation 58. If Si n is an eigenstate of H, who is an eigenvalue En, then we can form a new state, A times this state, and this state will be an eigenstate of the SUSY partner Hamiltonian H+ with the same energy. And this means that all eigenstates of H with non zero energy should be the degenerate with eigenstates of H+. And now SUSY can be unbroken or spontaneously broken. If it's unbroken, a ground state with zero energy exists. On the other hand, if the energy of the ground state is larger than zero, supersymmetry is broken. And it turns out that mechanisms of breaking supersymmetry in quantum mechanics and quantum field theory are the same mechanisms that lead to tunneling and escape from metastable potential. So how the escape looks like. If we go back to the language of classical statistical physics, then the process is described as an event when due to thermal fluctuations, a particle gets enough energy to jump over the barrier. And the probability of such event will be obtained as a product of two factors, the Arrhenius factor B and pre-factor A. The Arrhenius factor B is shown here in equation 60. And its exponential in parameter Eb that gives the height of the barrier. So if a barrier is very high then the actual escape probability can be very tiny. And vice versa, if a barrier is not too high, or the energy of a particle is such that it's near the top of the barrier then the escape probability might become quite noticeable. And the remaining pre-factor A can also be computed for one dimensional diffusion it turns out that this factor is proportional to the frequency of oscillations, omega near the bottom of the potential well. That's shown in equation 61 here. It turns out that the same expression can also be obtained from an equivalent quantum mechanical formulation. And in this case it turns out that tunneling can also be described by the laws of classical mechanics. But applied in imaginary time. In imaginary time, the kinetic energy becomes negative, and the action becomes imaginary. As you can see, if you look again at our equation, for, which I did here as equation 62, for your convenience. So for this case, the expression in the square root in there integral is negative and therefore the action itself is imaginary. But because the weight of the action is I times S, this produces exponentially suppressed tunneling in quantum mechanics. And finally, a few more words about the tunneling effect. This effect is non perturbative as we said, so it cannot be obtained as an expansion in small values of parameters kappa and g around a model with a trivial vacuum x=0. It turns out if we still start developing such perturbative schemes they become divergence series and the origin of these divergence of perturbative series and tunneling turns out to be the same. A mechanism for this is similar to divergence of Quantum Electro-Dynamics that was discovered by Freeman Dyson in 1950s. You can read more about such problems since statistical physics and quantum mechanics in your weekly reading quiz for this week. And for now we just want to summarize. So we saw that reinforcement learning and inverse reinforcement learning can be used not only to compute specific numbers in finals, but also to construct new models. And we presented one such simple model for market dynamics that is inspired or kind of derived from reinforcement learning in our previous course. Now, in this week we took another look at the same model and found the need to extend it by introducing the cubic non-linearity. And this falls from the analysis of behavior of the model and is needed for stability. This cubic non-linearity can also probably be derived directly from the enforcement learning approach. But here we take a simpler route and just added this to our phenomenological grounds using arguments based on asymptotic analysis on electricity and symmetries. And all these are common are useful tools in physics. In particular symmetries play a major role in determining characters or phase transitions between different phases of matter. Here we use such sort of analysis in a similar way to using prior Bayesian statistics. This is something that is not directly in the data but should hold anyway based on some more general arguments. In the course project for this course you will analyze the model that you did in your previous course, but this time with keeping a non-zero value of g. And hence with keeping a cubic non linearity. And this will be it for this lesson. And see you the next week.
