[MUSIC] Stability is probably one of the most popular ways to remove noisy concepts from the lattice, to reduce the size of lattice. But other criteria are also used. I'll mention a couple more. So the first one is separation index. Here that is that we want to look at concepts that are very different from the rest of the context. So if we have a concept (A, B), are the objects covered by A sufficiently different from other objects? Are the attributes from B sufficiently different from other attributes? This is a visual representation of the index. So the large square is our formal context. And A are the rows corresponding to the objects of our concept. And B is the set of columns corresponding to attributes. So on the intersection of A and B, we have crosses. So, every object from A has all the attributes from B, but we may also have crosses in other parts of this large cross. So objects from A may have other attributes not belonging to B. And other objects, those that are not from A, can also have attributes from B. So we would say that a concept is good if we have not too many objects outside A that have attributes from B and attributes outside of B characterizing objects from A. More formally, the separation index is defined as follows. In the numerator, we have the size of the formal concept (A, B); so we have the number of objects A multiplied by the number of attributes B. So if we look at our cross table just as a table, then the numerator of this formula is just the size of the subtable formed by our concept. And it's completely filled with crosses. In the denominator, we have the number of attributes that all objects from A have and the number of objects that have at least some attributes from B. So we take the size of g' for every g from object set A, and we also count the number of objects that have some attributes from B. So this is the sum of the size of m' for every attribute m from B. And it so happens that in this case we count the area given by the concept (A, B) twice, so we remove the product of |A|, |B| here in the denominator. This is the definition of the separation index. And, well, there are various ways it can be used. The most obvious way is to keep only formal concepts with a high separation index. But how do we check if this procedure works, if it gives us the result that we want to see? Well, if we have real data then we can simply apply it and look at what we get in return. If it makes sense from the domain point of view, then okay. But this is kind of subjective. A more objective way to check whether the filtration, whether the noise removal works is like this: we take a certain formal context... Well, in this case, we took a formal context that has 300 objects, and here we have its concept lattice. So 100 objects have attributes A, B, and C. Another 100 objects have attributes D, E, F. Then we have 70 objects with attributes H, I, G. And we also have 30 objects with attributes J, K, L. This is the concept lattice of this formal context, which is not shown here. And now we do the following. We alter every cell value in our formal context with probability, well, let's say 10%. So we add 10% of noise into our data. Then when we compute a concept lattice, we can get many more concepts than we initially had. For example, in one particular experiment we got 324 concepts. And then we select concepts with the highest value of the separation index, and this is what we get in return. Note that this lattice is certainly not identical to the one from which we started. But its concepts can be clustered in such a way that, well, the structure of these clusters is the same as the one from which we started, as in the original lattice. Well, but separation doesn't have to be applied alone. We can combine it with other indices. So we can combine separation with stability. And so if we're on the same experiment, but we select concepts satisfying certain thresholds for both separation and stability, then it turns out we can reconstruct the same concept lattice. We can reconstruct the same structure. Just one more experiment. So here we have something a bit more complicated. In the previous slide, all our concepts were kind of disjoint. Their extents were disjoit, except for the top concept. So here we again alter every cell value in the context with probability 10%, and then we get 16 concepts. Not as many as in the previous picture. Why? Well, because here we have only four attributes. And if we have four attributes, then 16 is simply the maximum number of concepts that we may have in our lattice. So what we got in this case is Boolean lattice. But for each concept, we have different number of objects associated with it. So we can use separation. We can use stability and other indices to try and filter out concepts that are due to noise. And this is what we can obtain with separation. It's not completely identical to this lattice, but it's almost the same. [MUSIC]
