[MUSIC] Stability is probably one of the most popular ways to remove noisy concepts from the layers, to reduce the size of layers. But other criteria are also used. I'll mention a couple more. So the first one is separation index. Here that is that we want to look at concepts that are very different from the rest of the context. So it will have a concept AB. Are the objects covered by A sufficiently different form other objects? Are the attributes from B sufficiently different from attributes? This is a visual representation of the index. So the large square is our formal context. And A are the roles corresponding to the objects of our concept. And B is the set of columns corresponding to attributes. So on the intersection of A and B, we have crosses. So, every object from A has all the attributes from B, but we may also have crosses in other parts of this large cross. So, objects from A may have all the attributes not belonging to B. And other objects those that are not from A can also have attributes from B. So we would say that a concept is good if we have not too many objects outside A that have attributes from B and attributes outside of B characterizing objects from A. More formally, the separation in X is defined as follows. In the numerator, we have the size of the formal concept A, B. So we'll have the number of objects A multiplied by the number of attributes B. So if we look at our cross table just as a table, then the numerator of this formula is just the size of the subtable formed by our concept. And it's completely filled with crosses. In the denumerator, we have the number of attributes that all objects from A have and the number of objects that have at least some attributes from B. So, we take the size of g prime for every g from objects at A and we also count the number of objects that have some attributes from B. So this is the sum of the size of m prime for every attribute m from B. And it so happens that in this case we count the area given by the concept AB twice, so we remove the product of AB here in the denumerator. This is the definition of the separation index. And, well,, there are various ways it can be used. The most obvious way is to keep only formal concept with a high separation index. But how do we check if this procedure works, if it gives us the result that we want to see? Well, if we have real data then we can simply apply it and look at what we get in return. If it makes sense from the domain point of view, then okay. But this is kind of subjective. A more objective way to check whether the filtration, whether the noise removal works. It's like this, we take a semi formal context. Well in this case we took a formal context that has 300 objects and here we have its concept lattice. So 100 objects has attributes A, B and C. Another 100 objects has attributes DEF. Then we have 70 objects with attributes HIG. And we also have 30 objects with attributes JKL. This is the concept lattice of this formal context, which is not shown here. And now we do the following. We alter every cell value in our formal context with probability, well let's say 10%. So we add 10% of noise into our data. Then when we compute a concept lattice, we can get many more concepts than we initially had. For example in one particular experiment we got 324 concepts. And then we select concepts with the highest value of the separation index and this is what we get in return. Note that this lattice is certainly not identical to the one from which we started. But its concepts can be clustered in such a way that well, the structure of these clusters is the same as the one from which we started, as in the original lattice. Well, but separation doesn't have to be applied alone. We can combine it with other indices. So we can combine separation with stability. And so if we're on the same experiment, but we select concepts satisfying certain thresholds for both separation and stability, then it turns out we can reconstruct the same concept lattice. We can reconstruct the same structure. Just one more experiment. So here we have something a bit more complicated. In the previous slide all our concepts were kind of disjoined. Their extents were disjoined, except for the tall concept. So here we again alter every cell value in the context with probability 10% and then we get 16 concepts. Not as many as in the previous picture. Why? Well because here we have only four attributes. And if we have four attributes, then 16 is simply the maximum number of concepts that we may have in our lattice. So what we got in this case is Boolean lattice. But for each concept we have different number of objects associated with it. So we can use separation. We can use stability and other indices to try and filter out concept that are due to noise. And this is what we can obtain with separation. It's not completely identical to this lattice, but it's almost the same. [MUSIC]
