![]() ![]() So, entropy simply quantifies the uncertainty or ignorance in a statistical sense, which is equivalent to the possible configurations of possibilities of the system. When entropy is zero, this is the case of maximum information, where there is no need to carry out the experiment since we know the outcome is always 1! When we have more certainty about the outcome, entropy is lower. Another way of looking at this is that maximum entropy is when all the outcomes are equally likely and the degree of uncertainty or ignorance about the outcome is therefore the highest. So the entropy is maximal when the number of possibilities is highest and lowest when there is only one possibility. Now it turns out that H=0.69 which is in between. Let’s consider one more case, where you can pick either 1 or 2 but not 3, then probabilities are This would have the following probabilities for which it turns out that H = 0. If instead you could only pick the number 1 which means there is only possibility. Plugging these values to the equation above, the entropy H turns out to be 1.09. Say in one case we have the equal probabilities for picking each number as follows: To make the concept of statistical entropy more intuitive, consider an experiment of picking a number from a set S= and the probabilities of picking each number. Shannon introduced this concept into the field of information theory and defined what is commonly known as statistical entropy, The basic idea of entropyĮntropy as a concept first originated in the field of thermodynamics and a typical, physical interpretation of entropy is the disorder of a system, described by the distribution probabilities of molecules of gaseous or fluid systems. However there are many method and parameter choices that can fundamentally change the result and meaning. Entropy measures quantify the uncertainty in the EEG, which roughly equates to the possible configurations or their predictability. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |