{"id":194,"date":"2009-05-31T23:45:06","date_gmt":"2009-06-01T03:45:06","guid":{"rendered":"http:\/\/kmci.org\/alllifeisproblemsolving\/archives\/black-swan-ideas-mediocristan-extremistan-and-randomness\/"},"modified":"2009-06-04T21:15:59","modified_gmt":"2009-06-05T01:15:59","slug":"black-swan-ideas-mediocristan-extremistan-and-randomness","status":"publish","type":"post","link":"https:\/\/kmci.org\/alllifeisproblemsolving\/archives\/black-swan-ideas-mediocristan-extremistan-and-randomness\/","title":{"rendered":"Black Swan Ideas: Mediocristan, Extremistan, and Randomness"},"content":{"rendered":"<p align=\"center\"><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/www.dkms.com\/kmci\/alllifeisproblemsolving\/wp-content\/themes\/cutline-3-column-split-11\/images\/burningofparliament1834JWMTurner.jpeg\" alt=\"burningparliament\" height=\"356\" width=\"475\" \/><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">There&#8217;s a good reason why <a href=\"http:\/\/www.amazon.com\/Black-Swan-Impact-Highly-Improbable\/dp\/1400063515\/ref=sr_1_2?ie=UTF8&amp;s=books&amp;qid=1243826742&amp;sr=1-2\" title=\"The Black Swan\">The Black Swan<\/a> is a best seller. It&#8217;s written in a very lively style with great narratives, literary images, and vivid terms and phrases. Nassim Nicholas Taleb (NNT) talks about scalability, non-scalability, Extremistan, Mediocristan, the fallacy of silent evidence, confirmation error or platonic confirmation, epistemic arrogance, future blindness, the lottery-ticket fallacy, the ludic fallacy, Mandelbrodtian randomess and Gray Swans, the narrative fallacy, Platonic folds, Platonicity, randomness as incomplete information, retrospective distortion, and the round-trip fallacy. In future Knowledge Management blogs, I&#8217;ll be commenting on many of these ideas. In this blog, I&#8217;ll begin with NNT&#8217;s distinction between Medocristan and Extremistan, and will also comment on the idea of randomness as incomplete information.<\/font><\/font><!--more--><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">Both Mediocristan and Extremistan are Weberian Ideal Types. They are \u201cutopian\u201d places, \u201ccountries,\u201d \u201cprovinces,\u201d or \u201clands.\u201d What distinguishes these, first of all, is the idea of \u201cnon-scalability\u201d vs. \u201cscalability.\u201d Non-scalability exists when the consequences of an activity are dependent on the labor or effort invested. The remuneration of a Doctor, Lawyer, or industrial worker isn&#8217;t scalable because its based on the number of hours worked. The work of an author, however, may produce gains that are unrelated to its quantity. These gains are scalable in the sense that a relatively small amount of work can lead to a huge return. NNT has had a career as a trader. Traders can make a fortune in a single day if they can sense the direction of the market and ride a wave. <\/font><\/font><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">In Mediocristan, nothing is scalable, everything is constrained by boundary conditions, time, the limits of biological variation, the limits of hourly compensation, etc. Because of such constraints and the limits of our knowledge, random variation of attributes exists in Mediocristan, and can be usefully described by Gaussian probability models (the bell curve or other distributions having a family resemblance to it). In such \u201corderly\u201d randomness models, probability distributions are such that no single instantiation of the value of an attribute can greatly affect the sum of all values in the distribution. Even the most extreme attribute values do not materially affect the mean value of a distribution, because the more extreme any value is, the more improbable it is that the extreme value will actually occur in nature. Games of chance are the paradigmatic generators of phenomena found in Mediocristan. The distributions they produce provide the best fit to Gaussian models. And individual Black Swan events almost never occur because their probability, according to Gaussian models, is so low. When they do occur, it is because someone thinks that events are generated deterministically, when they are actually generated by Gaussian random processes, and so they encounter the unexpected events that violate their deterministic expectations.<\/font><\/font><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">In Extremistan, variation within distributions, is far less constrained than in Mediocristan. It is the land of scalability. Generators of events produce distributions with very large or very small extreme values, relatively frequently. And those extreme values often affect the sum of attribute values in a sample distribution, and the mean value of such distributions. The probability of occurrence of extreme values varies greatly from Gaussian models. In fact, many attribute value distributions in Extremistan do not fit any known models well. Examples of them include sales distributions for books per author, wealth and income distributions for individuals and businesses. Since extreme occurrences can greatly affect statistical properties of distributions from Extremistan, it is hard, in contrast with data from Mediocristan, to make reliable inferences from sample data. <\/font><\/font><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">The values in Extremistan distributions represent events that are Black Swans in NNT&#8217;s sense. But there are also rare events in Extremistan that are not Black Swans. They are Gray Swans in the sense that they are somewhat tractable scientifically, and reflect what NNT calls \u201cMandelbrodtian randomness.\u201d These distributions follow or approximate \u201cscalable, scale-invariant, power laws, Pareto-Zipf laws, Yule&#8217;s law, Paretian-stable processes, Levy-stable, and fractal laws.\u201d (p. 37)<br \/>\n<\/font><\/font><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">Now, the importance of the distinction between Extremistan and Mediocristan is that for NNT, \u201cwhile weight, height, and calorie consumption are from Mediocristan, wealth is not. Almost all  social matters are from Extremistan.\u201d (p. 33) That&#8217;s because, he says, social quantities are informational, and can take on any value without expending energy. A consequence of this is that attempts to predict social phenomena based on techniques using Gaussian models will render us \u201cfooled by randomness\u201d because they won&#8217;t either account for or lead us to expect Black Swans or Gray Swans. We may, at first, and for some time use these models to help us predict successfully. But they will lull us into a false sense of security, and ultimately we will invest or otherwise act on a prediction from the model that will cost us dearly. On the other hand, if we keep in mind that social events are from Extremistan, where Black and Gray Swans occur frequently, we will be alert to the possibility of their occurrence, and will expect them to sooner or later occur. This will affect our strategies for investment, or action and will allow us to build in hedges that allow us to cope much more successfully with the unexpected.<\/font><\/font><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">The distinction between Extremistan and Mediocristan is thought-provoking and perhaps even a bit compelling. But I wonder what it adds to the idea that social phenomena can&#8217;t be described or explained by models using Gaussian probability or related models, while they can sometimes be described and explained much more successfully using models from what NNT has called the category of \u201cMandelbrodtian\u201d randomness. That is, I can&#8217;t see what the invocation of the provinces of Extremistan and Mediocristan really accomplishes by way of explanation of distributions that don&#8217;t fit either Gaussian or other probability models with a family resemblance to them.<\/font><\/font><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">Moreover, in pointing out that Black Swans can occur in Mediocristan due to errors in models generating erroneous expectations, NNT reminds us of the subjective aspect of his notion of \u201crandomness\u201d. He views it \u201cas incomplete information: simply what I cannot guess is random because my knowledge about the causes is incomplete, not necessarily because the process has truly unpredictable properties.\u201d (p. 308) Thus, for NNT, whenever a model about a process is contradicted by data and violates our expectations we have \u201crandomness.\u201d So random events are always Black Swans, or at least Gray ones whether they occur in Mediocristan or Extremistan, so long as they don&#8217;t fit the particular models one has been using to study events in the domain in which the unexpected occurs.<\/font><\/font><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">One of NNT&#8217;s favorite philosophers is  Karl Popper, whom he praises effusively, for his skeptical empiricism and emphasis on conjectures and refutations. But he seems unaware of, and does not discuss, Popper&#8217;s lifelong opposition to the idea than randomness is about incomplete information, that the world is fundamentally deterministic, and that, if we have enough information, everything can be explained with reference to efficient causation. In <a href=\"http:\/\/www.amazon.com\/Open-Universe-Postscript-Scientific-Discovery\/dp\/0415078652\/ref=sr_1_1?ie=UTF8&amp;qid=1243827274&amp;sr=1-1\" title=\"The Open Universe\">The Open Universe (1982)<\/a> and in an Appendix to recent editions of <a href=\"http:\/\/www.amazon.com\/Logic-Scientific-Discovery-Routledge-Classics\/dp\/0415278449\/ref=sr_1_1?ie=UTF8&amp;s=books&amp;qid=1243827377&amp;sr=1-1\" title=\"The Logic of Scientific Discovery\">The Logic of Scientific Discovery<\/a>, Popper gives his account of the propensity interpretation of probability which views both probability and randomness as occurring as a function of the generating conditions of processes.<\/font><\/font><\/p>\n<p style=\"line-height: 150%\" align=\"left\"><font face=\"Arial, sans-serif\"><font size=\"3\">Randomness, according to the propensity interpretation and Popper&#8217;s axiom system for probability, is not, in general, a function of our ignorance, but is an objective attribute of certain real world processes. If this account is right, and NNT doesn&#8217;t confront it in the Black Swan, there are not two kinds of randomness, one most characteristic of Mediocristan, and the other most characteristic of Extremistan, but only one kind of randomness, the kind that is generated by certain real world processes. Errors in both \u201cstans\u201d are caused by our ignorance, and not by randomness, and they are to be cured in both cases by creating better models, theories, conjectures and other formulations of knowledge claims that can survive our best tests, criticisms, and evaluations.<\/font><\/font><\/p>\n","protected":false},"excerpt":{"rendered":"<p>There&#8217;s a good reason why The Black Swan is a best seller. It&#8217;s written in a very lively style with great narratives, literary images, and vivid terms and phrases. Nassim Nicholas Taleb (NNT) talks about scalability, non-scalability, Extremistan, Mediocristan, the fallacy of silent evidence, confirmation error or platonic confirmation, epistemic arrogance, future blindness, the lottery-ticket [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[14,16,3,8],"tags":[1618,1594,19,3531,1596,1595,1382,1597,1598,124,1381,99],"class_list":["post-194","post","type-post","status-publish","format-standard","hentry","category-complexity","category-epistemologyontology","category-knowledge-making","category-knowledge-management","tag-1618","tag-extremistan","tag-karl-popper","tag-knowledge-management","tag-mandelbrodtian-randomness","tag-mediocristan","tag-nassim-nicholas-taleb","tag-objective-randomness","tag-propensity-interpretation-of-probability","tag-randomness","tag-the-black-swan","tag-the-open-universe"],"_links":{"self":[{"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/posts\/194","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/comments?post=194"}],"version-history":[{"count":0,"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/posts\/194\/revisions"}],"wp:attachment":[{"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/media?parent=194"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/categories?post=194"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kmci.org\/alllifeisproblemsolving\/wp-json\/wp\/v2\/tags?post=194"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}