Is there inherent intelligence in probability?
You have a jar. It's full of a lot of sweets/candies. In order to win the jar you must correctly guess the quantity in the jar. There are many players. You make a semi-educated guess and the likelihood of you being correct is pretty low.
But what happens at the end of the game is mysterious and amazing. After a large number of players have guessed, suddenly an accurate answer is tangible. How? The average. The mean.
Due to probability a player is just as likely to overestimate as to underestimate the quantity in the jar. And due to this the more players that play, even if they are all wrong, the more accurate the calculation of the real answer gets. I find this remarkable. It's as if by understanding a simple rule of maths you can transform a whole lot of wrong into a single piece of right.
Why does a quantity of invalid information indicate the whereabouts of valid information? Is there some intelligence to this behaviour of maths and statistics or is it all simply a product of intelligence?
But what happens at the end of the game is mysterious and amazing. After a large number of players have guessed, suddenly an accurate answer is tangible. How? The average. The mean.
Due to probability a player is just as likely to overestimate as to underestimate the quantity in the jar. And due to this the more players that play, even if they are all wrong, the more accurate the calculation of the real answer gets. I find this remarkable. It's as if by understanding a simple rule of maths you can transform a whole lot of wrong into a single piece of right.
Why does a quantity of invalid information indicate the whereabouts of valid information? Is there some intelligence to this behaviour of maths and statistics or is it all simply a product of intelligence?
Comments (14)
The phenomenon of many guesses centered on the right answer may be called "The Bell Curve" or The "Wisdom of Crowds". Chaotic randomness has been found to have an intrinsic hidden core of Order. Chaotic systems are unpredictable, but deterministic. Opinions vary on an explanation for the ultimate cause of that emergent order within chaos, but without it our world would have decayed into dust long ago. I call that anti-entropy organizing force "Enformy". And it seems to be a sign of intelligence & intention underlying the laws of Nature. :smile:
Wisdom of Crowds : https://en.wikipedia.org/wiki/The_Wisdom_of_Crowds
Chaos Theory : https://en.wikipedia.org/wiki/Chaos_theory
Enformy : http://blog-glossary.enformationism.info/page8.html
If the opposite is true, then bigger crowds will give wronger answers.
I wonder if this is so...or if it is just something you are supposing is so.
Has it been tested?
There is something gnawing at me saying, "The general thesis here may be all wrong in many cases."
Even in variations on the example you gave, I doubt there is a bell curve resultant that aims at the correct answer.
Have you indication that it is so...or is it just something that randomly occurred to you?
This is so interesting
It has. Numerous times with replicable results
In case you missed the sidebar, here's a little more detail on the concept of positive Enformy, which is called "Negentropy" by physicists. They probably missed the progressive implications of evolution because they believed it to be totally random & directionless. :smile:
Entropy :
[i]A quality of the universe modeled as a thermodynamic system. Energy always flows from Hot (high energy density) to Cold (low density) -- except when it doesn't. On rare occasions, energy lingers in a moderate state that we know as Matter, and sometimes even reveals new qualities and states of material stuff .
The Second Law of Thermo-dynamics states that, in a closed system, Entropy always increases until it reaches equilibrium at a temperature of absolute zero. But some glitch in that system allows stable forms to emerge that can recycle energy in the form of qualities we call Life & Mind. That glitch is what I call Enformy.[/i]
http://blog-glossary.enformationism.info/page8.html
Does empirical statistical evidence count as rational? This intelligent "fluke" seems to be built-in to the mathematical foundation of Nature. :smile:
Wisdom of Crowds : "There is growing evidence that the wisdom of crowds can be really powerful," Kao says. "A lot of studies show that you can calculate the average of estimates and that average can be surprisingly good."
https://phys.org/news/2018-04-crowd-wisdom.html
"Do you believe that a crowd can be more intelligent than any individual in the crowd?"
https://nrich.maths.org/9601
Law of Large Numbers : https://en.wikipedia.org/wiki/Law_of_large_numbers
Your very article points out the biased nature of people's predictions.
"However," adds Berdahl, "there is a great deal of evidence that people have strong biases in estimation and decision tasks."
Hence the crowd might be wise, but it has nothing inherently to do with probability or the laws of nature, per se. All that the law of averages can say is that ignorant opinions cannot completely drown out expertise.
Sounds like fantasy to me.
IF what follows is the question proposed, are you saying that the guesses would lead us to "valid information" on the truth of the situation?
Are there any sentient beings living on any of the planets circling the nearest 25 stars to Sol?
And are you also suggesting that because 95% of Americans guess that a GOD exists...that represents "valid information" on the truth of whether or not a GOD actually does exist?
They argued that if you only know that your data is Gaussian distributed, but that you don't know it's location and scale parameters, then you should assign a particular 'non-informative prior' over these parameters, say a uniform distribution, and then integrate out these parameters analytically in order to obtain a 'predictive distribution' - which is essentially to accept the wisdom of crowds.
But ignorance about a distribution's parameters cannot imply any particular distribution of outcomes, rather it implies every distribution of outcomes that is consistent with one's state of ignorance.
For example, if we don't know anything about the bias of a coin, then we shouldn't assign a particular distribution to the probability of getting heads, say P(heads)= 0.5 - as is done by naive Bayesian practitioners who appeal to maximum entropy or the principle of indifference, rather we should assign the interval (0,1) i.e.
P(heads) = (0,1) i.e. 0 < P(heads) < 1
No because "means" refer to numerical values which are subject to probability in the statistical sense no in the philosophical/metaphysical sense. Plus these experiments encounter the phenomenon of "educated guesses" where direct information is being perceived but relatively poorly or inaccurately. It's as if the sum of all semi accurate perceptions balance eachother in such a way as to give a realistic perception of the quantity if something.
I would imagine it also works for some qualities. For example the ratio of different basic tastes in a foodstuff. If you had hundreds of people taste the flavour of a food and asked them to rate it in each category and you took the average of all subjective interpretations it's likely to result in a reasonably objective actual ratio of sugar (sweet) , salt (salty), glutamate (umami) etc flavours.
Basically there is an intelligence to collectives that is not possessed by any specific individual but is derived from the interaction of their personal guesses/ perceptions.
Okay. You do seem convinced of that. I am not.
But, it has no practical purpose for me in my life at the moment, so I'll leave it be.