So the question is how do we know when to trust a statistic?
Darrel Huff the author of How to Lie With Statistics, whom is also a mathematician writes,
"To be worth much, a report based on sampling must use a representative sample, which is one from which every source of bias has been removed." (Huff 20)
Examples of bias:
-using too small of groups
-favoring one group over another
The definition of bias is prejudice in favor of or against a person, or group compared with another.
Bias is not fair and greatly skews a statistics accuracy.
So consider this statistic
"The average Yaleman, Class of '24. makes $25,111 a year." This is from Time Magazine. (Huff, 13).
Is this statistic biased?
If you answered no, you are wrong. This statistic is extremely biased.
Darrel Huff lists many reasons for the Yale sample being biased.
Here are a few of them
-The statistic is to precise.
"There is a small likely hood that the average income of any far-flung group is ever going to be known down to the dollar." (Huff 13).
-People lie.
"Furthermore, this lovely average is undoubtedly calculated from the amounts the Yale men said they earned... Some people when asked their incomes exaggerate out of vanity or optimism. Others minimize." (Huff 14).
- The responses
"There are bound to many whose addresses are unknown twenty-five years later." (Huff 15).
"And, those whose addresses are known, many will not respond to the questionnaire."
"Those who are most likely to reply are those with incomes to brag about, such as the CEOs the executives of big companies, people who have made it in the world." (Huff 17).
Here I have made a chart of the Yale statistic clearly showing how the statistic is biased.
Other pointers on how to tell if a statistic can be trusted...
Don't trust them. Unless you know for sure that the source can be trusted and more often then not the statistic and the source that used it can not be trusted.
Very interesting post. Having read a little about statistics my self, including touching on Bayesian statistics I think that statistics are not inherently bad, in fact when done correctly they're actually quite reliable. The biggest problem I think, is that people generally do not know how to think in terms of statistics, and cannot determine what the statistic is saying and what is just a factor of luck and circumstance. Good job on pointing out that statistics in small groups are very unreliable and can be misinterpreted to a very large extent. That is absolutely true. Although I find your tag labeled "how to lie" to be misleading and unnecessarily cruel. It really dragged me in then backhanded me across the face when I learned the article wasn't about lying. I don't think its fair to do that to a person -Teran Long
ReplyDeleteThanks for your comment. I agree that statistics are not inherently bad, in fact I believe them to be quiet reliable if done correctly.
DeleteThanks for responding
I have to agree with Teran that the tag "how to lie" had me tricked: I was searching through the tags and right when I saw that one I knew immediately that I would enjoy the post, but then the post wasn't really about how to lie and I was a little disappointed... Nonetheless I believe that while statistics can be spun many ways to validate a claim, they can also be beneficial to understanding the world.
DeleteTerribly sorry about the confusing tag. It is meant as a reference to the book that I am reading which is titled How To Lie With Statistics.
DeleteI agree with you. Statistics can be beneficial to how the world works especially in such a fact based world. However if one takes all statistics as truth then they can also be dangerous.
It's hard to say that a statistic is unbiased, especially considering on the internet when you search a statistic you can find all kinds of results.
ReplyDeleteSourcing is definitely a key thing to keep in mind - for example trusting a textbook vs. trusting a magazine or online forum. I'll be sure to keep this information in mind when doing research.
Thanks for your comment. Please always keep in mind sourcing especially when using the internet. Because getting information of the internet is like trying to get a drink from a fire hydrant ( a quote a teacher of mine used to say all the time).
DeleteI am glad that I could be helpful in any future research that you may come across.
Interesting. I have also found that a person can find statistics to support pretty much anything they want. Please make certain to include a question with your post to facilitate discussion.
ReplyDeleteThank you for your comment. I agree that a person can find a statistic for almost anything. After all this is the internet. I will make sure to include a question next time. Thank you.
DeleteI find it extremely surprising that it is so easy to create a biased report to support their idea even if it isn't true. It makes me think how many times in my life I have possibly heard a biased report. It also is scary because you would think statistics to be truthful because they are just numbers.
ReplyDelete