Developed by engineers for engineers.

The truth about statistics

What do the issues of so-called Climate Change, Intelligent Design, Cold Fusion, and Gun Control have in common? Answer: Ignorance. A saying often attributed to Mark Twain is, “There are three types of lies: lies, damn lies, and statistics.” Since I recently discussed lies, I would like to address statistics. I think Twain meant that certain people can force both correlated and uncorrelated data to support any conclusion that they wish to draw. But the scientific method for analysis and synthesis depends on logic (common sense?), facts, and accurate and reliable data. Unfortunately, many non-technical savvy people (and some technical people) tend to fall into a trap set by the purveyors of false information (sometimes mixed with a little truth) to convince you of their agenda. Every day, as human beings, we have to resist the temptation to force relationships and draw conclusions among false or unrelated sets of data, which could lead to making some disastrous decisions. Our conclusions seem to naturally lean more toward intuition or emotion (not always reliable), personal biases, and false data correlation rather than on facts and first-hand observations.

The first line of this editorial lists some of my favorite topics (or pet peeves) that have become political issues, instead of remaining scientific investigations, in which much published information is based on false ideas, mistaken beliefs, incomplete data sets, and fallacies. Unfortunately, both proponents and opponents of these issues are guilty of manipulating unsubstantiated data to convince you to join them. In his book, The Art of Science, Joseph J. Carr lists some 26 types of arguments based on fallacies of relevance, including intimidation, ignorance, popular opinion, authority, oversimplification, ambiguity, and last but not least, prose or vague statistical comparisons.

For example, the data sets of rates of gun ownership and firearm-related crimes have no (actual, scientific, mathematical or logical) correlation. Yet, the gun-control politicians would have you believe that there are more “weapon-related” crimes (including suicides) in countries that allow citizens to own guns than in countries where guns are banned. This is simply not true. If you look up the raw data, you can see for yourself that there is no direct correlation; it is “…an argument in which all premises are true but do not properly infer the conclusion.” The data are sufficiently scattered to resist any sensible correlation. Yet, millions of dollars and man-hours are being wasted trying to convince us that we should ban private-citizen gun ownership based on these “statistics.”

Don’t misunderstand, I am not a gun-toting radical; I don’t own any weapon larger than a single-shot, .22 rifle that I used for practice when I trained as a sharpshooter in college Reserve Officers Training Corps (ROTC). But I recently joined the National Rifle Association (NRA) so I could subscribe to their publications and try to understand their position and arguments. One thing that I did learn was something about definitions. For example, “semiautomatic weapons” and “assault rifles” are not the same thing.

But the politicians have erroneously equated these two terms to support their argument for banning all weapons that contain more than a single round. Under that definition, both my grandfather and father owned “assault weapons” when they hunted ducks and pheasants. Other sportsmen used “assault weapons” to hunt deer, bear, elk, and other game. These are just some of the terms these people use to distort facts, whether they are aware of it or not. My conclusion is that we have adequate gun control laws. We should not change the Bill of Rights to satisfy some misinformed politicians who promote fear to get their votes. But, to be fair, I believe we have some room for fine-tuning the laws, such as where and when we can carry concealed weapons – a position moderately in opposition to some NRA beliefs.

I will not go into the myths surrounding Climate Change, Intelligent Design, and Cold Fusion in this editorial because of space limitations. But I will do so in the Engineering Exchange if you ask. In any case, the false premises widely published for these three other issues fall into the same category. Finally, Carr says that reason is the most powerful scientific tool you can use to get at the truth. And when you become skilled at recognizing fallacies, you considerably improve the probability of making the correct decisions. Statistics frequently lie.

References: 1. The Art of Science, by Joseph J. Carr 2. No. 07-290, Supreme Court of the United States; District of Columbia and Adrian M. Fenny, Mayor of DC vs. Dick Anthony Heller, Brief of International Scholars as AMICI CURIAE in Support of Respondent.

www.supremecourtus.gov/docket/07-290.htm

Comment

You need to be a member of The Engineering Exchange to add comments!

Join The Engineering Exchange

Comment by Steve Meyer on August 25, 2009 at 10:02am: Intelligent Design: None of the commentators appear to really know what this is about. Like most of the other controversial topics, each comment seems to simply assert their position. That's fine.

However as a point of information; Intelligent Design is simply looking at a field of knowledge and saying that from observable phenomenon the Theory of Evolution doesn't adequately explain the data, and further that an intelligent designer is more likely than random chance.

Example: It has recently been published that upon examination of the rate of genetic mutation and the birth rate cycle of whales, the theory that whales evolved from dogs or horses is physically impossible. It takes 99 years for whales to conceive and the rate of genetic mutation makes the number of years required for a whale to evolve from anything else is so large, that there is no possibility of this theory being real. There is simply not enough time in any system of estimated age of the universe. Let alone life on earth.

So if the Theory of Evolution is wrong on this account, only one of many, then it probably isn't a very useful Theory. The converse of which is an intelligent designer is much more likely than Evolution.

This is only one of dozens of situations in recent years where scientific inquiry by non-creation scientists has lead to conclusions which invalidate the theory of Evolution.

Another one is the recent Hyman Genome Mapping effort. Based on the latest study by people in this field, the oldest possible age for humanity is between 100 and 200 thousand years. This disqualifies most evolutionary theory because believers of this theory usually posit older Earth age (Billions of years) in order for Evolution to do its work.

As for evidence of evolution in humans, I would like to hear about one.

Another problem is the number of conditions required to sustain life of Earth. Distance from the sun, atmospheric pressure, composition of the atmosphere, and many other discrete factors which, when taken together, make the random confluence of these conditions statistically impossible.

As for Climate Change, there are many accredited professional who look at the data and can demonstrate that the temperature change is based on changes from the Sun. If we had that much extra CO2 in the atmosphere we would see increase plant productivity since plants use CO2 for growth. All the nurseries in my old neighborhood still bring in CO2. There have also been measurements of the energy per square meter on the Earth's surface and man's contribution is a fraction of 1%.

Do I think we ought to be driving electric cars? Absolutely. Polluting the air is crazy. But we have to have alternatives that make economic sense.

Comment by Matthew Lorono on August 6, 2009 at 5:00pm: For the record, evolution IS observable, even within the modern human species (hey, that's us!). Nonsensical is the idea that someone has to stand around and "watch" evolution happening life right in front of them in order for there to be proof. This is one form of observation. It is not the only form. Theory is a factual model that is based on repeatable observations to provide repeatable results. This is happening in the fields that have benefited from knowledge gained from evolution related studies. There are practical real world examples of this knowledge in use. There is no difference between supposed "micro evolution" and "macro evolution" at the cellular level. The same factors driving evolution in viruses and bacteria are also present within our own cells. Knowledge gained from understanding the relationship between bacteria species is no different than in scope than the knowledge gained from understanding relationships between the human species and other lifeforms. For example, it has allowed us to better understand how viruses are able to make the leap from bigs, birds and apes to humans. Another example may be that bacteria in the human body is easier to kill than fungus. We are more closely related to the fungi than to bacteria, so the number of poisons specific to fungi (that will not harm us) are fewer than the poisons we can use on bacteria. These are all at the microscopic level, but these all directly impact whether members of our species survive long enough to continue the next generation. Those who are better adapted are more likely to procreate, and that is the heart of natural selection.

Comment by Michael Eustice on August 4, 2009 at 5:02pm: EDIT to above post:

I always wonder what our fore-fathers had in mind when they came up with this one. <= sounds too sarcastic, which wasn't what I was going for. I sincerely do wonder what they meant, especially with the words 'arms'.

Comment by Michael Eustice on August 4, 2009 at 4:57pm: Climate Change: I've heard a lot of good arguments on both sides, but the one outstanding fact is that all of the respectable scientific communities in the world agree that it is man-made. The vast majority of scientists agree as well. Sure, you can find some dissenters looking for attention but most agree it is man-made. This includes the National Academy of Science and the Royal Society. I may not be able to see through the 'magical statistics', but I trust they are not easily deceived. The best argument I heard against this is that all of the scientific communities are in cahoots in some elaborate conspiracy.

Intelligent Design: LOL.

Cold Fusion: From what I've read I wouldn't hold my breath on this one.

Gun Control: Right to bear arms. I always wonder what our fore-fathers had in mind when they came up with this one. Should we each be able to own a nuke? Where do you draw the line? Thomas Jefferson wrote that he considered it important that the government remembers that the people have the power. He considered occasional rebellions 'healthy'. I don't know if he could have anticipated the level of mass destruction we are now capable of. Statistics that show gun-toting nations have no more murders than the other option would be good to see? Can you link such statistics?

Comment by Martin Howell on August 3, 2009 at 8:24am: The problem with using reason as the only basis for what is true is that, like statistic, it too can lead to false conclusions. As French mathematician and physicist Blaise Pascsal once said "Contradiction is not a sign of falsity, nor the lack of contradiction a sign of truth". I think most of us can find examples of something we know to be true yet has contradictions (eg. F=ma doesn't work as velocity approaches the speed of light). Absolute truth can only be revealed by a higher authority than our limited experiences.

Comment by Alan Pitas on August 3, 2009 at 8:23am: Dear Mr. Kessler,

My apologies for the tone of my reply. My anger at Mr. Gyorki's article got the best of me. I failed to disagree repectfully. In rereading my post this morning I feel ashamed.

Again, please accept my humble apologies.
Alan Pitas

Comment by Alan Pitas on August 3, 2009 at 8:11am: Hi Tyler,

Thanks for thinking about my post.

I'm sorry if I didn't lay my thoughts out clearly. Let me try again. The article is about the misuse of statistics. But the author never properly documents an example. Rather he does exactly the thing he says he's against. He quotes a single data point without references and without background. He is continuing the kind of speech that he is supposedly arguing against.

Now we could follow him down this path and start throwing single data points at each other, but that is what 95% of talk radio is about. What I would call "Drive by statistics."

Your example of "the Antarctic has more than doubled in size." is a classic point. A single data point that is so badly removed from context that, as stated, it is patently false. The correct statement is: Areas where ice is increasing out number areas where ice is decreasing by 2:1. Note that the relative rates of increase are not included and so no overall surface area estimate is implied. (There must be some talk show pundit out there that misquoted this thing because you are not the first person I've heard get this wrong.) In a logical discussion about Antarctica's usefulness as a measure of global warming one would want to compare this data point with other measurements like ice volume for the continent (Ice volume is actually decreasing) and then tie the data to a model. Perhaps a lower volume of ice means that Antarctica is getting colder and so less snow is falling. (The same could be said for summer melting of the Arctic.) Perhaps a larger surface area results from Antarctica warming causing ice to flow to the sea faster. These are subtle issues that require a detailed discussion. Not dueling data points. The author begins by saying that he supports detailed discussion, then violates his own request.

This obvious cognitive dissonance begs the question "Why?". Why begin by castigating others for misusing statistics and then do that very thing yourself? We all have these areas where unresolved emotional conflict leads us to hypocrisy. Most of us don't put it on display as the lead item in a technical magazine.
My suggestion was to apply logic and rigor to the authors choice of topics. My analysis suggests he chose those topics because it will enrich him personally. You are correct that I went beyond the author's putative topic.

You are right that his actions make me angry. When I lived in Europe I often found myself involved in discussions about a variety of issues from the nature of God to American foreign policy in the Africa. At no time did these discussion fall to the level of dueling data points. In general each participant could accurately defend each of the other points of view present. These discussions were more of a way of understanding the nuances of each position rather than verbal attempts at King of the Mountain. I find American discussion has not reached this level. And I miss those discussions. When Mr. Gyorki starts with such a promising title and then dives down into a pit of self-righteousness, I get mad. This approach poisons the life of the mind.

So, if the point of this discussion is to fire isolated facts off into the ether and enjoy the rush of adrenaline then I shall bow out. If instead we want to discuss how to avoid the misuse of statistics when discussing social hot button issues, then I will be happy to participate.

Comment by Tyler Kawahara on July 31, 2009 at 4:50pm: I'm not sure why Alan Pitas is so angry, but he completed neglected to make any points against what the poster stated. I'm not sure he even understood what the topic of the discussion was. The topic is about how statistics can be used to support an erroneous conclusion. As far as his choice of subject matter: it's irrelavent beyond its ability to serve as an example to help him explain the concept he is trying to discuss.

In a strange and clumsy attempt to join in the discussion, he seemed capable only of name calling and insults. First, he claims the author is a dumb person who only thinks he is smart; that the only possible motivation the author had in his choice of examples was simply to make more money.

Then he begins to deconstruct the author's very personality and character traits. He gleaned all this from the few examples the author used to help communicate the topic of the essay! Highly trained indeed!

Furthermore, the poster seemed to be of a liberal bent given the reaction he had. Yet how quickly he turns into a fiscal conservative when it comes to his money going to help others who lost thier homes in floods (possibly even floods caused by human driven global warming!), It's not ok for the author to be influenced by money, but it's apparently OK if Alan's is.

Having said that, anyone who thinks 'the debate is over' regarding global warming is simply not paying attention to the vast numbers of scientists who have gone on record to dispute the claims that man is the cause, and the solution to global warming. Someone mentioned that man's affect on the environment is 'huge'. I don't really think that the .05% (that's 1/2 of 1 percent, not 5 percent) of greenhouse gases man is responsible for can be called 'huge'. As if to prove this point, note the change in terms: 'global warming' has been abandoned for 'climate change' in light of the recent reports that the earth is actually in a cooling phase according to recorded temperatures. Sure the Northern polar cap may have shrunk, but the Antarctic has more than doubled in size. And while my daughter's teacher might think it's fun to assign a homework assignment on the plight of the Polar Bear, she doesn't think it's ok for me to help my daughter to include the fact that the Polar Bear population has doubled in the last 50 years. Perhaps Alan can use his enormous intellect to determine the motivating factors for people like Al Gore who neglect to discuss the whole issue and ignore the real 'inconvenient truths'. It's simply disgusting behavior from a giant group of elitists who are incapable of any real debate.

Comment by Alan Pitas on July 31, 2009 at 10:22am: Dear Mr. Kessler,

It seems you got out of college without studying several topics that you, never the less, feel comfortable expounding upon.

Let's begin with the idea that "Events simply do not happen without a cause/reason." Really?? Have you done an experiment to prove this? Do you know of an experiment to prove this? The greatest minds from the last two thousand years of Western civilization would love to hear from you. The fact is that causation is one of the founding assumptions of science. No one,so far, has thought of a way of proving causation. It is one of the reasons that physics was first called natural philosophy.

Please try and get your head around this: Causation is not a fact. It is something most of us believe in. It has been a very useful belief, but it is nothing more.

Second , your idea that randomness does not exist conflicts with the very heart of quantum mechanics. Quantum mechanics states that certain processes can not be known by underlying rules but instead are inherently random. One can only measure the distribution of results. Predicting the outcome of any one experiment can only be given by a probability and nothing more. By the end of the 19th century it was looking like the whole universe was a big clock that would run to its conclusion following a well defined set of rules. Quantum mechanics destroyed this view. This is why Oppenheimer said "God again has something to do."

To be fair, Einstein objected to this aspect of quantum mechanics. He said "God does not play at dice." However, to this point, it looks like She does. You might want to update your world view to at least 1930.

Finally, your differentiation between a theory and a law is layman's distinction not found in science proper. If someone tells you they know a law of science it just means they have an emotional attachment to a theory. Newton's "Law of Gravitation" turns out to have been theory. Einstein refined it. Someone in the future will refine it again. The fact is the words "law" and "science" do not belong in the same sentence. There is only our current best guess. Many of our guesses are very powerful. But they are only a best approximation. With time they be refined again.

Understood in this light, climate change and evolution are our best approximations. They have been refined to the point that they are powerful tools generating new insights of their own accord. But they are still theories. Those who claim that they are more demonstrate the existence of unresolved personal issues. Those who claim that they are "just" theories demonstrate a shameful ignorance of the language of science. The fact that so many young people graduate from college without this understanding speaks to the emphasis on training over education and the anti-intellectual nature of American culture in general.

Comment by Alan Pitas on July 31, 2009 at 6:57am: So how far are you willing to go in being logical?

How about analyzing why you chose that particular list of issues? There are, after all, so many misconceptions out there. Issues like intelligent design, gun control, and cold fusion have almost zero effect on your life. It looks to me like your motivation is primarily emotional. Essentially an irrational cry for others to be rational.

Consider that civil nuclear power exists only because Congress capped the liability for a nuclear accident so that power generating companies could get insurance. Without this exemption, worth trillions of dollars, there would be no nuclear industry. Yet this give away is never discussed, never priced, never reviewed. Much as I love nuclear power, I have to admit that truely insuring our nuclear reactors will require a huge investment. As a tax payer, I would have thought that you would be pretty concerned about this hugh liability hanging over your head. But then again, perhaps your list wasn't chosen logically.

Consider the growth of disaster relief as a federal priority. In Wisconsin, if your town allows you to build in a flood plain then the state will not provide flood insurance. Whole communities have been moved out of river bottoms to avoid future drains on the state treasury. Yet the federal government continues to subsidize the building of trailer parks in regions prone to hurricanes. As a tax payer this issue will impact your wallet much more than whether or not intelligent design is taught in what passes for schools in Kansas. Why doesn't it merit inclusion in your list?

I could go on, and on with issues that have a significant direct impact on our lives, yet never merit a tirade in your editorials. So why did you pick those topics? I'm not sure. It's so hard to differentiate mendacity from incompetance.

Are you, on the one hand, part of that massive herd discourged from our colleges and universities each year who thinks of himself as educated when in fact you are only highly trained? As a builder of cutting edge instrumentation I have met far too many scientists and engineers who love to feel the adrenline rush of pointing out the errors in other's logic, but never turn their logic on themselves. An educated person watches their inner self with the same scutiny that they bring to the outside world. So perhaps you are simply an unconscious hypcrite. Perhaps no one has ever challenged you to bring intellectual rigor to your own motivations. Perhaps you chose that list because it felt good. I would like to think that's the case.

But you are a magazine editor. Which means you have a monetary interest in your writings. So my guess is you are an engineering version of Rush Limbaugh, chosing your list for its societal punch. By picking hot button topics you hope for a larger readership, increased advertising revenue, and a better salary. I have no problem with such an approach, but then you need to say that from the start. Leading off with: "I'm going to say some things that will get some of you riled up so I can make more money." would be far more honest than decrying the ignorance that surrounds your list.

Bringing logic and rigor to bear on how you chose your list, I find another student of "The Prince". Machivelli would be proud. Unfortunately, I do not support forces that lead to a less civil society. I don't buy any thing that is advertised on Limbaugh like talk shows. Your magazine advertises a number of products that I own or hope to buy in the near future. Since I want these products, my only choice seems to be to stop reading your magazine. I'll think about my options for the next month or so, but your editorials are leading me to cancel my subscription.