It is a common belief among certain skeptics that one can prove nearly anything with ‘statistics.’ Unfortunately, such skeptics voice a real and pertinent concern: journalists, public speakers and government officials daily use altered, misinterpreted or simply flawed statistical figures to prove their side of an argument. Joe Horgan in “Your Analysis is Faulty (How to lie with drug statistics)” shows how in 1990 Michael Walsh, the “director of the Division of Applied Research and the Office of Workplace Initiatives at the National Institute on Drug Abuse (NIDA), the chief federal drug research agency,” used faulty statistical studies to justify and promote “programs in which the urine of working people is searched for signs of illegal drugs. Walsh designed the drug-testing program for federal employees mandated by Ronald Reagan … [in the late 1980s and advised] business leaders on how to test their workers. He has argued in favor of testing before Congress and federal judges, on national radio and TV shows, and in countless other public forums.”
According to Horgan, Walsh’s argument was simple and straightforward: “[1] drug users, from crack addicts to weekend marijuana smokers, make less productive workers than non-users. So [2] employers are justified in using drug tests … to root out all users from the work force.” The only problem was that the studies that proved Walsh’s ideas were not abundant and their results were usually not nearly as strong and convincing as Walsh hoped. But he found a way to make the statistics advocate his methods. Horgan explains how he did it.
In 1987, Walsh presented in the Supreme Court that drug abuse cost the US industry nearly $50 billion per year. Two years later, president Bush used this figure in a cavalier manner, saying that the costs ranged from $60 to $100 billion. Horgan explains why this statistic is flawed.
The statistic was taken from a 1982 study by Research Triangle Institute (RTI), a NIDA contractor in North Carolina, which analyzed incomes of 3700 households in the US. The results showed that “the household income of adults who had ever smoked marijuana daily for a month (or at least twenty out of thirty days) was twenty-eight percent less than the income of those who hadn't. The RTI analysts called this difference ‘reduced productivity due to daily marijuana use.’” They calculated the total ‘loss,’ when extrapolated to the general population, at $26 billion. Adding the estimated costs of drug-related crimes, accidents, and medical care produced a grand total of $47 billion for ‘costs to society of drug abuse.’” According to Horgan, the researchers interpreted a correlation between drug use and decreased productivity as a cause-and-effect relationship. Horgan argues that the behavior of poorer individuals differs from that of the richer individuals in many areas: possibly, the poor are more likely to watch “The Wheel of Fortune” much more frequently than the rich, but that fact does not provide sufficient evidence to state that watching “The Wheel of Fortune” decreases the viewers’ productivity or makes them poorer.
The fact that the study based its findings only on loss of productivity of people who have for at least one month in their lifetime smoked marijuana daily also raises Horgan’s suspicions. Such a strict and strange criterion was chosen, as Horgan points out, because the data supplied for less frequent use of other drugs (among which were heroin, LSD and cocaine) showed no correlation between drug use and lowered income. In order to produce the desired statistic, NIDA simply had to alter the variables under consideration.
Horgan cites other examples of statistically flawed studies that Walsh used in his drug-testing campaigns. One study analyzed the effect of marijuana on work productivity of 500 Navy officers who were employed by the Navy even though their marijuana tests were positive. In a period of 30 months, 43% of those 500 were discharged, compared to only 13% of the officers who tested negative when they were recruited. Although the numbers look decisive, their significance needs to be carefully assessed. For example, the ‘positive’ discharged officers were twice as likely not to have had a high school diploma than the ‘negative’ officers. “Why not blame poor education … for their high discharge rate?”, Horgan asks. An even more serious problem with the study lies in the fact that the control group of 500 Navy officers was heavily tested for drug use during the 30 months the study took place, and one third of the discharged officers were discharged only because they failed another drug test, while not exhibiting any loss of productivity at the workplace. Horgan notes that from these results it is reasonable to conclude that those who have used drugs already are more likely to continue using drugs – but not that the drug use affects work efficiency. However, Walsh thought otherwise.
Another study Walsh cited was performed by the Utah Power and Light Company. The results showed, as Walsh said, a “significant difference between drug users and nonusers in terms of being involved in accidents, being absent from work, and overutilization of health benefits.” The study, however, involved a miniscule sample of drug abusers – twelve users in total – and eight of them were tested for drug use after a car accident, after which some were injured and needed to take time off. Walsh’s study ignored these factors; Horgan claims that based on these factors the study itself should be ignored.
However, through like means of using incorrect statistics, Walsh succeeded in implementing the system for employee drug-testing in many big companies, including The New York Times. So maybe skeptics are right – it is possible to support almost anything with statistics; it just depends on the kind of statistics one uses.