Informed Speculation

Those who waved (and continue to wave – see McGovern, George) the Lancet’s 600,000+ excess-Iraqi-deaths number around as a blunt instrument to strike down defenders of the Iraq War had one advantage over us: they could point to the ‘statistically sound’ methodology and fall back on arcane statistical niceties while we could only say “It’s not logical – it’s not believable – it doesn’t pass the smell test”.

That advantage has now completely evaporated:

A new survey estimates that 151,000 Iraqis died from violence in the three years following the U.S.-led invasion of the country. Roughly 9 out of 10 of those deaths were a consequence of U.S. military operations, insurgent attacks and sectarian warfare.

The survey, conducted by the Iraqi government and the World Health Organization, also found a 60 percent increase in nonviolent deaths — from such causes as childhood infections and kidney failure — during the period. The results, which will be published in the New England Journal of Medicine at the end of the month, are the latest of several widely divergent and controversial estimates of mortality attributed to the Iraq war.

The three-year toll of violent deaths calculated in the survey is one-quarter the size of that found in a smaller survey by Iraqi and Johns Hopkins University researchers published in the journal Lancet in 2006.

Both teams used the same method — a random sample of houses throughout the country. For the new study, however, surveyors visited 23 times as many places and interviewed five times as many households. Surveyors also got more outside supervision in the recent study; that wasn’t possible in the spring of 2006 when the Johns Hopkins survey was conducted.

In other words, using the same methods, but more carefully, and with a much larger sample size, a result was obtained that is only 1/4 as large as that of the Lancet study.

Now, let’s put the obligatory disclaimer up front: 150,000 dead is, of course, a tragedy.  That’s not just lip service, that’s a heartfelt response.  And, to be honest, I think this number may even be a bit low.  However, knowing what we know about the population of Iraq, reported casualties, other studies, and numbers from sources as varied as Iraq Body Count and the White House, this one does indeed pass the elusive, undefinable, yet undeniably present ‘smell test’.

That is to say, while it represents a tremendous loss of life, it does not seem out of proportion to common sense.  One reason may be that the survey was allowed to ‘cross-reference’ other sources:

The new study was conducted between August 2006 and March 2007 in all regions of the country, including the Kurdish northern area. Surveyors visited about 1,000 randomly selected geographic areas (called “clusters”) and interviewed people in 9,345 households. They were asked whether anyone in the household — defined as people living under the same roof “and eating from one pot” — had died from June 2001 through June 2006.

Each death was assigned to one of 23 causes. “Violent death” covered shootings, stabbings, bombings and other intentional injuries, and included civilian, military and police deaths but not suicides and traffic fatalities unrelated to roadside bombs.

Danger prevented surveyors from visiting 11 percent of the chosen clusters. Deaths in those areas were estimated using the ratio of deaths in the region to deaths in other regions as found in the Iraq Body Count, a continuous count of reported and verifiable violent deaths of civilians kept by an independent, London-based group. (That count, which even its organizers agree misses many deaths, registered 47,668 deaths from the U.S.-led invasion through June 2006).

Previous research has shown that household surveys typically miss 30 to 50 percent of deaths. One reason is that some families that have suffered violent deaths leave the survey area. Demographers think that as many as 2 million Iraqis have fled the country since the war began, and the 151,000-death estimate includes an adjustment for this.

Let’s be clear on another point: this is not the gospel.  It is an estimate.  And while the researchers involved in this study were, I’m willing to wager, opposed to the war, the study seems devoid of the partisan edge that colored the Lancet study from the beginning, such as the intentional release timed to influence U.S. elections, a point the study’s authors readily concede.  And the reaction from others has reflected this:

Iraq’s health minister, Salih al-Hasnawi, in a conference call held by WHO yesterday morning, said: “Certainly I believe this number. I think that this is a very sound survey with accurate methodology.”

Other experts not involved in the research also expressed confidence in the findings, even though, as with the earlier survey, the 151,000-death estimate has a wide range of statistical uncertainty, from a low of 104,000 to a high of 223,000.

“Overall, this is a very good study,” said Paul Spiegel, a medical epidemiologist at the United Nations High Commission on Refugees in Geneva. “What they have done that other studies have not is try to compensate for the inaccuracies and difficulties of these surveys, triangulating to get information from other sources.”

Spiegel added that “this does seem more believable to me” than the earlier survey, which estimated 601,000 deaths from violence over the same period.

U.S. military officials yesterday pointed to the great disparity between the two estimates, noting privately that it underscores the potential for inaccuracies in such surveys. The Defense Department has not released any estimates of civilian deaths and has said often that the military takes precautions to prevent civilian casualties, while the United States’ enemies in Iraq deliberately target civilians.

“It would be difficult for the U.S. to precisely determine the number of civilian deaths in Iraq as a result of insurgent activity,” said Lt. Col. Mark Ballesteros, a Pentagon spokesman. “The Iraqi Ministry of Health would be in a better position, with all of its records, to provide more accurate information on deaths in Iraq.”

I titled this post “How To Spot A Hack”, so I think it’s time I delivered.  When someone who is against the war throws out the Lancet numbers as if they were completely uncontroversial and widely accepted, when anyone with the ability to do a Google search knows otherwise, guess what?  You’ve spotted a hack…and with the release of this new study, that’s truer than it has ever been…

UPDATE 10:02 p.m.: Hack sighting!  The Washington Post, cited extensively above, rightly makes the comparison between this study and the most often refererenced, that of the Lancet.  The New York Times, unable to bear the thought of giving any ground in its ardent pursuit of hackery (and apparently unable to work out the whole font thing), makes the totally fallacious comparison with Iraq Body Count, which was never intended to be a statistical survey at all, but rather, as its name suggests, a counting of actually confirmable reported deaths.  Ding ding ding!  You can collect your prize now…