Uses and abuses of the "impact factor"

Crooked Timber has a post about an article in the latest Chronicle of Higher Ed regarding uses and abuses of the concept of the "impact" factor by journals in or wanting to be in the ISI's listings of "important" journals. (ISI--owned by Thomson--produces the SSCI, SCI, and A&HCI, as we'll recall.) One interesting place in the Chronicle article is where Eugene Garfield, who helped develop the concept of the impact factor, "compares his brainchild to nuclear energy: a force that can help society but can unleash mayhem when it is misused":

"We never predicted that people would turn this into an evaluation tool for giving out grants and funding," says Mr. Garfield.

One abuse of the impact factor is the pressure to self-cite (that is, cite other articles published in the same journal) in order to boost the journal's impact factor. Another problem has to do with the shaky relationship between journals' impact factors and that of the articles published in those journals:

Mr. Garfield and ISI routinely point out the problems of using impact factors for individual papers or people. "That is something we have wrestled with quite a bit here," says Jim Pringle, vice president for development at Thomson Scientific, the division that oversees ISI. "It is a fallacy to think you can say anything about the citation pattern of an article from the citation pattern of a journal."

Such warnings have not helped. In several countries in Europe and Asia, administrators openly use impact factors to evaluate researchers or allocate money:

• In England, hiring panels routinely consider impact factors, says Mr. Nevill.

• According to Spanish law, researchers are rewarded for publishing in journals defined by ISI as prestigious, which in practice has meant in the upper third of the impact-factor listings.

• In China, scientists get cash bonuses for publishing in high-impact journals, and graduate students in physics at some universities must place at least two articles in journals with a combined impact factor of 4 to get their Ph.D.'s, says Martin Blume, editor in chief of the American Physical Society, who recently met with scientists in China.

The obsession with impact factors has also seeped into the United States, although less openly. Martin Frank, executive director of the American Physiological Society, says a young faculty member once told him about a policy articulated by her department chair. She was informed that in order to get tenure, scientists should publish in journals with an impact factor above 5.

"We are slaves to the impact factor," says Mr. Frank, whose organization publishes 14 science journals.

Although the article is focused on the sciences, one might wonder if there are similar problems with the abuse of this approach in the social sciences and humanities. (It's not hard to guess what my answer to that would be...)

There's an online discussion hosted by the Chronicle. (Begins at 1 p.m., US Eastern time)

[academic publishing] [gaming the system] [impact factor]