By A Nose
Jockeying in the rankings race


Vol. 8 No. 2
October/November 2005

Return to Contents


By A Nose
Jockeying in the Rankings Race

The Current Standings

Whither the NRC Study?

"I am not going to change our methods of calculation just in order to try and achieve a ranking higher than another institution."

"Part of the reason educational reputation is so important is because people—students, faculty, and administrators—derive much of their status from the status of their institution."

Graduate School and College Excellence
Does research reputation influence undergraduate rankings?

Peer Scorings and Rankings of Colleges and Graduate Programs and Research

Appendix


The “Lecture Track” Reconsidered
Professional identity and aspiration among non-tenure-path faculty

Tales from the Lecture Track: Kristin Wendland, Music

Tales from the Lecture Track: Sheila Tefft, Journalism


Virtue and the Stewardship
of Academic Freedom

Reflections on ambition, conversation, and community

Endnotes

 

 

Over the summer, a framed copy of the cover of the U.S. News
& World Report
issue ranking “America’s Best Graduate Schools” for 2006 appeared on the wall of the psychology department’s main office. Underneath, an engraved plaque congratulated the Clinical Psychology Program for making the magazine’s top twenty.

Did anyone notice?

“I think academics pay a tremendous amount of attention out of the corner of their eyes,” says Drew Westen, Professor of Psychology and director of the lauded program. “They often
pretend that it doesn’t matter, especially if they’re not in the top twenty (myself included). And as soon as they hit the top twenty, they start watching the charts. It makes a huge difference in the way a program is perceived nationally.”

Emory, along with most other colleges and universities in this country, has long nurtured a love-hate relationship with U.S. News
& World Report
and other entities that publish similar rankings.
We celebrated in 1997 when Emory College leapt from nineteenth to ninth in the U.S. News rankings of “America’s Best Colleges,” mostly due to a tweak in the magazine’s methodology. We fretted the following year when it sank to sixteenth place. Then-university president Bill Chace wrote a blistering open letter to U.S. News’s editor about its ever-changing methods. Since then, Emory College has hovered near the bottom of the top twenty.

Emory administrators choose their words carefully when discussing the rankings. The press release announcing Emory College’s position at twentieth in the 2006 edition of U.S. News supplied this comment from Provost Earl Lewis: “While we appreciate this external validation, we have a strategic plan that prepares us for an even brighter future, one that requires that we invest in the individuals who comprise our community, and in the ideas they will develop to make a difference in the world.”

As Westen suggests, institutions and faculty crave the national recognition that high rankings bring. Yet they also pose a dilemma: Can Emory in fact make a difference in the world and make its mark on it? Do we need U.S. News and the National Research Council (NRC) to bestow prestige? What are the real stakes in the rankings game?

Bryan Noe, Interim Dean of the Graduate School of Arts and Sciences, whose office is responsible for submitting data for the National Research Council’s ranking of U.S. research-doctorate programs, has a pragmatic answer: “Part of it is marketing. To have outstanding graduate programs you want to attract the best-qualified students possible. And part of the decision-making process for the prospective student is looking at rankings. We need to be in the mix.”

Daniel Teodorescu, director of the Office of Institutional Research,
which compiles the data for the U.S. News “America’s Best Colleges” issue, cites an annual survey conducted here as part of a national survey of college freshmen. Last year 46 percent of Emory freshmen considered rankings in national magazines a “very important” factor in their decision to attend this university. Rankings appear to be more important than the will of relatives or the advice of high school teachers and counselors. “The average for our peers, the highly selective schools, is 37 percent,” Teodorescu says. “This tells us that despite all our unhappiness with the way these rankings are, we should pay attention.”

But we can get carried away, says Laurie Patton, professor and chair of the religion department. The Graduate Division of Religion, which includes faculty from religion, the theology school, and other departments such as philosophy and Middle Eastern and South Asian Studies, was fifth in the nation according to the last NRC ranking in 1995. “In a sense Emory is still adolescent, and adolescents do tend to care about image. Comparison can be a form of being connected, and that’s great, but it can also lead to lots of anxiety without much else to show for it. And then we become reactive rather than proactive. We don’t get on with our intellectual work.”

To build a reputation

One of the most contentious factors in both the U.S. News and the last NRC rankings is the peer assessment score—derived from surveys of randomly selected university presidents, provosts, deans, department chairs, and other academics asking them to rate subjectively other institutions on a scale. In the U.S. News rankings, peer assessment, or the reputational survey, accounts for 25 percent of the overall score. The 1995 NRC rankings, intended for publication every decade, relied heavily on its reputational survey to measure the quality of graduate programs (see sidebar).

Dean of Admissions Daniel Walls always fills out the U.S. News survey, but not without some soul-searching. “The survey really
reinforces that the schools you tend to have heard of the most, that have been around the longest, are the most wealthy or the most selective,” he says. “There are some institutions doing some really good things that aren’t in the Ivy League but deserve recognition.
“I always consider just not responding. But the reality is that others are going to, and I would like to get my two cents’ worth in.”

Professor of Psychology Darryl Neill feels less ambiguity. “When U.S. News called me trying to bug me into filling out the survey, I told them to please go to hell,” he says. “I’m a behavioral neuroscientist. I don’t know ding about many psychology departments, and psychology is an extremely broad field. Do I know how good a department is in social psychology? No way.”

Apparently, a growing number of survey recipients share Neill’s inclination. According to the institutional research office of Johns Hopkins University, the response rate to the U.S. News academic reputation survey has declined steadily over the past seven years, from 68 percent in 2000 to 57 percent for the 2006 issue.

The reputational measure also subtly steers institutions toward a certain kind of growth. “The larger the department, the more likely they are to have stars,” explains biology department chair George Jones, adding that schools rely on academic stars to draw attention and recognition. “Larger programs are almost always going to get higher scores than smaller ones who, for what they do, might be just as good. That is a particular way in which Emory has been disadvantaged. We don’t have the critical mass.”

Noe points out that the same tendency applies to other kinds of rankings. “The ones I’m more familiar with are the total grant dollar tallies that characterize the sciences,” he says. “Even though in the 1990s Emory was among the fastest-growing schools in the country in terms of increases in federal research dollars, to make that quantum leap from twentieth into the top ten would require more researchers. On a per-capita basis, Emory faculty are generating just as much grant income as Hopkins; we’re just smaller. How do we get into the top ten? More buildings, more people.”

“Mission Creep”

As the strategic planning process nears its end and a major seven-year capital campaign begins, is Emory poised for such a herculean investment?

“We are what we are, we’re good at what we do, and we’re happy with that,” Noe replies. “That doesn’t necessarily help us in the rankings game, and to some extent we’ve got to play it. But if we can make it clear why we are special, we can overcome that, perhaps.”

There’s the catch: in order to join the ranks of the nation’s preeminent institutions, Emory must be like them. But in order to get noticed among the nation’s preeminent institutions, Emory must be different from them.

Some call it “mission creep”—the tendency for all research universities to become increasingly alike in size and profile, all in the name of the rankings competition. And the gains are relative: the standards are continually raised for what makes a top-ranked institution.

“I once heard [former Emory provost] Billy Frye say that it’s all well and good to want to be in the top ten, but who do we bump out so that we can move in?” Jones says. “I think the short answer is that anything we do, the other schools are going to do, too. The top ten is likely to be the same institutions, no matter what Emory does. So it seems to me our resources would be best used by making us the best we can be.”

Even so, administrators are investing much to assure that Emory is not only excellent but also perceived as excellent. One of the top priorities for the Office of Development and University Relations has been to create the new position of Vice President for Marketing to advance the “Emory brand” nationally (in late September, Home Depot marketing executive Ellen Dracos Lemming accepted the position).

Another factor that some say puts Emory at a disadvantage is southern stereotypes, compounded by the absence of NCAA Division I athletics to attract the national spotlight. “There are some people who are voting in the U.S. News survey, both college presidents and admission deans, who certainly have negative stereotypes of the deep South,” says Dan Walls. “They are not accurate stereotypes, but they’re there nevertheless. And the fact that every kid has a Duke cap on or is following Duke basketball—a national phenomenon—it’s hard to deny that the name recognition plays some role.”

According to Teodorescu, however, Emory can offset such issues indirectly by attending to other metrics where the university lags. “You can improve on factors like alumni giving and graduation rates, which in turn could raise your overall rank in the immediate future, which in turn, in a couple of years, will affect your peer assessment,” he explains. He points to Emory’s alumni giving percentage, another U.S. News metric, which was 19 percent in the latest ranking (a slip from 25 percent the year before), placing it seventy-third among national universities.

Are you still reading?

Ultimately, faculty may wonder whether the rankings angst bears at all on their work and lives. Laurie Patton says that when she is recruiting faculty, “I do talk about it.”

Drew Westen adds that while he and his colleagues don’t spend time worrying or congratulating themselves on the U.S. News spot, “certainly, when we’re considering who the next hire is, we’ve got our eye on who is going to move us up rather than down in those rankings. It’s basic primate psychology, if you ask me.”

But to Darryl Neill, the game goes to the soul of the institution. “Emory has gone from a university where teaching was not only highly valued but was basically ‘it,’ to having the problem of how to lure and keep high-powered, high-profile researchers—the people who get you these rankings—while still having a good undergraduate college. We have no real policy on that. We’re just feeling our way.”
—A.O.A.