Lilienfeld spots big problems in Rorschach tests

By Rachel Robertson

Imagine losing custody of a child based largely on the results of the Rorschach inkblot test, a projective measure whose validity has been questioned by researchers since the 1950s. Imagine also that other diagnostic measures disputed the Rorschach results, but they were ignored by the clinician in favor of the inkblots.

Scott Lilienfeld, associate professor of psychology, is not at all surprised to get a phone call from someone describing such a case. Although he cannot vouch for the accuracy of the details of this particular phone call, he said, “That kind of story is not all that atypical, so I wouldn’t find it terribly implausible.”

Stories like this illustrate the importance of examining the validity of such diagnostic instruments. Lilienfeld did just that, along with two other collaborators, James Wood of the University of Texas at El Paso and Howard Garb of the Pittsburgh Veterans Administration Health Care System and University of Pittsburgh. Their recent article published by the American Psychological Society reviewed three projective instruments commonly used by clinicians: the Rorschach inkblot test, the Thematic Apperception Test (TAT), and the Draw-a-Person Test. A shortened version of their article appeared in the May issue of Scientific American.

Many people are familiar with the Rorschach, in which subjects are asked to describe what they see in 10 different inkblots. This kind of test is referred to as a “projective” technique, the idea being that the subject is projecting personality traits and life experiences onto something ambiguous, thereby giving the clinician a window into the workings of his or her mind.

The TAT presents ambiguous scenes—such as a distraught woman holding an open door—and asks people to generate a story for each scene. The Draw-a-Person Test is fairly self-explanatory; the examinee draws a picture of a person.

Clinicians use responses to these tests to detect mental disturbance, and they often look for specific indicators. Focusing on random details of a Rorschach inkblot may indicate obsessiveness, for example. In the TAT, a clinician might look for recurring themes. Also, drawing large eyes in the Draw-a-Person test could be construed as expressing paranoia.

Lilienfeld and his colleagues have concerns about the accuracy of these measures to make mental diagnoses. There are generally two problems with these tests: scoring reliability and validity. A measure is reliable when different people who tabulate the responses arrive at similar conclusions; all three psychological measures showed problematic or even poor reliability, according to the researchers.

In order for a measure to be considered valid, it must either correspond well with other measures or predict future behavior. None of the three measures showed consistent validity, Lilien-feld found. However, a few of the Rorschach variables appeared to be more valid, for example, in detecting schizophrenia.

Some proponents of the Rorschach claim they do not use it in isolation, but Lilienfeld is of the opinion that including it rarely adds much of value to a diagnosis.

“In fact,” Lilienfeld said, “there are some data to suggest that when you give clinicians the Rorschach when they have other information available, it may actually decrease the validity of their prediction; it may make it worse rather than better. So, more is not always better in the area of psychological assessment. It’s a common misconception.”

As in the reported custody case, it also happens that clinicians commonly ignore other test information in favor of the Rorschach. In fact, some psychological literature recommends accepting Rorschach interpretations over other methods. Lilienfeld said some clinicians believe “that the Rorschach is plumbing some deep depths of personality that the other measures can’t access.”

Despite his findings, Lilienfeld is not totally opposed to using projective techniques, but he feels more research is necessary.

“These kinds of projective techniques can work in principle,” he said. “There is nothing wrong with the idea of having a projective technique; the rationale behind them is not an implausible one, and it may be a very meaningful one. So we really want to encourage people to build and construct better projective techniques.”