|
By Susan H. Frost and Daniel Teodorescu January 1999 |
|
Many would argue that the ways research university faculty allocate their time to teaching and to research is a topic of continuing concern. Partially in response to this concern, scholars have developed various profiles of faculty work. Some examine, for example, faculty workload, motivation, and expectations (Blackburn and Lawrence, 1995) or the relative importance of teaching, research, and service in determining the compensation faculty receive (Fairweather, 1993). For the most part, scholars use quantitative methods to answer such questions as: How do faculty use their time? What incentives influence the choices they make? As the answers to such questions become clearer, it may become easier to explain some profiles of faculty work. While scholars attempt to understand these current profiles, some universities are working to change them. For these institutions, it is not enough to know about current realities. The important questions concern the nature of the university we intend to become and how we can use our resources to achieve this vision. For example, if the balance of teaching and research is wrong, what should it be? If we know what it should be, how can we use our resources to achieve it? This study reports one research university's effort to explore these questions. The study had four purposes: first, to gather the opinions of faculty about the university's plan to achieve greater excellence in teaching at the same time the university is achieving greater excellence as a top research institution; second, to extract meaning from the body of faculty opinions and organize it to inform decision making; third, to structure the extracted meaning into a general framework that suggests paths to teaching excellence at this university; and fourth, to demonstrate how qualitative analysis can be used to investigate the broad, deep, and often unorganized data gathered during focus group discussions. We used qualitative data analysis software to produce the analysis. Typically, focus group discussions are rich in both context and the array of opinions explored during the exchange (Albrecht et al., 1993). Quantitative methods are not designed to capture these elements of data. In our analysis, a sense of the array of opinions was a central outcome of the investigation.
Review of Literature Because this study concerns both the outcomes and the research methods we used, several areas of past research are relevant. Among these are faculty attitudes toward teaching improvement and institutional change, and qualitative studies of faculty views. For at least ten years faculty and academic leaders at research universities have sought ways to improve teaching and reward it more appropriately. For example, since 1989 Syracuse University has sponsored several projects that opened the debate over teaching and research balances at other research institutions. One study examines how Syracuse professors, department heads, and deans viewed the relative importance of teaching and research. Although each group claims to favor a balance, each sees the others as favoring research (Mooney, 1992). Following this work, other universities including Emory, Northwestern, University of California at Berkeley, and University of Michigan joined the study. As a result, some faculties have worked to emphasize teaching on their campuses. However, few have based their actions on the systematic investigation of faculty views. Consequently, most strategies seem linked to the notion that the motivation of individuals can lead to improvement across an institution. For example, now some institutions use merit raises, promotions, public recognition, or prizes to reward excellence in teaching. Others establish teaching centers to award grants for innovative instruction or employ experts who give workshops to stimulate interest. These strategies, more focused on individual behavior or response than on some shared ideas about requirements for excellence, rarely consider what faculty believe about effective teaching support. Blackburn and Lawrence provide an exception. In a 1995 study of faculty perceptions of the teaching environment, they gathered faculty views about the support for teaching, the degrees to which their colleagues seem committed to teaching, and the portion of work effort administrators seem to believe should be made on instructional matters. In more recent work, Olsen and Crawford (1998) investigate discrepancies between expectations and realities of work attitudes and tenure acquisition for a cohort of junior faculty at a large public research university. Finally, in a study of six regional state universities in Tennessee, Tang and Chamberlain (1997) examine the differences between administrators and faculty regarding attitudes toward research and teaching. As they report, administrators tend to believe that research and teaching are mutually supportive; both are the mission of their universities. Faculty are less inclined to agree, however. Even at regional institutions we usually consider devoted to the teaching mission, some faculty believe that reward goes to research, not teaching. According to these faculty, research interferes with teaching. They perceive that faculty should do one or the other, but not both. These studies suggest a common theme as the essential component of change in the teaching-research balance: the active involvement of faculty. Understanding faculty goals and views may be keys to effective strategies to improve teaching (Eleser and Chauvin, 1998). However, neither resources and rewards for teaching nor the participation of faculty in workshops or seminars seem sufficient to bring systematic and permanent change in the status of teaching at a research university. Csikszentmihalyi (1997) addresses these issues in an analysis of intrinsic motivation and effective teaching. As he notes, rewards and feedback systems attempt to reconcile the behavior of teachers with some established criterion of effectiveness. However, university "teachers" are more than that. As professors, they are intrinsically motivated to instill in students a willingness to learn for its own sake. Administratively driven requirements about the frequency of talking with students, for example, may destroy this motivation and lead to outcomes opposite of those the requirements were intended to achieve. Such programs may fail to recognize that a university is a social system defined in part by the degree to which it is collegial, collaborative, and supportive. Perhaps some programs to improve teaching have failed to acknowledge the power of these forces in the intellectual environment. In his 1992 article, Bess seeks to clarify both the meaning and function of collegiality. By illuminating some complexities universities seek to shape, Bess suggests where new ideas and support for them might be found. As Bess indicates, although collegiality per se does not bring about institutional quality, it appears to provide a means of inducing members to cooperate and work in more integrative ways. Collegiality also seems to provide incentives for achieving institutionally sanctioned goals. Bess (1992) explores in some detail three distinct domains of meaning in which the term collegiality is embedded: academic culture, organizational structure, and social behavior. He argues that "disaggregation of the assumed singularity of the meaning of collegiality across the domains is necessary to appreciate its complexity and the uses to which people claim to put it" (p. 2). By claiming that faculty expectations -- about the improvement of teaching for example -- can be examined at three levels of social systems analysis, Bess offers one way to think deeply about helpful change. Frost (1998) draws from Bess and other scholars to consider the organizational learning that occurred in the years before this study began. In her view, three lessons seem clear: · "Processes that encourage the folding together of many points of view may be more useful than processes that allow ideas to be considered in more linear or isolated ways. Success seems to flow not from proceeding from one clearly defined concept or decision to the next, but from continual, though not always straightforward, progress toward outcomes many help to form. · Notions of balance seemed to help planners move from the necessity to choose among alternatives to a more imaginative goal of linking or synthesizing them. Strength seems to come not from a capacity to tolerate jarring change, but from the ability to connect past reality to a vision of the future. Modification, not deep or sudden change, may be the aspiration we should claim. · It has proved more productive to place conflicting views on the table than to shield them from discussion. However, we discuss conflicting views more productively, it seems, within existing networks of collegiality than across them" (p. 229). These lessons proved useful as the study developed. They provide evidence about the nature of change the university under investigation has accommodated, and thus serve a type of pilot for current efforts. Typically, qualitative research involves the collection and analysis of an unstructured exchange or set of exchanges for the purpose of detecting themes, categories, hypotheses, theories, or mere descriptions of social life (Kelle, 1997). Since the methodology was developed, most of the effort has been devoted to coding or indexing, an activity that requires extensive reading, rereading, interpreting, and comparing passages of text. In the early days of qualitative analysis, researchers coded text by hand, physically cutting apart passages of text and reassembling them in new ways. However, in the early 1980s, a tool was developed to ease this work. This tool, qualitative data analysis software, has eased researchers' coding tasks. It has also allowed researchers to investigate avenues of more complex inquiry than was possible during the cut-and-paste coding era of qualitative analysis. To date, most qualitative analysis has been conducted in sociology and ethnography, rather than education. When such studies occur in higher education, they usually explore trends in methodologies, themes, status of the profession, and paradigms employed by researchers in the field. For example, Volkwein et al. (1988) analyzes journal articles published in the journal Research in Higher Education over a 15-year period and Dimitroff and Davis (1996) explore the topics of journal articles in the field of undergraduate medical education. Researchers have also used content analysis to investigate collections of conference proposals (Faber, 1996), conference papers (Lincoln, 1984), doctoral dissertations (Kantorski, 1995), and even position descriptions in the Chronicle of Higher Education (Boyles, 1988). Other qualitative studies focus on students or alumni, often using data that are less typically thought of as documents: alumni entries to reunion class books (Zweigenhaft, 1992), reports of racially motivated harassment and violence experienced by minority students on predominantly white college campuses (Farrell and Jones, 1988), or student-instructor communication via computer conferencing (Mowrer, 1996). Qualitative analyses to inform policy-making in higher education are more scarce, due in part to the lower status most administrators attach to them compared to quantitative analyses. However, Frost (1997) use qualitative methods to investigate state policy and admission practices in the University of North Carolina system. In addition to analysis of interview data, they studied the content of official documents and press articles that concerned the issues under investigation. Goldsmith (1995) reports on an ethnographic study of the creation of a new public university, California State University at Monterey Bay, to highlight challenges the founders faced as they built a collective identity and to analyze the transformation of values into organizational realities. Grover et al. (1985) examines activities and trends at 53 offices of research in medical education by analyzing information on goals, organizational relationships, and funding efforts devoted to various educationally related activities and current research endeavors. Barrette et al. (1995) analyzes the perceptions of non-instructional staff at the University of Michigan. Bognanno, Estenson, and Suntrup (1978) study collective bargaining agreements by investigating the union management contracts in higher education negotiated between 1967 and 1975; more recently, Rhoades (1996) examines such agreements for 183 higher education institutions or systems. In recent years, many have written about using computer software to analyze qualitative data, and some critics have focused on the influence of the tool on the analytic process. Less common in the literature, however, are descriptions of precisely how researchers use a particular software package in an actual research project. Here we offer such an account.
Research Questions Specifically, we addressed the following research questions: (1) What are the themes of the discussions, and what were the range and breadth of each? Across the discussions, how frequently was each theme discussed? In each discussion, how many themes were addressed? (2) What was the nature of each theme? Across the themes, what problems, criticisms, agreements, and recommendations for implementation did the participants suggest? (3) What was learned that can inform the university's work to strengthen teaching excellence? Is there a framework to help guide useful change? In addition to answering these questions, we demonstrate how we used qualitative data analysis software to investigate data from focus group discussions. Based on our experience, we suggest some ways leaders and planners might use the software in similar studies to guide change. Before addressing these questions, it is useful to place the study in the context of the university and previous work to strengthen teaching excellence.
Background Over the last five years, Emory university has systematically conducted a series of discussions among faculty about the shape of its future and the nature of its community. Each year about ten percent of the faculty are chosen at random from employment rolls to be invited to participate in one of about 15 discussions. From year to year, the choice of the discussion topics has followed a natural progression, with each year's topics becoming increasingly focused on specific strategies to achieve excellence. In 1993, the scope of the first discussion series was relatively broad. The conversations were designed to gauge faculty opinion about the strengths and weaknesses of the university. Based on a manual analysis of a summary of each discussion, five areas emerged as the topics faculty raised most frequently: the balance between teaching and research, building a stronger community, encouraging interdisciplinary scholarship, keeping pace with the infrastructure needs, and assessing the university's external relationships (Office of the Provost, Emory University, 1994). These topics constituted the main themes for the 1994 discussion series; faculty who participated in that discussion series chose one of the five topics when they confirmed attendance at a focus group meeting. In 1994, approximately half of all faculty who participated in the focus group discussion series chose to discuss the balance between teaching and research. Accordingly, both the provost and the president of the university recognized the importance of this issue and created a faculty commission on teaching to address Emory's particular challenges and needs. They charged the commission to examine the most critical issues related to the support and improvement of teaching and make specific recommendations for improvement. To inform their work in 1996, commission members conducted the annual series of faculty focus group discussions, all of which concerned various aspects of teaching quality and improvement. After 18 months of work, the commission produced a report, Teaching at Emory (Commission on Teaching, Emory University, 1997). Based in part on the results of the 1996 discussion series, the report put forward recommendations for future action . In 1997, the discussion series concerned faculty reaction to the commission's recommendations. Using a sample stratified by discipline and rank, 485 of the 1772 faculty members were invited to participate in the series. Of these, 254 attended and took part. The summaries of each discussion, written by commission members serving as faculty facilitators, provide the data for this study. Before 1997, researchers analyzed summaries of the discussions manually, resulting in a less systematic and thorough approach. In 1997, we used qualitative data analysis software for the first time. It is important to note that in this project, all focus group discussions were structured around a set of planned questions. In our experience, the quality of the outcomes is related to this requirement. With established themes around which researchers can build the coding scheme, analysis can prove most useful. On the other hand, unstructured discussions are likely to yield few codes that are common across the groups. This characteristic could limit the researchers' capacity to make inferences from the data and thus restrict the usefulness of the project.
Method The data in this study are 24 transcript summaries of faculty focus group discussions of the report produced by the Commission on Teaching. The faculty facilitator of each discussion wrote each summary. Figure 1 shows the steps we used to analyze the data (See Figure1). Although qualitative studies in the area of university planning are relatively scarce, in this case it seemed the most reasonable and appropriate approach. We selected the qualitative data analysis software HyperRESEARCH for this study. HyperRESEARCH makes possible a thorough and systematic investigation of a volume of text within the time constraints that real world decision making imposes. HyperRESEARCH also allows researchers to change or refine code definitions easily. This software and other third generation qualitative data analysis tools allow for more complex retrieval strategies and hypothesis testing than manual coding can support. To code the data, first we read each summary several times to become familiar with the contents. Then, using a sample of four transcripts, we compared all texts, looking for similarities or patterns. We repeated the process for another sample and compared the two samples to produce preliminary themes. As the list of themes became complete, we discovered that most of them reflected major topics of the report upon which the discussions were based. Next, we refined the preliminary themes and created codes within each theme using a process similar to "open coding" described by Strauss and Corbin (1990). Specifically, we applied codes that were drawn from our common-sense knowledge of the topic, introduced by discussion participants, or appeared as main ideas in the commission's report. To manage the codes, we used the code-and-retrieve method (Richards and Richards, 1991). This approach produces a coding framework that can expand as researchers discover new themes. The method requires rsearchers to re-label data that is already processed when a more meaningful or comprehensive label is needed. Fortunately, HyperRESEARCH makes the expansion of themes relatively easy; as new codes are needed, old codes can be deleted, renamed, or collapsed. One can apply new codes to text already processed. The next step was to determine the reliability (or consistency of judgment of the coders) of the emerging codes. To verify the reliability of the codes and limit subjectivity, four researchers with different backgrounds and knowledge of the commission's report identified themes and coded the same transcripts independently. Then we compared the work of all researchers and found that although we used different words for some codes, the consistency of meaning was high. Thus, we inferred that the set of themes and codes was reliable. Next we applied the codes emerging from the two samples to the remaining transcripts. This stage also served as a test of code validation, confirming that the themes identified were not episodic or idiosyncratic occurrences. Initially there appeared to be eleven distinct themes: evaluation, making teaching a priority, faculty development, infrastructure, rewards for teaching, students, local implementation, teaching as a multifaceted activity, support for intellectual community, interdisciplinarity, and institutional mission and outcomes. Later we found that we needed several codes within each of the eleven themes to designate the ideas related to that theme. For example, the theme of teaching rewards contained comments about counting teaching in tenure and promotion decisions, offering salary increases based on teaching performance, and establishing teaching professorships. We used numbers 1 through 20 to designate codes within each theme. In addition to coding themes, we coded participant's comments by type. We began with four types: criticisms of the report; agreements with the report; problems not identified in the report; and recommendations for implementation. Later we added three new comment types to the matrix: quotes, bright ideas, and the name of the school of the university to which each comment pertained. We used the letters R, A, P, C, Q, B, and S to label recommendations, agreements, problems, criticisms, quotes, bright ideas, and the name of the school respectively. We organized the data as a matrix with codes as rows and types of comments as columns. To ease data manipulation and retrieval, we assigned to each cell in the matrix a unique alphanumeric label that allowed easy identification of both the theme of discussion and type of comments. For example, R21 Increase Staff Support indicates that the comment was a recommendation within the "infrastructure" theme. Later in the analytical process, this classification allowed us to group together all criticisms, for example. It also helped us structure the findings of the study. Throughout the coding process, we examined the transcripts for patterns and relationships among themes and types. Before finalizing the analysis, we completed three revisions. Had this process been manual, we would still be working on the project! However, HyperRESEARCH allowed us to revise and expand the coding scheme in minutes rather than hours or days.
An illustration of the coding process. One theme that occurred frequently in the faculty discussions was rewards for teaching; we use it here to illustrate in detail how a theme evolved. First, the theme occurred in almost every transcript in both sets of samples we coded initially, and thus it remained in the final set. We discovered it in both samples by investigating the similarities among the following codes: base tenure discussions on teaching performance, base promotion decisions on teaching performance, reward good teaching by establishing teaching professorships and chairs, and base pay increases on teaching performances. As one can see, the concept of reward for good teaching is the unifying theme across these recommendations. We developed each theme similarly. Each passed the reliability test because it appeared in the list of codes the three researchers developed independently. Next, we applied the rewards for teaching theme and its corresponding codes to the remaining transcripts. The theme occurred in 16 of the 24 transcripts. However, in some sentences we could not code phrases related to the underlying theme using the codes established in the initial samples. For example, "award free parking spots to good teachers" fits the theme rewards for teaching but does not fit an initial code. Therefore, we created a new code for the comment and returned to the sample transcripts to verify its occurrence there as well.
Findings After completing the coding steps, we were ready to address the first research question: What are the themes of the discussions, and what was the range and breadth of each? Across the discussions, how frequently was each theme discussed? In each discussion, how many themes were addressed? We used code grouping (using the "OR" Boolean operator) and frequency analysis (see Figure 2) to answer this question. Eleven themes seem to characterize the discussions. The most frequently discussed themes were evaluation, making teaching a priority, faculty development, infrastructure, and rewards for teaching. Notably, all 24 groups allocated a large part of the discussion to the issue of how to evaluate teaching (see Table 1). Often scholars criticize reliance on focus group discussion because of the possible "group thinking" effect that can occur in such settings. In this case, however, the breadth of the discussion demonstrates a lively interaction where problems, criticisms, agreements, and recommendations are not limited to only one or two themes. As Table 2 shows, all 24 groups engaged at least six major themes of the commission's report. Having identified the most frequently discussed themes and some patterns of discussion, we turned to a more detailed analysis to address research question two: What was the nature of each theme? Across the themes, what problems, criticisms, agreements, and recommendations for implementation did the participants suggest? Evaluation of Teaching. Discussed in 100 percent of the focus groups, faculty seemed to view the evaluation of teaching as either a "mission impossible" or an activity with questionable potential for teaching improvement. Many faculty expressed the view that effective professional development opportunities should be offered before a new evaluation system is adopted. The code evaluation difficult and/or not needed came up in ten of the 24 conversations. The reluctance many faculty expressed toward a more systematic evaluation of teaching might relate to a perception --commonly held among discussion group participants--that the intended purpose of such evaluation is to judge performance rather than to guide development or define support. For example, in six conversations faculty indicated that department chairs and deans might use the results of evaluation to make judgments about promotion, tenure, or salary increases rather than as a feedback to guide improvement. Faculty reluctance to embrace teaching evaluation also might relate to the perception that evaluation requires faculty to devote more time to assembling documents and less time to teaching (ten conversations). Faculty also expressed concerns that in most departments promotion and evaluation guidelines clarify neither the role of teaching nor its evaluation (eight conversations). Teaching portfolios received attention in 11 conversations. The main arguments against using such a tool cited the additional pressure it might create on faculty time. Peer evaluation ranked as one of the most frequently recommended tools (seven conversations); at the same time, participants criticized peer evaluation in almost the same number of instances. The ideas of interviewing exiting students and graduating seniors as well as conducting periodic surveys of alumni were well supported across the discussions.
Making Teaching a Priority. In almost a third of the discussions, faculty called on the university to increase its commitment to affirm the role of teaching. Some called for changes in existing incentive systems, improved physical infrastructure, or more effective faculty development programs. One of the most desired changes concerned greater commitment to protecting faculty time. As some participants suggested, this could be achieved either by increasing the size of the faculty (six conversations) or by decreasing course loads for each member of the faculty (four conversations). In general, faculty seemed to feel that past conversations about teaching were beneficial and should be continued (six conversations). However, in one third of the conversations, participants raised the possibility that the university advanced the perception that teaching is undervalued merely to stimulate conversation. According to these participants, at Emory teaching has always been excellent and therefore change is not needed. Some felt that the report that served the basis for discussions underemphasized the role research plays in achieving excellence or treated teaching and research as separate rather than related activities (eight conversations). Others objected to the report's description of teaching as a vocation (in seven conversations). The participants also identified obstacles to making teaching a priority, including conflicts between teaching and research (13 conversations) and constraints on faculty time (12 conversations).
Faculty Development. In more than half the discussions, faculty affirmed support for the concept of central resources to improve teaching. However, there appears to be support for such resources only if those resources are organized not from the top down, but at the local levels of the schools and departments. Some participants suggested the idea of a university center for teaching and offered recommendations regarding the roles that such a center might play (13 conversations). The most frequently requested assistance (seven conversations) was help for faculty to learn how to use the newly proposed evaluation methods, particularly in preparing teaching portfolios. Release time or sabbaticals for teaching improvement was the next most recommended action (mentioned in seven discussions). Some saw mentoring as an effective faculty development initiative (five conversations). Interestingly, the main reservation about its implementation had to do with the manner in which mentors would be selected: some protested that some senior faculty are not the most appropriate role models.
Infrastructure. Of the 24 conversations, almost half concerned Emory's buildings and classrooms. Some believed that existing facilities do not adequately support excellent teaching and the type of informal faculty-student interaction that is assumed to foster intellectual community. Specific criticisms concerned the perception that there is not enough classroom space and that the design and allocation of existing space are not well tailored to the needs of university programs. In addition to inadequate numbers and types of classrooms, a lack of basic teaching supplies and insufficient staff support (i.e., research assistants, teaching assistants, and secretarial support) also emerged as serious concerns.
Rewards for Teaching. As one facilitator noted, "The single most important factor in implementing the goals of Teaching at Emory [is] the development of adequate rewards." Faculty in 11 of 24 conversations noted that existing incentives to reward excellent teaching seem insufficient. Critics most frequently referred to the fact that decisions about tenure, promotion, and salary increases do not account for teaching excellence adequately. Specific recommendations included yearly raises, special parking spaces, teaching professorships or endowed chairs, and promotion and tenure, in some cases, based mainly on excellence in teaching. Counting teaching in tenure and promotion decisions and salary increases as well as establishing teaching professorships were the most frequently recommended incentives (8 times recommended).
Students. Two major issues of faculty concern were entering students who are unprepared for university work and grade inflation. Participants proposed the following strategies: expect more of students (six conversations); establish university-wide grade standards (two conversations); and emphasize learning over teaching (three conversations). Almost universally, participants noted that the report places the burden of change on faculty. Some participants noted that the improvement of student learning should be as important as the improvement of teaching, and that students should take some responsibility for their success.
Local Implementation. In almost half the conversations, faculty recommended decentralization of both the dialogue on teaching and the implementation of the report's recommendations. They expressed hope that change would take place not only at the university level, but also in the schools, departments, and programs. In almost one third of the conversations, faculty expressed concern that many initiatives to improve teaching recommended central implementation without specific attention to individual programs.
Teaching as a Multifaceted Activity. In ten conversations, faculty wished for more complete coverage of the issue of teaching as a multifaceted activity. Some asked for more thorough exploration of the implications of raising the priority of teaching in the schools and programs.
Support for Intellectual Community. In nine conversations, faculty identified several problems concerning the report's call to strengthen the intellectual community. Concerns ranged from the difficulty of measuring the culture to low participation in some campus events to the perception that parts of the undergraduate experience need more intense academic rigor. Recommendations included building informal places for faculty and students to interact, and studying other universities with strong reputations for lively intellectual communities.
Interdisciplinarity. In five conversations, faculty expressed concern with strategies to foster interdisciplinary teaching. Considering the diversity of the campus, some participants stressed the difficulty of reaching across different schools, the lack of sufficient incentives for collaboration, and the fear that emphasis on interdisciplinarity might deprive faculty of their more organic relationships.
Institutional Mission and Outcomes. In six conversations, faculty suggested that departments should develop consensus about their own teaching goals and communicate their views clearly to various public constituencies. Junior faculty in particular called for tenure guidelines that are clear about what is expected of them in the area of teaching. They also suggested that defining and disseminating to prospective students and to the university community the institution's mission and educational goals is essential to the implementation of the report.
Research question three concerns some ways the university might guide the direction of change: What was learned that can inform the university's work to strengthen teaching excellence? Is there a framework to help guide useful change? To organize such a framework, we used Bess's disaggregation of collegiality to cluster the themes of the study as they relate to behavioral change, cultural change, and structural change. Then we considered the clusters in light of earlier work at the university suggesting that change of the type we are discussing takes place in continual and interrelated ways; notions of balance allow one to synthesize or link useful alternatives rather than choosing among them; and conflict can become a useful tool, especially when explored within existing collegial networks rather than across them (Frost, 1997). See Figure 3 for a diagram of the clusters and some relationships they suggest.
Teaching Improvement as Behavioral Change. As Bess (1992) observes, both the structures and values of an organization influence the behavior of the individuals within it, and this in turn reflects and reinforces the influences of the culture. In this study, we are concerned with two types of behavior: organizational and individual. Although there is inevitable overlap, the data we coded supporting faculty development activities and providing effective rewards for teaching seem to concern organizational behavior. The data we coded teaching evaluation and participation in faculty development programs seem to concern individual behavior. As we noted earlier, many universities support faculty development workshops or seminars expecting that these offerings will contribute to the improvement of teaching by increasing the effectiveness of individual instructors. We found strong interest in such efforts, especially when faculty shape them and they are located at the local level of the school or department as well as at the university level. Perhaps one balancing point concerns the degree to which faculty development advances the mission of the university and the degree to which it meets the needs of individuals. The conversations also affirm the importance of rewarding excellent teaching. Across all 24 conversations, participants recommended both extrinsic rewards such as salary increases, promotions, and chaired professorships; and intrinsic rewards such as differentiated roles for faculty. Some participants suggested, for example, that departments name certain faculty primarily to teach and others to conduct research. Other participants worried that specifically assigning responsibility for teaching might diminish its status in a community already characterized by the primacy of research. To create a "teaching track" might relegate teaching to a lower status officially and exacerbate the problem instead of easing it. While the intrinsic rewards of teaching appear to motivate faculty, sustaining teaching excellence will require both intrinsic rewards and tangible incentives. Closely linked to recommendations about rewards are recommendations about evaluation. Although many participants favored a broadly based system of evaluation, some questioned the degree to which valid evaluation is possible. The commission recommended using two distinct forms, the teaching portfolio and the teaching dossier. A teaching dossier may consist of statements of self-evaluation, comments from students, written evaluations by one's peers, syllabi, course-development proposals, and a written philosophy of teaching. A teaching portfolio contains those elements from the dossier that candidates for promotion or tenure submit for evaluation by peers and administrators. Typically the teaching dossier is used in a formative process to identify teaching strengths and weaknesses for one's continued improvement in teaching; the teaching portfolio should provide a means of evaluation for hiring, tenure, promotion, and remuneration. The data suggest that the process of developing a more broadly based evaluation system should include opportunities for faculty to debate both the contents of a new system and the ways they are applied. Before portfolios are linked to promotion and tenure, for example, data sources and requirements should be discussed, debated, and made known in the community. Requirements should be applied evenly across schools and departments. Like the evaluation system, faculty development programs should have as goals helping faculty protect time and energy for research and teaching; and grow in ways that enhance intellectual excitement, accomplishment, and esteem (Menges, 1997). Such programs should evolve from existing aspects of faculty culture and use existing structures rather than duplicate them.
Teaching Improvement as a Cultural Change. To understand faculty views of the research-teaching balance and translate these views into useful change, one must consider the influence of culture. As individuals, faculty derive both meaning and motivation for teaching from the institutional cultures to which they belong. In the research university, permanently enhancing the value of teaching requires a cultural change. Faculty who value research more highly must come to place equivalent value on teaching. Teaching at Emory (Commission on Teaching, Emory University, 1997) describes this need: "We want to get beyond the notion that excellence in research must preclude excellence in teaching and that universities cannot support, evaluate, and reward teaching and research in equivalent ways" (p. 5). Our findings reveal five themes related to cultural change. We coded these clear institutional mission and outcomes, making teaching a priority, support for intellectual community, recognizing teaching as a multifaceted activity, and quality of the student population. The mission of a university is the public statement of its values. If a university fails to be clear about the value it places on teaching, then it will be difficult to build a culture of enduring support. For example, participants in this study noted a strong link between the mission of a university and the values each school or department affirms. They cautioned that a university cannot strengthen teaching excellence if its schools and departments do not reward the practices that define this form of excellence. Participants also recognized the link between teaching excellence and the strength of the intellectual community. Intellectual community provides an environment of stimulation that allows teaching to take place not only in the classroom but also at discussions over coffee, musical performances, and evening lectures. This type of environment in turn demonstrates the values on which all teaching excellence must rest: trust, honesty, free inquiry, open debate, tolerance for difference, and respect for others' convictions. If a university lacks the aspiration or the habits that support such a community, some necessary aspects of teaching excellence are missing. In our view, participants who spoke about the multifaceted nature of teaching recognized the interaction of teaching excellence and intellectual community. For example, some wished to move from the conventional image of teaching--with the lecturer "holding forth" in front of the class--to a more complex and multidimensional dynamic. As they noted, teaching happens in all interactions between faculty and students -- in offices, hallways, laboratories, patient rooms, and clinic offices; at seminars and during internships; over ethernet and telephone lines, and via e-mail. Increasing both the intensity and frequency of these interactions can contribute to a vibrant and rich intellectual community. A cultural shift toward valuing teaching appropriately involves not only improvement of teaching but also improvement of student learning. As the conversations indicate, strategies usually include recruiting students with high intellectual expectations and a willingness to view learning as a goal worthy to be pursued for its own sake. However, commission members rejected the idea that teaching quality should depend on the students the university admits. Rather, the university and the faculty are responsible for teaching excellence, and students have an equivalent responsibility for learning. Participants upheld these views. They recommended fostering a culture that not only values research and teaching equivalently, but values teaching and learning equivalently as well. These connections suggest that both teaching excellence and intellectual community can benefit from efforts to connect the values and activities of all parts of the university and put forth a clear and consistent message about ways to extend the values and participate in campus activities. This finding could give guidance to leaders and planners who aim to intensify, integrate, and maintain teaching excellence and intellectual community.
Teaching Improvement as a Structural Change. Recalling that research suggests strong links between structure and behavior, it is important to note that organizational structure can have a subtle but pervasive effect on faculty motivation to teach (Bess,1992; Hall and Bazerman, 1997). In most of the conversations, participants asked for new structures to support teaching. We coded these structures interdisciplinary teaching, local implementation, and physical infrastructure. Although they recognized that the balance between school or departmental and central resources for teaching improvement is a delicate one, many participants expressed the desire to extend to the schools and departments the conversations about teaching the university initiated. While departments are the administrative centers of teaching, too rarely they are a place where faculty discuss teaching. However, research university faculty seem to view departmental workshops related to teaching in the discipline as more useful than more general, university-wide workshops (LaCelle-Peterson and Finkelstein, 1996). Perhaps excellent teaching requires both local and central support. If so, that support should be integrative and complementary, not duplicative or competing. Although historically many universities have supported parallel structures, most can no longer afford them. Change should reduce redundancy and promote connected pieces that are more than the sum of parts. One helpful force might concern guidelines that regulate collaborative teaching and interdisciplinary course offerings. Although both seem to enhance learning and intellectual ferment, participants named several structural impediments to these activities. For example, at Emory and other private universities, most incentives reside in the hiring department, where disciplinary standards are measures of productivity. To persist with collaborative teaching with members of other departments or schools, faculty must be motivated intrinsically or eligible for resources and compensation that are assigned specifically to interdisciplinary initiatives. Few interdisciplinary courses seem to evolve and flourish in the face of these customs. If a university intends to increase interdisciplinarity, then departmental customs and rules must change. Perhaps notions of balance could enter the debate, allowing change to be understood as a natural evolutionary process, rather than a abrupt departure from customary practice and norms. We used Bess's components of collegiality to depict some of the interacting forces we have described (see figure 4). Perhaps leaders and planners can move existing strengths at the center of the diagram out along vectors of change.
Implications Recalling the real world activity this study was designed in part to inform, it is appropriate to ask: Of the directions of change the participants recommended, which have the capacity to increase teaching excellence at this university? How might the changes be achieved? Of course, the answers will vary from one institution to another, based on the prevalent culture and the levels of integration among schools and departments, resources, governance, leadership styles, and many other characteristics. However, as the commission noted, "the greatest impediments to valuing teaching appropriately lie not with the intentions of individual faculty but with institutional structures and cultural forms that frustrate, prohibit, render invisible, or even penalize teaching excellence and effectiveness" (p. 5). Bess (1992) seems to agree. He notes that "culture not only has a direct impact on behaviors; it generates structures that support values" (p. 7). Our findings suggest readiness at this university to: · move from existing strengths to new standards and levels of support for teaching excellence; · use aspects of collegiality to achieve these; and · develop new or stronger, shared or integrated structures to advance excellence in teaching. If these aims can be achieved, other advantages may follow. For example, both institutional and individual behavior may change to favor and advance teaching excellence more consistently and permanently. A stronger, more collaborative intellectual community may result. The first finding concerns what the next steps might be, and on some points the data are clear. For example, although some participants recognized that as a vocation, teaching depends on intrinsic rewards, others argued that tangible incentives are required to translate individual and random behaviors into more organic, systematic patterns and expectations. The participants also seem to recommend local rather than central structures to advance the behaviors they affirm. For example, although the teaching commission recommended that the university establish a center to support teaching excellence, participants argued for the resources to go to schools and departments. Perhaps departmental efforts should replace or supplement established practices at universities to hold centrally-sponsored teaching seminars and other programs. As far as cultural changes are concerned, although many individuals are strongly and deeply devoted to teaching, institutional norms appear to place higher value on research. That participants affirmed the commission's calls for more systematic evaluation indicates a desire to push the process of cultural change from a few individuals to the larger community. However, this change cannot make a real difference at one university in isolation. The foundation of evaluation is the standards against which individuals are judged, and the standards of the disciplinary guilds are at least as powerful as the standards of the university. To become persuasive and permanent, change must take hold at both levels. In addition to advancing desirable norms and customs at home, both the university and faculty who are leaders in the disciplines might direct national conversations as well. Findings two and three concern how such change might take place. As Figure 4 shows, at this university past success in emphasizing teaching seems to proceed from structural change to behavioral or cultural changes, and not in a linear fashion, but by integrating or enlarging existing components of the community (Frost, 1998). If the capacity to initiate structural support resides at the center of the university, and teaching improvement is best based locally in schools and departments, for example, what strategies can move support into the field? Just as incentives can encourage individuals to strengthen teaching excellence, so could incentives encourage departments and schools to strengthen and integrate teaching support. Our findings suggest that faculty should determine the nature of this support. Perhaps they should help guide its deployment as well. Taken together, these findings suggest that the work to strengthen teaching is and will remain an ongoing challenge, and all efforts should be both integrated and clearly understood. At times, progress may depend on the community's willingness to debate the questions before it, using conflicting views to expand rather than diminish the conversation. Faculty depend on conflicting views to expand knowledge in the disciplines. Perhaps their skills can be applied to this institutional setting as well. Our last observation concerns the use of qualitative data analysis software for studies like this one. As a analytic tool investigating the broad, deep, and often unorganized data gathered during focus group discussions, HyperRESEARCH seemed effective. In our view, leaders and planners can use it effectively in similar research studies or in less rigorous ways to guide change or explore current situations. For example, the software could help mine data from exit interviews of faculty or students, or meetings of boards of trustees or other policy setting groups. Such work could reveal facets of an issue that quantitative methods are not designed to explore.
Limitations Limitations of this study include restrictions imposed by our data gathering techniques and the degree to which we used the full capacities of the software. First, the data took the form of summaries of discussions and not full transcripts of each conversation. Had transcripts been available, we might have captured important and potentially insightful clues about both the development of ideas and the nature of faculty interaction. However, because complete transcripts were not available, specific exchanges between individuals were not part of the data. For this reason, code retrieval fails to capture some important events typically described in the market research literature on focus groups. For example, according to Gordon and Langmaid (1988), there is a sequence to a focus group discussion that can help explain the different kinds of interaction at the beginning (forming and storming), the middle (performing), and the end (mourning) of the discussion. Furthermore, participants sometimes contradict themselves in the course of a discussion or change their views and opinions in light of the opinions of others. Our methods did not allow us to document this interaction. Using full transcripts rather than summaries would have allowed us to investigate, for example, the extent to which faculty cite past experiences at other institutions in their recommendations for teaching improvement. Analysis of the tone of voice and degree of emotional engagement to contested recommendations in the commission's report could be equally revealing. As Catterall and Maclaran (1997) argue, by attempting to code the dynamics of a conversation, further analysis of the interaction could reveal: the shared language on the topic, what was taken for granted, and what clarification were asked; the beliefs and myths about the topic that are shared, taken for granted, and are challenged; the arguments participants use when their views are challenged; the sources of information one uses to justify views and experiences and how others respond to these; the arguments, sources, and types of information that stimulate changes of opinion or reinterpretation of experiences; and the tone of voice, body language, and degree of emotional engagement involved when participants talk to each other about the topic. Had we captured each complete exchange, we could have explored those dynamics in detail. It is important to note, however, that most software packages (including HyperRESEARCH) do not permit researchers to code both the content of the discussion and the nature of the interaction in focus groups and simultaneously. To identify the events described above, one would have to work with the complete transcript as an off-screen document. Instead of analyzing summaries from facilitators, one would have to use word-by-word transcripts of individual comments along with complete descriptions of the interaction. The second limitation concerns the fact that the analysis does not fully exploit the various possibilities for theory-building or hypothesis-testing that HyperRESEARCH and similar tools make possible. However, our purpose was to describe, not to explain. Explanatory attempts should employ more complex retrieval strategies or techniques that help locate text segments according to document-specific variables such as the department of employment, gender, or tenure status of a participant. Known as selective retrieval, the technique would allow one to compare systematically, for example, the attitudes toward evaluation of tenured versus non-tenured faculty or faculty views about teaching from different schools of the university. Another useful but complex retrieval technique uses information on whether text segments coded with certain codes co-occur in a given document; the goal is to "test" hypotheses which are derived from the emerging theory (Hesse-Biber, Dupuis, and Kinder, 1995). For example, having documented faculty opinions on teaching evaluation with codes such as evaluation difficult or not needed, fear of summative evaluation, or evaluation is time consuming, we might examine the hypothesis that the rejection of teaching evaluation is always or frequently accompanied either by the fear that its results will be used punitively or by the concern that it will take time from teaching. The co-occurrence of codes in a given case may indicate the presence of critical evidence for or against a hypothesis.
Conclusions For some years, research universities have talked about improving teaching. This study is an attempt to place the issues faculty view as important into a framework a university can use to guide change. Bess's analysis of collegiality suggests that, conceptually, universities exist in three overlapping areas: culture, structure, and behavior. We use these concepts to suggest paths to teaching improvement at Emory university. Perhaps other universities will find them useful as well. This analysis also shows how researchers can use a series of structured discussions among one constituency of a community to inform change in that community, and how this process can be completed in time to allow the results to influence real world activity. Although this example involves faculty at one research university, the same approach could be used with other groups in a university or in other types of communities. Methodologically the investigation is in many ways exploratory. It suggests that the content analysis of focus group discussions can be at least as useful as traditional survey research for analyzing opinion about change. One hoped-for outcome of our discussions is a more reflective and connected community. The study suggests that the process has some capacity to build community, especially when discussion is unconfined and feedback is timely and leads to change. For this reason, the study opens avenues of future research on the nature and process of change.
References Albrecht, T.L., Johnson, G. M and Walther, J.B. (1993). Understanding communication processes in Focus Groups. In D.L. Morgan (Ed.), Successful focus groups: Advancing the state of the art. Newbury Park, CA: Sage. |
|
|
