Appendix A: Report of the Subcommittee on the Evaluation of Teaching

In drafting this report, we have reviewed the literature on teaching evaluation and other materials on current related practices, criteria, and resources at Emory and at other universities. We also have considered comments and suggestions from faculty gathered during a 1995-1996 luncheon series organized by the provost, from the executive committee of the Commission on Teaching, and from subcommittee members. In making our recommendations, we recognize and affirm the diverse characteristics of the various schools at Emory. Consequently, these recommendations do not include strict criteria for all units to follow. Rather, we are setting forth global criteria and recommend that each school have some flexibility in designing its approach. For example, we recommend that each tenure and tenure-track faculty member have a teaching portfolio, and we list types of teaching activities and materials that may be included. We leave it to the individual school, department, or program, however, to specify how the portfolio is maintained, what information is appropriate, what form the information should take, and what schedule for review should be adopted.

I. DIMENSIONS OF TEACHING

We use the term dimensions to emphasize the variety of approaches that exist within each item listed. For example, teaching in the classroom can be carried out through lecturing, viewing and discussing audio-visual presentations, or using interactive approaches such as round-table seminars, supervised laboratory work, or hands-on computer tutorials. Mentoring can be carried out for either students or faculty and can involve thesis or dissertation supervision, academic or career counseling, or clinical consultation. Some schools or departments might consider textbooks, teaching materials, and publications on teaching as work done toward improvement of teaching. Even though they do not involve face-to-face contact with students, such scholarship or research may have an important impact on how teaching is carried out by professionals in one's field. Similarly, there are settings outside the traditional classroom where face-to-face teaching can have an important impact-- for example, in short courses at the national or international level.

II. Elements of "Good Teaching"

Outcomes desired by the student. The first task of a good teacher is to transmit information and knowledge to students in a manner that communicates the essential facts and the range of interpretation of those facts. Second, a good teacher ought to help students learn how to think and apply knowledge both to historical and contemporary problems, situations, or challenges. Third, the teacher ought to motivate students to learn more about the subject and-- what is even more significant-- should pique their curiosity and help them appreciate learning for its own sake. Fourth, students ought to be taught that a major part of education is learning to collaborate with other students and faculty. Fifth, students ought to learn how to be more responsible and to cooperate with the instructor and other students as part of the educational experience. And finally, a good teacher ought to instill in students a sense of the value of learning as important to their education and to their lives. Education should not end with the college or postgraduate degree; a student's curiosity should lead to further learning.

Outcomes expected of the teacher. A person is not born a good teacher, although there are people who bring to the profession certain qualities of inspiration, vigor, and enthusiasm that would make them good teachers. Whether the teacher is born with those qualities or tries to learn them, a good teacher needs to prepare thoroughly. We have identified the following eight indicators of good teaching. First, a teacher needs to be well organized; he or she needs to plan thoroughly for the course and choose the appropriate teaching methods. Second, the teacher should state clearly the goals of the course and should communicate the material to the students in a clear and effective way. Third, a good teacher should use the best teaching materials-- e.g., texts, case studies-- and should design assignments that evoke thoughtful analyses. Fourth, exams should be fair and appropriate; grading should be fair; and teachers ought to find ways to provide feedback to students regarding both correct and incorrect answers. Fifth, teachers ought to cultivate good rapport with students. Sixth, a good teacher will leave such a positive impression on students that they would recommend the course to other students. Seventh, students ought to master the material of the course. And finally, at the end of the course, students ought to have a more elevated view of the value of learning.

III. Goals of Teaching Evaluation and Focus of This Report

The first two goals are the foci of this report.1 As foundation for goals one and two, it is helpful to clarify two distinct purposes of evaluation. Traditional evaluation can be of two types: summative and formative. Summative evaluation is designed to provide a foundation for decisions about advancement (promotion, tenure, or salary enhancement) or, at times, about retention of a faculty member in association with a program or position. Formative evaluation is designed to bring about improvement in performance over time.2 It is important to note that the nature of goal one-- to improve the classroom teaching of individual faculty-- is formative. Goal two-- to assess the teaching performance of individual faculty with regard to appointments, promotion, tenure, and salary issues-- is by nature a gate-keeping activity that requires summative evaluation.

In the course of our work, the subcommittee has learned that, particularly when compared to evaluation of research performance, many Emory faculty view evaluation of teaching as a subjective process that has unclear or narrowly defined purposes or places too much value on one information source: student opinion. In this report, we espouse a more complete and objective assessment of teaching-- one that is collaboratively defined and purposeful. It may have both formative and summative aspects and-- in addition to student evaluations-- should include peer review, student exit interviews, and documentation of teaching activities in the four dimensions of teaching.

Peer review is a form of teaching evaluation that can provide valuable information to promote teaching improvement (a formative process). When carefully crafted and well understood by the evaluating unit and the teacher being evaluated, it also may be possible to use peer review to evaluate teaching performance for the purpose of tenure decisions, renewal of contract, and/or salary adjustments (a summative process). Although some respondents report success in using peer review to serve these dual purposes, others caution that peer review is weakened when the summative aspect is included. Those who use peer review seem to agree that its integrity depends in part upon preliminary discussion about the purpose or purposes of the review and the use or uses of the information gathered.3

IV. Recommendations Regarding Evaluation to Improve Classroom Teaching of Individual Faculty

First, the university should develop and maintain a listing of current Emory- and school-specific resources that support teaching and should give clear information about how individual faculty can access such support. The list should be updated at least yearly and made available to faculty. The subcommittee encourages the university, the schools, and other groups that provide tangible support for teaching to articulate the purposes that support is designed to accomplish. In some cases, the sponsoring group will need to define more clearly the nature of the support to assure that it will advance the intended purpose.

Second, the resources should be expanded to include a university teaching center . This center should be funded by the university and designed to serve its constituents. Faculty representing each school and certain interdisciplinary programs should be appointed to guide the center. The subcommittee urges the university to organize the center primarily around goals of nurturing and improving teaching over time rather than providing a basis for judging the outcomes of instruction. The center should provide instructional specialists or other educational consultants for those who seek their services.

Third, each school should develop a locally organized center or office for teaching support similar to the Emory College Center for Teaching and the Curriculum. These centers or offices should coordinate work with the university center and should enrich overall support for teaching by providing school- or discipline-specific expert help.

Fourth, all schools and departments should develop and implement peer review as a means to improve teaching performance. When used to improve teaching performance rather than punish poor teaching, peer review provides a way for faculty at all ranks to exchange ideas as well as offer feedback and suggestions in a helpful but nonjudgmental way.4 The exact structure of the review may vary widely. Some departments may find it most helpful to ask faculty to observe their peers in the classroom and offer feedback in conversation;5 others may ask faculty to provide a somewhat informal document that can be included in a teaching portfolio. Some universities with an informal peer-review system ask senior faculty to help junior faculty identify teaching weaknesses before they are reviewed for tenure; others have one or more faculty visit and critique every new course, regardless of the rank of its professor. In any case, the procedures of peer review should involve coordination among faculty being reviewed, faculty reviewers, and educational specialists or consultants who can provide guidelines for the review.

Although the subcommittee recommends that peer review take place across the university, it suggests that each school determine the appropriate structural design of the process and implementation of both kinds of review. It is important that each school, department, or program make clear the purpose of the review-- that is, whether it is for strictly developmental or for summative purposes.6

When peer review is used to inform decisions about tenure, renewal of contract, or salary adjustments, the process becomes summative and therefore a more formal structure is required to ensure fairness and reliability. Again, the diverse missions and goals of the schools and departments demand that the review be acknowledged as an activity that supports decision making and that its specific form be designed at the local level. When peer review is used to inform decision making, it is critical that (1) each school, department, or program include its faculty in the process of designing the peer-review system; (2) the system be tested before officially being adopted; (3) faculty are trained properly in the principles and practice of peer review; (4) peer reviewers work in teams of at least two colleagues and/or one colleague and one education specialist; (5) peer reviewers use a standard form to report in order to ensure maximum uniformity among observers; (6) procedures require preobservation and postobservation conferences between the visiting team and the teacher; (7) copies of peer-review reports are sent to the teacher and maintained in the teacher's dossier (for his/her reflection and response) and in the teacher's portfolio (for use in tenure, promotion, and salary reviews); and (8) procedures for departmental or school peer review are coordinated with university or school instructional-support resources when possible.

V. Recommendations Regarding Evaluation to Assess Teaching Performance of Individual Faculty

A. Current Practices of Teaching Evaluation at Emory

In Emory College, professors distribute the Emory College Course and Instructor Evaluation form to students in each of their classes. These forms are collected and scored by computer. Each department chair receives summaries for courses in his/her department. These summaries provide feedback to instructors about their teaching performance and are used to evaluate classroom teaching for purposes of decisions about salary and promotion/tenure. As a matter of policy,7 promotion and tenure are based upon either a combined strength in teaching and research requiring "excellent" achievement or promise in either teaching or research and at least "very good" achievement and promise in the other. With only a single departmental exception, faculty perceive teaching to carry substantially less weight than research in considerations about annual salary and promotion.8

The Graduate School of Arts and Sciences expects each department or program to have a formal procedure for evaluating teaching that is consistent with the principles set forth in the Report of the Committee on Evaluation of Graduate Teaching.9 The report recommended that all graduate courses and seminars be evaluated in a written form explicitly different from the form used to evaluate teaching at Emory College.

Teaching within the School of Medicine comprises a broad spectrum of activities, ranging from large, didactic lecture courses to clerkships in which students rotate through clinical disciplines in close association with residents and faculty members. The quality of individual teaching receives attention primarily within departments and varies accordingly. Considering the size of the school and the number of departments, it is not surprising that practice differs widely. For example, some departments do extensive student and peer faculty evaluations and give systematic feedback to faculty. Other departments present evaluation forms to professional school students as the sole source of information about teaching effectiveness and restrict peer evaluation to informal evaluation by course directors. Still other departments do nothing at all. The importance of teaching in salary and promotion/tenure decisions is determined by the departmental chairperson, thereby causing additional variation. For example, most faculty in School of Medicine basic science departments (rather than in clinical departments) do some or all of their teaching in the Graduate School Division of Biological and Biomedical Sciences (GDBBS). However, all review for promotion and tenure takes place in a faculty member's School of Medicine department, which may place less value on teaching in the GDBBS, as its students are not enrolled in the School of Medicine.

In the Law School, each student in every class is asked to complete a Student Evaluation of Teacher (SET) form, which includes several standard questions and invites general comments. These forms are available to review by the instructor, the dean and, where appropriate, the members of the promotion and tenure committee. SET forms are reviewed by the dean as part of the annual review of each faculty member and can be useful in identifying problems in classroom teaching. Members of the promotion and tenure committee regularly visit the classes of junior faculty; junior faculty are invited to visit classes of more senior professors; and tenured faculty periodically visit each other's classes. These visits are intended to lead to fruitful discussion about teaching methods. The candidate is given a candid summary of the evaluations both of students and faculty colleagues. Both forms and class visitations are important in the promotion process.

In the School of Nursing, student evaluations of course/teacher are obtained for each course on a standardized form. These forms are considered part of the annual merit review of each faculty member. The forms and criteria for promotion and tenure, as they pertain to teaching, are currently under revision.

Near the end of each semester, faculty in the Goizueta Business School distribute a standard evaluation form to students in their classes. Results are summarized and forwarded to the appropriate area coordinator and to the dean. Each faculty member writes a commentary on teaching as part of his/her annual report. Student ratings and faculty commentaries are used to evaluate teaching on an annual basis.

In the Rollins School of Public Health, new criteria for assessing teaching as part of the overall criteria for appointment, promotion, and tenure are currently under review by department chairs and center directors. Salary decisions are made by the department chair (with approval of the dean), and there are at present no formal guidelines for factoring teaching performance into this process.

In the School of Theology, junior faculty are required to have students complete teaching evaluation forms for each class taught. Summaries of these forms are submitted with other materials during tenure review. Senior faculty have one course per year evaluated, and summaries are used in promotion review. Peer reviews of academic performance are held every three years for non-tenure- and tenure-track faculty; peer reviews for tenured faculty are held every five years. Faculty are expected to report on their own teaching for purposes of this review. Each year the associate dean of academic affairs makes recommendations to the dean about annual salary adjustments for each faculty member. These recommendations are based in part on an evaluation of teaching performance.

At Oxford College, teaching ability is evaluated primarily on professional competence and on teaching effectiveness; however, also considered are other important activities, some of which are considered equivalent to teaching (for which released time is given). Evaluation of these activities may involve present and former students, other faculty members, the dean, and outside professional evaluators. In addition to the individual's self-evaluation, the college considers all teaching evaluations previously collected from students, former students, and peers; three letters of recommendation from former students; and two letters of recommendation from professional colleagues. (Letters of recommendation are submitted to the person being evaluated.) Other pertinent materials such as teaching aids (e.g., textbooks, lecture notes), student papers, other work supervised by the individual, the development of new courses, and attendance at training institutes also may be included.

B. Recommendations for the Evaluation Process

While it is essential to allow each school flexibility, we also recognize the need for the university to require that certain basic standards be met by each school or program. We therefore recommend the following:

• Each school, department, or program should adopt clearly defined guidelines for the evaluation of teaching performance, taking into account the dimensions of teaching relevant to the given school, department, or program.

• Each tenured or tenure-track faculty member should maintain a teaching portfolio for purposes of review. This portfolio should be updated periodically (e.g., yearly) and should provide documentation of teaching performance and activities related to the school-specific dimensions of teaching.

• Each portfolio should include (with specific format determined separately by each component of the university): (a) results of student course evaluations; (b) results of peer review; (c) faculty statement of teaching philosophy and goals, documentation of teaching activities and accomplishments, comments from student evaluation and/or peer review; and (d) letters from current students, student exit interviews, statements of alumni or colleagues about teaching ability and performance.

• Each school, department, or program should conduct exit interviews with graduating students to assess the quality of their educational experience. Students may identify faculty who are exceptional teachers.

• Each school, department, or program should survey alumni periodically about their educational experience at Emory. Alumni should be asked which instructors were particularly helpful to them as teachers and mentors.

• Student evaluations should continue to be an important-- though not the only-- method of evaluating teaching performance. In particular:
• Each school, department, or program of Emory College should review and update the form used for student evaluation to ensure that the form appropriately measures teaching performance.
• The evaluation subcommittee has reviewed the form used by Emory College and recommends modification of what is measured, how and when to measure, how to report results, and to whom results should be given. We recommend that the form be modified to ensure that the elements of good teaching listed in section two are measured. The provost should appoint a committee to carry out this task.
• The extent to which results of course evaluations should be made available to students and/or the public needs to be addressed.
• Each instructor and department chair should be allowed to comment on the results of student evaluations and note what modifications are planned for future sections of the course. (In some schools-- e.g., the Rollins School of Public Health-- such written statements are required of each instructor and department chair.)
• A set of guidelines should be developed to help departments and administrators interpret and evaluate the student evaluations.
• Summaries of student evaluation forms that are sent to faculty and departments should include the average grade and distribution of grades in the course.
• The problem of "grade inflation" should be addressed in the following ways: first, the provost should establish a small faculty advisory committee to propose guidelines for grading (e.g., what should be the average grade and distribution). These guidelines would not be mandatory, but rather would offer faculty a general idea of how they should grade. Second, the provost should commission a detailed study of factors other than good or bad teaching that might influence student evaluation-- in particular, the relationship between evaluations and grades should be investigated. C. Discussion

  These recommendations are designed to encourage each school first to consider carefully how teaching performance may best be evaluated and then to prepare written, publicly available guidelines for evaluation. Such guidelines should consider the dimensions of teaching important to each discipline in order to give a more complete and objective accounting of a faculty member's teaching contributions. At their best, student ratings of a professor's classroom performance provide only a limited picture of faculty teaching. Unfortunately, however, most schools at Emory currently have little or no formal accounting of teaching outside the classroom.

  In recommending that each faculty member have a teaching portfolio, we suggest that this portfolio be a careful selection of evidence (as judged by the school, department, or program and the faculty member) organized around agreed-upon categories representing key dimensions of teaching. We believe that a teaching portfolio, when properly prepared, can capture more effectively than any single measure of teaching the substance and complexity of an individual's teaching. As discussed by Edgerton, Hutchings, and Quinlan,10 use of the teaching portfolio places responsibility for evaluating teaching in the hands of faculty (rather than the students), prompts faculty reflection on teaching practice and ways to improve it, and can promote thoughtful discussion among faculty about the elements of good teaching. The portfolio is an appropriate place for summaries of student and peer reviews of teaching; the faculty member's statement of teaching philosophy, goals, and any other accomplishments related to teaching; summaries of exit interviews with students; and statements by alumni and/or colleagues about teaching ability and performance.

  Student evaluations of teaching performance can provide useful information about student perceptions of courses and teachers. As discussed in the 1993 Report on the Quality of Teaching in Emory College, student ratings can be correlated with grades and class size: professors who give lower grades may get lower student rankings, and student ratings of smaller classes tend to be higher than for larger classes. However, as reported in Emory College, these factors taken together account for no more than 10 to 18 percent of the variance in mean course and instructor ratings. Consonant with the conclusions of the Report on the Quality of Teaching in Emory College, student ratings should be an important part of any system for evaluating classroom teaching and summaries of these results should be part of an instructor's teaching portfolio. The opinions of Emory students do matter, and each school should develop and use the results of systematic student review of courses and instructors. Procedures and forms should be reviewed periodically to ensure that they meet the needs they are intended to serve. Each instructor and department chair should receive summaries of student evaluations and comment about them in writing for the instructor's teaching portfolio.

  VI. Criteria for Adequacy and Excellence in Teaching Appointments, Promotion, and Tenure

  We recommend a university-wide set of guidelines for the review process that makes use of information contained in the teaching portfolio. In proposing these guidelines, we have been careful to allow flexibility in the approach taken by each school while at the same time asserting global criteria applicable to all schools. We leave it to each school to specify what information from the teaching portfolio is appropriate for the school's evaluation process. As an example, the Rollins School of Public Health (RSPH) report contains a listing of the dimensions of teaching, information that may be used to document teaching activities for each dimension, methods used for evaluation of teaching performance, and criteria for adequacy and excellence in teaching included for the purpose of administrative review. The report also contains scenarios about real teaching experiences to promote discussion by RSPH's review committee on how to apply the teaching criteria.

  For the university as a whole, we recommend that:

  1. Each school, department, or program adopt its own guidelines for adequacy and excellence.

  2. Guidelines be based on documentation in the teaching portfolio of each faculty member.

  3. Guidelines consider teaching activities of the candidate in all dimensions of teaching identified as relevant to the given school, department, or program and therefore need not be restricted solely to performance in the classroom.

  4. To satisfy promotion and tenure criteria, excellence in teaching should include a reputational component; that is, the faculty member should have earned school-wide, local, national, and/or international recognition from peers in the profession. Excellence in teaching connotes an objective, current, accurate, and balanced command of the subject matter of the field, an integration of knowledge, effectiveness in communication, innovation in course syllabi, and willingness to interact and exchange views with students and colleagues at the highest levels of intellectual stimulation and inspiration, as judged by students and peers.

  5. When assessing adequacy or excellence in the classroom, attention should be given to the number of courses a faculty member teaches, the degree to which a faculty member has sole or shared responsibility for each course, and the quantity of classroom instruction actually performed by the faculty member. A specific formula for quantity and/or degree of responsibility is not suggested. However, the length of service, previous teaching experience, and (for the professional schools) the proportion of salary covered by research activities should be considered carefully. Assessing dimensions of teaching other than classroom instruction are recognizably more difficult to evaluate than classroom instruction, and evidence in these two dimensions is likely to be based primarily on student exit interviews or written testimonials or letters from students, alumni, or other faculty.

  VII. Reward

  Although incentive and reward is not the topic that this subcommittee was charged to consider, we recognize that evaluation and reward are linked to teaching. Thus, we recommend that teaching rewards be based on well-defined, publicly articulated criteria and that the university treat teaching and research equally when considering reward. We support the recent Teaching Fund initiative to provide financial support to individual faculty as a first step toward balancing the way teaching and research are rewarded. Nevertheless, support for a one-year project is not equivalent to support of salary for an extended appointment. More meaningful, longer-term support might take the form of creating "teaching chairs," particularly in schools with graduate programs. We note that in units such as Rollins School of Public Health and the basic science departments of the School of Medicine-- where a faculty member annually must replenish about 50 percent of his or her salary from external (e.g., governmental) funding sources, the creation of teaching chairs would allow faculty with records of outstanding teaching to devote more of their time and energy to creative teaching activities. Therefore, we encourage the university to broaden its financial commitment by considering longer-term rewards that recognize teaching accomplishments over time.

  ---

  Notes to Appendix A

  1 Goals three and four are not the subject of this report. However, the committee notes the following. Goal three-- to assess the success of a teaching program in a department, school, or the college-- is typically the responsibility of the school, department, or program, as required for accreditation purposes. Goal four-- to provide a basis of information for rewarding outstanding teaching-- is being considered by the Subcommittee on Support, Rewards, and Incentives for Teaching. See also the note about reward in section seven of this appendix.

  2 Evaluation to improve current teaching practice should address, for example, lecturing technique, use of alternative approaches to lecturing, overall course organization, developing new teaching materials, using computer technology, techniques about exam construction, and course (homework) assignments. An important prerequisite for carrying out this formative evaluation is that the university should have the resources necessary to support the activity. In addition, four conditions should be met: (1) the teacher must acquire new knowledge about his or her teaching performance; (2) the teacher must value the information; (3) the teacher must understand how to bring about change; and (4) the teacher must be motivated to make change. See Richard J. Light, "Explorations with Students and Faculty about Teaching, Learning, and Student Life" in J. S. Stark and A. Thomas, eds., Assessment and Program Evaluation: ASHE Reader Series (Needham Heights MA: Simon and Schuster, 1994) 789-96.

  3 See, for example, L. Keig and M. D. Waggoner, Collaborative Peer Review: The Role of Faculty in Improving College Teaching, ASHE-ERIC Higher Education Report No. 2 (Washington: George Washington University) vii-ix.

  4 Implicit in these conditions for formative evaluation is the requirement that the teacher have confidence in the sources of new knowledge about teaching performance and in the evaluation process. New knowledge can come from many sources, including colleagues, chairs, students, specialists in instruction, and the teacher himself or herself. In fact, new knowledge needs to come from more than one source. It is possible, for example, that faculty colleagues are the best sources of information about the nature of course design, the appropriateness of pedagogy, and the quality of learning that results; students may give reliable evidence about the effectiveness of teaching technique. When the conditions for formative evaluation are sound, useful information from a number of sources can lead to positive change.

  5 After analyzing systems of peer review, Patricia Hutchings has noted that classroom observation is a commonly shared component. Classroom observation has three aims: (1) to prompt concrete, substantive discussion of teaching and learning; (2) to create an occasion for reflection and self-assessment; and (3) to foster collegiality and community among faculty. See Patricia Hutchings, Making Teaching Community Property. Preface by Russell Edgerton. (Washington: American Association for Higher Education, 1996) 17.

  The New Jersey Master Faculty Program is one peer-review process that has been documented by Hutchings. Here, pairs of faculty work as a team to conduct reciprocal visits to each other's classes and interview one another's students. The process results in an ongoing conversation about what is happening in the classroom and how it can be improved. A peer-review program also is in place at the University of North Carolina at Chapel Hill.

  6 To advance these recommendations, the committee offers the following brief descriptions of exemplary activities at Emory, at other universities, or at the national level. We encourage Emory to take appropriate advantage of the evaluations of the programs being assessed as this document goes to press.

  Bok Teaching Center, Harvard University. The Bok Center began as the Danforth Center in 1976. Today the mission of the center is to improve undergraduate teaching at Harvard, where the medical school and other graduate programs have their own centers for support of teaching. The Bok Center involves itself with formative evaluation only; it works with faculty, section leaders, and tutors to bring about improvement. The center advises start-up groups on other campuses to begin small and grow in the direction of clear need and to encourage the involvement of the departments whenever possible. Two years ago, the faculty council of the university mandated training for all new teaching fellows and required that each department organize a training program. After being involved with a number of these programs, the center reports the greatest success when center personnel work with, and not in place of, the department.

  Harvard Assessment Seminars. In 1986 Harvard President Derek Bok asked why faculty and administrators did not engage routinely in research and assessment of their own programs and focus their efforts in ways that would be likely to influence policy decisions. To address the question, he organized the Harvard Assessment Seminars. Unlike other assessment programs-- which seek to determine how much students know or merely evaluate the teaching of an individual instructor-- the seminars are designed to encourage innovation in teaching, curriculum, and advising, and then to evaluate the effectiveness of each innovation. Seminar findings include information about the characteristics of highly respected courses, the nature of faculty innovation in teaching, gender differences in college experience, and the value of small groups to enhance learning. Richard Light, director of the seminars, offers the following lessons:

  • Faculty initiative is imperative. Faculty must take a central role in shaping the important questions and in directing the investigation.
  • Assessment must be clarified. Harvard colleagues define assessment as a process of evaluating and improving current programs, encouraging innovation, and then evaluating the effectiveness of the innovation.
  • Students should be involved in every step of the enterprise. Graduate students interested in research, statistics, or higher education as a substantive field and undergraduates who make seminar topics the subject of an honors thesis can make valuable contributions.
  • Senior administrators can help get assessment started. Bok advocated the value of the activity and provided seed money from central funds.
  • Projects should lead to real policy change. Work that will actually improve education should be emphasized.
  • Leaders should work to create a climate that encourages evaluation and innovation rather than only rewarding successful outcomes.

  The American Association for Higher Education Peer Review of Teaching Project. This project is based on research by Lee Shulman which suggests that high-quality teaching is a subtle and complex activity requiring both a deep understanding of a particular field of knowledge and a capacity to transform this knowledge in ways that connect with students' diverse worlds. This notion contradicts the conception of teaching as technique concerning, for example, how to lecture to large classes or manage small discussion groups. Instead, project directors understand that neglect of teaching might be rooted not in the nature of teaching, but in the conception of teaching that has come to dominate current thinking. With this realization as an underlying factor, AAHE designed and sponsored a collaborative effort involving four academic departments (chemistry, engineering, history, and mathematics) at thirteen research universities (including Stanford and Northwestern) for the purpose of evaluating habits and practices of faculty collaboration that would lead to genuine improvement in teaching and learning. See Edgerton in Hutchings, Making Teaching Community Property, v-vi for a review of preliminary findings.

  Candler School of Theology Peer Review. A system of peer review that serves the ends both of formative and summative evaluation is already in place at Emory in the Candler School of Theology; the system bridges our discussion of evaluation for improvement and evaluation as a foundation for decision making. Although most representatives at the universities using peer review for improvement stress that the dual functions cannot be served by one process, the Candler process appears to function successfully at both levels. At Candler, peer review covers research and service activities as well as teaching performance. The process is viewed as an extension of the procedures and criteria for promotion and tenure, and has as its goal the establishment of a regular pattern for individual faculty to consult with their colleagues concerning their work, with a view toward professional development. Nontenured faculty are reviewed every three years and tenured faculty every five years.

  7 Principles and Procedures for Promotion and Tenure, Emory College, 1991.

  8 Report on the Quality of Teaching in Emory College, Emory College, 1993.

  9 Report of the Committee on Evaluation of Graduate Teaching, Emory University Graduate School of Arts and Sciences, 1994.

  10 Russell Edgerton, Patricia Hutchings, and Kathleen Quinlan, The Teaching Portfolio: Capturing the Scholarship in Teaching, AAHE's Teaching Initiative (Washington: American Association for Higher Education, 1991).

  ---

  Table of Contents