|
Report of the Task Force on Assessing and Improving Teaching and Learning at Indiana State University October 1998
Executive Summary |
Most faculty and administrators readily acknowledge that the current systems for evaluating teaching are unsatisfactory. Given that teaching is a major part of the mission of Indiana State University, it is clear that a comprehensive, systematic, campus-wide model is needed to provide faculty adequate feedback on their pedagogy and to allow the campus community to examine its teaching strengths and weaknesses. In response to this need, the Provost appointed a university task force to develop a model designed to assess and improve teaching at our university. This report is the result of the work of this task force.
The task force's proposed model is grounded on a framework of Guiding Principles. It uses the scholarship on college teaching to identify six major areas where faculty expertise will enhance the learning success of our students and uses them as the components around which a review of teaching is constructed. It acknowledges that there are multiple sources of data that contribute to making fair and consistent evaluations regarding these components and offers a summary of how each of these sources can contribute to the evaluation process. Finally, using a common rating system, the model presents a matrix which allows enough flexibility for each unit to discuss and select the weights of components and data sources that match departmental values.
This report concludes with the recommendation that, after some campus discussions, select units first adopt this model as a pilot program before the model is applied on a university-wide scale. Doing so would ensure a careful and well-thought out implementation of the proposed model. This purposefully deliberate implementation should be closely monitored by both the faculty and the administration to assure that the resultant teaching evaluation system becomes a valuable part of a university culture committed to student success.
Unmet Needs
Academics tend to regard teaching as an unaccountable art. You either have the gift or you don't. If you possess the ability to inspire students, to organize knowledge into teachable moments and semesters of instruction, the assumption is made that this admirable ability will be recognized intuitively by peers and probably reflected in student answers to standardized evaluations. Moreover, conceived of as a talent expressed in many fashions, teaching also enjoys tremendous protection in the name of academic freedom, which provides cover from quantifiable and externally validated measurements, even to the point of obscuring the need to assess and evaluate teaching with the same rigor as scholarship. Many tacitly agree that publishing demonstrates academic competence, whereas teaching is approached so variably and conceptualized so vaguely that no one can agree on how to gather information about it ("assessment") and how, reliably, to interpret the resulting data ("evaluation").
A review of our assumptions about assessing and evaluating teaching is warranted by new evidence of increased focus on pedagogy and decreased satisfaction with the balance of teaching and research in rewards systems (Dey et al. 1993; Boyer 1990, 23-41). These national doubts call attention to issues at home, such as pay-for-performance. Our campus has observed warning signs about perceived inequities between recognition for teaching achievement versus scholarly production.
Because pedagogy is considered complex and unknowable, the perfect time to re-examine teaching always seems to be "next year" or "never." Such delays are no longer tolerable, given improved tools for assessing and more thoughtful, systematic methods of evaluating teaching. A reasonable starting point is a survey of the teaching culture on our campus.
Local History
Indiana State University, despite having advanced its goals in the 1970s beyond the "Teacher's College" level, still pursues the idea that the classroom counts. Recruiters point to comparatively low faculty-to-student ratios and market the University by touting an unintimidating 1950s' mood of "college the way it is supposed to be." Traditional teaching styles, like the lecture method or the master/apprentice relationship, seem to earn high marks from parents, though their 1990s children have been struggling to keep up grades and graduate, or even to persist in college. Whether ISU's troubling retention level is influenced more by teaching methods or by inadequate academic preparation coupled with distracting financial and social factors is the focus of close scrutiny by the Lilly Project (Chesebro, 1997). Added to the traditional-aged students who are at risk of "slipping out" of academic life, the student population includes a cohort of displaced workers who attend ISU to upgrade job skills, plus another cadre of mothers and fathers who never had a chance to go to college.
Change is a fact of university life, on both sides of the desk. Professors teach different students than they ever met in graduate school. Information-intensive teaching methods aren't working as well as they apparently did during the "golden years" of the G.I. Bill, an era of post-WWII educational enthusiasm. Renewed attention to pedagogy is one response to the search for equilibrium between attracting the students whom professors wish for and serving the ones their universities can retain. Some teachers, who are reluctant to blame the students for underpreparedness, seek changes in teaching approach, learning communities, and classroom assessment; other teachers urge different recruiting strategies and admissions policies. Within the instructional realm--the area classroom teachers can most readily affect--new methods and measures are needed to discover which approaches should be kept and which can be improved.
The lack of a carefully examined and adapted system of evaluating teaching prevents intelligent analysis of rich opportunities for self-improvement and enhancement of the learning environment. No longer trustworthy are the instruments and practices inherited from a time when failed students would be replaced by other diploma seekers. And in the context of that anxiety-producing pressure induced by linking salary adjustments to demonstrable performance, teaching effectiveness is hard enough to document within one's own unit and harder still to compare among departments and across schools.
The Three Wise Professors of Gotham
Consider this accumulated worry from the viewpoint of three typical university teachers. Full Professor Oldcastle is a well-respected, capable lecturer. She seriously doubts whether high schools and community colleges give enough of a head start to students generally, and particularly to the ones who can't write, don't think, and won't come to office hours for enlightenment. She is frustrated that her teaching has stagnated. Associate Professor Henryson approaches his tenure year with some apprehension because his modest publications aren't balanced by any indicators more substantial than Student Instructional Reports, whose averages for "overall quality of instruction" his department's personnel committee had not found very impressive. He wonders whether he can sneak through the screening process on the basis of minimal evidence and the hope that rumors of his teaching skill will be heard by the right ears. Assistant Professor Hotspur is having second thoughts about her new employers at Indiana State. Will they reward the cutting- edge pedagogy she began to use as a T.A. in grad school? Without any endorsed indicator of teaching effectiveness, and lacking a teaching support service beyond a drop-in Learning/Teaching center, she wonders whether she should have taken that job at Mega Junior College. Once, the lure of teaching and doing scholarship at a medium-sized university had overcome most of her doubts. Now worries remain.
The hypothetical cases of each of these professors in early, middle, and later stages of their careers will convey the practical meaning of high-level choices about teaching evaluation. Teachers' needs differ depending on their place in the span between first paycheck and the retirement tea.
Call and Response
In January 1998, Provost Wells solicited sixteen faculty members for a task force to look again at teaching's effects on learning, within the context of assessable goals and appropriate rewards. He charged the group 1) to describe a conceptual model for the improvement of teaching assessment and evaluation,
2) to propose a set of steps for implementing the recommended model, and 3) to put forward incentives that would reward validated achievement.
Alan Lacy chaired this group of independently-minded, vocal, and dedicated teacher-colleagues. We met regularly during the spring semester, to debate guiding principles and investigate particular practices and constituencies. During the summer, a quintet drafted sub-committee reports and composed a working manuscript. Then, at the start of the fall semester, the entire group criticized and revised the document, always mindful that other universities' best practices might not fit our needs. Abstract schemes would not necessarily work here, either.
The following survey of options and recommended approaches to teaching evaluation was conducted with a realistic appreciation of effects a new evaluative tool will have on teachers who have the honor of working at the Indiana State University in this stressful yet stimulating stage of its history.
A program for evaluating teaching must be designed to inform and guide faculty and administrators in making judgments affecting the quality of the university's teaching and learning environment. This assessment system should reflect best principles and practices. It should be grounded in the scholarship of effective teaching and learning. It should apply fairly and reasonably to all segments of the teaching faculty.
A. An effective teaching evaluation program must be comprehensive and systematic.
Because teaching is the central aspect of the university, it is essential that everyone engaged in teaching be evaluated annually. While evaluation procedures may necessarily be adapted according to teaching areas, the overall procedures for conducting teaching evaluations should be integrated into a campus-wide evaluation system so that results fairly represent the quality of teaching throughout the campus.
B. A sound teaching evaluation program must be designed according to the scholarship on teaching and learning.
A teaching evaluation program should measure qualities that are known to affect the success of learners. Despite the fact that there is lively debate among pedagogical scholars, there are also broad areas of consensus about teaching practices and teacher qualities. These areas of agreement should be reflected in the design of a teaching evaluation program.
A teaching evaluation system must integrate two distinct, but connected, aspects: formative and summative evaluations. To do one without the other produces a less effective evaluation system.
A. One purpose of a teaching evaluation system must be to provide valid, summative information regarding the quality of teaching.
An evaluation program must provide a report that offers valid indicators of the quality of teaching. These measures must provide comparable ratings for faculty at all levels and across all teaching areas. These ratings should be the primary summative documentation used for promotion, tenure, and performance evaluations.
B. The second purpose of a teaching evaluation program should be to provide formative direction for faculty improvement efforts.
Evaluation of teaching should not exist separately from a faculty development program. The design of faculty evaluation procedures and reports should provide information about areas of strength and those that could use improvement. Campus faculty development programs should be related to this process.
An effective teaching evaluation model relies on procedures that assure the collection of reliable, valid, and adequate information. It also ensures the generation of reports reflecting fair and responsible interpretations of that information
A. Evaluations should include multiple sources of data collected from students, peers, administrators, and self-evaluation.
Because teaching involves many dimensions, it is best to gather information from a number of sources. Each different source should provide that information about the instructor's teaching with which it is most familiar.
B. Every course must be evaluated using a standardized university procedure.
To assure continual improvement of instructional quality, it is essential to collect information about teaching in an ongoing, systematic way. It is invaluable to generate consistent, comparable results over time and across courses, so that patterns may be discerned and acted on. While faculty or departments may also choose to administer more specific evaluations, a standardized form administered by the university should be used in each course.
An evaluation system must be organized, managed, and adjusted to assure its integrity as well as its ability to enhance the teaching and learning environment at the university.
A. The primary authority to interpret the data from teaching evaluations lies with individual faculty members and his or her department.
Because there are multiple and complex factors involved in the interpretation of information, particularly in relation to the educational character and mission of each department, the evaluation system must be responsive to the individual's and the department's priorities, norms, and
positions--provided this responsiveness exists within the general parameters of the overall evaluation system itself.
B. Faculty are entitled to a fair and consistent evaluation of data regarding their teaching.
In order to reduce over-reliance on subjective representations of individuals' teaching performance, the evaluation system must aim to utilize reliable data collection methods and clear criteria for making interpretations and judgments. It is the primary role of the administration to ensure the integrity of this process when summative evaluations are made.
Scholarship on the Components of Effective College Teaching
Teaching evaluation systems should be organized around characteristics crucial to the success of the teaching endeavor. As John Murray asserts, "If colleges and universities intend to include standards for good teaching when making personnel decisions, they need to be able to recognize good teaching" (1995: 61). He claims, further, that along with general research of learning and teaching, cognitive psychology is helping to reveal demonstrable characteristics of good teaching. This section aims to categorize some of the qualities scholars have identified as fundamental to student success. The sources used in this section are drawn from studies that summarize the relevant literature. What follows is an effort to consolidate this information into clusters of specific areas known to have an impact on the quality of teaching and learning.
I. Teacher's Content Expertise
A teacher's knowledge base in a subject area is fundamental to the creation and enhancement of students' opportunities to learn well.
A. Effective teachers understand their academic field well.
It is obvious that teachers must know the material well in order to instruct students accurately. More importantly, a teacher's expertise assures that content is current and taught in adequate depth. Expertise in a subject can provide a teacher with the flexibility to design and adapt material for the learner. The instructor can use this background to respond to a range of learners' needs.
Supporting claims:
-- An instructor has a good background in the course concepts, principles, and paradigms. (Feldman, 1988)-- Excellent teachers have a strong command and organization of their subject. (DuBois, 1993)
-- Competencies for good teachers include demonstrated mastery of the subject. (Smith and Simpson, 1995)
-- Good teachers are knowledgeable about their work areas and disciplines; they integrate current subject matter into their work. (McCabe and Jenrette, 1990)
A teaching assessment program must include some way to measure and rate a faculty member's expertise in the content area. Competence includes not only content knowledge but the ability to organize, integrate, adjust, and adapt this content in ways that make it accessible and thought-provoking to the learner.
The organization of course materials and activities has a profound effect on a student's possibility to succeed in a course. Effective organization goes beyond the orderly arrangement of content to involve the design of experiences that assist students in their learning. A number of course design principles that need attention.
A. Effective teachers have a clear purpose that organizes course elements.
A teacher needs to provide an organizing framework that orients students to the course's ideas, materials, and activities.
Some research that supports the need for planned organization:
-- An instructor prepares and organizes the course. (Feldman, 1988)-- Good courses are crafted. (CTL, 1997)
-- Excellent teachers spend considerable amount of time preparing course presentations. (DuBois, 1993)
-- Excellent teachers do their work in a well-prepared and well-organized manner. (McCabe and Jenrette, 1990)
The assessment program would search for evidence of careful planning. The quality of this planning would be one indicator of the likelihood of successful learning.
B. Effective teachers communicate high but realistic expectations.
The goals of a course must be challenging enough to motivate students, yet not so demanding as to overwhelm them. The teacher must identify what Lev Vygotsky called the "zone of proximal development." The research on college teaching recognizes this balance between high expectations and adaptation to students' ability level.
The studies claim:
-- The teacher sets and maintains high but realistic expectations & goals. (Angelo, 1993)-- Good practice communicates high expectations. (Chickering and Gamson, 1987)
-- Excellent teachers have the ability to motivate students to form goals and succeed academically. (DuBois, 1993)
-- Good teachers communicate and manage appropriate expectations for achievement in the course. (Smith and Simpson, 1995)
-- Excellent teachers project a positive attitude about students' ability to learn. They set challenging individual and collective performance goals for themselves. (McCabe and Jenrette, 1990)
-- Good teachers set challenging performance goals for students. (McCabe and Jenrette, 1990)
Realistic challenges are those deemed appropriate for students. Effective teachers are aware of their students.
-- The teacher shows a sensitivity to and concern with students' level and learning progress. (Feldman, 1988)-- The teacher selects course material suited to students' backgrounds, abilities, and interests. (Smith and Simpson, 1995)
This balance allows for intellectual and personal growth.
-- Students feel good courses are those where they learn a lot. (CTL, 1997)-- Build confidence in students by helping them to successfully meet learning objectives. (Smith and Simpson, 1995)
-- Students reach beyond course content and grow as human beings. (CTL, 1997)
-- Good teachers enhance student motivation by demonstrating the subject's relevance to their future needs and goals. (Smith and Simpson, 1995)
Accordingly, an assessment program must be designed to determine how well suited the course design is to students' abilities and interests. A well-constructed course (or set of activities) that is too simplistic or too challenging will not enhance students' success. It is necessary to gather and rate the appropriateness of the course's teaching materials.
C. Effective teachers match the instruction to students' learning needs and interests.
The design of a course must include deliberate connections between the subject matter and students' needs and interests, which engage them in the learning process. The various research statements below describe a number of points at which the course material should be designed to work with students' experiences.
Research on teaching suggests:
-- Good courses are those where students can connect new information to prior knowledge. (Angelo, 1993)-- Good courses are those that are relevant to students' needs and interests. (CTL, 1997)
-- Students become aware of their values, beliefs, preconceptions, and prior learning - and are willing to unlearn when necessary. (Angelo, 1993)
-- The teacher focuses student attention on what matters most. (Angelo, 1993)
-- Students seek and find real-world applications of what they are learning. (Angelo, 1993)
The teaching assessment system needs a method for determining to what degree a course provides opportunities for students to develop valuable personal connections to the course content. A straightforward organization of content may ignore the need to establish bridges to learners' experiences. Good courses should demonstrate deliberate strategies for determining student needs and interests regarding content. They should be designed to help students extract main points and they should incorporate activities that connect learning to applications.
Student learning is also affected by academic interactions between teachers and learners as they engage the subject matter. The various strategies and techniques that teachers use to implement the course design are critical to the learner's success. There are several key points that help teachers strengthen student learning.
A. Effective teachers use good communication skills.
It is necessary for teacher to communicate ideas clearly.
Research indicates:
-- The [effective] instructor is clear and understandable (Feldman, 1988)-- Excellent teachers communicate effectively in both written and oral form. (Smith and Simpson, 1995)
-- Effective teachers have the ability to present material interestingly and clearly. (Bernoff, 1992)
-- Excellent teachers present ideas clearly. (McCabe and Jenrette, 1995)
-- Excellent faculty are talented in clarifying difficult subject matter. (DuBois, 1993)
An effective teaching assessment process should determine that the ideas and directions presented in a class are clear and understandable. Miscommunication, confusion, or failure to convey key information leaves students at a distinct disadvantage. Good communicators go beyond clear information delivery to create environments that encourage comfortable, two-way communication between students and teacher.
B. Effective teachers design learning environments that encourage time on task.
Simply because time is spent in a classroom chair does not mean that time is being spent learning the subject. Successful classrooms spend time on tasks that aid learning. An effective teacher uses what is known about how people learn to design productive learning time.
Reports state:
-- Learning is enhanced when students invest as much time and high-quality effort as possible. (Angelo, 1993)-- Good practice emphasizes time on task. (Chickering and Gamson, 1987)
-- Good teachers manage the learning environment so that maximum learning will result. They lead class discussions that stimulate learning and enhance the goals of the course. (Smith and Simpson, 1995)
The assessment of effective teaching should be able to characterize the productiveness or lack of productiveness of a class period. There should be some way to determine how teaching and learning efforts are committed to the most important information.
C. Effective teachers engage students to actively use knowledge.
Learning is enhanced when students are engaged in active cognitive processes. Teachers need to design learning activities that require thoughtful information processing.
On this point, researchers say:
-- Students are (cognitively) actively engaged in their academic work. (Angelo, 1993)-- Good practice encourages active learning. (Chickering and Gamson, 1987)
--Excellent faculty encourage student participation and see themselves as student-centered teachers. (DuBois, 1993)
-- Good teachers promote students' individual involvement through learner-centered teaching methods. (Smith and Simpson, 1995)
-- Excellent teachers encourage independent thinking, analytic listening. (McCabe and Jenrette, 1990)
-- Good new teachers lecture in a relaxed style that provides opportunities for students' comprehension and involvement. (Boice, 1991)
Using course materials, an assessment program should be able to identify activities that involve students in active engagement with course material. Straightforward presentations of materials are not, in themselves, supportive of solid learning. Excellent teachers can describe specific ways in which their understanding of cognitive processes shapes the design of instructional activities to ensure that students encounter the subject thoroughly. An assessment process should be able to discriminate among teachers at either end of this continuum.
D. Effective teachers use an appropriate array of methods.
Because of the semester has its own rhythms and because students' needs change, teachers must be prepared to alter instructional methods to suit immediate goals. Methods must also be varied to accommodate students' different learning styles or developmental levels. However, it is obviously the aptness of methods that is important to learning, not simply the presence of different teaching techniques.
Researchers report the following:
-- Excellent teachers match varying teaching methods with specific instructional objectives. (Smith and Simpson, 1995).
-- Excellent teachers provide students with alternative ways of learning. (McCabe and Jenrette, 1990)
-- Good courses contain varied instructional methods. (CTL, 1997)
-- Students encounter a balance of intellectual challenge and academic social support. (Angelo, 1993)
-- Good teachers present material that is sequenced and paced appropriately for learners. (Smith and Simpson, 1995)
Teaching assessment procedures should indicate the degree to which the teacher makes deliberate use of varied techniques to engage students. The mere fact that there are varied techniques is not as promising as deliberate use of variation. It is important that a teacher respond to student needs and to the changing focus of the curriculum. The review process should develop methods to identify and acknowledge teachers' success in adapting teaching methods to meet student needs.
E. Effective teachers encourage students to work together to learn.
Peers are one of the most powerful learning aids. Effective teachers structure activities that use peer relationships to assist in the learning enterprise.
Research studies indicate:
-- Students work regularly and productively with other students. (Angelo, 1993)-- Good practice encourages cooperation among students. (Chickering and Gamson, 1987)
-- Excellent teachers provide cooperative learning opportunities for students. (McCabe and Jenrette, 1990)
-- Good teachers encourage cooperation and collaboration among students. (Smith and Simpson, 1995)
A complete teaching assessment program should thus be able to indicate how well a teacher creates conditions that enable students to work together productively, to improve their understanding of the material. A course should not be composed simply on teacher talk. Classroom activities that teach students how to work together effectively should yield high rates of learning success. Protocols for determining whether adequate levels of cooperation exist should be established within the assessment rubric.
F. Effective teachers give regular, helpful evaluations of learning.
To improve the quality of their work, students need continual, immediate, and helpful feedback. The teacher must incorporate an effective system for giving feedback to students, if they are to reach their potential.
The research states:
-- In good courses, the quality of examinations is good. (Feldman, 1988)-- Effective learning occurs when students are assessed and evaluated on what matters most (main points) and understand the criteria and standards against which they will be evaluated. (Angelo,1993)
-- Students become aware of their own ways of learning, so they can better monitor and direct their energies and efforts. (Angelo, 1993)
-- Good practice gives prompt feedback. (Chickering and Gamson, 1987)
-- Good courses have timely, informative feedback/evaluations of academic work. (CTL, 1997)
-- Good teachers provide helpful feedback to students in a variety of ways. They construct valid and reliable tests and administer other evaluation measures fairly. (Smith and Simpson, 1995).
-- Excellent teachers evaluate their students frequently and always let them know where they stand with regard to academic performance. (DuBois, 1993)
-- Excellent teachers give constructive feedback promptly to students. They are fair in their evaluations. (McCabe and Jenrette, 1990)
In order to assure that students have the information necessary to make progress in the subject matter, the teacher must have a thoughtful plan for assessing students' knowledge and abilities. In contrast to a generic letter grade, this evaluation system must provide specific information that both confirms knowledge gains and highlights the next steps for improvement. A teaching assessment system should be able to determine how well course evaluation methods contribute to enhancing the students' learning.
A special dimension of academic support integral to high student performance is the relationship between the teacher and the students. Teachers who can develop relationships that foster and encourage student engagement will enhance learning.
The researchers claim:
-- Excellent teachers are open, have respect for opinions of others, and encourage questions and discussions. (Bernoff, 1992)-- Excellent teachers never embarrassed or berated students (DuBois, 1993)
-- Excellent teachers create a climate that is conducive to learning. They treat individuals with respect. (McCabe and Jenrette, 1990)
-- Good new teachers have positive attitudes about students. (Boice, 1991)
A. Effective teachers and students are enthusiastic.
Clearly, motivation is a vital source of energy in learning activities. The effective teacher is energetic and enthusiastic and finds ways to create student interest and commitment to the learning task.
Reports indicate that:
-- The effective instructor shows an enthusiasm for the subject and for teaching. (Feldman, 1988).-- In good courses students and teachers are enthusiastic about learning. (CTL, 1997)
-- Effective teachers have enthusiasm about/interest in the subject matter, they are dynamic, energetic, and stimulate interest. (Bernoff, 1992)
-- Excellent teachers exhibit enthusiasm about their discipline and class presentations. They are able to convey a strong sense of presence in the classroom to elicit students' attention and stimulate their emotions. (DuBois, 1993)
-- Good teachers enhance students' motivation through personal enthusiasm for the subject. (Smith and Simpson, 1995)
-- Excellent teachers are enthusiastic about their work. (McCabe and Jenrette, 1990)
Classrooms with enthusiastic learners are not hyped-up circuses but places where dedicated, sincere efforts are being made to learn the subject. While strategies for creating this enthusiasm may vary, the degree to which a classroom generates this dedication should be evident and should be assessed as an important quality in promoting learning.
B. Effective teachers and students need to know and respect each other.
Students are unlikely to learn from people they do not know or respect. This does not mean that teachers are required to reveal their personal lives to students (or vice versa), but it does suggest that students learn best when they see how scholarship is connected to dimensions more personal than the mere repetition of content.
Researchers claim:
-- The relationships between students and teacher matter. (CTL, 1997) -- Learning is enhanced when students work regularly and productively with instructors. (Angelo, 1993) -- Good practice encourages student-faculty contact. (Chickering and Gamson, 1987) -- The instructor is available and helpful. (Feldman, 1988) -- Effective teachers are interested in, have concern and respect for students. (Bernoff, 1992) -- Excellent teachers have an approachable and friendly style with students ( DuBois, 1993) -- Excellent teachers are responsive to students' needs. (McCabe and Jenrette, 1990)
Teaching assessments should determine the degree to which a constructive relationship was established between students and teacher. At the very least, it is important that students feel welcomed to talk to the instructor. It is even better when students understand and respect the teacher because he or she models a commitment to scholarship (Smith and Simpson, 1995). Teachers' ability to connect with students constitutes a significant factor in learning success.
C. Effective teachers acknowledge and adjust to student differences.
Because students come from a variety of class, race, gender, ethnic and lifestyle backgrounds, because they manifest a variety of learning styles, and because they are often at various developmental learning stages, it is imperative that teachers be responsive to student differences.
Research demonstrates:
-- Good practice respects diverse talents and ways of learning. (Chickering and Gamson, 1987)-- Good teachers accommodate students' different learning styles by using a variety of teaching methods. They deal appropriately with issues that relate to various aspects of diversity. (Smith and Simpson, 1995).
-- Excellent teachers respect diverse talents. They regard students as individuals operating in a broader perspective beyond the classroom. They provide perspectives that include a respect for diverse views. (McCabe and Jenrette, 1990)
-- Excellent teachers understand [students' backgrounds]. (DuBois, 1993)
The degree to which the classroom provides ready access to multiple world views in its material, activities, and relationships represents an important marker of students (equal) opportunity to succeed. Teachers who address students' different needs for support are generally perceived as fair. Assessments of teaching should recognize teachers' efforts to assure that all students feel welcome and can be successful in learning the subject matter. The assessment should incorporate a rating of the degree to which they reach these goals.
D. Effective teachers are fair and impartial in dealings with students.
For students to succeed, they must trust that their efforts will be treated with the same respect as those of all other students. Perceived unfairness can destroy student interest and motivation.
The researchers state:
-- The teacher is impartial when grading students. (Feldman, 1988)-- The teacher shows overall fairness to students. (Feldman, 1988)
-- Effective teachers are fair. (Bernoff, 1992)
The quality of fairness is also mentioned in McCabe and Jenrette (1990) and Smith and Simpson (1995) as cited in section III F (above).
Students' perception that the instructor treats all equally is fundamental to the integrity of a class. An assessment program must therefore collect data that indicates how fairly the teacher treats students, if it is to assure this dimension of effective teaching.
E. Effective teachers are open to receiving feedback and adjusting courses appropriately.
When there is a clear and open channel for communication, teachers and students can modify and adjust to circumstances that might otherwise obstruct learning. Without such channels, confusion and misunderstanding may lead students and teachers to work in a too restrictive environment.
The reports indicate:
-- The teacher provides, receives, and makes use of regular, timely, specific feedback about course procedures (Angelo, 1993)-- Two-way communication is evident in good courses. (CTL, 1997)
-- Excellent teachers give consideration to feedback from students and others. They listen attentively to what students say. (McCabe and Jenrette, 1990)
-- Good teachers develop a reflective approach to teaching by collecting feedback and using it to continually modify the approach to teaching. (Smith and Simpson, 1995)
Therefore, teacher assessments should gauge how well teachers and students communicate on matters dealing with the classroom environment. The assessment process should be able to determine whether there are mechanisms in place to collect feedback and how well students and teachers use them to maximize learning.
Obviously, logistical and administrative components of a course must be handled efficiently and effectively. Even though the research on college teaching and learning largely overlooks course management, the smooth functioning of a course is an important factor in creating an environment conducive to learning. Course management encompasses a range of issues such as: regular meetings with classes, timely assignment of tasks and return of feedback, etc. The way in which the classroom is organized as an environment determines how effectively it will support learning.
The reports do suggest:
-- Good teachers manage the process of planning, teaching, and evaluating in a timely manner. They deal appropriately with matters of discipline, academic honesty, and legal information. (Smith and Simpson, 1995)
Nearly all the researchers recognize the importance of being available to help students (McCabe and Jenrette, 1990; Bernoff, 1992; DuBois, 1993; Smith and Simpson, 1995).
No single arrangement of teaching and learning components will be successful in all circumstances. Rather, good teaching requires that educators have a rich understanding of pedagogical theory and practice and are able to engage in the on-going development and adaptation of their teaching approach. Stephen Brookfield (1995) describes the importance of critical reflection to college teaching. Mary Ellen Weimer (1990) and Anthony Grasha (1996) have detailed the processes that help faculty improve their teaching practices. Moreover, Raoul Arreola (1995, pp. 78-79) has documented how faculty evaluation scores improve when they participate in faculty development initiatives. Faculty members who hold themselves to high professional standards in their role as teacher exhibit qualities which attest to the dynamic nature of effective teaching.
Researchers claim:
-- Excellent teachers recognize and accept teaching as a fundamental and challenging dimension of scholarship. They use research in teaching as it applies to instruction in their field. They use a reflective process. (Smith and Simpson, 1995)--Excellent teachers have a strong sense of commitment and dedication to college teaching. (DuBois, 1993)
-- Excellent teachers are committed to education as a profession. They display behavior consistent with professional ethics. They work collaboratively with colleagues. They provide clear and substantial evidence that students have learned. (McCabe and Jenrette, 1990)
-- Good new teachers show evidence of actively seeking advice about teaching from a colleague. They demonstrate a great readiness to become involved in campus faculty development programs (Boice, 1991)
-- Excellent teachers have a distinct identity as a teacher, have been inspired by past teachers, and need their students as much as students need them. (DuBois, 1993)
It is reasonable to expect that faculty be able to document their scholarly efforts to enhance pedagogical knowledge and to develop practices that enrich their teaching and learning environments. Portfolios (Seldin, 1997) or other reports can be used to demonstrate such scholarly efforts in the area of good college teaching. An assessment program should incorporate some tool to encourage professional development, to measure progress, and to reflect areas for further growth.
Conclusion
Researchers argue that there are multiple components involved in effective college teaching. The major research has been summarized in six categories to better focus our efforts to collect and rate performance levels. Figure 1 presents the six categories and the components that make up each. These categories form one of the underlying structures from which a sufficiently complex evaluation system can emerge.
The Components of Effective College Teaching
|
I. Teacher's Content Expertise
|
|
II. Course Design
|
|
III. Instructional Delivery
|
|
IV. Instructional Relationships
|
|
V. Course Management
|
|
VI. Professional Development
|
|
|
Using Multiple Sources for the Assessment and Improvement of Teaching
True objectivity in a faculty evaluation system is an impossible goal; in a process that, by necessity, relies on the subjective judgement of peers, administrators, students, and self-evaluation to gather its data, subjectivity is unavoidable. Informed, personal assessment is indeed required to examine the complex human interactions of education. Yet the majority of faculty must believe that the system used is fair and impartial. Unfairness arises when evaluations are based on biased, capricious, hear-say comments about teaching. Thus, the goal of the evaluation system should be to reduce and control the negative aspects of subjectivity. By using multiple sources in appropriate ways, subjectivity can be controlled, and a fairer and more equitable system can be developed.
Current practices for evaluation of teaching undoubtedly vary from department to department on our campus. However, in all cases, students, peers, and administrators are the most common sources of information for assessing teaching, whether data are systematically gathered or randomly acquired. Depending on the situation, the credibility of these sources varies. The credibility of a source depends on the opportunity that the source has (had) to be a firsthand observer of the teaching component in question, as well as on the willingness of the person being evaluated to accept and believe what that source has to say (Arreola, 1995).
As stated earlier in the Guiding Principles, teaching evaluations should include multiple sources of data including students, peers, administrators, and self-evaluation. This type of evaluation must provide valid, summative information regarding the quality of teaching and formative direction for faculty improvement efforts. Finally, faculty are entitled to a fair and consistent evaluation of their teaching. With these principles in mind, this section will discuss issues related to the various sources and their roles in the assessment and improvement of teaching.
Students play an essential role in the evaluation of teaching. They are, after all, the group most directly affected by the quality of teaching. As firsthand observers in classes, students are in the best position to evaluate specific and critical aspects of classroom teaching. Students are the only individuals who regularly observe us in action, and it is for their benefit that we teach. Murray (1995, p. 50) states, "This symbiotic relationship between professors and students means that it is not only in our best interests to respect what they can tell us about our teaching, but also in their best interests to assist us to improve our teaching."
Techniques for formative evaluation can range from very formal to more informal. At the formal end of the spectrum are instruments that look very similar to student evaluation instruments used for summative purposes at most U.S. institutions of higher learning. Informal techniques using such strategies as student journals, open-ended questions, writing an unsigned letter to a fictitious friend who is thinking of taking your class, etc. can be valuable formative tools (Murray, 1995). Many of the formative evaluation techniques should be implemented after students have had enough time to become familiar with a teacher's style, but no later than midterm to allow for mid-semester modifications. Several studies (McKeachie et al., 1980; Overall and Marsh, 1979; Stevens and Aleamoni, 1985) have reported that midterm student feedback helps faculty improve teaching. Brinko (1993) suggests that this is particularly true if such feedback is combined with advice from a faculty development specialist. By providing this type of early formative evaluation, the same students who provide the feedback will gain some of the benefits from any course modifications the faculty member makes.
Formal student evaluations given at the end of the semester are typically used for summative purposes, but can also be used in formative ways as well. Student ratings are one of the most common features of faculty evaluation systems. Seldin (1993) reports that more than 85% of all faculty evaluation systems make regular use of student ratings. Unfortunately, these ratings are often the only component used to evaluate teaching. Given that this information is used for personnel decisions that are not always positive for the faculty member, it is not surprising that student rating forms are not met with universal acclaim by faculty! This misuse of limited perspectives has led to many myths and misconceptions about student ratings.
It is critical that the instrument used be both reliable and valid, and provide meaningful feedback that can be used for making improvement. Because locally developed rating forms may not possess the necessary psychometric qualities of reliability and validity, it is generally a good idea to adopt an existing form rather than developing one from scratch (Arreola, 1995). Many believe that student rating forms are unreliable and invalid. Since many forms are "home-made," this perception is often reality. However, well-developed, professional student rating forms have been shown to be valid and reliable (Aleamoni, 1978; Aleamoni and Hexner, 1980; Costin, Greenough, and Menges, 1971; Marsh, 1984; Burdsal and Bardo, 1986.)
According to Arreola (1995), there have been more than 70 years of research on student ratings, and it continues to be one of the most heavily studied topics in faculty development literature. Space does not allow for a complete review of this research, however Cashin (1995) and Arreola (1995) both provide excellent summaries of the research literature that examines variables related to student ratings. In brief, they report:
Variables not consistently related to student ratings include:
Variables consistently related to student ratings:
Given these types of variables, it is critical to use a well-constructed student evaluation capable of taking these variables into account. In order to reduce subjectivity and to make this process fair and equitable, it is strongly recommended that institutional procedures for administering the end of course survey be adopted and carefully followed by all faculty members. The Course/Instructor Evaluation Questionnaire (CIEQ) and the Instructional Development and Effectiveness Assessment System (IDEA) instruments have been shown to be reliable, include national norms, and provide flexibility to faculty in the type of questions included. Self-developed instruments often lack some or all of these qualities (Arreola, 1995, p. 63).
As major participants in the educational process, students represent a valuable source of information for teaching assessment and improvement efforts. They are firsthand observers of course design, instructional delivery, class management, and teacher-student relationships -- key elements of effective teaching. Well-designed student rating forms can provide valid information concerning these critical components of teaching. A wide variety of other formative assessment techniques can also provide important feedback as well. When a teaching assessment system is implemented, it is essential that students understand how the process works and how the information they provide will be used.
Peer evaluation typically takes the form of classroom observations, review of instructional materials, and critiques of course design. These types of review have traditionally been used for both formative and summative assessment. Using colleagues as a source of assessment has both advantages and disadvantages. It is certainly advantageous that colleagues, who are usually from the same discipline, are familiar with departmental goals, programs, and priorities. However, such familiarity can also have negative consequences resulting from bias due to previous evaluations, personal relationships, and peer pressure (Arreola, 1995).
While commonly used, classroom visitations are not recommended for summative assessment unless certain guidelines are followed that result in satisfactory inter-rater reliability. The peer should be a part of an observation team whose members have been trained in the use of a standardized observation tool. Each peer must make multiple observations produce adequate inter-rater reliability. Centra (1975) reported that visitation ratings had less inter-rater reliability and a greater bias toward leniency than student ratings. He concludes that colleague visitation ratings are not typically reliable enough to use for personnel decisions. He suggests that reliability of raters can be improved by increasing the number of observers, increasing the number of visits per observer, and by providing special training in classroom observation technique. Each of these factors, however, can be prohibitively expensive in both faculty time and effort.
Other research findings have supported Centra's work. Braskamp, Brandenbury, and Ory (1984) reported that almost all peers rated their colleagues as excellent or good instructors. Furthermore, they reported that the relationship between observed instructor behavior and student learning was weak. Findings like these make the validity of peer ratings highly debatable, and researchers suggest that colleague visitation is more appropriate for faculty development activities than for summative evaluation. Finally, it has been suggested that colleagues can provide valuable, reliable, and valid assessments of course materials such as examinations, syllabi, course assignments, and instructional media (Kulik and McKeachie, 1975).
Based on a review of literature, Cohen and McKeachie (1980) concluded that peers can reliably evaluate:
Departments will need to select which areas (from the list above) peers should evaluate. Further, decisions on appropriate methods will need to be made. Recommendations about methods for effective peer review are included later in this document. Thus, it is suggested that peer evaluation can be used effectively in the assessment of content expertise of the instructor and course design. Instructional delivery should be assessed by peers for formative assessment linked to professional development activities, but not for summative evaluations used for personnel decisions.
It should be noted that peer evaluations may also involve professionals from other universities, industry, government or private agencies. While this method may need to be used selectively because of the costs and logistical demands involved, they provide an way to assure that the university's internal reviews are in line with professional standards outside the department.
According to the American Association for Higher Education (AAHE) Joint Task Force on Student Learning, the "whole college" of teachers, administrators, students, and career professionals must collaborate on the assessment and improvement of teaching and learning. Their draft position paper states that "It will take the whole college, and those affiliated with our institutions as members of the larger community, to ensure that we achieve our mission of increased higher learning" (1998).
Whetten and Whetten (1985, p. 35) declared that "...the most powerful predictor of organizational effectiveness in colleges and universities is administrative behavior. Results from research show that administrators are more important than environment, structure, institution type, and control in accounting for performance." However, the AAHE (1998) position paper contends that "Administrators frequently fail to focus resources on those functions at the college and university that are most effective in producing learning. Indeed, they may not even know what those functions are, having little information about results on the basis of which to plan and make decisions."
Thus, administrators must play a key role in creating and nurturing a "culture of teaching and learning" (Hutchings, 1996). There must be a long-term commitment on the part of upper administration to plan, support, and properly implement any proposed assessment plan if it is to be effective. The President, Provost, Deans, and Chairs all have essential roles in the assessment of teaching and learning.
The Role of the President and Provost
As Whetten and Whetten (1985) point out, administrative behavior is likely to result in improvement of teaching and learning if that behavior creates a culture that values teaching and learning. The founding and continued support of the Center for Teaching and Learning, as well as the Lilly Project are steps toward establishing such a culture at ISU. The continued role of the President and Provost in providing leadership for faculty evaluation of teaching should include enforcement of standards and criteria used by deans and chairs, as well as the review of deans and chairs with regard to their appropriate use of both developmental (formative) evaluation and personnel decisions (summative evaluations).
The Role of the Dean
The dean should ensure that faculty development and faculty evaluation are carried out in a fair, impartial, and systematic way at the departmental and school/college level within guidelines endorsed generally by the AAUP policy on academic freedom (ISU Handbook 3-1) and specifically by the AAUP's statement on teaching evaluation (1990). Because the dean is seldom a "firsthand observer" of teaching, s/he is not directly involved with the assessment of teaching. S/he ensures that teaching assessments follow the stated school mission and evaluation criteria and that evaluations follow explicit procedures. The dean should encourage a culture within the respective school/college that values teaching and learning.
The Role of the Chair
If an organization's effectiveness largely depends on administrative behavior, it follows that chairs, if for no other reason than sheer numbers, must play a crucial role in the assessment of teaching and learning (Gmelch and Miskin, 1993). It is estimated that chairs outnumber all other administrators by five to one (Murray, 1995). Since most of the choices within higher education occur at the departmental level, it is reasonable to ascertain that chairs are key players in evaluating and providing formative direction to faculty.
A primary role of the chair is to oversee departmental procedures for both summative and formative evaluation. The chair must work to build trust and integrity into these ongoing processes. As a source for evaluating teaching, the chair shares many of the same advantages and disadvantages of peers in the department. Generally, it is logistically impossible for chairs to visit classes of all faculty members frequently enough to achieve reliability. As with colleague visitations, occasional observations by the chair can be biased by personal relationships, common values, favored teaching methods, etc. Class visitations for professional development reasons may be appropriate, but they are problematic for summative purposes. In most cases, chairs should be able to adequately evaluate course design and content expertise. The chair, moreover, is in the best position to evaluate the teaching component of course management.
The principal characteristics for chairs who oversee teaching evaluations are impartiality, vigilant protection of faculty rights, and active support of teaching improvement. The chair should work with departmental personnel committees to ensure proper procedures are being followed in summative evaluations. Providing leadership and direction in this process is critical. The chair should try to facilitate the work of the committee. While the chair can provide data input into these evaluations, s/he should be limited to that which is observed "firsthand," The most important role a chair can play is providing leadership for collegial professional development activities within the department. This can prove to be a very challenging task if faculty perceive all evaluation activities to be linked with personnel decisions. Ideally, both summative and formative evaluations should facilitate professional development activities.
Self-evaluation of instruction is based on subjective evaluation of classroom performance by the individual professor. Self-evaluation that results in improvement in the classroom requires both reflections (self assessment) and input from other sources. There is a fine line between self-assessment and reflective practice. Self-assessment usually indicates that one is applying some measure to one's own work and passing judgment. However, engaging in self-assessment does not necessarily imply that one has fully reflected on one's work. Neither does reflective practice necessarily imply that one will perform an assessment.
College professors are primarily motivated by the intrinsic rewards of academic work, including intrinsic rewards gained from teaching...(they) want to feel competent and have a sense of self-determination...The need for self-determined competence also prompts individuals to scan the environment for feedback that informs them of the results of their performance -- producing a critical psychological state that an intrinsically motivated person continually seeks through work. (Paulsen and Feldman, 1995)
This continual feedback is what most faculty consider as a first (and sometimes only) step in self-evaluation. Without deeper reflection, it tends to confirm the faculty member's theories of instruction, which are often anecdotal and faulty (Paulsen and Feldman, 1995). This somewhat sustained reflection requires more structure, such as record keeping, and may lead to systematic change. There are a variety of techniques that are recommended for complete self-evaluation -- videotapes, journals, teaching portfolios (which can include such things as tapes and journals, collegial evaluations to name a few -- that are fully documented in the literature (Murray, 1995; Travis, 1995).
The strength of self-evaluation is that faculty are most likely to act on data collected by themselves. The various means of gathering information to assess performance relative to personal needs, goals, and objectives may be used as an ongoing program of continuous formative assessment. The weakness of the process is that most faculty tend to rate themselves higher than students do (Arreola, 1995). Centra (1993) reported that several studies have found low correlations of self-evaluations with those collected from other sources. Despite this research, Centra contended that self-ratings could be very useful for instructional improvement efforts and that self and student ratings tended to identify the same relative strengths and weaknesses. After a thorough review of literature, Brinko (1993, p. 577) pointed out that "feedback is more effective when information is gathered from oneself as well as from others."
Arreola (1995) identifies several conditions for effective use of self-evaluation. First, instructors need to understand precisely how the information is going to be used in order to feel confident and secure in the process. The exact form and substance of the self-evaluation would likely differ for faculty at different stages of their career. Faculty need to have skills in identifying goals and collecting appropriate data. Chairs should help facilitate this process, and the CTL should be a valuable resource in providing this type of training.
Effective self-evaluation requires self-reflection and self-assessment. Further, it should be used in conjunction with administrator/colleague observations and student feedback to determine if the instructor's self-perceptions of classroom performance are consistent with more objective observers. Differences in perceptions can be used to target weaknesses and strengths in teaching, the recognition of which is integral to the improvement of instructional effectiveness. In the best instances, faculty will reflect thoughtfully upon their performance and their students' outcomes, compare this self-reflection with more objective perceptions of colleagues and students, and modify their teaching as needed. A teacher should continually consider both the quality of the knowledge, values, and skills being conveyed as well as the pedagogical background and skill they bring to the delivery of this content.
Summary
Because teaching is a complex, multifaceted endeavor, it must be assessed from many viewpoints. An evaluation system that fails to gather data from qualified, first-hand sources would necessarily be compromised. Likewise, a system that restricts its view to only those components fairly measured by one data source would neglect vital information on teaching effectiveness. It is necessary to choose from multiple sources in order to receive the most appropriate data on each of the components of quality college teaching. Doing so eliminates obvious mismatches (where students would interpret a teacher's content expertise, for example), reduces the use of methods that are complicated, inefficient, or potentially unfair (such as summative use of classroom observations by peers), and emphasizes reliable procedures that can help reduce the arbitrary and confusing dimensions of teacher evaluation.
The summary of data sources outlined above describes the sources essential to a comprehensive evaluation system, while outlining some of the qualifications that are needed to keep the system consistent and fair. Clearly, different teaching conditions require some flexibility in selecting what combination of sources and what methods are most suited to particular disciplines or even courses. However, the general parameters described above offer a sound framework for assuring a flexible but consistent evaluation system for the campus. Figure 2 illustrates how these components can be viewed in reference to the multiple dimensions of effective teaching, allowing a complex system for measuring the various components of effective teaching appropriately.
Assessing the Components of Effective College Teaching
COMPONENTS / SOURCES Students Peers Admin Self
|
I. Teacher's Content Expertise
|
|
II. Course Design
|
|
III. Instructional Delivery
|
|
IV. Instructional Relationships
|
|
V. Course Management
|
|
VI. Professional Development
|
|
|
Introduction
An evaluation system must acknowledge the six components of effective college teaching described in part 2 as well as the various sources from which suitable data can be gathered summarized in part 3. The challenge is to develop a way in which this array of information can be brought together in a report that permits comparisons to specific standards. We recommend the following assessment process in order to combine the teaching components and multiple data sources described in the previous two sections. The process allows departments a range of flexibility in determining the value weighting of components and sources. At the same time, it asks departments to take some steps that permit consistent assessment reports for all its faculty. In general, this process involves four steps:
1. Discussing the academic unit's relative valuing of components that the pedagogical literature associates with effective teaching
2. Distributing these choices on a grid that incorporates the possible assessment sources
3. Deciding how much weight to give to each assessment source, in relation to a given component.
4. Determining what instruments should be applied.
These phases are explained in more detail in the remainder of this section.
Departments / units discuss their professional approaches and traditional obligations in the six areas:
1) the content knowledge teachers need to carry out their various assignments; 2) course designs appropriate to disciplinary integrity; 3) various approaches to delivery, from among the best practices in the field; 4) the different manners and degrees of accommodation by which teachers relate to students; 5) reasonable expectations for a well-managed course; and 6) professional teaching development
Although some of these attributes may be recorded in departmental by-laws and informal traditions, it is crucial for departments to review the full scope of faculty members' practices in tutorials, classrooms, labs, lecture halls, seminars, and anywhere else that directed learning occurs. CTL staff could help facilitate departmental exchanges, so that colleagues in a unit would debate and decide for themselves, within their discipline, the relative significance of the six components in their discipline.
The participants must distribute their agreed-upon component interpretations into an array of appropriately matched sources. This relationship has been called an Assessment Matrix (Arreola, 1995). A blank grid is shown in Figure 3.
Each row of the grid represents a component of effective instruction. Each column of the grid shows an assessment source. The formatted matrix displays cells for reporting standardized scores determined by interpreted data that have been collected about that teaching component. Since not all sources are appropriate for assessing all of the teaching components, blanks will appear in the final grid. Also, the deliberative process allows departments, as well as individuals (if the unit so decides), to weight the value of cells according to their own commitments. However, no unit should be able to ignore any of the six components nor shall their weights fall outside the range adopted by the university.
|
Components |
|
|
|
|
|
|
I. |
Content Knowledge |
||||
|
II. |
Course Design |
||||
|
III. |
Instructional Delivery |
||||
|
IV. |
Instructional Relationships |
||||
|
V. |
Course Management |
||||
|
VI. |
Professional Development |
Phase 3
Deciding Source Weights
The matrix also allows each teaching component to be rated according to some or all of the data sources. Each source may be weighted differently to reflect the degree to which that data source should be utilized in the evaluation of that teaching component. Figure 5 provides a list of recommended weight ranges for each source.
|
Components |
|
|
|
|
|
Content Knowledge |
||||
|
Course Design |
||||
|
Instructional Delivery |
||||
|
Instructional Relationships |
||||
|
Course Management |
||||
|
Professional Development |
||||
|
TOTAL WEIGHTS |
|
|
|
|
With these two parameters set by general debate within the unit, it is possible too use the matrix to give a value for each cell. The percentage that each source contributes to the evaluation of the teaching component appears in the small boxes and add up to 100% for that row. In the sample provided in Figure 6 the crossed-out cells represent sources that are not deemed appropriate in evaluating that teaching component.
The total value allocated to the sources of evaluation can be seen by multiplying the percentages and summing down the columns. For example, the student source accounts for 35% of the evaluation [.15(.0) + .20(.20) + .20(.70) + .20(.85) + .05(0) + .20(0) = .35]. With these weights, the sources have the percentage impact listed in Figure 6. These final weights should fall within the range indicated in Figure 5.
SAMPLE APPLICATION OF SOURCE WEIGHTS
|
Components |
|
|
|
|
|
||||||||||||||||
|
Content Knowledge |
|
70% |
|
|
|
||||||||||||||||
|
Course Design |
|
|
|
|
|
||||||||||||||||
|
Instructional Delivery |
|
|
|
|
|
||||||||||||||||
|
Instructional Relationships |
|
|
|
|
|
||||||||||||||||
|
Course Management |
|
|
|
|
|
||||||||||||||||
|
Professional Development |
|
|
|
|
|
||||||||||||||||
|
Total Source Weights |
|
|
|
|
|
Figure 6: The Assessment Matrix with Source Component Weights
Adding Scores to the Matrix
With values adjusted according to the individual's and the unit's commitments, the matrix can serve as a tool for developing an overall measure of a faculty member's teaching performance. An evaluation process can place a rating in each cell to represent an interpretation of how well a teacher has performed in this component, according to that source. When the ratings in each cell are based on the same scale they can be multiplied by the cell's value factor to produce an overall score.
Consider, for example, that each cell is rated using a 6-point system. In this scale, a score of 1 or 2 would reflect below average performance; 3 or 4 would represent average performance; and 5 or 6 would indicate above average performance. Student responses on the standard instrument will yield a set of scores where predetermined thresholds would translate student rankings into this scale. Peers, administrators, and self assessments would be established to translate the data gathered into this rating system. After a score from 1 to 6 is determined for each cell, the score is multiplied by the cell percentage and by the row percentage. These numbers are then summed to give an overall score for teaching. As an example, consider the professor whose scores were as recorded in
Figure 7.
|
Components |
|
|
|
|
|
|
||||||||||||||||
|
Content Knowledge |
|
70% |
|
|
|
|
||||||||||||||||
|
Course Design |
|
|
|
|
|
|
||||||||||||||||
|
Instructional Delivery |
|
|
|
|
|
|
||||||||||||||||
|
Instructional Relationships |
|
|
|
|
|
|
||||||||||||||||
|
Course Management |
|
|
|
|