Essays on Teaching Excellence

Toward the Best in the Academy

 

A Publication of The Professional & Organizational Development Network in Higher Education

 

Vol. 19, No. 8, 2008-2009

 

The U   seful, Sensible, No-Frills Departmental Assessment Plan

Barbara E. Walvoord, University of Notre Dame

 

Academic departments from physics to philosophy to physical therapy face new demands for Òassessment of student learning.Ó  ItÕs hard to argue against the basic idea of assessment: when a department invests time and resources trying to nurture student learning, it should ask itself: Are they learning?  Yet departments may also fear that assessment will require them to dumb-down their teaching; use standardized tests; teach alike; or compromise academic freedom. Every department wonders how it will find the time and resources for one more thing.

 

This essay suggests a simple, sustainable, and useful departmental assessment plan that capitalizes on what departments are already doing or should be doing, that can help improve student learning, and that can meet the requirements of accreditors. The basic plan includes three elements that are common to the requirements of virtually all accreditors, both regional and disciplinary:

 

  1. Written learning goals (sometimes called objectives or outcomes) phrased: ÒWhen students complete this program of study, we want them to be able toÉ.Ó
  2. Measures that indicate how well the learning goals are being met (These measures need not dumb-down learning or use standardized tests.  They can be based on classroom assignments and exams. They can seek indications about studentsÕ achievement of ineffable goals like creativity, ethical sensibility, or ability to work well in diverse groups.)
  3. Ways of using the information for improvement (Òclosing the feedback loopÓ)

 

First, the department should construct written learning goals for each of its distinct courses of study, e.g., certificate program, major, masterÕs, and doctorate. Different tracks (e.g., music history and music performance) may require somewhat different goals. It is important that these goals include the departmentÕs highest aspirations.  For example, a swine management department listed a number of very practical learning goals such as identifying and treating common swine diseases, developing a financial plan for a swine operation, and so on.  But its ultimate goal was Òappreciate the pig!Ó  Departments in a religiously-affiliated institution wanted students to develop Òsensitivity to injustice.Ó  You canÕt ÒproveÓ learning in these areas, but you can get indications about whether students are developing in the ways you wish, and if you donÕt articulate and share your highest goals, you risk undermining your most important mission.

 

Next, the department should institute an annual meeting of at least two hours, in which it reviews one of its programs (for example, the undergraduate major). Hold the meeting even if you think you have no measurements or evidence, and even if you have only a partial or imperfect list of learning goals.

 

The purposes of the meeting are (1) to consider whatever evidence you have about how well students are meeting the learning goals; and (2) to generate one action item, for which you assign responsibility and a timeline.  You should allow no other concerns on the agenda.  This is the time when the department sets aside all the other concerns that crowd its time, and steps back from the daily race to ask, ÒHow well are we doing?Ó and ÒWithin our limits of time and resources, is there one action we could take that might improve student learning?Ó

              

Once the meeting is established, what are the minimum types of evidence that might be most helpful in defining an action item? The basic no-frills plan might have two types of evidence:

 

  1. An evaluation of the quality of student work as students complete the program.  This can be a sample of student classroom work in course(s) taken by students at their end of their course of study; an evaluation of an ultimate clinical or internship experience; a standardized exam if relevant; a licensure exam; or a qualifying exam and theses for graduate degrees.  In programs with many students, a sample of student work can be used.
  2. Response from students about what they thought they learned and about their perception of the programÕs effectiveness for their learning.

 

Additional types of evidence might include alumni surveys, employer/industry feedback, studentsÕ job or graduate school placement rates, or, especially in graduate programs, awards and/or publications by students.  But in most cases, it is better to have the first two types of evidence working well than to proliferate assessment measures beyond what the department can fund, sustain, or effectively use.

 

The most basic assessment plan can be illustrated by a political science department that was highly successful: it was rapidly increasing its number of majors; it was known throughout the university for the high quality of its teaching; and it maintained a high rate of publication and professional activity.  The smart, effective faculty members of this department hated Òassessment.Ó  They viewed it as an attempt to diminish the high goals they held for their students, as an attack upon their autonomy, and as a foolish waste of time.  They did agree, however, that despite demanding schedules, it would be helpful to sit down for two hours once a year and examine evidence of student learning in one of their programs. 

 

For the first year, they chose the undergraduate major.  During the meeting, they brought no rubric scores (most of them hated rubrics) and no written preparation.  Instead, each faculty member who taught a senior capstone course briefly spoke about two strengths and two weaknesses that she or he had observed in senior student research projects.  These were listed on the board.  One weakness that a number of faculty mentioned was that as students began their senior research projects, they did not know well enough how to frame a question for inquiry in the discipline.  The department decided to work on that item.  They discussed where in the curriculum students were taught to frame research questions and given practice and feedback in doing so.  A committee was designated to suggest where and how this aspect could be strengthened in the curriculum.  Changes to the earlier courses then provided more instruction and practice in constructing research questions.  Now the department waits to see whether future cohorts of students seem to be better prepared.

 

At the end of the annual meeting, the department should ask itself what additional or better information it might want to collect in future years.  The political science faculty noted the lack of student input for their data, and they wanted to know whether students experienced disjuncture between their earlier training and their senior research and if so, what students might suggest as remedies.  It was proposed that each teacher of a capstone course, during the first week in May, would administer a 3-question survey to seniors enrolled in the course. The survey would ask students: (1) what aspects of the senior research project they had found most difficult; (2) what earlier training in the department had best prepared them for these difficult areas; and (3) what their suggestions were about how earlier work might better have prepared them.  Several faculty were concerned that the survey would take more time and effort than it was worth, so it was decided to administer the survey only in the classes of a few volunteer faculty, as a pilot, to determine whether reliable and useful information could be gathered. The department assigned responsibility for constructing, administering, and analyzing results of this pilot survey.

 

As this story suggests, an action item chosen in one year may take more than a year to fully implement.  In that case, the annual meeting is devoted to tracking progress and planning further steps on a continuing action item.  As it feels ready, the department may also begin work on another program. For example, the political science department might gather its graduate faculty for a review of its Ph.D. program. Some departments may prefer to do part of their review of learning through a committee structure and bring reports and recommendations to the department as a whole.

 

At the assessment meetings, the department should take written minutes, which can serve as a reference for their own future actions, and which, as needed, can be the basis of reports to the universityÕs assessment committee and accrediting bodies.  The minutes provide the data to demonstrate that effective assessment is taking place.

 

The key is to institute the annual assessment meeting immediately, no matter how incomplete or inadequate the assessment data are.  Use the data available to generate an action item, and also discuss how you want to improve the quality of the data.  The annual meeting provides an ongoing structure that most departments can manage, and that helps the department step back, consider the big picture, bring in evidence of student learning, and make good decisions about how to help their students learn more effectively. Assessment Clear and Simple (Walvoord, 2004) gives more detail and shows how to write up such plans for accreditation.

 

Resources

 

Banta, T. W., ed. Assessment Update Collections.  Series of booklets containing articles compiled from the journal Assessment Update.  Jossey-Bass, various dates. Practical, short, on-the-ground descriptions of assessment practices and principles.

 

Banta, T. W., Lund, J. P., Black, K. E., and Oblander, F. W. (1996). Assessment in Practice: Putting Principles to Work on College Campuses. San Francisco: Jossey-Bass. Contains 82 case studies of best practice, each in 2-3 pages.  Though now more than ten years old, still a wealth of practical ideas. 350 pages.

 

Palomba, C. A., and Banta, T.W., eds. (2001). Assessing Student Competence in Accredited Disciplines: Pioneering Approaches to Assessment in Higher Education. Sterling, VA: Stylus Publishing, LLC.  At 350 pages, it gives more extensive details on many of the subjects covered in this volume, and it is organized as a manual of advice to practitioners.  The single most useful reference as an accompaniment to WalvoordÕs short guide.

 

Suskie, L. (2007). Assessing Student Learning: A Common Sense Guide. Jossey-Bass, 2007. A 300-page guide with many good ideas and illustrations.

 

Walvoord, B. E. (2004). Assessment Clear and Simple: A Practical Guide for Institutions, Departments, and General Education. Jossey-Bass.  In 79 pages plus appendices, I try to give institutions, departments, and gen ed programs all the basics they will need.

 

Walvoord, B. E., and Anderson, V. J. (1998). Effective Grading: A Tool for Learning and Assessment. San Francisco: Jossey-Bass. Shows how the classroom grading process can be enhanced and how it can be used for assessment. Helps classroom teachers make the grading process fair, time-efficient, and conducive to learning. Contains a case study of how a community college used the grading process for general-education assessment.

 

Barbara Walvoord (Ph.D., University of Iowa) is Emerita Professor of English, University of Notre Dame.

 

______________________________________________________________

Essays on Teaching Excellence

Editor: Elizabeth OÕConnor Chandler, Director

Center for Teaching & Learning

University of Chicago

echandle@uchicago.edu

 

Subscriptions: Member, $100, or nonmember $120 annually (campus-wide reproduction rights); Individual, $15 annually.  Teaching Excellence is published eight times annually.  To order, send check or P.O. to POD Network at the address below, or contact us for further information.

 

The POD Network facilitates the exchange of information and ideas, the development of professional skills, the exploration and debate of educational issues, and the sharing of expertise and resources.  For further information, contact:

 

POD Network in Higher Education

P. O. Box 3318

Nederland, CO 80466 U. S. A.

 

303.258.9521 (voice)

303.258.7377 (fax)

podnetwork@podweb.org

 

The posting and use of this publication on an institution's WWW server is covered by an End User License Agreement (EULA). By the terms of this agreement, the essay must be located on a secure server that limits access to your institution's students, faculty, or staff. Use beyond your institutional boundaries, including the linking of this page to any site other than this one, requires the express permission of the POD Network.

 

This publication is part of an 8-part series of essays originally published by The Professional & Organizational Development Network in Higher Education. For more information about the POD Network, visit  www.podnetwork.org.