Evaluation

Grading & Proctoring

Grading

Getting back a test is really stressful -- especially when the average is 30 and I got a 20.
-- Junior Business Major

Be fair and reasonable and maintain grading standards which you can defend if challenged. In your syllabus, be very clear about your grading policies and criteria. It is also helpful to specify the types of tests, quizzes, and assignments along with their point distributions. You may also want to include dates of tests and due dates of projects and papers. The more details you give about your grading criteria, the fewer hassles you will receive from students throughout the semester.

GENERAL GRADING TIPS

Construct answer key
Assign points in advance

SPECIFIC GRADING TIPS

Papers -- Good feedback enhances learning
Essay exams -- One question at a time
Problem sets, short answer, multiple choice
Analyze answers to determine question clarity

GENERAL GRADING TIPS

CONSTRUCT THE ANSWER KEY PRIOR TO GIVING THE TEST.

The best way of doing this is by taking the test yourself (as stated in previous section).

ASSIGN POINTS AND PARTIAL CREDIT BASED ON YOUR KEY PRIOR to looking at student tests. Include the point distribution on a test so that students can budget their time accordingly.

SPECIFIC GRADING TIPS

Papers

MAKE WRITTEN COMMENTS. There is nothing more arbitrary to a student than a paper passed back with just a grade on it and either no comments or just perfunctory ones. When grading papers, write comments judiciously and legibly. Do not obliterate the text: use the back or append a note. Try to say enough so that the student has a reasonably good chance of doing better next time. If you find that you are saying similar things to several students, prepare a handout on whatever the students are stumbling over; for example, how to write a review, or how to develop an argument.
JUDGE ON CONTENT, ORGANIZATION, AND PRESENTATION. Often it is useful to the student if you evaluate the paper in each of these areas and assign a mark on the basis of some combination of these factors. Also, some teachers find it helpful to ask students to write papers twice. The first draft is submitted and subjected to constructive criticism on both content and style. The second draft is graded and usually shows some kind of improvement that is quite satisfying to student and teacher alike.
WORK TO MAINTAIN OBJECTIVITY. Both papers and essay exams involve a lot of subjective judgment. The following suggestions may help with the problem of maintaining consistency. You are more likely to be stringent with the first few papers you read than with the rest, and less likely to be careful about comments and such when you are tired. To avoid such problems, read a few papers before you actually start grading to get an idea of the range of quality, and stop grading when you get tired or start to undergo personality changes due to boredom. When you start again, read over the last couple of papers you graded to make sure you were fair.

Essay Exams

PLAN WAYS TO DIVIDE GRADING WITH OTHER TAs. Usually the problem here is how to wade through all those booklets while remaining both consistent and sane. When there are a number of TAs assigned to a course, the course supervisor can divide the workload. If each TA has had a section and all of you have covered the same basic material, then you may prefer to mark the exams of the students in your own section. The problem here, of course, is that objectivity may be hard to achieve since you may feel close to, or even partial toward, your own students. Grading question-by-question rather than student-by-student may help. This will allow you to give credit for material that you presented in section and it will give you feedback on whether the ideas you have emphasized have actually registered. At the same time, you should be guided by a grading standard that has been mutually agreed upon by all TAs and the supervising faculty.

If each TA has dealt with specialized topics in lecture and section, then it is probably better to split the exam questions up so that each TA grades questions about what he or she taught. Dividing the exam questions this way ensures that each question will be marked consistently across the class. However, reading 200 answers to the same question one after the other has its drawbacks: it can affect your mental health and your grading range. This is less likely if you pace yourself, mark questions that you are interested in, and switch questions every once in a while.

GET TOGETHER TO RESOLVE DIFFICULTIES. When the exams have been marked, get together with the other TAs to discuss and resolve any problems you have encountered. Then add up the total scores, check your addition (this saves a lot of trouble later), and plot the distribution. Now you are ready to present the results to the supervising professor or to assign the grades yourself.

Problem Sets, Short Answer Questions, and Multiple Choice

DIVIDE EXAM GRADING.Although these tests usually take longer to make up than the others, they are easier to grade. But problems can still arise. For the same reasons as those mentioned above, it is often a good idea to divide the exam questions among the TAs. Consistency is more likely and deviations easier to spot.
BE PREPARED FOR ALTERNATIVE ANSWERS. You may think that you have written the perfect question with only one correct answer, but you must always be prepared for alternative answers. In the case of multiple choice questions, for example, if the students are doing worse than chance on a particular question, it is likely that the question was poorly worded. In this case you must either give credit for more than one answer or toss the question out (for example, by giving everyone credit).

Grading can be a constructive process both for you and your students. It can give them the opportunity to improve their knowledge and writing skills, and it can give you feedback on your teaching and evaluation methods. By being consistent and fair, you can minimize the inevitably unpleasant aspects of passing judgment on someone's efforts.

Proctoring Duties & Tips (prepared by Amy Cass, Kelly McHugh & Justin Alms 8/15/2006)

A proctor is an individual who is responsible for monitoring students while taking exams and ensuring proper exam protocol and etiquette.

BEFORE AGREEING TO PROCTOR AN EXAM:

Know how long the test will last. Make sure you are available to proctor the entire exam.
Ask the course instructor what is expected of you and what policies and protocol you will have to enforce.

BEFORE THE EXAM IS DISTRIBUTED:

Some courses might ask you to identify each student by photo ID.
Make students aware of the test policies and protocol. Verbally restating these policies at the beginning of the exam will help avoid misunderstandings.
If closed book exam, verify that PDA computers, reference books, notes, backpacks, purses, hats, etc are not opened or used in the test room.
Affirm that the test environment is a quiet area with adequate space and comfort for taking the exam.
Try to space student out in effort to reduce temptation to cheat and to give yourself enough room to move around the class.
Check with the course instructor to make sure testing accommodations have been made for students with documented disabilities.

DURING THE EXAM :

Try not to have the students leave the room except for emergencies.
Do not simply read and sit at the front of the classroom. Walk up and down the aisles and scan the room to discourage cheating.
If you suspect a student is cheating, first watch them closely and take notes. Then hover near them a minute. This discreet gesture may warn the student and discourage this behavior. Try not to publicly embarrass the student.
Attempt to answer student questions in a minimally disruptive way. In a small or cramped room, having students come to the front of the room to ask the proctor questions may be best.
Keep students aware of how much time they have left by either keeping a 'countdown' on the board or giving verbal warning such as "the exam is about halfway over" or "there are 10 minutes left."
For large enrollment classes, when the allotted time for the exam has ended, tell students to finish writing their last sentence and pass the exam to one side of the room for collection.

What is the "Truth"?

A long time ago a king complained to Nasrudin, "My people do not always tell me the truth. This bothers me."
Nasrudin answered. "It does not matter whether something is absolutely true or not. What matters is that something is true in relation to other things."

The king was not pleased. "This is just one of your tricks. A thing is true or it is not true."

The king thought of a plan to make his people tell him the truth. He had a gallows built just inside the city gates. He told the heralds to announce, "If persons want to enter the city, they must first answer a question asked by the Captain of the King's Guards. If the answer is not truth, the person will be hanged."

Nasrudin came forward. "I want to enter the city."

"Why do you come?" asked the Captain.

"To be hanged," answered Nasrudin.

"This is not true," said the Captain.

"If I am not telling the truth, you must hang me," explained Nasrudin.

"But, this would make it the truth," said the Captain. "I cannot hang you if you tell the truth."

"You must decide, which truth is the real truth," replied Nasrudin.

--Iranian fable

You, or your professor, have written the perfect exam question. You have constructed your answer key and determined the points for full and partial credit. And then you begin grading the papers. By and large, you get what you expect -- more or less, better and worse than the solution you had outlined. However, one student has an original and rational answer from a different perspective. We encourage you to consider a different "truth." Fortunately, unlike the captain, you need not face this dilemma alone; talk with fellow TAs and your supervising professor about the answer and how to grade it.

Testing

The one question students always ask is, "Will this be on the test?" Can't students realize which material is important and likely to be asked on a test?
--Graduate TA

Testing and grading are necessary ingredients of teaching and learning. Tests give teachers a way of measuring student learning. Grading gives students feedback on their academic performance.
(See the section on University Policies & Guidelines on academic dishonesty.)

TESTING

What to test: Learning objectives, major topics, assignments
How to test: Frequently, cumulatively, clearly
Types of questions: Multiple choice, short answer, essay, problem sets, completions, matching
Total test: Knowledge requirements, level of thought, cognitive performance levels

WHAT TO TEST

IMPORTANT LEARNING OBJECTIVES AND COURSE GOALS. Your exams should test learning objectives and instructional goals that you have emphasized to students throughout the course.
If your course goal is to analyze a marketing situation and recommend alternative solutions, don't ask students to perform simple memorization, plug-and-chug problems. You will give a mixed message and students will learn to just memorize, not to analyze.
MAJOR TOPICS OF DISCUSSION. Test topics that were discussed in detail during class.
You generally devote class time to discussion of material only if you consider it important to your course goals.
ASSIGNED OUT-OF-CLASS READINGS OR PROJECTS. Testing out-of-class readings and projects that are important but have not been thoroughly covered in class emphasizes their importance to students.

HOW TO TEST

TEST FREQUENTLY. Research has shown that frequent testing with a quick turn-around on feedback enhances student learning and allows students to learn from their mistakes.

TEST CUMULATIVELY. Frequent (shorter) tests and quizzes give students good feedback, but students also need to be given tests in larger, more cumulative chunks. Students who have to study for cumulative tests remember the material longer and are better able to integrate the new knowledge with old.

WRITE EXPLICIT DIRECTIONS. Go over the directions with students prior to starting the test.

CLEARLY STATE THE QUESTIONS. Remove as many sources of ambiguity as you can and be careful not to introduce new or difficult terminology into your test questions.

Sometimes what is clear to you may not be clear to your students, so you might ask a colleague to read your questions. During the test, if a question statement seems confusing to some students, clarify the question for the whole class.

TAKE THE TEST YOURSELF. In addition to checking test construction, this technique helps you to generate your answer key.

This is an effective way to check for typos or problem questions. Also time yourself and check for proper length of test. A general guideline is that for a 50 minute test, you should need no longer than 20 minutes.

TYPES OF TEST QUESTIONS

MULTIPLE CHOICE. It is difficult to construct good questions, but they are quicker to grade. Appropriate for measuring both recall of information and discipline vocabulary.

SHORT ANSWER. These questions are easy to grade, but questions need to be clearly worded. It may be necessary to accept alternatively worded answers. This form of question evaluates information recall and analytic skills.

ESSAY. Such tests are hardest to grade, but have the best educational value. These questions challenge students to organize, evaluate, and integrate ideas, as well as demonstrate logical thinking and writing skills.

NUMERICAL OR LOGICAL PROBLEMS. Grading difficulty depends on the complexity of the problem. Problems test understanding of the material and the ability to apply it.

COMPLETION QUESTIONS. It may be necessary to accept reasonable alternative answers. Completion questions test for recognition of key terms and concepts.

MATCHING. Give enough answers so students can't guess by elimination. This form of test evaluates recognition of relationships between pairs of words or between words and definitions.

TOTAL TEST

CLASSIFY QUESTIONS ACCORDING TO KNOWLEDGE REQUIREMENTS. You may find that some questions involve only rote memorization and that with a small change, you can produce a question which requires a higher level of thought.

	Knowledge Requirement	Level of Thought
Lower	Memory	Recall or recognition of information
	Comprehension	Deciphering symbols or technical language
	Application	Discovery of relationships among facts, generalizations, definitions, values and skills
	Analysis	Breaking down a problem with conscious use of defined forms of thinking
	Synthesis	Solving problems by creative thinking. Integrating concepts into new ideas
Higher	Evaluation	Judging according to rational standards

This system, developed by Benjamin Bloom, is one of the most widely used and is based on the notion of a hierarchy of thought processes. Each category requires more complex thinking than the one preceding it, and also builds on or incorporates the preceding types of thought in order to proceed to the "higher" levels. This in itself suggests a teaching strategy: In the early stages of a topic, there should be more emphasis on lower, more basic thinking processes; as this is mastered, students will be able to move up the hierarchy toward more complex ways of dealing with the material.

Descriptions of the Major Categories in the Cognitive Domain

Memorization is defined as the remembering of previously learned material. This may involve the recall of a wide range of material, from specific facts to complete theories, but all that is required is the recall of the appropriate information. Memorization represents the lowest level of learning outcomes.
Sample Performance Statements
- State names of 4 European rulers during World War I.
- Describe how to titrate.
- Define deviance.
Comprehension is defined as the ability to grasp the meaning of material. This may be shown by translating material from one form to another (words to numbers), by interpreting material (explaining or summarizing), and by estimating future trends (predicting consequences or effects). These learning outcomes go one step beyond the simple remembering of material, and represent the lowest level of understanding.
Sample Performance Statements
- Summarize Plato's views on good and evil.
- Describe the composition of the nucleus of an isotope from its position in the periodic table.
- Restate the concept of the superego in your own words.
Application refers to the ability to use learned material in new and concrete situations. This may include the application of rules, methods, concepts, principles, laws, and theories. Learning outcomes in this area require a higher level of understanding than those under comprehension.
Sample Performance Statements
- Calculate the volume of a sample of gas under any conditions.
- Write a coherent paragraph, as given by the basic rules, to express an idea.
- On the basis of a description, classify a society as embodying organic or mechanical solidarity.
Analysis refers to the ability to break down material into component parts so that the organizational structure may be understood. This may include the identification of the parts, analysis of the relationships between parts, and/or recognition of the organizational principles involved. Learning outcomes here represent a higher intellectual level than comprehension and application because they require an understanding of both the content and the structural form of the material.
Sample Performance Statements
- Identify the ions of an unknown using a chemical qualitative analysis scheme.
- Identify the key themes in a short story and show how they are interrelated in producing the outcome.
- Identify several central values in a non-western cultural system and show how they are expressed in economic and religious activities.
Synthesis refers to the ability to put parts together to form a new whole. This may involve the production of a unique communication (theme or speech), a plan of operations (research proposal), or a set of abstract relations (scheme for classifying information). Learning outcomes in this area stress creative behaviors, with major emphasis on the formulation of new patterns or structures.
Sample Performance Statements
- Devise a scheme for the synthesis of an organic compound from a given starting material.
- Working from original documents, construct an account of how European intervention affected the Civil War.
- Design an experiment for establishing in what ways certain members of an ecological system are dependent on one another.
Evaluation is concerned with the critical ability to judge the value of material (a statement, novel, poem, research report) for a given purpose. The judgments are to be based on definite criteria. These may be based on internal criteria (organization) or external criteria (relevance to the purpose) and the student may determine the criteria or be given them. Learning outcomes in this area are highest in the cognitive hierarchy because they contain elements of all the other categories, plus conscious value judgments based on clearly defined criteria.
Sample Performance Statements
- Choose the most efficient scheme of synthesis for an organic compound from among several schemes.
- Evaluate the quality of several poems on the basis of internal structure and harmony of imagery.
- Compare the adequacy of 3 theories of personality in explaining a specified type of behavior.

Testing Yourself - Student Feedback

There are several strategies for getting feedback so that the students who give it will benefit from it. Whatever format you choose, ask your students to be specific. "This course is the best ever!" can make you feel really good, and "This course stinks!" can make you feel really bad. But neither comment is especially helpful in maintaining excellence or improving instruction.

FEEDBACK

Informal Strategies:

Chain letter
Dial-a-prof
Dear Professor
Feedback form
One Minute Paper

Formal Strategies:

Student Rating Instruments
IDEA Form

INFORMAL STRATEGIES

CHAIN LETTER: Send a sheet of paper (or several, for larger classes) around the room with a few survey questions at the top and space for comments. Each student puts a tally mark to indicate responses to questions, then writes comments. As the sheets circulate, students respond to each other's comments, creating revealing dialogue for the professor.
DIAL-A-PROF: An answering machine on your telephone allows students to ask questions any time they come up. If there is a flood of calls on the same subject, you know you have a "hot" topic that needs clarification in class. E-Mail also works well for this purpose.
DEAR PROFESSOR LETTERS: About three weeks into the term, ask students to write you a letter (anonymously, if they prefer), telling you (for example) what they find most interesting about the course so far, what questions they hope the course will answer, any problems they are having with the readings, or a concept they are having trouble understanding. You may also ask them to comment on materials and assignments, solicit what you could do to help and request that they think about what they can do themselves to improve their learning.
FEEDBACK FORM: Take a few minutes at the end of a class, after the third week and perhaps again at mid-semester, to have students complete a feedback form. It is best if you leave the room. Ask one of the students to collect the feedback sheets and return them to your office in a provided envelope. Make sure that you stress to your students that their comments will have no bearing on their grades and that you want to find out how you can best meet their learning needs. A copy of an informal feedback form is at the end of this section. You may copy this form and distribute it to your students.
ONE MINUTE PAPER: At the end of any class ask students to write down brief answers to two questions: (1) What is the main point you learned in class today? (2) What is the major unanswered question you leave class with today?

Having students do a one minute paper regularly in class has several purposes:

It focuses student attention on the "big idea" of the session.
It focuses student attention on unclear points.
It informs you of student questions and general misunderstandings.
It builds continuity over time.

Include several examples of what students say they like about the class.
Choose one or more suggestions for change that you also believe will improve the course and that you feel you can implement. Also, let the students know how you plan to do that.
Choose one or more suggestions that, for practical or other considerations, cannot be implemented. Inform the students of the reasons.

One final note:

There are other forms of feedback, such as a class interview facilitated by a teaching consultant, a videotape review, or analyses of student work which require the help of the Center or a colleague. If you are interested in one of these, or if you have questions about the others listed here, call CTE at 831-2027.

FORMAL STRATEGIES

STUDENT RATING INSTRUMENTS. All departments require course evaluations to be conducted at the end of the semester. About three weeks before the last class, ask about details for administering your department's evaluation instrument.

Some departments will send someone to distribute and collect the forms from students, others will expect you to find a student to do this.

It is important that you do not stay in the room while the students complete the ratings. You should also make it very clear to students that you will not see the results until after grades are in. Their feedback will not influence their grades in any way.

IDEA. The Center for Teaching Effectiveness (CTE) makes available, at no cost to faculty and TAs, the IDEA (Individual Development and Effectiveness Assessment), a nationally-normed student rating instrument. IDEA is based on student progress on instructional goals and controls for class size and student motivation levels (two factors that influence ratings of instruction). IDEA provides a diagnostic section for faculty development and space for additional questions designed by the instructor.

If you are interested in using the IDEA, call 831-2027 for information. If you are unfamiliar with IDEA, we will be happy to talk to you and provide further information on the instrument and have you look at a sample packet.

Sample Student Feedback Form

1. How satisfied are you with the class (discussion section/lab section) so far this semester? (Circle one.)

1                                2                           3                                    4                            5
    Very                                                                                                                     Very
Dissatisfied                                                                                                             Satisfied

2. What helps you to learn in this class (section/lab)?

3. What hinders your learning in this class (section/lab)?

4. What could you do for yourself to make this class (section/lab) go better for you?

5. What specific questions/content are still unclear to you?

6. Other comments.

Excellence in Teaching Awards

The University of Delaware Faculty Senate annually awards the Excellence in Teaching Awards that give recognition to exceptional performance in teaching. Awards for both faculty members and graduate teaching assistants are made each year. The award winners are chosen by the following procedure: The Faculty Senate Committee on Student and Faculty Honors solicits nominations from the students in February. Nomination forms are available at the Faculty Senate Office, department offices, and the Student Center. The nominees should be highly knowledgeable about their subject matter and intellectually challenging; be deeply committed to teaching and to students; be skilled in communicating with students and finding ways of promoting learning and understanding; and have a positive and lasting impact upon students and their intellectual development. The nominations are submitted to the Faculty Senate by early March for review. The Award Recipients are announced and recognized during ceremonies on Honors Day in May.

On the basis of letters of support from students, each TA reward recipient received $1,000 from funds made available by the University of Delaware Alumni Association. Consult your department for further information about the Excellence in Teaching Award.