Public university leaders and state legislators have gone assessment-crazy in recent years to answer the question: “What did students learn in the course or their degree program?” My question is different: “Does it make sense to assess student learning using methods that most students perform poorly on?” It seems to me that inaccurate assessments of learning are generated if the assessment method used is one in which most students are weak.
There are many books that describe the various forms of assessment that faculty can use to evaluate classroom learning. The basic assumption is that all assessment methods , if used correctly, are good ones. What they are missing is the students’ view of assessment methods. For example:
- Do students like or dislike the assessment method?
- Does the assessment method work to students’ strengths or weaknesses?
- Does the assessment method help the student learn the most they can in a course?
- Can assessment be customized to individual students’ strengths?
I want to know the answer to these questions. So, earlier this semester, I surveyed the students in the four undergraduate and graduate courses that I teach. I asked two simple questions, in relation to their undergraduate education experience (n = 79):
- What three mode(s) of learning assessment are most beneficial for you personally, to help you learn the most in a course?
- What three mode(s) of learning assessment do you hate the most?
The results are shown below, and they are remarkable.
This data, which I plan to continue to collect in future semesters, leads me to the following preliminary observations:
1) These assessment methods are typically imported from K-12 education and carry over into graduate education. What it reveals is an indiscriminate application of assessment methods: “My teachers used XYZ assessment method on me, so I’m going to use it on my students.” So much for critical thinking.
2) The continued use of assessment methods that do not strongly contribute to individual students’ learning experience (or greatly detract from it) constitute a repetitive error by faculty. Faculty should not make repetitive errors.
3) Some assessments methods are clearly more contributive to learning than others, for most students. This suggests assessment methods that detract from learning should be abandoned: pop quizzes, team projects, and in-class tests. Pop quizzes and in-class tests consume valuable face-to-face class time that is better spent imparting tacit knowledge, while team projects perpetually suffer from free-rider problems that greatly annoy students.
4) The majority of students will learn the most they can from a course using these just three assessment methods: small weekly assignments, practical problem-solving (inclusive of structured problem-solving processes such as PDCA, formal root cause analysis, A3 reports, etc.), and individual projects.
5) Assessment can be customized to individual student’s strengths; e.g. small weekly assignments or practical problem solving or individual project. Assessment for the few students who like to write could be a research paper. Or, students can pick the method(s) by which they would like to be assessed from a menu of 4 to 6 items. This would demonstrate that professor respects students (congruent with the “Respect for People” principle) and their individual learning capabilities, interests, and differences. This changes the paradigm from professors pushing their preferred form of assessment onto students, to each student pulling the method(s) of assessment that best demonstrate what they have learned. Pull systems are a fundamental feature of Lean management, and now of Lean teaching!
6) The top three assessment methods – small weekly assignments, practical problem-solving, and individual projects – are consistent with how people are taught in workplaces, especially those that correctly understand and practice Lean management. The classroom is a contrived learning environment, one that is relatively short-lived in terms of a student’s life and career. The popular modes of classroom assessment – pop quizzes, team projects, and in-class tests – are widely disliked by students and also diminish learning outcomes. It also suggests that faculty are not preparing students for the workforce in terms of how learning is commonly assessed in that environment.
Obviously higher ed need not do everything the way industry does, and some types of classroom evaluation may be necessary though disliked by some students, but we should not be stupid about this. Instead, we can learn from the data and make improvements. Below is a form that I plan to introduce next semester in an undergraduate course that will allow students to select the method(s) of classroom assessment that results in the greatest learning outcomes for them.
The logic for the assessment methods selected in the table is as follows: Small weekly assignments, practical problem-solving, and individual projects are given as choices because that is what most students want. Individual take-home test is given as a choice because it scored highly (4th place). Team project is given as a choice because that is how people learn in the workplace (note that the free rider problem should be eliminated because students selecting this choice are intrinsically motivated to do a team project). Research paper, though low scoring, is given as a choice because that is how those students who like to write prefer to learn.
Given the choices shown in the above table, most of my students indicate a preference for the following three assessment methods: small weekly assignments, practical problem-solving, and take-home tests. We’ll see if this holds true when I deploy pull assessment next semester. I’ll let you know how it goes in a future blog post.
Finally, what effect do you think pull assessment might have on enrollments, student committed to their major, retention, time to graduation, and graduation rates?