We think that if a student gets a problem right, then they most likely know what’s going on. I suspect this really isn’t so.
I’ve discussed what might go on when a student get’s a problem wrong, but what about when they get it right. It is easy to think that all is good and that the student has the appropriate understanding of the underlying material. Here’s some things to think about:
- Could a student memorize enough specific problems, and is this problem the same or nearly the same, so that he or she could just spit out the answer without really knowing? For example, in one of our lower level courses, we typically include the proof that the square root of 2 is irrational, and often it is done in class multiple times and shows up on a quiz before it shows up on an exam. A student could just memorize the proof to produce on the exam, but wouldn’t necessarily really understand it.
- Could a student get similar problems or problems based on similar ideas correct just as well? Following the previous problem, if a student showed that is irrational, could they show that is irrational? Would they use the same proof structure to show that is irrational?
- Are there certain phrases or notations that you look for in high quality solutions that are phrases or notation that you always use and emphasize in class? Again, this reflects the possibility of memorization without understanding. It can also happen when you think that the material is really difficult for students and so you generously reward any type of sense making, i.e. if they just say something relevant to the problem.
- Could a student cheat in some form to get this answer? In a graduate course I taught years ago I asked this pretty complex question on a homework assignment. What I didn’t realize is that a colleague who had taught this course before had basically solved the problem in class and had posted his notes. What the students who found and used this posting didn’t realize is that I had modified the question slightly and so my colleague’s notes weren’t quite a perfect match (this was also how I discovered the copying).
- Could a student develop or learn from other sources, shortcut or simpler methods for solving the problem? You’d probably pick this up in their solution, but it goes to the heart of this discussion of our association between getting the correct answer to a problem and really knowing what is going on.
I don’t think this is an easy problem to address as there will always be students who score well but who may not know the material at the level you think they do, no matter how you design your testing. However, it doesn’t mean we shouldn’t try, and I’d suggest at least thinking about this issues and trying different and multiple modes of assessment to see if you can gain more insight into your students’ true knowledge.