They weren’t bad — not nearly as bad as I was expecting. Not very many negative, ad hominem-style comments at all, and several compliments. The numerical responses were as good as I’ve gotten since I’ve been here, although part of that is due to the merciful retirement of some evaluation items that have always lowered my numbers. (Like the one that used to say “My instructor senses when some people don’t understand”, which has been replaced with “My instructor answers questions clearly when they arise”. Thanks, P&T committee, for realizing that professors don’t have ESP.)
Unfortunately, though, I’m not getting a lot of good information right away from the comments or the numbers either. There’s not a lot of comments that point to specific elements of the class that are working especially well or poorly . Mostly the comments are general in nature (“He explains things well” or “He knows his stuff”), immediately dismissable because the student is betraying a lack of understanding about the setup of the course (e.g. “His office hours are in the middle of practice” — so why didn’t you schedule an appointment?), or one comment cancels out with an equal but opposite comment (“I like the daily homework assignments” followed by “You need to get rid of the daily homework assignments”). So I’m pleased with the nature of my evaluations but I wish I could identify more clear patterns of stuff working or not working, so I can improve what I’m doing.
Here are a few things that did emerge from my two sections of calculus:
- There seems to be a widespread desire for more group work or in-class hands-on work. There were over half a dozen distinct comments to that effect, which out of 40-some students constitutes a trend. I’m encouraged by this, because I like group work and dislike lecturing, and I happen to think active learning is the only kind of learning there is. However, I wonder if students are counting the cost of doing more in-class work. A trend that almost cohered in the comments was that students want to spend more time in class discussing questions they had with homework, spend more time in class learning to use the computer algebra system, and so on. You begin to see the economic picture here — we already have a far-too-full syllabus for the semester, and the time spent doing these other things has to come from somewhere. I’m absolutely in agreement with my students on their comments, but I don’t think they understand the balancing act we profs have to perform just to make the class work out right.
This blog would be a great place for students to learn about that angle of college instruction. I should try harder to get them to read it.
- I invented/adopted a system of “Daily Preparation Assignments” last semester to serve as a sort of homework requirement. Students would be given 1–3 exercises from the book after each class meeting and would have to hand in solutions at the next meeting. I would grade these on a scale of 0–3. If there were any issues that a lot of students (i.e., more than five) had that needed to be addressed, we’d take time to do so. Otherwise we’d save questions for office hours. I believe this system worked very well, and the students who commented on it at all were mostly positive about it. It allowed me to use the initial minutes of the class to address the questions that were actually coming up for more than just one or two people — as opposed to beginning a class by opening the floor for questions and having only the most vocal students control the time we spent. It forced students to spend time on the class every day, and the 0–3 grading scale (with a “3″ meaning “entirely correct with maybe a couple of goofs”) was simple enough to allow fast grading and allow room for mistakes with new material. Definitely keeping this assessment the next time I do calculus (which will be in the summer).
As for MAT 240, the Methods of Problem Solving course in which I debuted the POGIL framework, the numerical scores were quite good, with low scores happening in somewhat predictable places. For example, the lowest marks came from the items “My instructor answers questions clearly when they arise” and “Exams are well coordinated with course content.“
I’m not surprised at the low score on the first question, because the whole point of the POGIL framework is to get students to develop the skills for answering questions on their own, without dependency on the professor. In fact during the POGIL activity sessions, I would always begin by declaring that I AM NOT GOING TO ANSWER ANY MATH QUESTIONS TODAY. And I would use a Socratic style elsewhere in the course. Maybe not a good pedagogical technique for all courses at all levels, but it’s simply the way it has to be in a course like MOPS where the criterion for success is how well you can determine an answer without an outside authority helping you. Still, some students in the written comments wanted me to answer their questions “correct” or “incorrect” all the time.
The second question about exams being well-coordinated with content seems problematic because (1) we didn’t have much in the way of exams, (2) the course wasn’t about content, and (3) the exams were aimed at applying problem-solving skills to new situations, not the regurgitation or practice of content. I can’t say that the students were wrong in answering that item the way they did; it’s more of a case of the item itself assuming a particular structure to the course that MOPS just doesn’t have.
But other than that, the MOPS/POGIL marriage seems to have been comsummated well, and students overall seemed to “get it”, which is gratifying. (Especially considering that when you do a Technorati search for “POGIL”, and ignore the results from my blogs, 3/4 of the results consist of outraged rants by students taking POGIL courses somewhere.)
I might do a more detailed breakdown of the POGIL experience in MOPS and my POGIL plans for the future in another post.
Overall — a nice way to end a very hectic semester (at last). Now onto Spring 2006!
Filed under: Education, Life in academia, Teaching
In my opinion, student evaluations are nearly useless for the reasons you describe. I’ve discovered this to be true across the board in my educational/training experience. Whether the venue is an IT training class, a software dev class delivered over a period of a semester for a local non-profit, or teaching a Sunday School class (I haven’t done college instruction yet, but it appears that the same dynamics prevail), student input is almost always too general or contradictory to enable one to develop a specific plan for improvement. This isn’t entirely the students’ fault. After all, they are given a fairly vague set of questions right at the end of the course, when they tend to want to just get the thing done and leave.
I suspect a peer evaluation model would perhaps work better, where one periodically had another teacher sit in on the class and observe. Good luck getting anyone to sign up for that, though.
Rob, we used to do something like a peer review for classes taught by TA’s at Vanderbilt. The Master Teaching Fellows would go in to the TA’s class upon request and conduct a discussion about the instruction. It was several orders of magnitude more informative than a standard evaluation (which all classes did in addition to the optional discussion thing). For one thing, I found that the whiners who rip into professors on anonymous evaluations were frequently shouted down by other students in the discussions. I really wish we did more stuff like that nowadays.