The case of the curious boxplots

I just graded my second hour-long assessment for the Calculus class (yes, I do teach other courses besides MATLAB). I break these assessments up into three sections: Concept Knowledge, where students have to reason from verbal, graphical, or numerical information (24/100 points); Computations, where students do basic context-free symbol-crunching (26/100 points); and Problem Solving, consisting of problems that combine conceptual knowledge and computation (50/100 points). Here’s the Assessment itself. (There was a problem with the very last item — the function doesn’t have an inflection point — but we fixed it and students got extra time because of it.)

Unfortunately the students as a whole did quite poorly. The class average was around a 51%. As has been my practice this semester, I turn to data analysis whenever things go really badly to try and find out what might have happened. I made boxplots for each of the three sections and for the overall scores. The red bars inside the boxplots are the averages for each.

I think there’s some very interesting information in here.

The first thing I noticed was how similar the Calculations and Problem Solving distributions were. Typically students will do significantly better on Calculations than anything else, and the Problem Solving and Concept Knowledge distributions will mirror each other. But this time Calculations and Problem Solving appear to be the same.

But then you ask: Where’s the median in boxplots for these two distributions? The median shows up nicely in the first and fourth plot, but doesn’t appear in the middle two. Well, it turns out that for Calculations, the median and the 75th percentile are equal; while for Problem Solving, the median and the 25th percentile are equal! The middle half of each distribution is between 40 and 65% on each section, but the Calculation middle half is totally top-heavy while the Problem Solving middle half is totally bottom-heavy. Shocking — I guess.

So, clearly conceptual knowledge in general — the ability to reason and draw conclusions from non-computational methods — is a huge concern. That over 75% of the class is scoring less than 60% on a fairly routine conceptual problem is unacceptable. Issues with conceptual knowledge carry over to problem solving. Notice that the average on Conceptual Knowledge is roughly equal to the median on Problem Solving. And problem solving is the main purpose of having students take the course in the first place.

Computation was not as much of an issue for these students because they get tons of repetition with it (although it looks like they could use some more) via WeBWorK problems, which are overwhelmingly oriented towards context-free algebraic calculations. But what kind of repetition and supervised practice do they get with conceptual problems? We do a lot of group work, but it’s not graded. There is still a considerable amount of lecturing going on during the class period as well, and there is not an expectation that when I throw out a conceptual question to the class that it is supposed to be answered by everybody. Students do not spend nearly as much time working on conceptual problems and longer-form contextual problems as they do basic, context-free computation problems.

This has got to change in the class, both for right now — so I don’t end up failing 2/3 of my class — and for the future, so the next class will be better equipped to do calculus taught at a college level. I’m talking with the students tomorrow about the short term. As for the long term, two things come to mind that can help.

  • Clickers. Derek Bruff mentioned this in a Twitter conversation, and I think he’s right — clickers can elicit serious work on conceptual questions and alert me to how students are doing with these kinds of questions before the assessment hits and it’s too late to do anything proactive about it. I’ve been meaning to take the plunge and start using clickers and this might be the right, um, stimulus for it.
  • Inverted classroom. I’m so enthusiastic about how well the inverted classroom model has worked in the MATLAB course that I find myself projecting that model onto everything. But I do think that this model would provide students with the repetition and accountability they need on conceptual work, as well as give me the information about how they’re doing that I need. Set up some podcasts for course lectures for students to listen/watch outside of class; assign WeBWorK to assess the routine computational problems (which would be no change from what we’re doing now); and spend every class doing a graded in-class activity on a conceptual or problem-solving activity. That would take some work and a considerable amount of sales pitching to get students to buy into it, but I think I like what it might become.


Filed under Calculus, Clickers, Critical thinking, Inverted classroom, Math, Teaching, Technology

7 responses to “The case of the curious boxplots

  1. Cathy

    It is often disheartening when you realize how superficial many students’ knowledge is. I do think that problem solving is the last item to fall into place in the learning process, though. Unfortunately, it seems that many students never get to that step.

    Cornell University has a large bank of concept type questions for calculus that they make available on the web. I think they are called “Good Questions”. I looked at them once, but unfortunately they were mostly too advanced for the courses that I teach.

  2. Cathy

    Btw Robert, I am not very familiar with maths education in America – just out of interest, what is the mathematical background of the students in this class? Are they seeing calculus for the first time?

  3. I’ve checked out the “Good Questions” file and they are, um, good questions. But like you mentioned, some are way over the heads of my students.

    The math background of these students is all over the place. One student — the one who pushed the whiskers on the box plots all the way into the 90’s — is a home-schooled 17-year old who hasn’t finished the high school curriculum yet. But another is on his third time through the class; still others are on their second tour. They have all have all had high school algebra, geometry, and either a precalculus course or a calculus course in high school. The background is there… on paper.

  4. Jami

    It kind of amazes me… but I think I understand. I know that I didn’t really comprehend how important and interesting my education was until I was out of it for 5 years, but it still amazes me how awful students are at math. Most of my bad grades came from laziness and not caring, but I was still able to pull off a C at my worst.
    I guess the more I read your posts about this stuff, the more motivated I get about teaching middle or high school. I really want to know where the problems are coming from, and I want to help fix them. The stigmas about math are horrible! My current professor (of college geometry) was talking to our small class about how math is the only subject in college that is not “required” for everyone at a college level. All liberal arts schools make you take college-level english, writing, history, humanities, etc. classes, but if you’re not in a science major, you can get by with just taking advanced algebra (if that). Its pathetic if you really think about it that way. Math is everything! It has been a required subject of learning for thousands of years, and now no one seems to care about it.
    By the way, I just read that article about the teaching residency program in Boston. Wow! I think that’s awesome. It reminds me of what I’ll be doing, and makes me feel better about getting into it. I’m excited to see the changes they are making to teacher education programs. I just hope it actually makes a difference. The numbers will tell in time, I guess! 🙂

  5. Jami, the one and only time I’ve taught the liberal arts math class at Franklin (back in your day it was called LA 103, Math Models and Applications) the biggest challenge for me as a teacher was remembering that it was a college level math class, not a high school or remedial level course. The book we used then took the high school/remedial approach and nearly all of the students did too. To the students’ credit, they responded really well to high standards. I kind of suspect that all honest students would do the same under any teacher. It takes work to teach a general ed math course with high standards, and you have to have a thick skin when teaching it, but it can be done if we have the will to seek students’ long-term good and not just make them happy in the moment.

    The school I worked at prior to Franklin had “college algebra” as the math requirement, which is sad in and of itself, doubly so when you realize that “College Algebra” really means “10th grade algebra”.

    On a side note, you really need to watch this video: That’ll get you fired up about teaching math.

  6. Jami

    I think I’ve been looking at one too many “community college” transcripts lately. Not quite college level math transferring in here… At least people like you try to make it college level. I just wish everyone did.
    Thanks for the link! He’s quite a smart guy. Gets me excited to see this stuff. I’m so glad its starting soon. It’ll be nice to have an intellectually challenging career for once. 🙂

  7. Pingback: The semester in review « Casting Out Nines