Category Archives: Tenure

Course evaluations: The more, the merrier

This post at ProfHacker reminded me to write about something I’m trying this semester in my calculus classes (the only freshman-level class I have right now). I’m giving not one but four course evaluations during the semester. I’ve given midterm evaluations on occasion in the past, but it seemed to me that even twice a semester isn’t really enough. So, I’m giving evaluations at the end of the third, sixth, ninth, and twelfth weeks of the semester.

The first three of these are informal and very loosely structured. They each have three basic questions:

  1. What do you LOVE about this course?
  2. What do you HATE about this course?
  3. If you could change ONE THING about this course, what would it be?

The 6- and 9-week evaluations have two additional questions: What’s changed for the BETTER since the last evaluation? and What’s changed for the WORSE since the last evaluation? In week 12, students will do the official college evaluation for my course which has all the usual questions on it that probably are found anywhere.

To give credit where it’s due, I stole this idea from Eric Mazur in his book Peer Instruction: A User’s Manual, wherein he steals it again from another professor at UC Berkeley. I modified the idea and form so it works for a series of evaluations rather than just a one-time evaluation. I really like the questions, even though they’re loaded and ambiguous, because the elicit only the one or two most strongly-held opinions about the course from students rather than a laundry list of minutiae or slogans that often show up on written evaluations. When you bring students to the point, the things they often complain about the most are not things that they really feel all that strongly about — certainly not strongly enough to call it “hate”. Similarly for “love”. It’s easy to write about nice little things that happen in a class, but students aren’t often asked to reflect on what they “love” about a class.

We’re about to start the eighth week of the semester, so I’ve already given the first two of these evaluations. I posted them as a questionnaire on the course Moodle site and let students fill in their responses over a Friday-Monday period. They were not required to do so, but I ended up with nearly 100% participation. What’s been great about the results is the change in the responses from week 3 to week 6.

In week 3, the responses were all over the place. The students — mostly freshmen, some of them still unpacking from moving-in day — loved a lot and hated a lot. About half the class loved the technology focus of my class. The other half hated it and wanted it all gone. Many students hated that I wasn’t collecting homework from the book and grading it, hated that the class didn’t consist entirely of examples being worked on the board for them and that the tests weren’t exactly like the homework, and so on — in other words, they felt strongly about how the class was going, but it sounded a lot more like the ongoing struggle to cope with college intellectual culture than it did a serious beef with me or my teaching.

I took the results from the week 3 evaluation and discussed them with our associate dean, who works with faculty on teaching issues, to get his perspective on things. After we met, I blocked off 25 minutes — half a class — before the week 6 evaluations to debrief the students on their responses. The responses on Moodle were anonymous, so I just put them all up on the screen for students to see. This allowed students to see two complementary things: That some of the things they thought everybody hated were really just issue that they alone had, and that some of the things they thought they were alone in loving were actually shared by others. The anti-tech faction saw that there was a significant pro-tech faction, and vice versa, and so the notion of abolishing all technology and using only pencil and paper (one of the actual “one things you’d change” repsonses) became suddenly complicated. During this debriefing session, I showed them some of the things I’d done to respond to the things to change that made sense to change, and I made my case for not changing the things that didn’t make sense to change.

After the week 6 evaluation, it was clear I hadn’t made everybody happy, but the “love” section of the evaluation almost doubled in size, and the “hate” section was about half its previous size — and consisted in large part of the response “I don’t really hate anything about this course”. Given that it’s freshman calculus and I have a reputation, well-deserved, for being a hard professor, that’s kind of shocking. The statements in the “hate” or “change” section that were substantive took on a different tone: Instead of “We should stop using WeBWorK!” it because “I wish WeBWorK weren’t so hard to use.” Through this reflection and evaluation process, and my responses and ongoing conversation related to it, students are refining their ideas about what they like and dislike about a course.

These shifts are crucial for getting students to think clearly on the official evaluation of the course, which is coming up in week 12. I’m not guaranteed to get all-positive evaluations, but I think that after three practice rounds of informal evaluations, after each of which I demonstrate my seriousness in listening to their concerns and doing things about the stuff that can or should be changed, students should be able to write on the official evaluations in a serious, mature, and meaningful way — rather than latching onto one little thing in the course that bugs them and turning it into a wholesale rant, or letting feelings get the better of their judgment.

This entire process is just an example of using both formative and summative evaluations in a class, which is a mirror of the kinds of assessments we should be giving students. The formative part — my informal evaluations — let the students act as “spotters” for the course while it’s developing and running its course. The summative part — the official evaluation — is for students to look back over the entire course and evaluate it. I think that without at least one, preferably two or three, formative evaluations, it’s hard for novice learners like (most) college freshmen to know how to write a good summative evaluation.

I’m tenured, so I’m only doing this to make my students’ learning experience better. If you’re not tenured, this kind of formative-summative evaluation scheme is even more important. Having been on my college’s Promotion and Tenure Committee for five years now, I can definitely say that evaluations of a single course don’t usually provide much meaningful data. It’s the changes from one course to the subsequent ones that matter. Every instructor is going to have one course every now and then that just doesn’t work out, and the evaluations are miserable. The question that the P&T committee has is: What did that faculty member do about it? Did the same complaints crop up over and over again? Or was there some effort expended to address the issue (if the issue is worth addressing)? By giving multiple evaluations in a semester, faculty get the chance to scale the multiple-evaluation process down to fit within a single semester rather than across semesters.

Do you have a similar experience doing something like this? How did it go?

 

Enhanced by Zemanta
Advertisements

2 Comments

Filed under Education, Higher ed, Life in academia, Profhacks, Tenure

Working and having a life, redux

Harvard Yard. Photo taken by Dudesleeper on Se...
Image via Wikipedia

The Chronicle has an article on a Harvard survey of Gen-X professors and their attitudes toward the balance of work and the rest of life. The professors surveyed indicate that they want to be successul in their careers but don’t want to sell out their personal lives in the process. The main survey report is here (PDF, 2.1MB). Here’s a representative quote from one of the interviewees, a business professor, talking about the perils of overwork that Gen-Xers perceive in their older colleagues:

There’s really nothing to be gained by closing your door and working until 11:00 o’clock at night, other than the tenure hurdle that is somewhere out there. If you want to pole vault over it, you go right ahead, but no one here is going to back up the Brinks truck and start dumping all this cash on you, simply because you’ve decided to work like you have three jobs. So that’s the approach I take – sometimes you have to know when there’s this point of diminishing return, where if I keep pounding at this one front, then yes, I may nail it, but at the same time, it will then for a very high cost in other areas.
Although the sample size for this study is painfully small — just 16 professors (the Chronicle article says 12) — the responses are nonetheless fascinating to read and range across a wide variety of work/life balance issues. It’s worth reading the whole thing.

The study is from the same group at Harvard to which I referred in this post from 2006. There, I was responding to comments form some older (or “embedded”) faculty who took the reluctance of Gen-Xers to work until 11:00 PM every night as some form of laziness. Some of the comments at the new Chronicle article tend in that direction also, and conversely there are comments from Gen-Xers that lob equal and opposite stereotypes back at the older faculty.

Unfortunately, until COACHE comes out with a scientific nationwide  study on this issue (with, at the very least, n > 16), all we can do is rely upon anecdotes to understand the issues. But it does seem that most GenX faculty I know share my incredulity at the priorities of some other faculty who place work as the be all-end all of their lives. We also share an extreme irritation toward the inefficient use of time that seems endemic to academia. I shudder to think about how many meetings have I been forced into that have no agenda, spend 45 minutes in chit-chat or irrelevant philosophizing, and accomplish nothing.  And — very especially — we share a kind of hopelessness in considering the rewards structure of academia that gives the loudest applause to those faculty who cut the most out of their lives and say “no” to work the least.

I can only speak for myself (until COACHE gets more data), but I have learned that the best sacrifice to make is not to take time away from your wife and kids so you can get another publication out or hold office hours at 10:00 PM, but rather to lay down hard boundaries around your family and make the crossing of those boundaries by work to be unacceptable. I have learned to say a resounding “no” when work gets to be too much. I have tenure, and surely if I can get tenure then anybody can, but I am coming to understand that I will probably never win one of those prestigious teaching or service awards at my college simply because I maintain those boundaries and protect my family time ruthlessly.

And you know what? So be it. I have three happy and healthy kids who see a great deal of both Mom and Dad every day, who never want for play time or story time, and who know without question that they and their Mom are top priority in Dad’s life. This is more important, more satisfying, and ultimately more crucial to the well-being of the next generation than anything I can possibly crank out in my career. And if it ever gets to the point where my job and my family life cannot coexist, guess which one I’ll jettison without a second thought?

Although hopefully it will never come to that, and I have no reason to think that at my current place of employment it will. And hopefully higher ed as a whole will begin to see that there are a lot of people like me out there and learn to respect our boundaries even as we work to respect the mission of the academy.
Reblog this post [with Zemanta]

3 Comments

Filed under Family, Higher ed, Life in academia, Tenure, Vocation

Must the tenure process really be like this?

Like a lot of people in higher ed, I’ve been following Friday’s deadly shooting at the University of Alabama-Hunstville. (Click the link for background in case you missed the story. I have no idea how much press it is or is not getting in the national mainstream media.) It’s known that Amy Bishop, the UAH biology professor being charged with the shooting, was denied tenure in April and had made an unsuccessful appeal regarding her tenure denial. It’s not clear that the shooting was related to the tenure situation, but the speculation — especially in the article at the second link — is that there’s a connection.

What is clear, at least from my perspective as a professor and as somebody in the fourth year of a five-year appointment to my college’s Promotion and Tenure Committee, is that something is really badly wrong with UAH’s tenure system, and perhaps with tenure as a concept. Listen to this description of Prof. Bishop’s situation from William Setzer, chemistry department chair at UAH:

As for why she had been turned down for tenure, Mr. Setzer said he had heard that her publication record was thin and that she hadn’t secured enough grants. Also, there were concerns about her personality, he said. In meetings, Mr. Setzer remembered, she would go off on “bizarre” rambles about topics not related to tasks at hand — “left-field kind of stuff,” he said. […]

While there were those who supported her tenure and promotion, Mr. Setzer said, he didn’t believe she had any friends in the department.

There was no doubt, however, about her intelligence or pedigree. “She’s pretty smart,” said Mr. Setzer. “That was not a question. There might have been some question about how good of a [principal investigator] and mentor she was. Yeah, she knows her stuff, and she’s a good technical person, but as far as being the boss and running the lab, that was kind of the question.”

Mr. Setzer might not be giving an accurate description of how people get tenure at UAH, but is this really what tenure is all about? Publication records? Grants? Personalities? Whether or not you have enough friends, or the right friends?  UAH does state up-front that it is a research-intensive university, but where is teaching in all of this?

Now look at the depressing remarks of  Cary Nelson, president of the American Association of University Professors, about the realities (?) of tenure:

“The most likely result of being denied tenure in this nonexistent job market is that you will not be able to continue teaching,” said [Nelson]. “You probably can’t get another job.”

Nelson, who teaches at the University of Illinois at Urbana-Champaign, said the review is a period of great stress for even the most likely candidates. They feel judged. Denial can lead to isolation.

“If you have underlying problems,” Nelson said, “then there’s a good chance that they will surface during the tenure process because you are under so much stress.”

We do not know, yet, just how much Prof. Bishop’s tenure denial contributed to her actions, which (I stress) are not justifiable under any circumstance. But honestly — if this is what getting tenure is like at your school, then your school is doing it wrong.

The tenure process can, and should, be an open and transparent process whereby junior faculty are guided in their professional development by senior faculty with a view towards making positive contributions to their institution for 30 or 40 years or more. Done right, tenure can be a transformative and powerful experience for faculty, institutions, and students alike. Done as it is described above, though, it is bound to be petty, political, focused on all the wrong things, and producing professionally unbalanced faculty who have merely learned to play the game properly. No wonder so many schools are considering dropping tenure. But isn’t there some middle ground where tenure can be redeemed?

Reblog this post [with Zemanta]

8 Comments

Filed under Education, Higher ed, Life in academia, Tenure

The blogging VPAA?

I was thinking over the session coming up at Blog Indiana by John Oak Dalton titled “Chancellor 2.0” which promises to address “existing and emerging obstacles of CEO-grade context” [sic? Was that supposed to be “content”?] for Twitter. In other words, it sounds like the session will be about how to get your college’s upper administration up and running with blogging and tweeting. I’m curious to see what Dalton makes of this, because his home institution seems to have embraced blogging and Twitter at a scale you don’t normally see from a university. Even the chancellor tweets.

I’d love to see more college administrators blogging or twittering, using their real names, making no secret of their institutions, and writing honestly about their successes and struggles in the work that they do. There’s no faster track to giving higher education a measure of transparency that it badly needs than this. That transparency is needed both inside and out.

On the inside, faculty benefit from having a window on what the administration is doing, rather than having an administration that lives and works behind a wall of separation. Students, for whom college administration is especially important but also mysterious, would benefit too. And as faculty have a tendency to objectify administrators and turn them into lay figures to complain about — a mirror image of what many students do to faculty — anything that administrators can do to show people their human side (up to a point, of course; there’s still such a thing as “too much information”) helps the organization operate better.

On the outside, the general public has cultivated such a distrust and dislike for higher education — and can they be blamed, the way we act sometimes? — that giving them that same window on administrative operations would be an honest, unilateral step towards reestablishing the trust that ought to be shared between town and gown. And if I were a parent with a child about to start college, the administrator and faculty blogs would be a valuable source of information about what the college is really like.

If I were a college administrator (not that I’m looking to become one), not only would I be blogging and Twittering regularly, I’d encourage the people who work under me as well as faculty to do the same. I’d be trying to make sure the resources are there to make it happen — dedicated server space for faculty and staff to have their own WordPress installations, and so forth — and most important to make sure that they have permission to speak freely. Imagine what it would be like if your official college blog posts or tweets could be used for your benefit towards tenure.

Are there other college administrators out there who blog or tweet? Or any administrators out there reading this post who don’t, and would care to explain why not?

2 Comments

Filed under Academic freedom, Blogging, Higher ed, Life in academia, Social software, Technology, Tenure, Twitter, Web 2.0

Linkages, 29 December

The perils of graphing, from xkcd.

Erin at Critical Mass claims that tenure is untenable and has already been phased out without our even knowing it. Good discussion in the comments.

The New York state comptroller says that New York state college students could save $500 per year on textbooks by buying them online rather than in a campus bookstore. He gives a list of recommendations to universities to make this easier to do. Right on.

Possibly the most comprehensive and mathematically thorough analysis ever of how to curve grades.

The trouble with GTD.

An observation about the changing demographic of mathematics education researchers.

Finally, some number-theoretic trivia about the number 2009, just in time for the new year.

2 Comments

Filed under Education, Life in academia, Links, Math, Technology, Tenure

Four questions to ask about tenure

Over at the Young Mathematicians’ Network, I have an article today on four revealing questions that young faculty should ask about tenure. Since you have to have an account to post comments at the YMN web site, and since some readers who aren’t mathematicians might want to discuss this stuff, I’m going to reprint the article below the fold and open comments for it. Enjoy!

Continue reading

Comments Off on Four questions to ask about tenure

Filed under Higher ed, Life in academia, Tenure, YMN

Monday GTD moment: Scholarship and GTD

This is the third installment of Monday GTD Moment, where I take a post to blog about Getting Things Done and how it applies in an academic setting. If you’re unfamiliar with GTD, here’s a good overview, and make sure to read David Allen’s book that started it all.

Last week I wrote about grading and GTD. I noted that grading is kind of a poor fit in traditional GTD. A prof can grade anywhere, so the idea of contexts fits awkwardly; and grading “tasks” are usually projects, although we think of them as tasks and although the next actions contained in those projects are usually nothing more than smaller projects. GTD wasn’t really made for the academic profession, and so the staple activities of academics don’t often fit well.

Another area similar to grading in its relatively poor fit within the canonical GTD philosophy is research, or more generally scholarship. By “scholarship” I am including not only the usual pure research that most profs do (at least while they are getting their terminal degrees) but also any significant creative activity based on one’s expertise that contributes to a discipline or the application of a discipline. This is Boyer’s model of scholarship, and it is finding increasing purchase at colleges and universities all over. So, for example, an applied mathematician who contributes her skill by consulting “on the side” for some external project is scholarship; so would be research that she does on the teaching of applied mathematics or using her expertise to  do a math workshop with a bunch of elementary school kids.

Scholarship is hard for many reasons, perhaps the main one being that original creative work is amorphous. When I was working on my dissertation, the hardest aspect of that work was that at any given point in the process, although I could always measure how much work I’d done, I had no way to tell how much more work I needed to do, what course of action was really the best next action, or even if my previous work was going to remain or be wiped out by the discovery of a mistake or a more elegant and general theorem. It wasn’t like reading a book, where you knew not only how far you’d read but also how far you had left to go and which page was next, and you had a reasonable assurance that the pages you’d read wouldn’t disappear from the book once you’d read them.

Even now, I have all these scholarly projects I want to finish (or start), and I struggle to GTD-ize them. For example, I’m currently working through this book on probability as a self-study course. Some parts of this project are pretty linear and predictable and therefore have clearly-defined next actions —  “Read and work through examples in section 1.2”, for instance. But then the nonlinearity hits. I will eventually read through all of Chapter 1 and will need to work through the exercises. Do I do them all in order, or do I skip around? Would it be fair to lump exercises 1-5 together as a single task, or is that another project? Is even a single exercise a task, or a project, and how can you know in advance? Academic or intellectual work is a black box — you have no idea how long it’s going to take, what resources you will need, whether it is properly thought of as a task or a project, or indeed even if David Allen’s idea of a “task” (a single physical action) is even appropriate at any level.

Scholarship is highly nonlinear, which makes it much different from the business tasks for which GTD was originally created. But it’s what also makes scholarship fun and rewarding, and it’s why most of us eggheads went and got PhD’s in the first place. So, what’s a scholarship-enthused, GTD-powered prof to do in order to bring this important aspect of his work under the GTD framework? Here are some thoughts.

1. Use the review process — all six levels — to craft a coherent and realistic scholarship plan. The heart of GTD is the weekly review, but don’t forget the other kinds of review that Allen talks about in the book. Specifically, Allen gives a six-level model for review in terms of altitude: 50000+, 40000, 30000, 20000, 10000 feet and “runway”. The weekly review often tends to stay on the runway — which, if your work looks like mine, resembles the runways at O’Hare around Christmas — but those higher-level reviews are important. Scholarly directions change rapidly and often at the discretion of the individual. In the business world, you don’t often get the opportunity to change the fundamental direction of your work on your own initiative. But in academia, every day has the potential for such change. If I decided tomorrow to stop studying cryptology and start doing mathematical finance, I could do that. That ability is liberating but also a recipe for stagnation. It’s important to see your scholarship, regularly, not only in terms of current projects and areas of responsibility but also in terms of where you want it all to head in the next year, the next two years, and so on. The tenure and promotion process at an institution, if that process is well-designed, will help profs to think in these terms, but only once a year or (post-tenure) every five years. GTD, done with these higher altitudes in mind, would say to think about the big picture a lot more regularly so that your overall plan is more coherent.

2. As much as possible, concretize your research agenda. Since scholarship is amorphous, once you get down to the level of 10000 feet and lower, some superimposition of structure on scholarship is necessary. It doesn’t always fit well, like a nice suit on an unruly young boy. But it’s still important to break the scholarship plan you’ve created down into manageable projects with a list of concrete next actions. Having “Write a paper” as a project will lead nowhere; most scholarly activities are projects within projects, and at bottom you find one project that can finally be broken down into discrete next actions, each of which has a well-defined context. The challenge is to get to that point. (This is a major similarity with grading.)

3. Don’t be bothered if the plan changes. The nature of research puts all scholarship-oriented action lists into an automatic state of flux. That nice, tidy list of actions under the “Prove the twin prime conjecture” project stands a good chance of being brutally rearranged if, say, you discover a journal article that shows your main theorem so far (which you thought you proved 3-4 next actions ago) to be false, or somebody proves it first, or if you get an unexpected opportunity to work on something else which requires dropping or postponing the project. We all know that research and scholarship are highly volatile areas. But one of the strengths of GTD as a workflow management system is that GTD assumes that tactical decisions will change fluidly and constantly, and that’s OK. The system doesn’t fall apart if things change; you just adjust your next actions and move on.

4. Subdivide your Read/Review folder and make it more like an inbox. Read/Review means something very different to an academic than it does to a business person. The entire life of an academic could be summed up by the term “Read/Review”. So I think Allen’s conception of the Read/Review file needs to be expanded for academics. In my system, I’ve got three Read/Review folders for physical stuff and three for electronic stuff — the three folders in each medium being Teaching/Service (articles about the profession, articles about GTD, articles about pedagogy, etc.), Research (traditional research papers from journals), and Popular (math-related but not from journals; sometimes ed tech items make it in here). I treat these folders like inboxes in the sense that I make them part of my weekly review. Sometimes I gather articles that look good at the time, and I do intend to read them, but they get crowded out by something more urgent. I find that I need to go through Read/Review at least 2-3 times a week to process stuff. Expanding on the Read/Review idea helps keep fresh ideas coming onto your radar screen and into your brain.

5. Stick to your guns with GTD on everything else besides scholarship. Being a prof involves wearing lots of hats — we teach, we serve on committees, we grade, we mentor and advise students and colleagues, and many other things. In order to have the time and flexibility to carry out these amorphous, nonlinear scholarship projects, we have to exercise discipline in getting things done that are “morphous” and linear — stuff like grading, prepping courses, working on committee proposals, and so on. If a person can use GTD to get those tasks and projects under strict discipline and control, then there will (for the most part) be time and space in our schedules to do scholarship. But if the manageable stuff is running all over us, then we can forget about research, unless you are one of the tiny minority of professors who do research and basically nothing else.

I think there’s a great deal of connection between being happy in your academic work and being balanced. The more we enable ourselves not only to be excellent teachers but also active scholars, the more we benefit and so do our students and institutions. I think GTD can help in that regard.

Have a productive week!

[Photo by Jay Lichtman; artwork by ynot2006]

4 Comments

Filed under GTD, Higher ed, Life in academia, Profhacks, Scholarship, Study hacks, Tenure