Evaluating Student Evaluations

Prior to the pandemic, I was a big believer of conducting student evaluations in every class and during every semester. I wrote a post about this a bunch of years ago and compared the practice to stepping on a scale everyday. I was on a big dieting kick for a while and weighed myself everyday. I read somewhere that the accountability process of stepping on a scale daily can help dieters stay on target with their goals. I felt the same could be applied to the process of conducting regular student evaluations. By getting student feedback every semester, I argued, instructors could better monitor the changes in how students respond to their teaching and could better address them in subsequent semesters.

That was my practice until the pandemic started. When we were teaching remotely, I didn’t opt to do student evaluations. I felt so much of the teaching and learning processes were outside of my control that I wondered what the data could tell me. At least that’s how I convinced myself to stop doing student evaluations for the last two years.

Here we are in Fall 2022 and I’m up for my five-year review this year. As part of my review, I have to conduct student evaluations in all of my fall classes. Like the person who hasn’t stepped on their bathroom scale for a while, I’ve been pretty hesitant to see what the numbers might look like. Since it’s been so long since I’ve collected student evaluations, I worried that the data might show huge areas for growth. I’m not really worried that I’d lose my job or that I’d get a negative review from the administration or anything. More than anything, I worried that the data would communicate that I wasn’t supporting my students as much I should.

Through a technical glitch, I’ve already received the student evaluations for one of the classes. Every semester, I teach a seven-week graduate course that all of the students in one MEd program are required to take. Since the class ended a couple of weeks ago, the evaluation data was emailed to me last week. I held my breath for a second as I opened the attachments and saw the evaluation averages.

If you’ve been reading this blog for a while, you may remember a post I wrote where I discussed in great detail how I emotionally and professionally navigated the low student evaluations I received in Fall 2016. I wrote at the time that “seeing these low scores in print was definitely a bitter pill to swallow.” Unlike those low scores, however, the scores from my class this semester were great. Beyond the positive evaluation scores, the students also left really positive comments. One student commented that I was a “helpful instructor” while another wrote “Ollie is attentive to his students’ needs.”

It’s funny. When I received those low scores a bunch of years ago, it set me into a tailspin. After working through the emotions for twenty-four hours, I got to work on making improvements and addressing the students’ concerns. This semester, I read the student evaluations and comments and dismissed them. Not that they weren’t relevant or that students’ impressions weren’t valuable or anything. I just wasn’t prepared for the positive results and then, I came up with false narratives for why they could be positive or why they could be quickly dismissed.

Don’t get me wrong. I’d prefer to get positive student evaluations. I just wish my internal critic could accept the positive results as easily as they accept the negative ones.

Advertisement

Not in Trouble

A few weeks ago, I received an email from my dean inquiring about an assignment I had given to my students. Before getting too far into the nature of the email or my resulting reflections, I feel some context may be needed.

In the professional block of classes for our teacher candidates, we offer intensive, shortened classes to offer students more time out in their field placements. For me this semester, that meant condensing an entire course on assessment into seven weeks of instruction. Assessment is a pretty expansive topic, so I hit the ground running and use every minute of every class intentionally. It also means that I assign my students readings to tackle before coming to class. Even on the first day of class.

Beyond reading any assigned articles or chapters, I ask my students to complete “Close Reading” assignments which are designed to help them better engage with what they’re reading. I’ve written about these strategies before (see Reading More Closely and Literacy and the Collegiate Student). For the readings I assigned for the first day of class, I asked students to identify a word, a phrase, and a sentence from the readings and write a short paragraph defending their choices. They had to post these to a discussion forum before coming to class. Even on the first day of class.

I choose to have students post these Close Readings to a discussion forum for several reasons. One, I look over the posts prior to class and quickly assess which topics resonated with my students and which topics they misunderstood. Two, I can assess which students actually read to material. Three (and most importantly), the students have contributed to a large discussion forum where they’re all highlighting different aspects of the text. For the motivated student, they can review all of the posts and see how their classmates have processed the readings. This can foster a collaborative meaning making process of some challenging material. Considering the reasons, I feel the Close Reading assignment is an important one for my students to complete. Even on the first day of class.

So, that’s the necessary backstory. Now, let me get into the email from the Dean. The Dean received an email from the university Provost who received an email from one of my students who was stressed about my first day reading assignment. The student felt it was unfair that I gave an assignment on the first day of class. That student’s email prompted the email I received from my dean who was inquiring about my rationale for assigning work on the first day of class.

If you don’t work in a collegiate environment, you might not know what a dean or a provost does. In some ways, a dean’s work is similar to a principal in a high school. The dean oversees the teachers and students within a college. Continuing with the K-12 comparison, if a dean is like a principal, then a provost would be like an assistant superintendent. A provost oversees all of the academic activity across the entire university. Provosts are pretty high up in the chain of command in a university setting. I would argue they almost have as much influence as a university president. So, provosts are really important people in a university environment. And the provost was contacted about my first day Close Reading assignment.

I guess the challenging part for me is that while I’ve been able to explain my pedagogical rationale to my dean (and to all of you lovely readers), I haven’t really been able to explain my rationale to the student, who hasn’t been identified. I’m happy to report that after a quick conversation with my dean, she assured me that I wasn’t in trouble in any way and that she thought the Close Reading practice was pedagogically sound. That was comforting.

The lack of conversation with the student, however, has been unsettling for me. This student is training to be a teacher and I worry about the missed opportunity for a discussion. Given the chance, I’d like to have a chance to outline my pedagogical rationale for the assignment but also to discuss how best to navigate stressful situations professionally and appropriately. Beyond that, I’d also like to explain that some time down the road, they may have a student or a parent who will email the principal (or the superintendent) to complain about their own class and I bet they’ll wish they had the opportunity to clarify their reasoning directly.

The Cold Call

Of all of the educational experiences in my life, the one class whose memory still creates some anxiety for me is a History of Physics course I took in my sophomore year in college. The course was built around reading seminal works in physics and discussing them as a class. We read Millikan’s paper on the oil drop experiment which established the charge of the electron. We read Einstein’s paper explaining the photoelectric effect which detailed a connection between the energy of light and electron emission. We read parts of Newton’s Principia which outlined the laws of motion and formed the basis for all of calculus. The semester was a wild ride through primary physics works.

I’m sure that in itself sounds stressful to a lot of readers, but the content wasn’t the anxiety-inducing part for me. I thought it was cool to dig into old research articles and see how they set the foundation to our understandings hundreds of years later. For me, the stressful part was how the class discussions were led by the instructor, Dr. Wofford (not his real name).

Dr. Wofford was a physicist. If you were to close your eyes and imagine what a physicist working in the late 80s would look like, I’m sure your vision would be pretty close. Each day, Dr. Wofford would arrive precisely at the start of class, remove a chewed cigar from his mouth, and place it in the pocket of his brown tweed jacket. He sat down at the desk and got to work. No pleasantries. No greetings. Just physics.

While Dr. Wofford was intimidating in his demeanor, the structure of his class was also stressful. For each research paper that would we read, Dr. Wofford would hand out a series of questions that each of us needed to be prepared to answer during class. Some of the questions could easily be obtained from reading the article. Others were really complex and required a great deal of application and analysis to answer. During class, Dr. Wofford would sit behind his desk and cold call students to answer the assigned questions. He referred to everyone as Mr. or Ms. so each class went something like:

“Mr. Lewis, what is your answer to question number five?”

And the student would try his best to answer the question. If the answer was satisfactory, Dr. Wofford would move further down his class list and address another student.

“Ms. Smith, what is your answer to question number six?”

And the class would continue until all of Dr. Wofford’s questions were satisfactorily answered.

The real challenge came when his questions that weren’t satisfactorily answered. Dr. Wofford would berate and embarrass any student who didn’t answer his questions correctly. Even today, I clearly remember the first time this happened. I stared down at my desk as Dr. Wofford called a classmate “ill-prepared” for the demanding work in the field and advised him to choose another course of study. Thinking back, it could easily be labeled as pedagogical malpractice.

But a strange thing happened as the semester continued. Our class would gather early to make sure we all had answers to the assigned questions. We knew that Dr. Wofford would sometimes ask impromptu follow-up questions so we tried our best to make sure each of us clearly understood the articles. Because none of us wanted to feel the burning pain of Dr. Wofford’s public humiliation, we worked together to help each other learn. We also tried to hijack his discussion methods. While Dr. Wofford typically worked alphabetically down his class roster after starting randomly on the list, we began raising our hands to answer questions. At first, he seemed surprised to see our hands raised before ignoring them and returning to his roster. With time, however, he began calling on us. While our volunteering to answer questions didn’t completely diminish the berating or the humiliation, it gave us a little more control over a really stressful learning environment.

You may be wondering what prompted this walk down memory lane. A colleague and I are facilitating a reading group focused on the book The Missing Course: Everything They Never Taught You About College Teaching (Gooblar, 2019). Chapter 1 discusses the benefits of active learning and includes some research on the impact of “cold calling.” As a teacher, I rarely call on students who don’t volunteer. Mostly, I want to avoid creating an environment like the one in Dr. Wofford’s class. But the research by Dallimore and her colleagues has me re-evaluating that decision. In the paper, the authors examined sixteen sections of an accounting class and studied the impact on cold calling on students who hadn’t volunteered. Discussing their findings, the authors write:

“The results demonstrate that significantly more students answer questions voluntarily in classes with high cold-calling, and that the number of students voluntarily answering questions in high cold-calling classes increases over time. Furthermore, students in classes with high cold-calling answer more voluntary questions than those in classes with low cold-calling; this also increases over time. Finally, in classes with high cold-calling, students’ comfort participating in class discussions increases while in classes with low cold-calling, students’ comfort participating does not change.” (Dallimore et al, 2013, p. 305).

While I’m not equating this thoughtful research with my experiences in Dr. Wofford’s class, it is causing me to re-examine my use of cold calling. Thankfully, the authors provide some great suggestions for building a warm and supportive classroom environment which should help me avoid creating painful memories that my students would blog about thirty years from now. One can hope.

References:

Dallimore, E. J., Hertenstein, J. H., & Platt, M. B. (2013). Impact of cold-calling on student voluntary participation. Journal of Management Education, 37(3), 305-341.

Gooblar, D. (2019). The missing course: Everything they never taught you about college teaching. Harvard University Press.

A Rare Sighting

Collectors consider a Honus Wagner baseball card to be one of rarest.  A “near mint-mint” copy sold for $2.8 Million in 2007.

Action Comics #1 was released in 1938 and features the first appearance of Superman. While it originally sold for ten cents, collectors consider a near-perfect copy one of the rarest comic books.  A copy sold for $3 Million on eBay in 2014.

Reportedly, there are only six known copies of Shakespeare’s signature in existence, which is pretty surprising considering how much he’s written. Collectors consider his autograph to be one of the rarest.

This post isn’t about collecting things.  Or about Superman. Or Shakespeare. Or even about baseball.  It’s about rare stuff.  It’s about the things you don’t see everyday.

I started a new online class last week and the first assignment was due recently. One of the first activities I have my online students complete is recording an introductory video. The students share their majors, their hobbies and their goals for the class.  I usually pair this with some readings so the students have to make sense of course content at the same time. As I was watching the recorded introduction, one video stuck out.  While the class is a required course in several education programs, Mark (not his real name) isn’t pursuing a degree in the field. Although he’s a graduate student on campus, Mark is enrolled a program pretty far removed from education. I wondered whether he was misadvised or had accidentally signed up for the wrong class. As the add/drop period just ended, I worried that maybe Mark would be stuck in a class he hadn’t planned to take.  So, I emailed him.

My email went something like this.

“Mark, I’m wondering why you’re taking our Educational Technology class. Don’t get me wrong. I’m glad you’re in the class but I’m wondering about your future goals and how this class fits in with them. Did your advisor recommend taking this class? I’m trying to get a handle on your plans and how I can support them.”

Mark responded almost immediately. He shared a personal story about his journey to class. While I won’t go into a lot detail about that, his rationale for taking the class is what made this interaction (and him) really rare.  He wrote:

“I’m taking this class because I felt it would be a good fit for me. It’s not required for my program or anything but it will help me learn about some of the newer technologies that I’m unfamiliar with. This was a personal pick for the summer.  I felt it would be a positive addition to my existing knowledge.”

I’ve been teaching at the collegiate level for the last decade. This is the first time that I can remember that a student took one of my classes just for the sake of learning. The class wasn’t going to satisfy a check box on his transcript, help him maintain state certification or reach some contractual requirement at his school district.  While those are common reasons that people enroll in my classes, Mark was motivated by something different. He recognized gaps in his own knowledge and expertise and wanted to fill them. He was here just to learn. And that makes him pretty rare.

 

Be More Stubborn

When my wife and I first became parents, we’d get lots of advice from experienced mothers and fathers on how best to raise kids.  Some would say things like “provide lots of structure” or “tired kids are happy kids” or “let them experience the natural consequences of their choices.”  As new parents, we waded through these pieces of wisdom, looking for the ones that best reflected the types of parents we hoped to become.  Across all of the advice we received, the one that my mother-in-law shared stands as the keystone to our roles as parents.

“Be more stubborn than your children.”

As most parents know, kids can be pretty stubborn.  Children can get fixated on a toy or an activity and scream and yell until they get their way.  And they can be resolved in their emotion and steadfast in their desire.  They want their way and they’re prepared to fight it out and create a fuss until their parents give in. But that’s when the “be more stubborn” parenting mantra needs to kick in.  If a parent gives in to every demand a child makes, long term, the child can become selfish or lack respect for their parents or become undisciplined.  Being “more stubborn” means having faith in your choice as a parent and waiting it out.  While the child is focused on the short game, as a parent, you need to focus on the long game.  It’s not always easy to wade through the cries, screams and temper tantrums but, in most cases, the resolve pays off.

I was reminded about this parenting mantra recently after a meeting with some colleagues.  We were discussing a class that one of us was teaching and how the students were resisting the teaching strategies that my colleague was employing.  As she explained the goals with her assignment and the strategies she was using, I tried to alleviate her self-doubt and explain that what she was doing was pedagogically sound.  Despite her students’ resistance, my colleague was trying foster an active learning environment in her class which would ultimately lead to more student engagement and increased student learning.   I also shared the research on how active learning was a little like broccoli; students know that it’s good for them but they don’t always enjoy it.  I blogged about this research a few years ago in a response to our campus newspaper’s attack on faculty who “weren’t doing their job.”  Despite our best intentions, many students want us to lecture to them so they can passively receive information.

But that’s when we need to be more stubborn than our students.  If we know that the instructional choices we’re making are in the students’ academic interests, we need to face the resistance and be resolved in our expertise and decisions.  While I doubt that many of us will face temper tantrums from our students, we may face some individuals who don’t readily see the value in the assignments we’ve developed or the instructional techniques we’re using.  In these instances, we may need to patiently explain some of our overall goals to help build buy-in from students.  In the end, however, like the parent facing the cries and screams of a difficult child, we may need to be more stubborn than our students and remember that what we’re doing is in the students’ be interests, whether they recognize it or not.