Grading Drift

I’ve been reflecting on the concept of grade drift recently. My brain usually turns to grade drifting at the end of each semester as I evaluate how objectively I have scored the students. If you’ve never taught before, what I call “grade drift” is the tendency for an instructor to allow their grading standards to drift as they grade an assignment. I’m not sure how prevalent this is for assessments that are concerned with exact answers, such as math, science, engineering, programming, etc. But in assessments that deal with qualitative or creative responses, it’s common for you as an instructor to change your evaluation thresholds as you make your way through the pile of papers.

There are a few reasons why this happens. In my first courses, my grades would drift if I didn’t have a sufficiently robust marking guide to help me narrow down the kinds of answers I was looking for in a student’s response. Now that I’ve taught the course a few times, I know generally what I’m looking for and can go through the assignments with a checklist of items. This form of grade drift is usually related to lacking experience in teaching and grading, especially if your course delivery isn’t tied to your course learning outcomes.

However, there is another reason why your grades might drift. On the one hand, you want to approach your evaluations as if they are objective – that there are clear right and wrong answers. After all, your grading should be defensible, and one way to defend the grade you grant is by pointing to an objective standard. However, on the other hand, you want your evaluations of your student’s work to allow for imagination, creativity, and novel connections between ideas. You also want your evaluations to acknowledge that your classes have unique compositions of people, with their own experiences and their own progressions through your course. Just because you teach the same material each semester doesn’t mean your students progress through it at the same rate. Different students will take the material differently, which means the average response you read through will be different for each cohort of students.

This means that some groups will be “better” than others in how they perform. You can choose to penalize the students for not meeting an abstract standard that you’ve set, or you can meet the students where they are and make it your goal in the course to improve their performance as you go.

I know that some instructors will take the first approach. They will believe their teaching, evaluations, and courses are reflective of an external, eternal, objective standard; they don’t think they are being arbitrary. I’m willing to bet that these are also instructors who believe that a 70% in their course is “like a 50% in other courses” (a literal thing I’ve heard said by a peer). I think this approach is wrong and it reflects a misunderstanding of what it means to pass a course. When a student passes a course, it means they have met the minimum requirements of the course learning objectives. Anything above that should reflect varying degrees of competence and mastery. If your course is designed so that a student is only “just” meeting the standards with anything above a 70%, you haven’t calibrated the course or your expectations very well. At an undergraduate level, your job is to elevate your students, and improve their abilities while meeting the program learning outcome objectives.

I find myself on the other side of this issue. I understand what the goals are of the course I teach, and my aim is less concerned with ensuring my students can adequately explain all the theory they are exposed to in the course. Instead, my goals is to make my students better thinkers and writers from where they enter my course to their exit. If they can explain Aristotle’s ideas of tragedy by the end, great! But I won’t lose sleep if they can’t. My course is a general elective, and that means I’m supposed to round them out as students and people. I care less about them absorbing my esoteric knowledge and instead I care more about them learning how to think, reason, and communicate their ideas clearly.

And that’s why sometimes my grades drift as I score the student’s rubrics. I start off with some ideas of where they could be, then I calibrate my grading to meet them at their level. It’s a little more work for me, but I think it better captures the student’s performance.

As indicated above, there are a few strategies for overcoming this, such as having a clear rubric and clear notes on what you are looking for.  Also, marking all the students on one question, one at a time, rather than marking an entire submission will make it easy to compare student fairly.

While this is an unfortunate thing (the students like to think they are being objectively evaluated) this will happen as you come to understand the natural curve of your class.  You can either grade your students to one standard, or you can reflexively respond to your class’s own knowledge and aptitudes.  If your course objectives are clear, then you can feel free to adapt your grading to match the student’s progress.

Stay Awesome,

Ryan

Advertisements

Two Approaches to Intro Phil

By the time this is published, I will have finished my grading for the Winter semester, however as of drafting, I am procrastinating from finishing.  That means it’s time to let my mind wander and to blog about it.

The course I’ve been teaching since Fall 2016 is a light, multidisciplinary romp through philosophy that examines various topics in human experience through multiple lenses.  Often, this is one of the first general education courses the students encounter, and for many it is the first time they are getting into something approximating formalized philosophy.  It’s an online course, so the students work their way through the module material, post to discussion boards, and submit weekly asignments.  The modules are a mix of text, graphics, and embedded videos, and all things considered is a pretty marked departure from my first course in philosophy.

My introduction to philosophy was significantly more old school.  We all had copies of Cahn’s Classics in Western Philosophy (the sixth edition), and we started the semester on Plato.  It was my first class in university, and my first introduction to academic philosophy.  In a small way, that course has dictated how, in my mind, philosophy gets taught.  It is my default template for teaching – the professor stands at the front and pontificates in fifty minute blocks of time.  I never questioned it; I just tried to take notes, not really knowing how one takes notes about the lecture material.

Since then, I’ve seen a few different pedagogical approaches to teaching an intro phil course.  You can teach the material chronologically, topically, using ancient sources, using modern case studies, you can approach the different branches of philosophy thematically, or you could teach the material historically.

But there is another axiom I’ve been thinking about when teaching philosophy (or any topic for that matter):

Should I require the students to read the text before or after the lecture?

On first glance, this seems like a silly question.  Afterall, the lecture is about exploring ideas, clarifying thought, and demonstrating how to extend the ideas.  Students should grapple with the text and work to come to an understanding.  If there are problems with the text, they can bring their questions to class for rich discussions.  After reading the text, the students can teach the concepts to each other and have fruitful conversations about the ideas.  Why would you want to let the students not read the text?

On the other hand, it depends on what your goals are.  You see, the problem with requiring the students to read the text prior to the lecture, at least at an introductory level, is that the students often don’t possess the vocabulary or the historical schema’s to truly understand the work in front of them.  In an introductory philosophy course, I’m not looking for students to master the material.  Often times, I’m not even looking for them to be right in their arguments.  Instead, I’m looking for other skills, like close reading, comprehension, and (most importantly) the ability to read text charitably.  The hardest thing for students to grapple with in intro phil is that they are seeking to be right and rarely allow themselves to explore ideas.

Plato wrote more than 2000 years ago – what could he possibly have to say in his silly dialogues that helps me today?  Given the current political climate, I’d say his Republic has a lot to say (I’m looking at you, Thrasymachus).

If I give students a text, they don’t have the benefit of a historical view.  They don’t understand the context in which the author is writing.  They don’t understand the conventions of language and allusions.  They don’t see the historical dialogue that is unfolding on the page, where the author is addressing thinkers who came before them.  For instance, in my experience, the vast majority of students who read Descartes for the first time end up thinking Descartes literally believes an evil genius is out to deceive him.  They get so fixated on trying to keep up with what Descartes is doing that they lose sight of the argument being laid out before their eyes.

Reading the text before class is a great way to have students develop critical thinking skills, but if you want the students to understand what they are reading, it might be worth your time to consider helping them to explore the text together.  At least for an intro phil course.

Stay Awesome,

Ryan

 

 

Student Coaching

Lately, I’ve noticed that in addition to my roles of administration and teaching, I’ve been spending more time coaching students.  It tends to come up in small ways, such as offering suggestions on how to word emails or how a student should approach talking to their professor about something in their class.  At first I was a little uncomfortable with taking on a coaching role when it wasn’t really part of my job.  That’s not to say I’m uncomfortable with doing tasks that are not explicitly written into my job description.  Instead, I was uncomfortable because I wasn’t sure if it was my place to offer guidance.  Sure, I might be a decade older than most of the students, but I sometimes struggle with the impostor syndrome: what do I know?  I’m just a lowly administrative assistant.

Having said that, I recognized a fundamental truth that I think many people take for granted – students are young.  I don’t just mean young in age, but also young in experience.  Most students haven’t had the same experiences I’ve had, whether that is my post-secondary schooling, grad school, or work.  Things that I take for granted that come second nature to me are wildly new for students just coming into school.  It’s even worse for students who are first-generation college/university students (like I was).  For some, they haven’t had a lot of experience navigating systems on their own.

We bemoan the helicopter parents and make snide remarks that students don’t know anything (i.e. “life skills”), but I think we should have more empathy.  Post-secondary is a big, scary place to navigate.  If it’s your first time living away from home, having to manage your own schedule, finances, and life, would you not also feel overwhelmed?

Instead of starting with the assumption that students are lazy, or wanting everything fed to them on a silver spoon, I try to take the approach that students don’t know how to narrow down their options or choose a path.  They are the modern Buridan’s ass stuck between competing options with no practical way of making decisions or selecting priorities.  Instead they focus on what’s immediate and take the path of least resistance (for them at the time).  Without a longer view and a chain of successes, their choices may seem short-sighted, but in their context it makes sense to them.  If you couple that kind of decision framework with the complex, convoluted machinery that is “real life,” you can hardly be surprised when students make sub-optimal decisions.

In light of this, my response is not to infantilize them, nor chide them for what one would judge to be their bad decisions.  Instead, I offer my perspective and anecdotes to provide teachable moments.  I provide insight into byzantine rule structures and explain my reasoning.  And I ultimately leave decisions or action up to them.  They must take ownership of the process because they have to be accountable for the outcomes.  The point of education is to create a safe space to fail and learn.  Therefore, our goal should be to set students up for success but also provide them with the opportunity to learn through their mistakes, not by punishing them into doing the right thing.  Rote learning works in some contexts, but in order to make deeper connections between ideas and develop ones ability to reason, students need to be coached on how to shift their perspective and see new connections.

And so, I sometimes coach students.  It’s pretty interesting so far.

Stay Awesome,

Ryan

 

Policy vs. Guidance Pedagogy

During an ethics board meeting recently, we discussed ways of providing direction to faculty members who have student-based research in their courses.  For faculty who have research elements built into their courses, it can be a challenge to determine what counts as research, and whether said research is subject to the rules governing conducting research at an institution (specifically in our case, whether an ethics application would need to be submitted to the board).  Not every scholarly activity necessarily counts as research, and not every kind of research requires an approval from the institutional research ethics board.  Since this can be a bit of a murky area, we have been considering ways of providing direction.

The conversation abstracted away from the specifics of this case, and we discussed some of the issues concerning policy and guidance, which applies to education and pedagogy more generally.

The benefit of policy is that it spells out clear expectations of what is expected, what the division of responsibility looks like, and what consequences might be considered in the event of a policy breach.  Policy is designed to protect the institution through due diligence, and it focuses on expressing what rules need to be followed in order to not get into trouble.  Loopholes arise when the policies are not sufficiently rigorous the cover contingency cases and when policies are not harmonized laterally or vertically with other policies.  Policy documents focus on the “ends.”

On the other hand, guidance documents focus on the “means” by providing suggestions and best practices that could be followed.  Guidance documents typically do not include comprehensive rules unless it’s appropriate.  Instead, the purpose of the guidance document is to provide clarity in ambiguity without necessarily spelling things out.  They are deliberately left open because guidance documents are meant to supplement and add to ongoing conversations within a field or system.  While guidance documents also do not provide comprehensive options to contingent situations, the strength of the guidance document is that it’s educational in intent – it provides reasoning that helps the reader understand the position it takes, and paints a vision of what success looks like.

I realized in the meeting that this has a lot of crossover into considerations for teaching.  It’s is better, in my opinion, to teach students frameworks for thinking, rather than rules for success.  In the case of ethics, I would avoid teaching students what rules they need to follow, and what they need to do to avoid getting into trouble.  Instead, I would seek to build good practices and habits into the material I’m teaching so that I can model what success looks like and help them understand why.  This way of conceiving the material is forward-thinking.  It gets the students to envision what the end-step looks like, and allows them to work backwards to figure out how they want to arrive there.  By focusing on the principles you want the students to uphold (as opposed to rules to follow), the students learn to think for themselves and are able to justify the decisions they make.  This also has the benefit of avoiding the problem with prescriptive policies – students are prepared to reason through novel situations based on principles.

 

Stay Awesome,

Ryan

 

Study Strategies #5 – Application

One of the hardest things I find my students struggling with is not grappling with deep philosophical thoughts, or technical jargon (to be fair, they do struggle with these as well), but it’s in the application of course material.  Most often, when my students submit work for me to evaluate, they submit work that is either:

  • straight opinion (read: a submission that is not structured as an argument with evidence and connecting ideas through logic); or
  • an attempt to solve or provide a definitive response for all the problems of this philosophical issue in about a page.

The thing my students don’t realize is that I don’t care whether they “solve” the philosophical problem.  Granted, I don’t expect them to be able to solve the problem in a page, but that’s not the point.

The purpose of the exercise is for me to check the thought-process of whether they are able to understand the material and work with it.

I was recently chatting with a Program Chair about her time teaching engineering courses.  She noted that often she’d give problem sets that lacked defined measurements, and her students would pause to ask what the length or value of the unknowns are.  She was very frank that she didn’t care what number their calculator displayed because it was more important for her to see whether the students could think through the problem, manipulate the equations, and understand how to go about solving a problem.  For her, the solution was extraneous for the purposes of the class – it was a quick and convenient way to mark an answer right or wrong, but not entirely indicative of whether the student was understanding the concepts.

Now, you may say that this is all well and good for engineering, but how does that apply to philosophy (“But, philosophy has no right answers!!!) or any of the other soft sciences or humanities disciplines.

The truth is that the faster you try to apply the concepts, the easier it is to learn and make the concepts stick, and it’s not all that different across disciplines.  If you are trying to learn a concept, the best thing you can do is to try to take what you think you are learning, and apply it to a novel situation.  By focusing less on the details and working with the core concepts, you get a chance to see what makes sense to you and where your gaps in knowledge are.

For the course I teach, the students work their way through the online module materials, which includes extra readings, embedded videos, probing questions, links to additional sources, etc.  Then, after a round of discussion board posts, the students have a weekly essay prompt related to the week’s topic.

Early in the course, my students will often reply strictly to the essay question with what they think the right answer is.  Through my weekly rubric feedback and general emails to the class, I encourage them to go back to the module content and apply the concepts they are learning to the essay prompt.  What would so and so say about this concept?  How does this school of thought define this concept?  Do you agree with how this concept gets framed?

The point of undergraduate philosophy courses is not for students to generate original philosophical thought.  That is an aim, but it shouldn’t be the outcome.  Instead, the instructor should be guiding the students to think better and understand the concepts being covered so that they can then apply it in novel situations.

When studying, a good way to learn the concepts it to try and extract the ideas from how the author framed them and see how you can apply those ideas in new ways.  It reinforces the learning and helps to spot gaps in understanding in a way that straight memorization doesn’t provide.

Stay Awesome,

Ryan

The Arts of Learning & Teaching

I’ve been in the apprenticeship phase of teaching for the last year, so I’ve largely been gaining experience in how information is conveyed and how to give feedback to students.  While I have given some consideration to course design and what kinds of courses I’d be interested in teaching, my primary focus has been on ensuring the students receive good content and (more importantly) good feedback on performance. Good performance management involves timely and specific feedback to either reinforce good behavioural outcomes, or quickly identifying and redirecting bad performance outcomes. It’s a challenge to ensure that feedback is both timely and useful, but it’s an important step of the process. 

I’m currently working my way through the Art of Learning by Josh Waitzkin, and I’ve started thinking about the process of learning.  While learning and teaching are separate domains, they are interconnected since they share similar goals.  However, being able to translate learning (whether being taught by a teacher or through self-teaching) into teaching to others is something that I have a lot of gaps in my knowledge about.

The first time I taught in-class in the college setting, I quickly became aware that my experiences with formal education (the university style lecture) was not a good mode of delivery to copy. While I am comfortable in the lecture setting, I saw that my students did not excel in that environment. I wish I could say that I had fixed my delivery before the end of the semester, but the reality is that I didn’t fully appreciate the situation until after the course was over and I reflected on the term. An environment where I stood at the front and spoke at length for two-hours was not one which the students could effectively absorb the material.

The problem I found is that how I think and absorb content is different from my students. Rather than teaching them to my style, I need to be more mindful of their talents and experiences. Waitzkin discusses this in his book, where he contrasts two kinds of teachers he’s had. One is the kind that teaches his own strengths and relies on rote memorization of strategies and techniques. In chess, this teacher has you studying opening moves to take early advantage of the board.  The other kind of teacher allows the student to play to their inner style, and teaches by building up concepts atomistically. In chess, this kind of teacher strip the board of all the pieces and focuses on the relationships between pieces at the end of the game. By showing how individual pieces play off each other, the student becomes comfortable across the game and learns not only how pieces fit together, but how to set yourself up for control at the end of the game.

I think my teaching style should embrace this second kind of teacher. Instead of dictating knowledge, I should focus on breaking the knowledge down and building up understanding in ways that make sense to the student. I can’t assume my students will have the prerequisite knowledge to compile the facts together on their own. It’s also the case that if I can’t break ideas down simply, the students might not get it, nor may I truly know what I’m talking about.  Afterall, Einstein and Feynman believed that if you couldn’t explain something simply, you probably don’t understand it very well yourself.

Stay awesome, 

Ryan

The Silent Mentor

I stumbled across an interesting thought recently while browsing Quora.  Apologies for the morbid nature of this post.

A member of the Quora community asked about what happens to cadavers after medical students are finished with them, to which user Daniel Lim offer this answer regarding medical schools in Taiwan.  You can read his full answer linked here.

“The students spend a year dissecting the body, and at the end replace the organs and sew back the skin. They then conduct a mass remembrance ceremony and funeral for the Silent Mentors.”

The concept of the silent mentor is bound up in the following quote from Li He-zhen:

“I will give you my body to experiment; you can make as many mistakes on me, but never make a mistake on the patient.”

I found this to be a power quote that exemplifies an element of education that is sometimes overlooked in the modern economy.  From my experiences, higher education is often seen as training first, before considerations of growth and development.  When you complete your program, you will have been signed-off as competent in a field.  This competence is granted after a series of lectures and tests; tests that you must not fail.

But failure is almost always viewed negatively.  Bad grades are seen as a sign of deficiency – you are not smart if you are getting bad grades.  Failure is costly to students as it sets them back, which costs time, money, reputation, etc.  Education is cut-throat in the modern economy and everyone is in competition for a scarcity of jobs.  If you fail, you are moved backwards relative to the pack.

Yet, failure can be an opportunity.  It’s a chance to see where you have avenues of growth and development.  Rather than seeing failure as an end-point, failure should be viewed constructively as the points that we need to focus on.  Teachers shouldn’t be seen as punishing students for failing, nor should students be seen as inadequate for failing.  Students should have permission to fail.  School is the best time to fail, because the stakes (tuition notwithstanding) are so low.  It’s a chance to test ideas, try things out and learn from the outcomes.  Making mistakes should be instructive.  Expertise is not just knowing the right answers, but also about having a powerful command of all the mistakes that are possible, too.  Teacher have an obligation to instruct pupils properly, not to attempt to download the contents of their brains into the minds of the students.  Education does not work that way.

If we approach failure this way, and encourage making mistakes in safe environments like school, then students will be better prepared to succeed when something as precious as life is on the line.

You can read further on the topic of the medical education and use of cadavers in Taiwan here and here.

Stay Awesome,

Ryan