Citing Your Sources

writing-828911_1920
Image by Free-Photos from Pixabay 

While reading a post from A Learning A Day, I thought I’d keep the irony train rolling by linking to Rohan’s linked post from Derek Sivers about the perceived need to quote an idea’s source.  Specifically, I wanted to respond to this point:

2. School teaches us to reference. But we’re not trying to impress a teacher anymore. And every unnecessary fact dilutes our point.

I often reflect on the learning objectives I expect to achieve in a course lesson while teaching.  I try to parse out the meaningful things I want students to learn from rote procedural tasks that don’t serve a purpose.  The last thing I want to do is to reinforce the wrong lesson or derive the wrong conclusion from a student’s performance (e.g. did a student do well on a test because they understood the material, or because they are good at taking tests?).

Derek’s point above about references is well-taken and got me thinking: why do I want students to cite their sources?  I brainstormed a few reasons and listed them below with comments.

Reason

Thoughts

I want a student to be mindful of their research process (procedure). Having gone through writing my master’s thesis, it’s easy to lose track of references and citations if you don’t stay on top of it.  This isn’t super relevant to most assignment learning objectives, but it’s a good practice to have before launching into a bigger endeavor or capstone project.
2. I want a student to critically examine their own knowledge (what do they believe to be true facts, where did that fact come from, and why do they think it’s true). I’m not sure if making students cite their sources achieves this aim on its own, but I suppose I could use citation requirements to help guide them through this process.
3. I want a student to be mindful of idea ownership and give credit to people who have done work.

 

I’ve used this mostly in plagiarism cases where students copied work and submitted it as their own.  I try to distinguish between sloppy citing and outright theft, and I remind students that they shouldn’t get marks for work they didn’t do.  I’m still undecided if this is a rule of the academy or a legitimate thing to prevent fraudulently passing work off as your own in the future.  This point, though, is mostly relevant in academic contexts as opposed to Derek’s notes about doing this during conversations.
4. I want an easy way to see if the student did the work. This is a trick I’ve developed to see whether a student giving me their opinion is right by chance, or if they have informed their opinion by doing the course reading.  The same result could be gained if students inserted relevant information without citations, but the citations help to highlight this when I’m reading through their submission.  In other words, it makes my job easier.
5. I want to reinforce good academic writing habits. Using references is part of what it means to write academically, and is used as part of the integrity process.  This is only a good reason if my objective is to teach/reinforce academic writing for students.
6. This is the way it has always been done.

 

More cynically, requiring citations is part of the tradition, and who am I to question it?  It’s not a good reason to require it, but it is what it is.  I won’t included in the list to the left, but a more sadistic version of this is “I had to do it, so you have to (go through this rite of passage) too!”
7. I want to remain consistent with departmental policies and culture. Whether written or unstated, most departments adhere to some level of standards.  This was less the case for me in undergrad and it depended largely on the preferences of the prof.  By the time of my thesis, I ended up developing a hybrid referencing system that did not strictly follow any of the major citation methods.  I received no comments from anyone who reviewed my thesis on my citation practices.
8. It’s important to trace an idea’s lineage as much as possible to spot fabrication. If you are going to insert facts or conclusions into your work, it’s important to point to where you found them.  Without a citation or an adequate way of accounting for how you know what you purport to know, it’s possible that the information is made up.  Being able to trace these things helps, albeit this is more useful from a scholarship point of view, as I suspect a lay-reader isn’t concerned with checking a text for factual accuracy and instead takes it on authorial authority.

9.

Related – to see if a student is able to either properly reference work, or at the very least charitably restate ideas without dropping important content from the idea. This perhaps falls under sloppy citation practices, but on occasion students will misunderstand a piece of text and paraphrase or summarize information incorrectly.  Knowing where the student is drawing their source from can have pedagogical merit if you take the time to compare the student’s work with the source and discuss the divergence.

10.

Related – when an author cites their sources, a reader can use the bibliography of sources for further reading.

 

This is perhaps more for book nerds, but I love having references to be able to learn more for things that pique my interest.  This is, however, not the context Derek is referencing when he discusses giving citations during a normal conversation.  If Derek’s conversation partner was interested and want to know more, I’m sure they would ask Derek for more information.

11.

More abstractly, knowledge and academics is a web of mutually reinforcing facts, so academic writing is an extension of that reality. This one is a bit of a stretch as to why a student who is not adding to a body of knowledge is required to rigorously cite their sources in a pedagogical exercise, but I include this more epistemological point to try and be exhaustive.

12.

It’s a symbolic representation that the student (in most contexts) is not generating new or novel work/insights that creates new knowledge, but instead is remixing ideas from other sources. I think this is a good reminder of what the goal of the assignment should be (students are often far too ambitious in what they think they can reasonably achieve in x-number of pages), but I wouldn’t consider this to be an adequate reason to insist on proper citations.

13.

Like other skills, the act of referencing needs to be practiced. I’m sympathetic to this, but as Derek is implying, you should be practicing skills that transfer into other domains or that you will need.  In most instances, outside of school you don’t need to cite sources.

14.

Citing references is part of the argumentation process.  In order to build a successful argument, you must clearly express and state your premises, which includes any premises taken from the work of others (either their premises or their conclusions). I’m also sympathetic to this as I think everyone should keep in mind that arguments need to be made to help convey ideas.  It shows the logical chain from premise to conclusion and seeks to make the implicit explicit, and the unstated stated.

Other than a subset of the reasons above, a strict requirement for citations is often unnecessarily enforced in the classroom, and is almost never required outside of the academic setting.  I think there are some good pedagogical reasons to have students go through the effort  to cite their sources, but you should be intentional when teaching as to when those cases apply.  For instance, I am less strict about my students citing sources and instead I look for them to directly apply material from the course in their assignments (instead of giving me their opinions).

I enjoyed Derek’s point about how citing sources is a common trope in pop non-fiction, which sounds like a convergence on my ideas concerning animated bibliographies, or Ryan Holiday’s “15 academic studies” comment from a few weeks back.  Maybe Derek’s right – we should have more courage to integrate knowledge into our existing schema and be prepared to state things as facts instead of citing our sources.  I’m not sure I’m prepared to abandon the practice wholesale, but it has given me something to chew on.

Stay Awesome,

Ryan

Book Ideas – Types of Feedback

I’ve been reading Scott Young’s recently released book, Ultralearning, and I think it’s a pretty good summary of how one can take on an intense learning project for personal and professional development.  It functions like an autodidact’s road map with plenty of good tips, insights, and stories to round things out.  Elements of the animated bibliography are present, but I don’t find it contrived in its execution.  The stories help frame the chapter and serve as an introduction to the core material.

It’s funny how last week I was talking about mnemonic devices, because after drafting that post I ended up reading about the concept in Chapter 10 of the book as it dealt with ways of supporting retention of material you learn.

In chapter 9 of the book, Young talks about ways of providing feedback in the learning process, whether the feedback is provided from others or feedback you can use in your own learning process.  He parses out three kinds of feedback that I found interesting, not only for my own personal use in learning, but also as something I should keep in mind as a teacher.

The three kinds of feedback he outlines are outcome feedback, informational feedback, and corrective feedback.  Each type of feedback serves a specific purpose, and you should be mindful of the context the feedback is given, as the wrong type of feedback can set you back in your learning.

Outcome feedback – provides information on whether you are getting answers right or if you are meeting a pre-identified set of learning objectives.  It tells you that you are right but doesn’t give any indication of why (or why you are wrong).

Informational feedback – provides further information to explain the underlying reason why something is right or wrong.  It can be informative to re-affirm what you have learned, and can identify key areas of strength or weakness, however it does not create a concrete process forward.

Corrective feedback – provides, as the name indicates, a path forward for the learner in terms of how to overcome deficiencies.  It details not only how one is right/wrong, why they are right/wrong, but how to address or avoid being wrong.  This type of feedback not only requires a level of comprehension of the material, but requires sufficient understanding to teach the underlying processes to the learner through explanation, demonstration, suggestion, etc.

As a teacher, it’s important to know what kind of feedback is warranted and under what circumstances.  Most of us tend to focus just on outcomes, but students often don’t learn from pure outcome assessment.  Rather, you need to take the further steps to go beyond an evaluation and ensure you are addressing the underlying deficiency present in the student’s performance.  Outcome assessment is awesome because it’s quick and definitive, but it’s also lazy if your goal is to improve your students.  On the other hand, corrective feedback is desirable but it’s labour-intensive and must be done carefully so as not to remove critical thinking from your student – you don’t want them to merely follow your instructions but instead you want to promote their thinking and reasoning through problems without your guidance.

Stay Awesome,

Ryan

Memorization and my Campus Response Experience

In January of 2008, I was walking through my university campus’s student centre and passed by a table for the UW Campus Response Team, whom were recruiting volunteers for the new semester.  I doubled back, chatted with the team members, and signed-up to participate in their interview process.  I had taken first aid courses periodically during my cub scout and army cadet days, plus I had ran some basic first aid courses while abroad, so it felt like a good fit.

In retrospect, my “experience” was quite paltry, but I had shown the team managers that I had enough of the “right stuff” that they invited me to join the team and participate in the weekend training course they put on for new recruits.  It’s an intense crash course in first aid skills that were well beyond my experiences and the training spanned several hours Friday night and all days Saturday and Sunday, before you perform your final scenario test to qualify as a secondary responder.

The material covered was largely derived from emergency first responder courses, along with some material covered for pre-hospital trauma professions (e.g. fire fighters and paramedics).  The training was designed to create heuristics in the responder’s mind to quickly flow through critical details while gathering as much information as possible and start treatment momentum.  The last thing you want is for a responder to have to intentionally think through what steps they should follow, because it shunts cognitive capacity away from situational awareness and into operational procedures.

In an effort to automate one’s thinking, you end up doing a lot of mock scenarios and skill drills.  As a responder, you end up creating a script in your mind to follow.  The script is based on a common set of things to attend to, which you follow according to handy mnemonics and other memory aids.

Despite the mnemonics functioning to provide mental triggers for actions, you still need to learn the process to go along with the mnemonics, and from the start of training weekend, you only have precious few hours after training concludes for the day to encode the information out of your working memory and into longer term storage.

I needed a way to quickly drill myself and aid in recall.  The system I settled on was to get some window writable markers and write out my mnemonic devices on the bathroom mirror.  Every time I used or walked passed the washroom, I would attempt to fill in as many of the mnemonics as I could remember, and note where I made mistakes.  Through constant repetition, I was able to turn:

E
M
C
A
P
I
E

A
V
P
U

P
E
A
R
L

A
B
C
D

S
A
M
P
L
E

O
P
Q
R
S
T

Into

Environment check
Mechanism of injury?
Count the casualties
Allied agencies?
Personal protection
Introduce yourself
Events leading?

Alert
Verbal
Pain
Unresponsive

Pupils are
Equal
And
Reactive to
Light

Airway
Breathing
Circulation
Deadly bleeds

Signs and Symptoms
Allergies
Medications
Past medical history
Last meal/beverage intake
Events leading

Onset
Provocation
Quality
Radiation
Severity
Time

It was a quick and dirty way to give myself quick feedback on these concepts that I could readily apply to my first aid treatment during training and eventually on shift.  Any time I lost momentum or felt nervous about the judges evaluating me, I would mentally go back to my bathroom mirror and fill in the blanks.  I haven’t been on the first aid team in almost a decade but these concepts easily come back to me, even during my crazy nights at the bar.  It’s a testament to the stickiness of the ideas and the effectiveness of the drills.

Stay Awesome,

Ryan

Knowing When To Quit

A recent article talking about knowing when to quit/retire from teaching had me reflecting on my own experiences with quitting.  Truthfully, I can’t recall many instances where I quit something.  Often, I will drag out experiences long after they have been useful, and instead of quitting as an active decision, I’m more likely to let things fall away through neglect.  Perhaps there isn’t a strong difference between the two since my history is littered with things that I eventually stopped doing.  I suppose in my mind, the difference comes down to whether I made a decision to stop – whether I took ownership over the act.

The strongest instance where I actively made a decision was when I stopped hosting at the local karaoke bar.  I was three or four years into my tenure as a host, and for the most part I enjoyed the experience.  I had a regular crew of friends who would come in and make the night interesting.  However, towards the end, I grew to resent patrons coming in who weren’t my friends.  I worked the slowest night, so if things were quiet, we’d shut down early.  But if patrons filtered in and kept purchasing stuff, we’d stay open.  Catering to the average customer felt like a chore, rather than chumming with friends with our own song preferences and inside jokes.

I started to dislike going into work, and even to this day I don’t sing much like I did while I was a host.  I’ll grab the mic from time to time, but I don’t go out to enjoy karaoke anymore.  I still work security at the bar, but I stopped hosting all together.

I made the decision to stop hosting because a small part of me knew it was time to move on.  I learned what I could from the experience, cherished the memories it gave me, but I recognized that I no longer wanted to spend time doing it.  I think that’s the critical part in the art of quitting.  It’s not about actually quitting or the how.  Instead, it’s about recognizing when the time has come and why.

Sometimes we have to slog it out in things we hate.  We don’t quit those things because we assign value to the activity (or someone else has assigned value and we are dragged along for the ride).  But quitting is more than stopping a thing you don’t like.  It’s about recognizing when the thing is no longer of value to you; that it won’t take you where you need it to go.  It is the recognition that your time is better suited elsewhere.  The art of quitting ultimately comes down to taking an active role in how you choose to spend your time.

Stay Awesome,

Ryan

Books on Vocational Reflection

I finished reading Complications by Atul Gawande last week and really enjoyed it.  It was his first book and covered stories from his apprenticeship phase of becoming a surgeon.  I thought back to the first book I read from him, The Checklist Manifesto, and realized that while I enjoyed the topic Manifesto covered, I found it lacking a certain charm that Complications had.

Manifesto felt like a good idea that was stretched a bit too thin to fit the book format, and was heavily supplemented with references to studies done by other researchers.  This isn’t meant as a criticism – it was a good book!  But what I felt Complications (and his other book Better) had is the first hand reflection on one’s professional development.  It’s not just a memoir of one’s life, nor is it a tell-all, but instead it’s a focused meditation on the training, learning, failures, achievement, and lessons one gains from devoting themselves to their vocation.

Over the last three and a half years of reading, I’ve found I really enjoyed these kinds of  books.  I looked over my reading list and pulled a bunch of examples randomly below.  Some of them are about medicine, others are of actors, and a few books from the business world.  The common thread is that it’s less about the personal biography of the person and more about the development of the professional (for this reason, I didn’t include Elon Musk’s and Enrico Fermi’s biographies, or career retrospectives like the books from James Comey and Hillary Rodham Clinton).

It describes a world bigger than the person telling the story, and their attempt to grapple with the epistemological, ethical, and professional obligations that comes from entering a profession, and where their limits lie.  These are not stories about heroes – the stories are about human error and fallibility, and learning to deal with that revelation.  It also keeps its eye towards what it means to serve others, and where the profession should go in the future.

Ultimately, these books differ from the animated bibliography in one crucial area.  The animated bibliography is often a book that results from a person researching and stitching together the ideas of others.  In some cases, these books will require the author to attempt to put the ideas into practice, but in my opinion this is in service of selling the credibility of the book.  However, the books I’m discussing here and listing below are different because they are an account of people who are learning by doing.  They are applying what they previously learned during formal education, and reflecting on the outcomes to see what lessons can be derived.  In some sense, the books are an autopsy that try to tease out causes, or at least serve as a cautionary tales for those who come later.

 

A Thousand Naked Strangers – Kevin Hazzard

View this post on Instagram

23/ This book came from the @artofmanliness #podcast a few months back. Talk about an exhilarating read! I identify strongly with the author, Kevin Hazzard, as he spins his memoirs of life as an Atlanta paramedic in "A Thousand Naked Strangers." His listlessness and feelings of being unchallenged, not knowing just how much potential he had in him, or how far his limits went, are things I've grappled with recently. A lot of what he wrote resonates with me, but his tale also serves as a caution: Am I on the right path? Am I willing to make the necessary sacrifices? Am I strong enough? Can I hack it? Will I choke? It's a trial by fire in the purest sense and a book I intend to revisit. #reading #books #selfimprovement

A post shared by Ryan Huckle (@rhuckle) on

 

Doctored – Sandeep Juhar

View this post on Instagram

/29 – my trip to Scotland threw my reading schedule off, so I haven't finished a book since before I left. I picked this book up in the spirit of Larry Smith's advice to read broadly in your passions when looking at career options. Since I'm considering paramedicine, I thought this would be good to check out. Juhar writes a fantastic reflection on the 5 or so years after he finished his formal medical training, and the stress and burnout he experienced as the realities of a medical system with misaligned values is forced upon him. Money, family, and values all weigh him down as he is forced to work within the private medical industry in Manhattan. I very much enjoyed his prose and story. I'm very happy I picked this up from the library. #reading #books #selfimprovement #medicine

A post shared by Ryan Huckle (@rhuckle) on

 

I’d Like to Apologize to Every Teacher I Ever Had – Tony Danza

View this post on Instagram

-/32 I'd Like To Apologize To Every Teacher I Ever Had by Tony Danza. This was a spur purchase I made while browsing books on sale from Audible. I figured I could relate to his experiences, having just finished my first teaching experience this past year (albeit at the college level). The book was a lot more engaging than I expected! I felt absorbed as Tony told his story of his year teaching high school 11th grade English. While initially the set-up for a reality show, the experience quickly morphs into a huge learning experience for Tony as he learns what it means to be a teacher and what it takes to be effective in reaching students. It's a humbling story, and I am glad to have learned from it. @tonydanza #reading #selfimprovement #books #nonfiction #teaching #education #memoir

A post shared by Ryan Huckle (@rhuckle) on

 

Total Recall – Arnold Schwarzenegger

*Note – I include this mostly because of his telling of his time as a body-builder and actor.

View this post on Instagram

-/30 – Total Recall by Arnold Schwarzenegger. As a kid, I was into his movies. After I started getting into exercise, I got more into his bodybuilding achievements. While I didn't follow his political career, his recounting of his Governor years was quite interesting (though personally I enjoyed the first two acts of his career more). The book was quite long, but I enjoyed the story and got some good advice out of it. My only critique is that parts of the end felt more like he was writing the book for damage control of his personal life and career (i.e. his infidelity and son). Still, I'm glad I listened to the story. #books #reading #selfimprovement #audiobook #nonfiction #autobiography #memoir

A post shared by Ryan Huckle (@rhuckle) on

 

A Life in Parts – Bryan Cranston

 

The Art of Learning – Josh Waitzkin

 

Shoe Dog – Phil Knight

View this post on Instagram

-/31 – Shoe Dog by Phil Knight. This is a good book, not because I have any affinity for Nike shoes (I don't think I've ever owned a pair) but because it's a great story. The skeptical part of me knows that Knight controls the narrative, so as a narrator we can't fully trust his objectivity as he tells his side of some of the bad events in Nike's early years. Nevertheless, even if he's stretching events to make himself look good, the lessons of his missteps are worth learning from. That, and the charisma his prose evokes, made this a gripping story to listen to and one of my top reads this year. #reading #selfimprovement #books #nonfiction #learning #education #audiobook #Libby #management #business

A post shared by Ryan Huckle (@rhuckle) on

 

Creativity, Inc. – Ed Catmull

View this post on Instagram

-/35 – Creativity Inc. by Ed Catmull. I love love love this book! I've already recommended it to a few friends because it's that good. Catmull, one of the founders of Pixar, gives us an account of his early life and the history of Pixar up through 2014. I grew up watching their movies, so getting the behind the scenes stories was fascinating. The best part, though, is his reflection on what did and didn't work. From his experiences, he draws out lessons for running a company and promoting growth, sustaining output, ensuring transparency, and fostering creativity. The book is captivating and one that I plan to add a physical copy of to my shelf. #reading #selfimprovement #books #nonfiction #learning #education #audiobook #Libby #kpl #library #memoir #pixar

A post shared by Ryan Huckle (@rhuckle) on

 

The Checklist Manifesto – Atul Gawande

View this post on Instagram

-/29 – The Checklist Manifesto by Atul Gawande. I'm torn on this book. On the one hand, the ideas are good and he has some good insights into the process of making good checklists for complex situations, like medicine. On the other hand, I was really hoping for a more technical dive into the fields he discusses that involve the uses of checklists (aviation, medicine, and construction). It felt like his attempt to make the material accessible also made it superficial and too easily digestible; that he deliberately didn't go deep in order to push out an easier book (not accusing him of it, just the feeling I got from it). Still, I am glad I read it, and like Koch's 80/20 Principle, I have already tried applying concepts and learnings from the book to my job. #reading #selfimprovement #books #nonfiction

A post shared by Ryan Huckle (@rhuckle) on

 

Better – Atul Gawande

View this post on Instagram

-/8 – "Better" by Atul Gawande. This was a great listen. I first checked out his book, The Checklist Manifesto, a few years back. While I enjoyed it, I felt it lacked a certain depth I had hoped to have had in diving into the origins of some of the checklist systems he discussed. "Better" is the kind of book I had hoped for "Checklist." This is the book of a craftsman reflecting on his work. He muses on many critical areas of medicine and asks how doctors have tackled the idea of being better. Here, he runs the gamut of behaviour changes, adapting to impossible situations, monetary incentives ruining practice, the need for diligence, and a few other valuable essays. While he's a surgeon, he explores fields outside of his own to see what kind of lessons we can learn from their stories. The book is packed with valuable insights, which is unfortunate because I blasted through the book fairly quickly, so not all of it had a chance to stick while I mulled it over. A reread for sure! #reading #selfimprovement #books #nonfiction #learning #education #audiobook #medicine #reflection #craftmanship

A post shared by Ryan Huckle (@rhuckle) on

 

Complications – Atul Gawande

View this post on Instagram

-/17 – Complications by Atul Gawande. This is the third or fourth book I've read from Gawande. I first encountered him in the Checklist Manifesto, but like his last book I read I really enjoy his reflections on what it means to be a doctor. Something about the gravity of the work speaks to me, and his reflections on the gaps in what can be known by people expected to have all the answers is fascinating to me. I should seek out books by other authors in professions that spark deep reflections on the training and expertise intersecting with the quest for mastery. #reading #selfimprovement #books #nonfiction #learning #education #audiobook #medicine #reflection #craftmanship #memoir #surgery #Libby #kpl

A post shared by Ryan Huckle (@rhuckle) on

Stay Awesome,

Ryan

Livestreaming For My Students

Last week, I tried a new tactic to engage with my students.  I was inspired by two workshops I attended during Conestoga’s annual E3 (Employees for Excellence in Education) Conference.  The first workshop covered how to write good assignment prompts, with clarity and purpose in mind, and the second covered strategies for writing for online courses.  In the course I manage online, my students were preparing to submit their first major philosophical paper, and historically my students do poorly on the writing side.  I largely attribute this to it being their first time trying to write a philosophical paper and their only exposure to this point was either essays in high school or non-philosophy essays for other courses in college.  After sitting in on these two workshops, I reflected on what I could do, in an online course, that would improve my student’s ability to write.  It’s challenging to engage with online students for two reasons:

  • first, you (almost) never meet your students face to face, so you lose the ability to use tone, voice, inflection, and body language to convey information, and
  • second, online courses are atemporal, which means you don’t engage with your students at the same time.

An idea I’ve been kicking around for some time is creating a video for my students as an added bit of content for the course.  The problem with this option is it’s still fairly static and easy for students to skip if they feel it doesn’t contribute to improving their assessments.  It also goes in one direction, where I speak at my camera rather than engaging with the students.

However, I’ve been mulling over another option.  I have borrowed a web camera from my podcasting partner, I have a good microphone, and I delivered a webinar with a live Q&A in the middle of May.  I considered running a livestream last semester, however when I offered the option to the students, I had no requests for it.  But this semester, I decided to set it up and run it, regardless if students attended or not.  At worst, it would be a wasted hour of my time.  However, the benefits would be two-fold: my students would have a chance to interact with me and ask me questions about their assignment, and it would give me practice with a new skill set.

I picked a date and time, figured out how to broadcast (in the end, I went with Twitch, but next time I’ll test out YouTube Live) and went for it.  I had 4-7 students drop in, which is fairly low engagement, however the questions were really good and I had a lot of fun actively engaging with students again.

One unfortunate thing was I didn’t set up the system to auto-record, so I don’t have a copy of the livestream to review or upload.  I ended up recording a second (static) video to cover the main points so that my students had something to reference when they were completing and submitting their essays this past weekend.

It was a good experience and I plan to run at least one livestream per semester moving forward.  I have yet to grade the papers, so I don’t know if I had a material impact on their performance, but in time I hope that my students will get better with the added direction I can give them.  I also now have a video that I can post to help them think through the process of writing a philosophical paper.  If nothing else, it’s good to build handy resources and have them available for your students.  My goal is to help my students improve their thinking and writing as a result of taking my course.  Even if their papers are 1% better as a result of my direction, it’s worth it.

Stay Awesome,

Ryan

Grading Drift

I’ve been reflecting on the concept of grade drift recently. My brain usually turns to grade drifting at the end of each semester as I evaluate how objectively I have scored the students. If you’ve never taught before, what I call “grade drift” is the tendency for an instructor to allow their grading standards to drift as they grade an assignment. I’m not sure how prevalent this is for assessments that are concerned with exact answers, such as math, science, engineering, programming, etc. But in assessments that deal with qualitative or creative responses, it’s common for you as an instructor to change your evaluation thresholds as you make your way through the pile of papers.

There are a few reasons why this happens. In my first courses, my grades would drift if I didn’t have a sufficiently robust marking guide to help me narrow down the kinds of answers I was looking for in a student’s response. Now that I’ve taught the course a few times, I know generally what I’m looking for and can go through the assignments with a checklist of items. This form of grade drift is usually related to lacking experience in teaching and grading, especially if your course delivery isn’t tied to your course learning outcomes.

However, there is another reason why your grades might drift. On the one hand, you want to approach your evaluations as if they are objective – that there are clear right and wrong answers. After all, your grading should be defensible, and one way to defend the grade you grant is by pointing to an objective standard. However, on the other hand, you want your evaluations of your student’s work to allow for imagination, creativity, and novel connections between ideas. You also want your evaluations to acknowledge that your classes have unique compositions of people, with their own experiences and their own progressions through your course. Just because you teach the same material each semester doesn’t mean your students progress through it at the same rate. Different students will take the material differently, which means the average response you read through will be different for each cohort of students.

This means that some groups will be “better” than others in how they perform. You can choose to penalize the students for not meeting an abstract standard that you’ve set, or you can meet the students where they are and make it your goal in the course to improve their performance as you go.

I know that some instructors will take the first approach. They will believe their teaching, evaluations, and courses are reflective of an external, eternal, objective standard; they don’t think they are being arbitrary. I’m willing to bet that these are also instructors who believe that a 70% in their course is “like a 50% in other courses” (a literal thing I’ve heard said by a peer). I think this approach is wrong and it reflects a misunderstanding of what it means to pass a course. When a student passes a course, it means they have met the minimum requirements of the course learning objectives. Anything above that should reflect varying degrees of competence and mastery. If your course is designed so that a student is only “just” meeting the standards with anything above a 70%, you haven’t calibrated the course or your expectations very well. At an undergraduate level, your job is to elevate your students, and improve their abilities while meeting the program learning outcome objectives.

I find myself on the other side of this issue. I understand what the goals are of the course I teach, and my aim is less concerned with ensuring my students can adequately explain all the theory they are exposed to in the course. Instead, my goals is to make my students better thinkers and writers from where they enter my course to their exit. If they can explain Aristotle’s ideas of tragedy by the end, great! But I won’t lose sleep if they can’t. My course is a general elective, and that means I’m supposed to round them out as students and people. I care less about them absorbing my esoteric knowledge and instead I care more about them learning how to think, reason, and communicate their ideas clearly.

And that’s why sometimes my grades drift as I score the student’s rubrics. I start off with some ideas of where they could be, then I calibrate my grading to meet them at their level. It’s a little more work for me, but I think it better captures the student’s performance.

As indicated above, there are a few strategies for overcoming this, such as having a clear rubric and clear notes on what you are looking for.  Also, marking all the students on one question, one at a time, rather than marking an entire submission will make it easy to compare student fairly.

While this is an unfortunate thing (the students like to think they are being objectively evaluated) this will happen as you come to understand the natural curve of your class.  You can either grade your students to one standard, or you can reflexively respond to your class’s own knowledge and aptitudes.  If your course objectives are clear, then you can feel free to adapt your grading to match the student’s progress.

Stay Awesome,

Ryan