Book Ideas – Types of Feedback

I’ve been reading Scott Young’s recently released book, Ultralearning, and I think it’s a pretty good summary of how one can take on an intense learning project for personal and professional development.  It functions like an autodidact’s road map with plenty of good tips, insights, and stories to round things out.  Elements of the animated bibliography are present, but I don’t find it contrived in its execution.  The stories help frame the chapter and serve as an introduction to the core material.

It’s funny how last week I was talking about mnemonic devices, because after drafting that post I ended up reading about the concept in Chapter 10 of the book as it dealt with ways of supporting retention of material you learn.

In chapter 9 of the book, Young talks about ways of providing feedback in the learning process, whether the feedback is provided from others or feedback you can use in your own learning process.  He parses out three kinds of feedback that I found interesting, not only for my own personal use in learning, but also as something I should keep in mind as a teacher.

The three kinds of feedback he outlines are outcome feedback, informational feedback, and corrective feedback.  Each type of feedback serves a specific purpose, and you should be mindful of the context the feedback is given, as the wrong type of feedback can set you back in your learning.

Outcome feedback – provides information on whether you are getting answers right or if you are meeting a pre-identified set of learning objectives.  It tells you that you are right but doesn’t give any indication of why (or why you are wrong).

Informational feedback – provides further information to explain the underlying reason why something is right or wrong.  It can be informative to re-affirm what you have learned, and can identify key areas of strength or weakness, however it does not create a concrete process forward.

Corrective feedback – provides, as the name indicates, a path forward for the learner in terms of how to overcome deficiencies.  It details not only how one is right/wrong, why they are right/wrong, but how to address or avoid being wrong.  This type of feedback not only requires a level of comprehension of the material, but requires sufficient understanding to teach the underlying processes to the learner through explanation, demonstration, suggestion, etc.

As a teacher, it’s important to know what kind of feedback is warranted and under what circumstances.  Most of us tend to focus just on outcomes, but students often don’t learn from pure outcome assessment.  Rather, you need to take the further steps to go beyond an evaluation and ensure you are addressing the underlying deficiency present in the student’s performance.  Outcome assessment is awesome because it’s quick and definitive, but it’s also lazy if your goal is to improve your students.  On the other hand, corrective feedback is desirable but it’s labour-intensive and must be done carefully so as not to remove critical thinking from your student – you don’t want them to merely follow your instructions but instead you want to promote their thinking and reasoning through problems without your guidance.

Stay Awesome,

Ryan

Advertisements

Memorization and my Campus Response Experience

In January of 2008, I was walking through my university campus’s student centre and passed by a table for the UW Campus Response Team, whom were recruiting volunteers for the new semester.  I doubled back, chatted with the team members, and signed-up to participate in their interview process.  I had taken first aid courses periodically during my cub scout and army cadet days, plus I had ran some basic first aid courses while abroad, so it felt like a good fit.

In retrospect, my “experience” was quite paltry, but I had shown the team managers that I had enough of the “right stuff” that they invited me to join the team and participate in the weekend training course they put on for new recruits.  It’s an intense crash course in first aid skills that were well beyond my experiences and the training spanned several hours Friday night and all days Saturday and Sunday, before you perform your final scenario test to qualify as a secondary responder.

The material covered was largely derived from emergency first responder courses, along with some material covered for pre-hospital trauma professions (e.g. fire fighters and paramedics).  The training was designed to create heuristics in the responder’s mind to quickly flow through critical details while gathering as much information as possible and start treatment momentum.  The last thing you want is for a responder to have to intentionally think through what steps they should follow, because it shunts cognitive capacity away from situational awareness and into operational procedures.

In an effort to automate one’s thinking, you end up doing a lot of mock scenarios and skill drills.  As a responder, you end up creating a script in your mind to follow.  The script is based on a common set of things to attend to, which you follow according to handy mnemonics and other memory aids.

Despite the mnemonics functioning to provide mental triggers for actions, you still need to learn the process to go along with the mnemonics, and from the start of training weekend, you only have precious few hours after training concludes for the day to encode the information out of your working memory and into longer term storage.

I needed a way to quickly drill myself and aid in recall.  The system I settled on was to get some window writable markers and write out my mnemonic devices on the bathroom mirror.  Every time I used or walked passed the washroom, I would attempt to fill in as many of the mnemonics as I could remember, and note where I made mistakes.  Through constant repetition, I was able to turn:

E
M
C
A
P
I
E

A
V
P
U

P
E
A
R
L

A
B
C
D

S
A
M
P
L
E

O
P
Q
R
S
T

Into

Environment check
Mechanism of injury?
Count the casualties
Allied agencies?
Personal protection
Introduce yourself
Events leading?

Alert
Verbal
Pain
Unresponsive

Pupils are
Equal
And
Reactive to
Light

Airway
Breathing
Circulation
Deadly bleeds

Signs and Symptoms
Allergies
Medications
Past medical history
Last meal/beverage intake
Events leading

Onset
Provocation
Quality
Radiation
Severity
Time

It was a quick and dirty way to give myself quick feedback on these concepts that I could readily apply to my first aid treatment during training and eventually on shift.  Any time I lost momentum or felt nervous about the judges evaluating me, I would mentally go back to my bathroom mirror and fill in the blanks.  I haven’t been on the first aid team in almost a decade but these concepts easily come back to me, even during my crazy nights at the bar.  It’s a testament to the stickiness of the ideas and the effectiveness of the drills.

Stay Awesome,

Ryan

Grading Drift

I’ve been reflecting on the concept of grade drift recently. My brain usually turns to grade drifting at the end of each semester as I evaluate how objectively I have scored the students. If you’ve never taught before, what I call “grade drift” is the tendency for an instructor to allow their grading standards to drift as they grade an assignment. I’m not sure how prevalent this is for assessments that are concerned with exact answers, such as math, science, engineering, programming, etc. But in assessments that deal with qualitative or creative responses, it’s common for you as an instructor to change your evaluation thresholds as you make your way through the pile of papers.

There are a few reasons why this happens. In my first courses, my grades would drift if I didn’t have a sufficiently robust marking guide to help me narrow down the kinds of answers I was looking for in a student’s response. Now that I’ve taught the course a few times, I know generally what I’m looking for and can go through the assignments with a checklist of items. This form of grade drift is usually related to lacking experience in teaching and grading, especially if your course delivery isn’t tied to your course learning outcomes.

However, there is another reason why your grades might drift. On the one hand, you want to approach your evaluations as if they are objective – that there are clear right and wrong answers. After all, your grading should be defensible, and one way to defend the grade you grant is by pointing to an objective standard. However, on the other hand, you want your evaluations of your student’s work to allow for imagination, creativity, and novel connections between ideas. You also want your evaluations to acknowledge that your classes have unique compositions of people, with their own experiences and their own progressions through your course. Just because you teach the same material each semester doesn’t mean your students progress through it at the same rate. Different students will take the material differently, which means the average response you read through will be different for each cohort of students.

This means that some groups will be “better” than others in how they perform. You can choose to penalize the students for not meeting an abstract standard that you’ve set, or you can meet the students where they are and make it your goal in the course to improve their performance as you go.

I know that some instructors will take the first approach. They will believe their teaching, evaluations, and courses are reflective of an external, eternal, objective standard; they don’t think they are being arbitrary. I’m willing to bet that these are also instructors who believe that a 70% in their course is “like a 50% in other courses” (a literal thing I’ve heard said by a peer). I think this approach is wrong and it reflects a misunderstanding of what it means to pass a course. When a student passes a course, it means they have met the minimum requirements of the course learning objectives. Anything above that should reflect varying degrees of competence and mastery. If your course is designed so that a student is only “just” meeting the standards with anything above a 70%, you haven’t calibrated the course or your expectations very well. At an undergraduate level, your job is to elevate your students, and improve their abilities while meeting the program learning outcome objectives.

I find myself on the other side of this issue. I understand what the goals are of the course I teach, and my aim is less concerned with ensuring my students can adequately explain all the theory they are exposed to in the course. Instead, my goals is to make my students better thinkers and writers from where they enter my course to their exit. If they can explain Aristotle’s ideas of tragedy by the end, great! But I won’t lose sleep if they can’t. My course is a general elective, and that means I’m supposed to round them out as students and people. I care less about them absorbing my esoteric knowledge and instead I care more about them learning how to think, reason, and communicate their ideas clearly.

And that’s why sometimes my grades drift as I score the student’s rubrics. I start off with some ideas of where they could be, then I calibrate my grading to meet them at their level. It’s a little more work for me, but I think it better captures the student’s performance.

As indicated above, there are a few strategies for overcoming this, such as having a clear rubric and clear notes on what you are looking for.  Also, marking all the students on one question, one at a time, rather than marking an entire submission will make it easy to compare student fairly.

While this is an unfortunate thing (the students like to think they are being objectively evaluated) this will happen as you come to understand the natural curve of your class.  You can either grade your students to one standard, or you can reflexively respond to your class’s own knowledge and aptitudes.  If your course objectives are clear, then you can feel free to adapt your grading to match the student’s progress.

Stay Awesome,

Ryan

Two Approaches to Intro Phil

By the time this is published, I will have finished my grading for the Winter semester, however as of drafting, I am procrastinating from finishing.  That means it’s time to let my mind wander and to blog about it.

The course I’ve been teaching since Fall 2016 is a light, multidisciplinary romp through philosophy that examines various topics in human experience through multiple lenses.  Often, this is one of the first general education courses the students encounter, and for many it is the first time they are getting into something approximating formalized philosophy.  It’s an online course, so the students work their way through the module material, post to discussion boards, and submit weekly asignments.  The modules are a mix of text, graphics, and embedded videos, and all things considered is a pretty marked departure from my first course in philosophy.

My introduction to philosophy was significantly more old school.  We all had copies of Cahn’s Classics in Western Philosophy (the sixth edition), and we started the semester on Plato.  It was my first class in university, and my first introduction to academic philosophy.  In a small way, that course has dictated how, in my mind, philosophy gets taught.  It is my default template for teaching – the professor stands at the front and pontificates in fifty minute blocks of time.  I never questioned it; I just tried to take notes, not really knowing how one takes notes about the lecture material.

Since then, I’ve seen a few different pedagogical approaches to teaching an intro phil course.  You can teach the material chronologically, topically, using ancient sources, using modern case studies, you can approach the different branches of philosophy thematically, or you could teach the material historically.

But there is another axiom I’ve been thinking about when teaching philosophy (or any topic for that matter):

Should I require the students to read the text before or after the lecture?

On first glance, this seems like a silly question.  Afterall, the lecture is about exploring ideas, clarifying thought, and demonstrating how to extend the ideas.  Students should grapple with the text and work to come to an understanding.  If there are problems with the text, they can bring their questions to class for rich discussions.  After reading the text, the students can teach the concepts to each other and have fruitful conversations about the ideas.  Why would you want to let the students not read the text?

On the other hand, it depends on what your goals are.  You see, the problem with requiring the students to read the text prior to the lecture, at least at an introductory level, is that the students often don’t possess the vocabulary or the historical schema’s to truly understand the work in front of them.  In an introductory philosophy course, I’m not looking for students to master the material.  Often times, I’m not even looking for them to be right in their arguments.  Instead, I’m looking for other skills, like close reading, comprehension, and (most importantly) the ability to read text charitably.  The hardest thing for students to grapple with in intro phil is that they are seeking to be right and rarely allow themselves to explore ideas.

Plato wrote more than 2000 years ago – what could he possibly have to say in his silly dialogues that helps me today?  Given the current political climate, I’d say his Republic has a lot to say (I’m looking at you, Thrasymachus).

If I give students a text, they don’t have the benefit of a historical view.  They don’t understand the context in which the author is writing.  They don’t understand the conventions of language and allusions.  They don’t see the historical dialogue that is unfolding on the page, where the author is addressing thinkers who came before them.  For instance, in my experience, the vast majority of students who read Descartes for the first time end up thinking Descartes literally believes an evil genius is out to deceive him.  They get so fixated on trying to keep up with what Descartes is doing that they lose sight of the argument being laid out before their eyes.

Reading the text before class is a great way to have students develop critical thinking skills, but if you want the students to understand what they are reading, it might be worth your time to consider helping them to explore the text together.  At least for an intro phil course.

Stay Awesome,

Ryan

 

 

Student Coaching

Lately, I’ve noticed that in addition to my roles of administration and teaching, I’ve been spending more time coaching students.  It tends to come up in small ways, such as offering suggestions on how to word emails or how a student should approach talking to their professor about something in their class.  At first I was a little uncomfortable with taking on a coaching role when it wasn’t really part of my job.  That’s not to say I’m uncomfortable with doing tasks that are not explicitly written into my job description.  Instead, I was uncomfortable because I wasn’t sure if it was my place to offer guidance.  Sure, I might be a decade older than most of the students, but I sometimes struggle with the impostor syndrome: what do I know?  I’m just a lowly administrative assistant.

Having said that, I recognized a fundamental truth that I think many people take for granted – students are young.  I don’t just mean young in age, but also young in experience.  Most students haven’t had the same experiences I’ve had, whether that is my post-secondary schooling, grad school, or work.  Things that I take for granted that come second nature to me are wildly new for students just coming into school.  It’s even worse for students who are first-generation college/university students (like I was).  For some, they haven’t had a lot of experience navigating systems on their own.

We bemoan the helicopter parents and make snide remarks that students don’t know anything (i.e. “life skills”), but I think we should have more empathy.  Post-secondary is a big, scary place to navigate.  If it’s your first time living away from home, having to manage your own schedule, finances, and life, would you not also feel overwhelmed?

Instead of starting with the assumption that students are lazy, or wanting everything fed to them on a silver spoon, I try to take the approach that students don’t know how to narrow down their options or choose a path.  They are the modern Buridan’s ass stuck between competing options with no practical way of making decisions or selecting priorities.  Instead they focus on what’s immediate and take the path of least resistance (for them at the time).  Without a longer view and a chain of successes, their choices may seem short-sighted, but in their context it makes sense to them.  If you couple that kind of decision framework with the complex, convoluted machinery that is “real life,” you can hardly be surprised when students make sub-optimal decisions.

In light of this, my response is not to infantilize them, nor chide them for what one would judge to be their bad decisions.  Instead, I offer my perspective and anecdotes to provide teachable moments.  I provide insight into byzantine rule structures and explain my reasoning.  And I ultimately leave decisions or action up to them.  They must take ownership of the process because they have to be accountable for the outcomes.  The point of education is to create a safe space to fail and learn.  Therefore, our goal should be to set students up for success but also provide them with the opportunity to learn through their mistakes, not by punishing them into doing the right thing.  Rote learning works in some contexts, but in order to make deeper connections between ideas and develop ones ability to reason, students need to be coached on how to shift their perspective and see new connections.

And so, I sometimes coach students.  It’s pretty interesting so far.

Stay Awesome,

Ryan

 

Policy vs. Guidance Pedagogy

During an ethics board meeting recently, we discussed ways of providing direction to faculty members who have student-based research in their courses.  For faculty who have research elements built into their courses, it can be a challenge to determine what counts as research, and whether said research is subject to the rules governing conducting research at an institution (specifically in our case, whether an ethics application would need to be submitted to the board).  Not every scholarly activity necessarily counts as research, and not every kind of research requires an approval from the institutional research ethics board.  Since this can be a bit of a murky area, we have been considering ways of providing direction.

The conversation abstracted away from the specifics of this case, and we discussed some of the issues concerning policy and guidance, which applies to education and pedagogy more generally.

The benefit of policy is that it spells out clear expectations of what is expected, what the division of responsibility looks like, and what consequences might be considered in the event of a policy breach.  Policy is designed to protect the institution through due diligence, and it focuses on expressing what rules need to be followed in order to not get into trouble.  Loopholes arise when the policies are not sufficiently rigorous the cover contingency cases and when policies are not harmonized laterally or vertically with other policies.  Policy documents focus on the “ends.”

On the other hand, guidance documents focus on the “means” by providing suggestions and best practices that could be followed.  Guidance documents typically do not include comprehensive rules unless it’s appropriate.  Instead, the purpose of the guidance document is to provide clarity in ambiguity without necessarily spelling things out.  They are deliberately left open because guidance documents are meant to supplement and add to ongoing conversations within a field or system.  While guidance documents also do not provide comprehensive options to contingent situations, the strength of the guidance document is that it’s educational in intent – it provides reasoning that helps the reader understand the position it takes, and paints a vision of what success looks like.

I realized in the meeting that this has a lot of crossover into considerations for teaching.  It’s is better, in my opinion, to teach students frameworks for thinking, rather than rules for success.  In the case of ethics, I would avoid teaching students what rules they need to follow, and what they need to do to avoid getting into trouble.  Instead, I would seek to build good practices and habits into the material I’m teaching so that I can model what success looks like and help them understand why.  This way of conceiving the material is forward-thinking.  It gets the students to envision what the end-step looks like, and allows them to work backwards to figure out how they want to arrive there.  By focusing on the principles you want the students to uphold (as opposed to rules to follow), the students learn to think for themselves and are able to justify the decisions they make.  This also has the benefit of avoiding the problem with prescriptive policies – students are prepared to reason through novel situations based on principles.

 

Stay Awesome,

Ryan

 

Study Strategies #5 – Application

One of the hardest things I find my students struggling with is not grappling with deep philosophical thoughts, or technical jargon (to be fair, they do struggle with these as well), but it’s in the application of course material.  Most often, when my students submit work for me to evaluate, they submit work that is either:

  • straight opinion (read: a submission that is not structured as an argument with evidence and connecting ideas through logic); or
  • an attempt to solve or provide a definitive response for all the problems of this philosophical issue in about a page.

The thing my students don’t realize is that I don’t care whether they “solve” the philosophical problem.  Granted, I don’t expect them to be able to solve the problem in a page, but that’s not the point.

The purpose of the exercise is for me to check the thought-process of whether they are able to understand the material and work with it.

I was recently chatting with a Program Chair about her time teaching engineering courses.  She noted that often she’d give problem sets that lacked defined measurements, and her students would pause to ask what the length or value of the unknowns are.  She was very frank that she didn’t care what number their calculator displayed because it was more important for her to see whether the students could think through the problem, manipulate the equations, and understand how to go about solving a problem.  For her, the solution was extraneous for the purposes of the class – it was a quick and convenient way to mark an answer right or wrong, but not entirely indicative of whether the student was understanding the concepts.

Now, you may say that this is all well and good for engineering, but how does that apply to philosophy (“But, philosophy has no right answers!!!) or any of the other soft sciences or humanities disciplines.

The truth is that the faster you try to apply the concepts, the easier it is to learn and make the concepts stick, and it’s not all that different across disciplines.  If you are trying to learn a concept, the best thing you can do is to try to take what you think you are learning, and apply it to a novel situation.  By focusing less on the details and working with the core concepts, you get a chance to see what makes sense to you and where your gaps in knowledge are.

For the course I teach, the students work their way through the online module materials, which includes extra readings, embedded videos, probing questions, links to additional sources, etc.  Then, after a round of discussion board posts, the students have a weekly essay prompt related to the week’s topic.

Early in the course, my students will often reply strictly to the essay question with what they think the right answer is.  Through my weekly rubric feedback and general emails to the class, I encourage them to go back to the module content and apply the concepts they are learning to the essay prompt.  What would so and so say about this concept?  How does this school of thought define this concept?  Do you agree with how this concept gets framed?

The point of undergraduate philosophy courses is not for students to generate original philosophical thought.  That is an aim, but it shouldn’t be the outcome.  Instead, the instructor should be guiding the students to think better and understand the concepts being covered so that they can then apply it in novel situations.

When studying, a good way to learn the concepts it to try and extract the ideas from how the author framed them and see how you can apply those ideas in new ways.  It reinforces the learning and helps to spot gaps in understanding in a way that straight memorization doesn’t provide.

Stay Awesome,

Ryan