Book Ideas – Types of Feedback

I’ve been reading Scott Young’s recently released book, Ultralearning, and I think it’s a pretty good summary of how one can take on an intense learning project for personal and professional development.  It functions like an autodidact’s road map with plenty of good tips, insights, and stories to round things out.  Elements of the animated bibliography are present, but I don’t find it contrived in its execution.  The stories help frame the chapter and serve as an introduction to the core material.

It’s funny how last week I was talking about mnemonic devices, because after drafting that post I ended up reading about the concept in Chapter 10 of the book as it dealt with ways of supporting retention of material you learn.

In chapter 9 of the book, Young talks about ways of providing feedback in the learning process, whether the feedback is provided from others or feedback you can use in your own learning process.  He parses out three kinds of feedback that I found interesting, not only for my own personal use in learning, but also as something I should keep in mind as a teacher.

The three kinds of feedback he outlines are outcome feedback, informational feedback, and corrective feedback.  Each type of feedback serves a specific purpose, and you should be mindful of the context the feedback is given, as the wrong type of feedback can set you back in your learning.

Outcome feedback – provides information on whether you are getting answers right or if you are meeting a pre-identified set of learning objectives.  It tells you that you are right but doesn’t give any indication of why (or why you are wrong).

Informational feedback – provides further information to explain the underlying reason why something is right or wrong.  It can be informative to re-affirm what you have learned, and can identify key areas of strength or weakness, however it does not create a concrete process forward.

Corrective feedback – provides, as the name indicates, a path forward for the learner in terms of how to overcome deficiencies.  It details not only how one is right/wrong, why they are right/wrong, but how to address or avoid being wrong.  This type of feedback not only requires a level of comprehension of the material, but requires sufficient understanding to teach the underlying processes to the learner through explanation, demonstration, suggestion, etc.

As a teacher, it’s important to know what kind of feedback is warranted and under what circumstances.  Most of us tend to focus just on outcomes, but students often don’t learn from pure outcome assessment.  Rather, you need to take the further steps to go beyond an evaluation and ensure you are addressing the underlying deficiency present in the student’s performance.  Outcome assessment is awesome because it’s quick and definitive, but it’s also lazy if your goal is to improve your students.  On the other hand, corrective feedback is desirable but it’s labour-intensive and must be done carefully so as not to remove critical thinking from your student – you don’t want them to merely follow your instructions but instead you want to promote their thinking and reasoning through problems without your guidance.

Stay Awesome,

Ryan

Memorization and my Campus Response Experience

In January of 2008, I was walking through my university campus’s student centre and passed by a table for the UW Campus Response Team, whom were recruiting volunteers for the new semester.  I doubled back, chatted with the team members, and signed-up to participate in their interview process.  I had taken first aid courses periodically during my cub scout and army cadet days, plus I had ran some basic first aid courses while abroad, so it felt like a good fit.

In retrospect, my “experience” was quite paltry, but I had shown the team managers that I had enough of the “right stuff” that they invited me to join the team and participate in the weekend training course they put on for new recruits.  It’s an intense crash course in first aid skills that were well beyond my experiences and the training spanned several hours Friday night and all days Saturday and Sunday, before you perform your final scenario test to qualify as a secondary responder.

The material covered was largely derived from emergency first responder courses, along with some material covered for pre-hospital trauma professions (e.g. fire fighters and paramedics).  The training was designed to create heuristics in the responder’s mind to quickly flow through critical details while gathering as much information as possible and start treatment momentum.  The last thing you want is for a responder to have to intentionally think through what steps they should follow, because it shunts cognitive capacity away from situational awareness and into operational procedures.

In an effort to automate one’s thinking, you end up doing a lot of mock scenarios and skill drills.  As a responder, you end up creating a script in your mind to follow.  The script is based on a common set of things to attend to, which you follow according to handy mnemonics and other memory aids.

Despite the mnemonics functioning to provide mental triggers for actions, you still need to learn the process to go along with the mnemonics, and from the start of training weekend, you only have precious few hours after training concludes for the day to encode the information out of your working memory and into longer term storage.

I needed a way to quickly drill myself and aid in recall.  The system I settled on was to get some window writable markers and write out my mnemonic devices on the bathroom mirror.  Every time I used or walked passed the washroom, I would attempt to fill in as many of the mnemonics as I could remember, and note where I made mistakes.  Through constant repetition, I was able to turn:

E
M
C
A
P
I
E

A
V
P
U

P
E
A
R
L

A
B
C
D

S
A
M
P
L
E

O
P
Q
R
S
T

Into

Environment check
Mechanism of injury?
Count the casualties
Allied agencies?
Personal protection
Introduce yourself
Events leading?

Alert
Verbal
Pain
Unresponsive

Pupils are
Equal
And
Reactive to
Light

Airway
Breathing
Circulation
Deadly bleeds

Signs and Symptoms
Allergies
Medications
Past medical history
Last meal/beverage intake
Events leading

Onset
Provocation
Quality
Radiation
Severity
Time

It was a quick and dirty way to give myself quick feedback on these concepts that I could readily apply to my first aid treatment during training and eventually on shift.  Any time I lost momentum or felt nervous about the judges evaluating me, I would mentally go back to my bathroom mirror and fill in the blanks.  I haven’t been on the first aid team in almost a decade but these concepts easily come back to me, even during my crazy nights at the bar.  It’s a testament to the stickiness of the ideas and the effectiveness of the drills.

Stay Awesome,

Ryan

A Note to My Students

I closed out the winter semester last week, submitted my marks, and took a breather.  During the next week, I will be prepping my next semester and updating the course shell in anticipation of the start of the new semester.  During the interim, I’ve been reflecting on how the last semester went, and what I can do to improve the student experience in my online course.  My failure rate was higher than usual, and I want to make sure that I’m doing what I can to address those elements I can control.

Were I to give some advice to my students, this is what I would say:

Hello Students!

It is the start of a new semester, and I welcome you to the course.  If this is your first general education elective or philosophy course, I hope it meets your expectations.  Gen Eds tend to be a special kind of course.  This is one of the few courses you get a chance to choose, but it’s also one of the courses you’ll take that is a departure from your core major courses.  While you need the Gen Ed credit for graduating, you will feel the conflict over prioritizing the rest of the semester’s courses that will lead directly to your career.  It is very easy to let my course slip to the back-burner.  I recognize this and can understand your predicament.

In light of that, I want to give you some advice on how to do well on my assignments.  If there is one piece of advice I can give that will maximize your chances of passing my course, it’s that you have to do the work and submit your assignments.  As obvious as it might be that in order for me to give you a grade, you have to give me something to mark, I know that you will look at the assignment weightings and judge that the assignment is not worth your time to complete when you have other “more important” assignments to turn in.  Unfortunately, each submission you don’t turn in is essentially free marks that you will miss.  A good grade in this course is not won through stellar big assignments.  It’s about showing up consistently and slogging through the little assignment.  All those little assignments add up.

It can be intimidating to do philosophy.  If you are used to coming up with the right answers easily, you can face down a philosophy paper and become paralyzed by the weight of the work.  However, I want you to remember one important fact: doing good work in philosophy is not about thinking big ideas.  To do good work in philosophy, you must be good at communicating ideas.  I don’t expect you to have the “right” answer.  I don’t expect you to fully understand the concepts you are encountering.  Instead, I expect you to give an honest attempt to grapple with the ideas, and for you to use what you are learning in the module to play around with the ideas.  The more connections you can make between the ideas, the better.  Also, the more simply you can communicate those ideas, the better.  Don’t try to wow me with big vocabularies and vague writing.

I am generally not a hard marker.  I value progress and earnest intellectual work over feeding me the “right” answer.  Don’t give me what you think I want to read, and don’t give me your unsupported opinions.  Learn to play around with the ideas and explore topics you’ve never thought about before.  Make sure to attribute your ideas, and make sure you keep your reader in mind when you write your papers; explain the concepts to me as if I am a your grandmother.  If you do that every week, and put in an honest effort, I won’t let you fail my course.

Stay Awesome,

Ryan

Institutional Systems and Game Theory

One of the hardest lessons I grapple with is treating systems (especially bureaucracies) as a series of “games.”  By games, I’m treating it in the academic sense as a series of interactions between parties that has rules, outcomes/payoffs, and strategies.  Being the meek person that I am, I tend to default to the assumption that the stated rules are all that there is, and you are expected to follow the prescribed process if you are seeking an outcome.  The truth is, in most cases there are multiple strategies that you can use to seek out advantageous outcomes for yourself.  Depending on how the rules are set up, you can avail yourself of several options, both sanctioned and unsanctioned.

For instance, in the case of students, you need to achieve a certain grade to pass a course (say, a 55%).  There are a number of strategies you can use depending on what outcome you are seeking:

  • If you are seeking the highest grade possible – you study the textbook, attend lectures, attend office hours, learn the rubric, do well on assessments, and challenge grades to bump your marks up.
  • If you are seeking mastery of the content – you study the textbook, attend lectures, attend office hours to resolve unclear topics, research the topic, create good study notes, take practice tests, and learn from mistakes.
  • If you are seeking a moderate pass – you prioritize the work and tackle the highest value graded units to achieve at least a minimal passing grade, and you disregard low-return work that requires lots of effort for little ROI, you attend only the lectures required to get information you need, and likely get notes from peers.
  • If you are seeking a pass regardless of content mastery – you can cheat and hope you are not discovered by your professor, then deny any wrong-doing if caught or present excuses to justify your behaviour.  If that doesn’t work, you appeal using the institutions mechanism.

Something to keep in mind is that cheating is still considered at “legitimate” strategy as long as you don’t get caught, because the goal is to secure your desired outcome.  If you aren’t caught, it’s because your strategy beat out your opponent, and you won your outcome.  It might be that cheating goes against the system or the intended processes put in place, but if an adequate system to police the rules isn’t in place, you can exploit that strategy to your advantage.

I hope it’s obvious that I’m not advocating for academic cheating.  I do my best to guard against cheating because I think it runs counter to my goals as a teacher.  I want my students to learn to play the game as I see it should  be played, because the skills and strategies used for my class are both useful and valuable outside of my class – the ability to read a variety of perspectives with an open-mind, the ability to articulate your position with evidence, the ability to connect ideas across different knowledge domains, etc.

I exploit the same rules when I help students navigate their way through the institution’s byzantine labyrinths and silo’d departments when they come to me with problems in their program.  I want them to get through their education with the least institutional friction and cost possible – school is hard enough and I don’t want them wasting time jumping through frivolous hoops because the systems aren’t set up optimally.

I sometimes feel irked or offended when I catch a student cheating, or catch someone lying to me.  I try to check myself in those instances because I know it’s not meant as a personal slight against me when these things happen; it’s because of the incentive structures in place.  A legitimate strategy is not available to the person, so they seek an alternative strategy to get what they want.  They are playing a game and their strategy is competing against mine when they submit plagiarized work, or hand me a fake ID at the bar I work at.  If my strategy is sufficiently robust, I can catch and counter their strategy.  But if I’m also using a sub-optimal strategy, then it’s more likely the case that their strategy will exploit my complacency.

It’s nothing personal.  It’s just how the institutional games work.

Stay Awesome,

Ryan

Self-Education Resources

Post-secondary education has never been more accessible to the average person.  We may have a long ways to go in terms of making courses more accessible for learners and reducing the financial barriers that keep students from being successful in school, but it is nevertheless an undeniable fact that there are more people who have been to post-secondary schooling than the entire history of people attending higher learning.

One issue with the proliferation of access is that it’s getting harder to stand-out in the workforce.  With so many people carrying credentials, the golden ticket that a diploma or degree used to confer has lost some of its value.  Your choices are to either go to industries where they are starving for workers (if you are looking for a solid career with good prospects, you should become a welder NOW), or figure out a way to become a better problem-solver to stand out amongst the crowd.

Another issue that complicates matters is that industry and technology is changing at such a rapid rate that you can no longer rest on your laurels that your program of study will adequately prepare you for work in your industry.  The techniques, technologies, and skills you learn in your first year may be obsolete by the end of your final year.

Therefore, it’s important to develop your ability to self-educate.  Knowing where you can find free or cheap resources can be a huge advantage when developing yourself in your career.  Here are some of the resources I use to teach myself.

Top Spot: your Public Library

In my humble opinion, the public library is one of the greatest inventions of all time.  Whether you are taking classes they offer, using resources in their catalog, or availing yourself of the free access to materials like online journals and portals, there is almost no limit to  the access your library card can provide.  When my HVAC system went on the fritz, I was able to check out an HVAC manual to help me learn just what the heck an HVAC system does so that I could understand what repairs were needed, and how to better care for the system in the future.

YouTube

YouTube changed the game when it comes to sharing knowledge.  Don’t get me wrong, books are great (the necessary precursor to the greatest invention of all time; see: public library entry), but unless your book has incredibly detailed diagrams, the video format will always be the superior resource for teaching hands-on skills.  When I had to fix my roof, I turned to videos to learn how to remove individual shingles and replace them myself.

Coursera

Coursera is all the benefits of attending lectures without the associated costs.  Granted, if you want formal recognition of completing Coursera courses, you’ll need to pay for the access.  However, nearly every Coursera course has the option for you to audit the course for free, which gives you access to the lecture content and some of the supplementary material.

Reddit (and other specialty discussion forums)

I suppose I should have used “Google” as the category here since I often will search for solutions through Google’s indexed results.  However, dedicated online communities are some of the best resources to learn from.  They often post comprehensive resources and how-to manuals, and are usually great about providing solutions when you are stuck on specific problems.  If you can find a good community that isn’t locked behind a paywall, you can lose yourself for hours in it’s wealth of information.

Lynda.com

While not a free resource, this is something that my employer has provided to its employees at no cost.  You should check to see if your employer offers any services for employees to self-develop because you might be missing out on a ton of non-financial benefits.  Lynda is a great resource for comprehensive courses on a wide variety of tech and business topics.  It’s a bit restrictive if you are looking for non-business courses, but it’s worth checking out for learning the basics you’ll need to navigate your early career development.

Udemy

Another paid service, I find Udemy great for high tech courses where I want to develop specific skills, such as in Python or in using Adobe software.  I wait for courses to go on sale, and I snap up courses up to 90% off their full price.

Ask friends

My final suggestion is to tap your friends to see if anyone can help you learn new skills.  Obviously, you don’t want to exploit your friends – you should pay for their services where appropriate.  However, in some cases your friends can be great resources to tackle projects.  Not only do you get to leverage their unique skills or experience, but you also get quality time together.  My entire podcast and music run for Woot Suit Riot has been some of the most formative experiences I’ve had, all because I was making stuff with friends.

All of this is framed as advice to help you in your career, however the truth is that you should be seeking to educate yourself for any project your’re interested in, regardless of whether you can get paid for the skills or not.  I took painting classes earlier this year at my local art store because I wanted to learn how to paint.  This isn’t a skillset that directly will get me promoted, but it rounds me out and allows me to explore my creative side.

The point of self-education or self-development is for you to become more of the person you want to be.  It’s often hard work, but the experiences are well-worth the effort.

Stay Awesome,

Ryan

Thoughts on Providing Feedback

I had a heartwarming moment this past week where a student provided some positive feedback about my teaching.  I “teach” an online course through the college and have been one of the instructors for the past year.  I find online courses a difficult medium to engage with students because you never get face to face interaction with them, so you need to find other ways to reach out and build a rapport.

One of the methods I choose, and use when I teach in-class as well, is to take time to provide comments and feedback on assignments.  I think of assignments as ways for me to provide real, concrete feedback on student performance (rather than just a grade).  In addition to the marking rubric, I will give three kinds of feedback:

  • comments on how well a student engaged with material according to a certain criteria;
  • areas where the student did not measure up to the criteria and why;
  • (most importantly) guidance on how the student could improve in the future.

That last bullet point is important to me because when I reflect back on my own learning in school, I realize that grades are often a poor way of gauging how well you understand the material.  Instead, the numeric grade stands as a proxy for how many “answers” you got right.  For the purposes of education, I’m concerned of the other side of the grade – the marks that you didn’t get; why you didn’t get 100%.  If I, as an instructor, can fill in the details of what the student missed, I can guide them towards improving in the future.

It takes me a while to grade assignments because I spend the extra time giving constructive feedback to the students.  If I simply told them what they  got wrong, the student won’t have a sense of what steps they could take to do better next time.  I don’t want the students to use trial and error to figure out how to improve skills.  Instead, I will give them explicit feedback on what I want them to do differently in the future.

Perhaps you are thinking that I’m coddling the students too much.  Maybe you are right, but I assume the students are here to learn, that I’m positioned as the expert, and that teaching is more about molding and guiding students, not expecting them to stumble across the correct answers.  I want the students to be mindful of what I expect of them (and it forces me to reflect on what I want to see in the student’s progress, which is significantly harder than a capricious grade you slap on after reading through a paper).

If done right, you should see a general improvement in the quality of the assignments you receive over the semester.  Students who are looking to improve will pay attention to your feedback and will get better over time.

I’ll close off with an excerpt from the student who emailed me:

I struggle with philosophy but I wanted to try out this elective. Your feedback has really helped me to know where I have gone wrong and what to work on for the future assignments. I have never got so much feedback from teacher, so thank you.

Stay Awesome,

Ryan

Study Strategies #5 – Application

One of the hardest things I find my students struggling with is not grappling with deep philosophical thoughts, or technical jargon (to be fair, they do struggle with these as well), but it’s in the application of course material.  Most often, when my students submit work for me to evaluate, they submit work that is either:

  • straight opinion (read: a submission that is not structured as an argument with evidence and connecting ideas through logic); or
  • an attempt to solve or provide a definitive response for all the problems of this philosophical issue in about a page.

The thing my students don’t realize is that I don’t care whether they “solve” the philosophical problem.  Granted, I don’t expect them to be able to solve the problem in a page, but that’s not the point.

The purpose of the exercise is for me to check the thought-process of whether they are able to understand the material and work with it.

I was recently chatting with a Program Chair about her time teaching engineering courses.  She noted that often she’d give problem sets that lacked defined measurements, and her students would pause to ask what the length or value of the unknowns are.  She was very frank that she didn’t care what number their calculator displayed because it was more important for her to see whether the students could think through the problem, manipulate the equations, and understand how to go about solving a problem.  For her, the solution was extraneous for the purposes of the class – it was a quick and convenient way to mark an answer right or wrong, but not entirely indicative of whether the student was understanding the concepts.

Now, you may say that this is all well and good for engineering, but how does that apply to philosophy (“But, philosophy has no right answers!!!) or any of the other soft sciences or humanities disciplines.

The truth is that the faster you try to apply the concepts, the easier it is to learn and make the concepts stick, and it’s not all that different across disciplines.  If you are trying to learn a concept, the best thing you can do is to try to take what you think you are learning, and apply it to a novel situation.  By focusing less on the details and working with the core concepts, you get a chance to see what makes sense to you and where your gaps in knowledge are.

For the course I teach, the students work their way through the online module materials, which includes extra readings, embedded videos, probing questions, links to additional sources, etc.  Then, after a round of discussion board posts, the students have a weekly essay prompt related to the week’s topic.

Early in the course, my students will often reply strictly to the essay question with what they think the right answer is.  Through my weekly rubric feedback and general emails to the class, I encourage them to go back to the module content and apply the concepts they are learning to the essay prompt.  What would so and so say about this concept?  How does this school of thought define this concept?  Do you agree with how this concept gets framed?

The point of undergraduate philosophy courses is not for students to generate original philosophical thought.  That is an aim, but it shouldn’t be the outcome.  Instead, the instructor should be guiding the students to think better and understand the concepts being covered so that they can then apply it in novel situations.

When studying, a good way to learn the concepts it to try and extract the ideas from how the author framed them and see how you can apply those ideas in new ways.  It reinforces the learning and helps to spot gaps in understanding in a way that straight memorization doesn’t provide.

Stay Awesome,

Ryan