Citing Your Sources

writing-828911_1920
Image by Free-Photos from Pixabay 

While reading a post from A Learning A Day, I thought I’d keep the irony train rolling by linking to Rohan’s linked post from Derek Sivers about the perceived need to quote an idea’s source.  Specifically, I wanted to respond to this point:

2. School teaches us to reference. But we’re not trying to impress a teacher anymore. And every unnecessary fact dilutes our point.

I often reflect on the learning objectives I expect to achieve in a course lesson while teaching.  I try to parse out the meaningful things I want students to learn from rote procedural tasks that don’t serve a purpose.  The last thing I want to do is to reinforce the wrong lesson or derive the wrong conclusion from a student’s performance (e.g. did a student do well on a test because they understood the material, or because they are good at taking tests?).

Derek’s point above about references is well-taken and got me thinking: why do I want students to cite their sources?  I brainstormed a few reasons and listed them below with comments.

Reason

Thoughts

I want a student to be mindful of their research process (procedure). Having gone through writing my master’s thesis, it’s easy to lose track of references and citations if you don’t stay on top of it.  This isn’t super relevant to most assignment learning objectives, but it’s a good practice to have before launching into a bigger endeavor or capstone project.
2. I want a student to critically examine their own knowledge (what do they believe to be true facts, where did that fact come from, and why do they think it’s true). I’m not sure if making students cite their sources achieves this aim on its own, but I suppose I could use citation requirements to help guide them through this process.
3. I want a student to be mindful of idea ownership and give credit to people who have done work.

 

I’ve used this mostly in plagiarism cases where students copied work and submitted it as their own.  I try to distinguish between sloppy citing and outright theft, and I remind students that they shouldn’t get marks for work they didn’t do.  I’m still undecided if this is a rule of the academy or a legitimate thing to prevent fraudulently passing work off as your own in the future.  This point, though, is mostly relevant in academic contexts as opposed to Derek’s notes about doing this during conversations.
4. I want an easy way to see if the student did the work. This is a trick I’ve developed to see whether a student giving me their opinion is right by chance, or if they have informed their opinion by doing the course reading.  The same result could be gained if students inserted relevant information without citations, but the citations help to highlight this when I’m reading through their submission.  In other words, it makes my job easier.
5. I want to reinforce good academic writing habits. Using references is part of what it means to write academically, and is used as part of the integrity process.  This is only a good reason if my objective is to teach/reinforce academic writing for students.
6. This is the way it has always been done.

 

More cynically, requiring citations is part of the tradition, and who am I to question it?  It’s not a good reason to require it, but it is what it is.  I won’t included in the list to the left, but a more sadistic version of this is “I had to do it, so you have to (go through this rite of passage) too!”
7. I want to remain consistent with departmental policies and culture. Whether written or unstated, most departments adhere to some level of standards.  This was less the case for me in undergrad and it depended largely on the preferences of the prof.  By the time of my thesis, I ended up developing a hybrid referencing system that did not strictly follow any of the major citation methods.  I received no comments from anyone who reviewed my thesis on my citation practices.
8. It’s important to trace an idea’s lineage as much as possible to spot fabrication. If you are going to insert facts or conclusions into your work, it’s important to point to where you found them.  Without a citation or an adequate way of accounting for how you know what you purport to know, it’s possible that the information is made up.  Being able to trace these things helps, albeit this is more useful from a scholarship point of view, as I suspect a lay-reader isn’t concerned with checking a text for factual accuracy and instead takes it on authorial authority.

9.

Related – to see if a student is able to either properly reference work, or at the very least charitably restate ideas without dropping important content from the idea. This perhaps falls under sloppy citation practices, but on occasion students will misunderstand a piece of text and paraphrase or summarize information incorrectly.  Knowing where the student is drawing their source from can have pedagogical merit if you take the time to compare the student’s work with the source and discuss the divergence.

10.

Related – when an author cites their sources, a reader can use the bibliography of sources for further reading.

 

This is perhaps more for book nerds, but I love having references to be able to learn more for things that pique my interest.  This is, however, not the context Derek is referencing when he discusses giving citations during a normal conversation.  If Derek’s conversation partner was interested and want to know more, I’m sure they would ask Derek for more information.

11.

More abstractly, knowledge and academics is a web of mutually reinforcing facts, so academic writing is an extension of that reality. This one is a bit of a stretch as to why a student who is not adding to a body of knowledge is required to rigorously cite their sources in a pedagogical exercise, but I include this more epistemological point to try and be exhaustive.

12.

It’s a symbolic representation that the student (in most contexts) is not generating new or novel work/insights that creates new knowledge, but instead is remixing ideas from other sources. I think this is a good reminder of what the goal of the assignment should be (students are often far too ambitious in what they think they can reasonably achieve in x-number of pages), but I wouldn’t consider this to be an adequate reason to insist on proper citations.

13.

Like other skills, the act of referencing needs to be practiced. I’m sympathetic to this, but as Derek is implying, you should be practicing skills that transfer into other domains or that you will need.  In most instances, outside of school you don’t need to cite sources.

14.

Citing references is part of the argumentation process.  In order to build a successful argument, you must clearly express and state your premises, which includes any premises taken from the work of others (either their premises or their conclusions). I’m also sympathetic to this as I think everyone should keep in mind that arguments need to be made to help convey ideas.  It shows the logical chain from premise to conclusion and seeks to make the implicit explicit, and the unstated stated.

Other than a subset of the reasons above, a strict requirement for citations is often unnecessarily enforced in the classroom, and is almost never required outside of the academic setting.  I think there are some good pedagogical reasons to have students go through the effort  to cite their sources, but you should be intentional when teaching as to when those cases apply.  For instance, I am less strict about my students citing sources and instead I look for them to directly apply material from the course in their assignments (instead of giving me their opinions).

I enjoyed Derek’s point about how citing sources is a common trope in pop non-fiction, which sounds like a convergence on my ideas concerning animated bibliographies, or Ryan Holiday’s “15 academic studies” comment from a few weeks back.  Maybe Derek’s right – we should have more courage to integrate knowledge into our existing schema and be prepared to state things as facts instead of citing our sources.  I’m not sure I’m prepared to abandon the practice wholesale, but it has given me something to chew on.

Stay Awesome,

Ryan

“The Same Fifteen Academic Studies”

Despite my rant a few weeks back on the podcast-book marketing relationship, there are a few authors I will check out when they appear on podcasts I’m subscribed to.  For instance, Ryan Holiday just released his new book and is out promoting it to various podcast audiences.

He appeared on Rolf Pott’s podcast, Deviate, and had a conversation about what it’s like to write a big idea book.  Towards the end of the episode, he makes an off-hand remark on getting ideas from recently published books, and how he chooses not to do this because it tends to result in recycling the same academic studies.  Given how much I rant about animated bibliographies and short term content bias, I was happy to see some convergence in our ideas – that my amateur attempt on commenting on culture is shared by people I admire and hold in regard.

I’ve transcribed his remarks below, but you can go here to listen to the episode yourself.  Holiday’s comments begin at the 50:37 mark:

Potts: And that’s why you should feel blessed not to be an academic, right, because that’s such a useful model from which to write a book.  The academic world has these different hoops to jump through that often aren’t as useful.  And I would think that, sometimes, there’s types of research, like, do you do much Google or Wikipedia research, or is it mostly books?

Holiday: Yeah, I mean, you have to be careful, obviously, relying on Wikipedia, but yeah you do wanna go get facts here and there, and you gotta check stuff out.  I like to use obituaries.  Let’s say I’m writing about a modern person and they’ve died.  New York Times obituaries, Washington Post obituaries often have lots of really interesting stuff.   Then you can be really confident that dates and places and names are all correct because they’ve been properly fact-checked.  So I like to do stuff like that.  I watch documentaries from time to time.  In this book, there wasn’t really a great book about Marina Abramović, but there was some really great New York Times reporting about her Artist is Present exhibit, and there’s also a documentary with the same name.  So I’m willing to get stuff from anywhere provided I believe it’s verified or accurate, but you can’t be choosy about where your stuff comes from.  And in fact, if you’re only drawing from the best selling books of the last couple of years, just as an aside, an example, I find when I read a lot of big idea business books, it feels like they’re all relying on the same fifteen academic studies.  It’s “the will power” experiment, and “the paradox of choice” experiment, and the “Stanford Prison” experiment!  They think it’s new because it’s new to them, but if they’d read a little bit more widely in their own space, they’d realize that they’d be better off going a bit deeper or treading on some newer ground.

*****
(Note: I’ve lightly edited the transcript to remove filler words and some idiolects).

Stay Awesome,

Ryan

Podcasts and Book Marketing

I’ve noticed an annoying trend in the podcast marketing of new books.  The marketing itself works quite well – I’ve purchased a number of books based on authors who appear on podcasts I listen to and discuss the main ideas of the book with the host.

The problem I have is when it seems like a significant amount of the content discussed in the podcast accounts for large chunks of the book’s main ideas.  For instance, I picked up a book recently and all of the key ideas (the theoretical framework and main argument for why the topic is important) laid out in the first 70 pages were discussed on a podcast.  I’m curious whether the rest of the book will be similarly spoiled, but I’m holding out to be surprised.  From what I’ve seen so far, though, the next 150 pages are presented as tactics and strategies to apply the main ideas, so I’m not overly optimistic.

I understand that this is part and parcel of the marketing format for these books.  An author is invited onto a show with a certain audience reach, the two discuss the book as a common framing device for the conversation, and everyone is happy – the author gets promoted, the publisher gets advertising, and the podcast generates revenues on sales kickbacks and sponsored content.   The only person that loses is the reader who pays for a book that was effectively summarized an hour-long conversation for free.  This is doubly bad when the book is an animated bibliography, and you’ve read enough books in the genre to get the punchline from the stories cited.

I can’t put all the blame on the author.  After all, they are just trying to sell their book and there is going to be a lot of repetition of the talking points if you do a lot of interviews.  A lot of the blame, instead, falls on the quality of the podcasts.  I find that podcast hosts tend to stick to a common format of teeing up questions based on key points from the chapters.  Sometimes this is handled smoothly, as in cases where the host poses an interesting question that the author uses to circle back to something discussed in the book.  But I have listened to podcast episodes where the host states a thesis from the book without a question, and the author then just elaborates on that point.

In an ideal world, I would want an interview the way it used to be when I watched Jon Stewart’s run of the Daily Show, where the author would be invited as a guest to the show on the pretense of discussing the book, however the conversation would be about whatever Stewart wanted.  Often the conversation was an excuse to catch up, swap stories, and bore little direct connection with what the book was about; the book was usually mentioned as an afterthought as the interview wrapped up.   Maybe this wouldn’t sell as many books, but at least I wouldn’t feel cheated reading through a book I already had the conclusions for.

Or maybe I have too high of expectations of free content, which runs counter to the content farming that needs to occur to regularly post stuff for consumers.

Rant over (for now).

Stay Awesome,

Ryan

The Value of Probing Assumptions

I was on a consultation call a few weeks ago about an ethics application.  The project was seeking feedback from participants about access to specific mental health information, and in my feedback to their application, I noted that their demographic question concerning the gender of the participant was probably too narrow.  The applicant asked for some advice how to address the comment.

On the one hand, they considered dropping the question as it a.) didn’t obviously connect to their research question, and b.) the literature supporting this branch of mental health was pretty well-studied in terms of incidence rates for the condition along the sex dimension, so they might not learn anything new by asking for the participant’s gender or sex.  On the other hand, if they left it in, they had to contend with whether they should use sex or gender as the focus of the question.  Since the mental health topic they were researching was a medical condition, it seemed like (biological) sex was the more salient feature, whereas my feedback suggested that if they chose gender, they would need to ensure it was inclusive.

While discussing the implications on the phone, I tried to tease out what the purpose of the study was.  Their study was collecting qualitative information about how people access information.  In the context of the demographic information, they weren’t seeking to know how a person’s sex/gender relates to the condition itself.  But, I wonder aloud, it seemed the purpose of the study was to understand how people seek information, which could arguably be influenced by one’s culture, behaviour, socialization, and experience of how the world treats them.  In that way, you would want to focus less on a person’s physiology and instead you might discover interesting differences in how a person seeks information based on their life-experiences.

The applicant noted that they started the phone call intending to drop the question from the survey, and through my line of questions and probes, was convinced to keep the question and modify it to be more inclusive.

I am not telling this story as a normative push on how we should conduct inquiry (though by reading through the lines, you should get a sense of how I feel about the topic).  Instead, I share this story as an example of why posing good questions is important to remove ambiguity and clarify thought.  One of the goals of our ethics board when we review applications is to make implied premises explicit so that we can be sure of what we take as a given when we set out to study a research question.  We often default to accepted practice and proceed with common tools, but sometimes we don’t think carefully through the implications of what using those tools means.  By leveraging my outsider status, I have an opportunity to get the applicants to explain concepts and lines of reason without assuming I share the same understanding of the material that they do.  This helps to spot those areas where the project is weakened by unsupported claims and assumptions.

Stay Awesome,

Ryan

Beware the Salesman

Podcast ads.

Hacks and routines.

Blogs that promote and sponsored content.

Look to the incentives and see if they align with your own.

What’s in their interest might not be in mine.

This might change over time.  Sources that don’t sell might eventually sell.  There is a future where I might try to sell you something or join my mailing list.

Beware the one who sells you solutions for problems they don’t have.

Beware the sponsored content coming from authorities you like.

Beware the salesman who makes universal the particular.  The one who puts everyone into a clearly defined box.  The one who makes money on your problems.  The one who charges to join their community.

I’m not saying these are all inherently bad things; merely that they merit many second thoughts.

Secret shortcuts don’t exist.  If they were valuable, they wouldn’t be secret and they wouldn’t be a shortcut – they would be the norm.

Everyone has to make a living.  Everyone has something to sell.  But not everyone has your best interest in mind.

Caveat emptor.

Stay Awesome,

Ryan

Book Ideas – Types of Feedback

I’ve been reading Scott Young’s recently released book, Ultralearning, and I think it’s a pretty good summary of how one can take on an intense learning project for personal and professional development.  It functions like an autodidact’s road map with plenty of good tips, insights, and stories to round things out.  Elements of the animated bibliography are present, but I don’t find it contrived in its execution.  The stories help frame the chapter and serve as an introduction to the core material.

It’s funny how last week I was talking about mnemonic devices, because after drafting that post I ended up reading about the concept in Chapter 10 of the book as it dealt with ways of supporting retention of material you learn.

In chapter 9 of the book, Young talks about ways of providing feedback in the learning process, whether the feedback is provided from others or feedback you can use in your own learning process.  He parses out three kinds of feedback that I found interesting, not only for my own personal use in learning, but also as something I should keep in mind as a teacher.

The three kinds of feedback he outlines are outcome feedback, informational feedback, and corrective feedback.  Each type of feedback serves a specific purpose, and you should be mindful of the context the feedback is given, as the wrong type of feedback can set you back in your learning.

Outcome feedback – provides information on whether you are getting answers right or if you are meeting a pre-identified set of learning objectives.  It tells you that you are right but doesn’t give any indication of why (or why you are wrong).

Informational feedback – provides further information to explain the underlying reason why something is right or wrong.  It can be informative to re-affirm what you have learned, and can identify key areas of strength or weakness, however it does not create a concrete process forward.

Corrective feedback – provides, as the name indicates, a path forward for the learner in terms of how to overcome deficiencies.  It details not only how one is right/wrong, why they are right/wrong, but how to address or avoid being wrong.  This type of feedback not only requires a level of comprehension of the material, but requires sufficient understanding to teach the underlying processes to the learner through explanation, demonstration, suggestion, etc.

As a teacher, it’s important to know what kind of feedback is warranted and under what circumstances.  Most of us tend to focus just on outcomes, but students often don’t learn from pure outcome assessment.  Rather, you need to take the further steps to go beyond an evaluation and ensure you are addressing the underlying deficiency present in the student’s performance.  Outcome assessment is awesome because it’s quick and definitive, but it’s also lazy if your goal is to improve your students.  On the other hand, corrective feedback is desirable but it’s labour-intensive and must be done carefully so as not to remove critical thinking from your student – you don’t want them to merely follow your instructions but instead you want to promote their thinking and reasoning through problems without your guidance.

Stay Awesome,

Ryan

Memorization and my Campus Response Experience

In January of 2008, I was walking through my university campus’s student centre and passed by a table for the UW Campus Response Team, whom were recruiting volunteers for the new semester.  I doubled back, chatted with the team members, and signed-up to participate in their interview process.  I had taken first aid courses periodically during my cub scout and army cadet days, plus I had ran some basic first aid courses while abroad, so it felt like a good fit.

In retrospect, my “experience” was quite paltry, but I had shown the team managers that I had enough of the “right stuff” that they invited me to join the team and participate in the weekend training course they put on for new recruits.  It’s an intense crash course in first aid skills that were well beyond my experiences and the training spanned several hours Friday night and all days Saturday and Sunday, before you perform your final scenario test to qualify as a secondary responder.

The material covered was largely derived from emergency first responder courses, along with some material covered for pre-hospital trauma professions (e.g. fire fighters and paramedics).  The training was designed to create heuristics in the responder’s mind to quickly flow through critical details while gathering as much information as possible and start treatment momentum.  The last thing you want is for a responder to have to intentionally think through what steps they should follow, because it shunts cognitive capacity away from situational awareness and into operational procedures.

In an effort to automate one’s thinking, you end up doing a lot of mock scenarios and skill drills.  As a responder, you end up creating a script in your mind to follow.  The script is based on a common set of things to attend to, which you follow according to handy mnemonics and other memory aids.

Despite the mnemonics functioning to provide mental triggers for actions, you still need to learn the process to go along with the mnemonics, and from the start of training weekend, you only have precious few hours after training concludes for the day to encode the information out of your working memory and into longer term storage.

I needed a way to quickly drill myself and aid in recall.  The system I settled on was to get some window writable markers and write out my mnemonic devices on the bathroom mirror.  Every time I used or walked passed the washroom, I would attempt to fill in as many of the mnemonics as I could remember, and note where I made mistakes.  Through constant repetition, I was able to turn:

E
M
C
A
P
I
E

A
V
P
U

P
E
A
R
L

A
B
C
D

S
A
M
P
L
E

O
P
Q
R
S
T

Into

Environment check
Mechanism of injury?
Count the casualties
Allied agencies?
Personal protection
Introduce yourself
Events leading?

Alert
Verbal
Pain
Unresponsive

Pupils are
Equal
And
Reactive to
Light

Airway
Breathing
Circulation
Deadly bleeds

Signs and Symptoms
Allergies
Medications
Past medical history
Last meal/beverage intake
Events leading

Onset
Provocation
Quality
Radiation
Severity
Time

It was a quick and dirty way to give myself quick feedback on these concepts that I could readily apply to my first aid treatment during training and eventually on shift.  Any time I lost momentum or felt nervous about the judges evaluating me, I would mentally go back to my bathroom mirror and fill in the blanks.  I haven’t been on the first aid team in almost a decade but these concepts easily come back to me, even during my crazy nights at the bar.  It’s a testament to the stickiness of the ideas and the effectiveness of the drills.

Stay Awesome,

Ryan