“The Same Fifteen Academic Studies”

Despite my rant a few weeks back on the podcast-book marketing relationship, there are a few authors I will check out when they appear on podcasts I’m subscribed to.  For instance, Ryan Holiday just released his new book and is out promoting it to various podcast audiences.

He appeared on Rolf Pott’s podcast, Deviate, and had a conversation about what it’s like to write a big idea book.  Towards the end of the episode, he makes an off-hand remark on getting ideas from recently published books, and how he chooses not to do this because it tends to result in recycling the same academic studies.  Given how much I rant about animated bibliographies and short term content bias, I was happy to see some convergence in our ideas – that my amateur attempt on commenting on culture is shared by people I admire and hold in regard.

I’ve transcribed his remarks below, but you can go here to listen to the episode yourself.  Holiday’s comments begin at the 50:37 mark:

Potts: And that’s why you should feel blessed not to be an academic, right, because that’s such a useful model from which to write a book.  The academic world has these different hoops to jump through that often aren’t as useful.  And I would think that, sometimes, there’s types of research, like, do you do much Google or Wikipedia research, or is it mostly books?

Holiday: Yeah, I mean, you have to be careful, obviously, relying on Wikipedia, but yeah you do wanna go get facts here and there, and you gotta check stuff out.  I like to use obituaries.  Let’s say I’m writing about a modern person and they’ve died.  New York Times obituaries, Washington Post obituaries often have lots of really interesting stuff.   Then you can be really confident that dates and places and names are all correct because they’ve been properly fact-checked.  So I like to do stuff like that.  I watch documentaries from time to time.  In this book, there wasn’t really a great book about Marina Abramović, but there was some really great New York Times reporting about her Artist is Present exhibit, and there’s also a documentary with the same name.  So I’m willing to get stuff from anywhere provided I believe it’s verified or accurate, but you can’t be choosy about where your stuff comes from.  And in fact, if you’re only drawing from the best selling books of the last couple of years, just as an aside, an example, I find when I read a lot of big idea business books, it feels like they’re all relying on the same fifteen academic studies.  It’s “the will power” experiment, and “the paradox of choice” experiment, and the “Stanford Prison” experiment!  They think it’s new because it’s new to them, but if they’d read a little bit more widely in their own space, they’d realize that they’d be better off going a bit deeper or treading on some newer ground.

*****
(Note: I’ve lightly edited the transcript to remove filler words and some idiolects).

Stay Awesome,

Ryan

The Value of Probing Assumptions

I was on a consultation call a few weeks ago about an ethics application.  The project was seeking feedback from participants about access to specific mental health information, and in my feedback to their application, I noted that their demographic question concerning the gender of the participant was probably too narrow.  The applicant asked for some advice how to address the comment.

On the one hand, they considered dropping the question as it a.) didn’t obviously connect to their research question, and b.) the literature supporting this branch of mental health was pretty well-studied in terms of incidence rates for the condition along the sex dimension, so they might not learn anything new by asking for the participant’s gender or sex.  On the other hand, if they left it in, they had to contend with whether they should use sex or gender as the focus of the question.  Since the mental health topic they were researching was a medical condition, it seemed like (biological) sex was the more salient feature, whereas my feedback suggested that if they chose gender, they would need to ensure it was inclusive.

While discussing the implications on the phone, I tried to tease out what the purpose of the study was.  Their study was collecting qualitative information about how people access information.  In the context of the demographic information, they weren’t seeking to know how a person’s sex/gender relates to the condition itself.  But, I wonder aloud, it seemed the purpose of the study was to understand how people seek information, which could arguably be influenced by one’s culture, behaviour, socialization, and experience of how the world treats them.  In that way, you would want to focus less on a person’s physiology and instead you might discover interesting differences in how a person seeks information based on their life-experiences.

The applicant noted that they started the phone call intending to drop the question from the survey, and through my line of questions and probes, was convinced to keep the question and modify it to be more inclusive.

I am not telling this story as a normative push on how we should conduct inquiry (though by reading through the lines, you should get a sense of how I feel about the topic).  Instead, I share this story as an example of why posing good questions is important to remove ambiguity and clarify thought.  One of the goals of our ethics board when we review applications is to make implied premises explicit so that we can be sure of what we take as a given when we set out to study a research question.  We often default to accepted practice and proceed with common tools, but sometimes we don’t think carefully through the implications of what using those tools means.  By leveraging my outsider status, I have an opportunity to get the applicants to explain concepts and lines of reason without assuming I share the same understanding of the material that they do.  This helps to spot those areas where the project is weakened by unsupported claims and assumptions.

Stay Awesome,

Ryan