Taking Notes from Audiobooks and Podcasts
It's been over 5 years since I started listening to audiobooks.
And I just did my inventory.
Out of ~150 books I've listened to since then:
- I've completely forgotten ~10%
- I've mostly forgotten ~25%
So, over a third of the books I've listened to have vanished from my mind.
And the remaining 70%? With a dozen or so exceptions, I can recall one or two main ideas at best.
Fortunately, it's not me being a chump, and it's not just audiobooks.
According to Dale's Cone of Experience and the Learning Pyramid, we don't retain a lot by reading and listening. Yes, the models behind these studies are being re-evaluated by contemporary researchers, but the point still stands.
So I might have as well read 150 good ol' paperbacks and forgotten most of it.
The one benefit of physical books is that you can at least highlight passages for later review, no matter how inefficient. But there's no highlighting of audio.
This fleetingness and in-the-momentness is what makes audio a particularly forgettable medium to me. Not dissimilar from life itself, which is why I've kept a daily diary for 2 years straight now.
So this is the reason why I finally decided to start taking notes from audiobooks, and it turned out harder than I had expected.
I tried almost everything, but I think I've finally found the solution while writing this article.
tldr; The Solution to Taking Notes from Audio: use an iPad. Really, it's that convenient. But keep reading.
Let's talk about information retention a little more.
If you're studying for an exam, taking notes isn't the best use of your time. For maximum retention, you want active recall and spaced repetition. Great proven studying techniques.
But I don't listen to audiobooks and podcasts to take exams, so I'm good with notes.
The main purpose of my notes is to make the distilled version of what I've just listened to available to me when I need it. For uploading it into my brain at request, not for keeping it there at all times. Just-in-time learning, or something.
You might say there are millions of summaries available online. True. But reading someone else's summary isn't the same as processing the whole thing, then distilling a version for yourself. (Insert alcohol joke here.) What's important to me isn't necessarily as important to you. Not a fan of summaries.
Taking notes it is then.
Now, I'll have to confess. I'm not a great note-taker in general. But the audio format poses a couple unique challenges.
Here's an example of a note I took a few years ago from David Perell's North Star podcast.
Not too shabby, right?
But this habit didn't stick back then, and I stopped taking notes shortly thereafter because there was a catch.
The catch was, I typed this note on my laptop while listening to the podcast. Not exactly how people listen to podcasts, as far as I know. At least that's not how I do it.
In fact, I generally listen to my audio whenever I'm AWAY from my laptop. Like, during a walk, or while cooking, or eating the results. Lying on an airport floor waiting for the boarding call. Anything except for driving. Driving is music time.
And typing is by far my biggest challenge with taking notes from audio material.
Since I'm away from my laptop, I have to resort to typing on my phone. Which is kind of torture because my phone typing speed is nowhere near my usual 120+ words per minute. And the accuracy is even worse. And it takes that much more effort overall.
It's why I don't use my phone for chatting, generally. Yes, I know I can improve, I had "learn to type faster on my phone" on my 2021 todo list. Yet here I am. (And even if I became the fastest smartphone typist in the world according to the Guinness Book of Records, I'd still only get around 110 WPM, which makes laptops roll on the floor laughing, honestly.)
The second challenge to audio note-taking is the Need for Hands (Unleashed).
As you might have guessed, I use hands for typing. And more often than not, I don't have them available when listening to audio. Sitting on a commode is one of the few exceptions, but everything else I do while listening generally requires all the hands I got. And pausing isn't a great workaround if you're taking lots of notes.
The final challenge is the Need for Attention (Most Wanted).
Note-taking aside, this is why I don't listen to books or podcasts while driving. Research says driving increases mental effort, even when it's music. And I don't retain that much from audio anyway, as we've already established. So doing it to the detriment of my driving performance feels pointless. Driving is music time.
And the Need for Attention isn't an issue only when driving. Listening to contemplate versus simply listening takes more mental effort anyway. And then some more to put it down, if you decide to take notes.
That's another benefit of physical books. You get all the time in the world to process each sentence. Which is why I'm also not a fan of the whole "listening at 3x" idea. Seriously, at this point just go read a summary. (Not a fan of summaries.)
ANYWAY, what I'm saying is, I got issues with note-taking from audio:
- I'm a slow smartphone typist. I'd rather type on my laptop, but I rarely have it around when listening to audiobooks.
- It takes hands to take notes, and I don't always have my hands at hand (sheesh).
- Listening takes mental effort. Listening to contemplate with the intent of taking notes takes even more.
- Switching between listening and taking notes breaks the flow. Pausing to take a note breaks the listening experience.
Well, here's the thing:
At least for me, not taking notes is NOT an option. It's the lesson I learned the hard way.
So, I didn't have a choice but to go through every possible way of taking audio notes to see what works best given the limitations.
I used a couple criteria to roughly assess each method:
- How quickly can I take a note? How many notes can I take per minute?
- Do I need my hands? How much effort and attention does it take?
- Does it affect the listening experience? Does it break the flow?
- How do I manage and organize my notes later?
Without further ado, here are the note-taking methods I tried:
Since I listen to audiobooks on Audible, its clip & bookmark feature was the obvious first choice.
And it's pretty convenient in some ways. You can add a bookmark with a note attached, and Audible pins it to the current timestamp. This lets you go back and re-listen to the whole section of the book around your note to retrieve valuable context.
But clips & bookmarks aren't without drawbacks. For one, taking an Audible note pauses playback. If you take tons of notes as I do, the process of pausing and unpausing becomes a pain in the ass. It's an overhead that results in a fragmented listening experience. There's also the Need for Hands, so dishwashing isn't an option. Finally, surprisingly, Audible has no official way of exporting notes into a text format. No organizing, no backups. I've found at least one unofficial method, but that's more overhead.
The next thing I tried was typing my notes into Evernote. Simple!
This has some benefits compared to Audible. There's usually no need to pause playback, so it's faster and more convenient. The notes are easier to work with since it's just plaintext, and it's all backed up if your note-taking software supports it. And it's a more universal approach. A note is a note, whether you're listening to an audiobook, a podcast, or a YouTube video.
And there are drawbacks. Unlike Audible, this way doesn't give you timestamps, context, or replays. The Need for Hands is still there. And the need for typing is still there, which is my major challenge. It's so sssllllooowww. Plus, too much smartphone kills my eyes.
What could possibly be a better alternative? Dictation!
Dictating into Evernote is what I tried next. And it's not too bad. Dictation solves at least one of the problems: typing. Dictating longer notes is definitely faster and less taxing than typing them on my phone.
But dictation has the drawbacks of both Audible and regular typing. First, you pause playback to dictate. Not optimal. Second, it's not completely hands-free. You still need your hands to some extent, even if not for actual typing. And it might not always be the most convenient option, unless you like to dictate like no one's watching on a packed bus on a Monday morning.
By the way, I don't own an Apple Watch, so I'm curious: can you dictate into your watch without pausing playback on your phone? Let me know! That would be a slight improvement.
Finally, I of course tried asking Siri to take my notes for me.
What can I say, she's not the best note-taker I've had the pleasure to work with. The upshot of Siri is that she's completely hands-free. Here, the benefits end.
The downsides are plenty. Talking to Siri pauses playback (again!), and she's slower than dictation. I can't imagine taking multiple notes per minute for a book that lasts 25 hours. Also, as of today Siri only works with Notes, not Evernote. And she can create new notes, but not update existing ones. So I would end up with hundreds of notes per listening hour that I'd have to bring together later. I know there are some Shortcuts for Evernote support, but they're not helping.
There are a few things that I haven't tried.
I haven't tried taking normal, physical notes. As in, writing in a notebook. The paper format doesn't appeal to me that much. I'm a slow writer, and I like my notes to be available anywhere, at all times. I have friends who swear by paper notebooks, though. If you're like one of them, it could be a decent alternative. Especially if you then digitalize your notes via OCR.
I also haven't tried Kindle in general, and Whispersync in particular. I know it allows you to read, listen and highlight simultaneously, and it sounds like a great idea. But I don't have a Kindle, and I don't have Kindle versions of my books, and at least as of this writing, there's no way to add a Kindle version to an existing Audible purchase, only vice versa.
To recap, here are the ways of taking notes from audio that I came up with that don't involve a laptop:
- Using Audible clips & bookmarks.
- Typing my notes into Evernote on my phone as I listen.
- Dictating my notes into Evernote on my phone as I listen.
- Asking Siri to take my notes for me.
- Taking physical notes, also known as writing.
- Kindle & Whispersync.
And I assume there are other ways.
If we get creative, I can imagine recording voice memos of your notes, then auto-transcribing them into text format. And I'm sure services like Airr, Readwise, and IFTTT can be used to build a better, more automated note-taking process.
As of today though, I must admit I would prefer typing my audiobook notes on my laptop if it didn't involve the "on my laptop part" that defeats the whole purpose. Taking my laptop into my bathtub sounds like a disaster waiting to happen.
And a month ago, this is where the article would have ended with something like "all of these methods suck, but I had to stick to typing on my phone".
But there's a twist, because a few weeks ago...
I bought an iPad. For the first time in my life. And it has truly saved my audio note-taking experience.
My iPad origin story is one for another time. The quick recap is I've been fascinated by the technology, but never needed a tablet. Finally, I got one for sketching and decided to take Audible for a spin while waiting for the pencil.
And turns out, this iPad Mini 5 is PERFECT for taking notes while listening to audiobooks and podcasts. Perfect, I tell you. As soon as I tried it, I realized I'm never going back.
Yes, iPad typing is still slower than laptop typing. But I'm still faster than on a phone, it's much easier on my eyes, and the Mini form factor is perfect for holding it in my hands as I type.
Yes, it's still not hands-free. No dishwashing. But it's still miles better than Siri or dictation, which makes it the best portable note-taking experience I could hope for. Did I mention it's perfect?
I'll keep looking for ways to make note-taking for the audio format more efficient, but for now iPad Mini 5 + Evernote is my choice.
