Friday, April 5, 2013

Exploiting Your Social Network Through Delay Tolerant Talk Prep (DTTP)

What I Tried - Delay-Tolerant Talk Prep (DTTP)

While prepping for my NSDI talk, I ran into a variety of (relatively uninteresting) problems scheduling dry runs.  So I tried something new (to me and folks with whom I've spoken about it) - I recorded myself giving versions of my talk, distributed it to my social network, got feedback, and iterated.  While it started as a standard kludge, after thinking about it a bit more, it struck me that there might be more to it - so this post shares my thoughts on that, some questions to (hopefully) start up an interesting discussion, and details as to precisely how I implemented DTTP (tutorial below).  My hope is this will be of interest to the community and helpful to others (particularly those of us who are early in our career trajectories) in approaching what can be one of the most important and challenging parts of research - conveying one's ideas effectively in a live format.  


Why DTTP could be more than a kludge

Let's briefly consider some of the drawbacks of current talk prep mechanisms.  The three dominant ones which come to mind are:
  1. A live dry-run: closest to what you'll actually be doing when you give your talk - both in terms of the delivery and interaction with an audience (answering questions, debugging verbal and non-verbal delivery/presentation issues).  But difficult to schedule and you are likely to get lots of overlapping feedback.  Its difficult to include colleagues who aren't geographically convenient - thus you can't make optimal use of your professional social network.  Moreover, most of the folks you'll get to attend will be from your research group and thus already fairly familiar with the work (and not much like your audience, most of whom will likely be seeing your work for the first time). 
  2. A sit down one-on-one, walk through each slide: Very targeted, but requires a colleague who is willing to invest a lot of time - again won't leverage the power of your social network - and you won't get many of these for any given talk.  Doesn't necessarily give you a great idea of how things will actually flow.  
  3. Asynchronous sending-out-a-slide-deck, receive written and/or verbal responses: Much lighter weight, fewer scheduling issues, get geography-independent feedback, and finally a much more continuous feedback loop (w/ correspondingly less overlapping feedback if you iterate quickly)!  But lots of context is lost. E.g., slides w/o narration can be confusing - especially to colleagues who aren't very familiar with your work, restricting again how much of your social network can be leveraged.  Moreover the feedback you get might be "say these five things in your verbal delivery" (which you'd already planned to say), instead of information that will actually be helpful to you.
Considered in this context, DTTP provides another point in the talk preparation mechanism design space.  By utilizing electronic capture, and delay tolerant dissemination, you get all the benefits of (3), but b/c you've included the full audio narrative and visual slide flow you get many of the advantages of (1) without its drawbacks.  Further, you make it possible to obtain meaningful feedback from far-flung colleagues who know relatively little about your work (and thus match your likely audience better) - greatly expanding the portion of your social network you can effectively leverage. 

Of course, what you don't get is the experience of speaking in front of a live audience including practice answering questions on the spot.  Nor will you get (at least with my method of recording) useful feedback on non-verbal presentation issues (which can be just as important as the verbal ones).  So while I definitely wouldn't recommend DTTP as a replacement for in person dry runs (particularly for those who are just starting out), I think it could definitely be a good option to add to our talk prep toolbox. 

What do you think?  Directly below are a couple of issues I've been mulling, after which I provide a tutorial for implementing DTTP on Mac OS X (anyone want to write one for Windows, Linux, or Haiku?)

Some questions for discussion

For what types of talk does this type of prep make sense?
  • uninterrupted conference talk (good fit w/ recording)
  • research group presentation, w/ lots of interruptions and possibly even dialogue expected (a less good fit, but still worthwhile?)
How wide a net to cast?
  • Wide: wider range of feedback (less echo chamber), higher probability of smoother feedback loop. 
  • Narrow: you might not want folks who know you less well and are still forming their opinion of you seeing you at less than your best yourself.  Making sure you don't make folks you know less well feel imposed upon?
Time-dependency?
  •  Maybe increase the size of your net as your talk gets better?
  • Maybe ping more junior colleagues (e.g., fellow students) first to fix obvious flaws to obtain greatest value density of feedback from the scarcer/more-valuable resources of more senior colleagues? 
Where does this fit with other talk prep mechanisms / when should it be done?
  • Early on?  Feedback early is always good, but maybe it's better to iterate on an outline first.
  • In the middle? Build up to a version that's worth getting spending a large group's time on.
  • Close to the talk?  To fine tune.

How I Did It - Tutorial

Dependencies

  • A reasonably up-to-date version of Mac OS X 
  • Microsoft Powerpoint for Mac 2011
  • SoundFlower - a "Free Inter-application Audio Routing Utility"
  • Quicktime Player 10.2

Focus

On current students, though hopefully useful for others as well.  Many of the advice points may seem obvious, particularly to those who have already progressed in their research career.   The technical points are basically one way of obtaining a pretty good video with fairly standard tools (which is way more of a pain than it should be).

Steps

Step 1) Record yourself giving the talk (make a copy of your talk file to do this, b/c after you've done the recording the PowerPoint file will be bloated - and you don't want to have to go through and deleted the recorded audio from each page)










Step 2) Create a Movie from that recording. 
Step 2a) Set audio output to Soundflower (2ch)

Step 2b) Set Quicktime to use Soundflower (2ch) as input

Step 2c) Select the area to be recorded, start recording, and then play slide show starting from the first slide.  Don't forget to make sure "Play Narrations" and "Use Timings" are selected (also don't forget to disable these before actually giving your talk in person!).

Step 2d)  Walk away, set a timer to remind you when the playback will be done.  Take a break, put your feet up - you deserve it!
Step 2e)  Stop the recording, trim any extra recorded material at the beginning and end of your recording, save it with a clever and descriptive name (use a version number).  Also, don't forget to undo step 2a, resetting audio output to "Internal Speakers" or you'll find your video (and everything else) on your computer has no audio!

Step 3) Upload it to a read-only, private, shared folder which you've provided to your social network - I used SkyDrive.  Include the pre-recorded slides (so your friends and colleagues don't have to download a huge file if they want to do some marking up)

Step 4) Send out an email telling your social network what you've done.  Ask for their feedback and the meanest nastiest questions they can think of asking you.

Step 5) Wait for feedback, while doing something other than fretting about what you'll hear.

Step 6) Consider received feedback carefully, really thank each person who responded (after all they took time out of their undoubtedly busy schedules for you!), and revise accordingly.  If revision will be significant, ping your social network and let everyone know you've gotten enough feedback to realize how much better a version you will have in X hours, so, if-all-things-are-equal-for-them, please wait while you get that done.

Step 7) GOTO Step 1

4 comments:

  1. I've done this as well on a Mac, and there's a much easier way to capture your talk. Simply give the talk while recording the screen (as you've shown in 2c), but use the internal microphone as the sound input (as you've almost shown in 2b).

    This is an interesting idea, but it didn't work so well for me. In my experience it's much easier to get people to listen to a talk when you've made them be somewhere physically than it is to get them to listen to a talk at their convenience.

    ReplyDelete
    Replies
    1. Your method for capturing the talk is definitely easier - thanks for posting! I'll make a quick comment on the tradeoff between that and my method and respond to your other point separately, since it's similar to Keith's.

      The main reason I provided a more complex setup for recording the talk, is that recording within PowerPoint first (and then screen capturing) has a couple of advantages. The biggest is that at any point during the record you can stop the record and save the partial recorded talk. You can then restart recording on any slide of your choice and the previously recorded slides won't be changed. Thus if you mess up a slide, you can just redo it, instead of starting from scratch. You can also record each slide separately - which comes in handy for earlier versions of the talk - so that you don't need to waste time polishing your delivery for slides that are likely to be signficantly changed, replaced, or eliminated; but your audience still gets to see something that is reasonably smooth. There are also some more minor advantages, like getting to see the individual timings for each slide which powerpoint automatically displays in Slide Sorter View - which I find helpful in assessing whether I'm spending too much (or too little) time.

      Delete
  2. This is a cool idea -- I wonder if I could get people to listen to a 20-minute practice talk in advance though. (Ok, let's be honest: I wonder if I could force myself to *prepare* a 20-minute practice talk in advance.)

    I think ideally you could do the video Usenix-style, with video of yourself inset in the corner and the slides taking up most of the screen.

    Josh, I am curious how helpful you found this technique in the end. I sometimes find that I can't really discover the confusing points of my work (or the best way to present the material) unless I actually just have a conversation with somebody and let us go back and forth until they really understand.

    I recently heard about something called the "Cephalonian method," where you actually hand out index cards to pre-load the audience with scripted questions to ask, which you (the presenter) then answer during the talk! Apparently this is a good scheme for getting the audience engaged. I wonder if anybody will have the guts to try this in a technical conference presentation.

    ReplyDelete
  3. So I actually found this technique really helpful in prepping for my talk. My full process consisted of: (1) put together an outline and go back and forth w/ co-authors over email. (2) draft a first set of slides based on the output of the first step. (3) did two one-on-one conversations/presentations with individual coauthors (iterating between each), then (4) switched to DTTP. (I actually never managed to do a live dry-run, which I wouldn't recommend for someone who hasn't given a couple of previous public talks).

    A couple of things popped out as interesting. The first was that despite the attention given in the one-on-ones (which really helped the talk). There were some subtle-but-significant structural and other issues with the talk draft that came out of the one-on-ones that were immediately caught by the DTTP responses. Likewise, when I fine-tuned, the multiple DTTP responses received for the revised version caught the exact same hiccups in flow and pointed out the same minor sources of confusion in a handful of slides. So I was really thrilled by this.

    The second, w/ directly addresses your and Wyatt's comments about actually getting responses, was that while the yield rate was lowish (maybe 15% of the folks I emailed actually responded), the 5 responses I did get, spread out over the course of a couple of days, proved to be more than enough to take my talk from something I thought was fine to something I felt was really sharp.

    That said, two factors that could be used to increase yield rates could be (1) advisor/mentor-level encouragement both inter and intra research group and (2) the natural reciprocity that arises from getting thoughtful feedback. I know that if anyone of the folks who took the time to send me feedback subsequently sent me out a DTTP video, I'd put watching and responding fairly high up in my priority queue.

    BTW, I really like the "Cephalonian method" idea you described. I'm going to check it out - though I'd probably try it at a group talk or relaxed workshop setting before trying it during a full technical conference ;-)

    Two final items:
    - I completely agree that a USENIX-style video with an insert of one's head in the corner would be great. Any thoughts on a not-too-complicated strategy for doing that?
    - You write that you aren't sure you could force yourself to prepare a 20-minute practice talk in advance. How do you prep currently? Your talk at NSDI was great - where does that super-smooth (and funny) delivery come from? Also, note my earlier resonse to Wyatt, that by recording in PowerPoint you can actually record each slide or set of slides individually so you don't need to do a full single-take to get a reasonable video out.

    ReplyDelete