What Do People *Really* Think About That Course You’ve Designed?


Sometimes we can learn a great deal when and where we least expect it. This past week I learned some things about eLearning while at, of all places, Starbucks.

I, like many people, enjoy the ambiance of Starbucks for getting work done. Last week was one of those times that I found myself sitting strategically at a table next to an outlet, laptop plugged in and Venti sized tea at the ready as I prepared to dive in for a few hours of productivity.

Unfortunately for me, the person sitting next to me wasn’t making that easy. She too was working from the Starbucks, but did not seem to be enjoying it.

It started with her talking to her screen, cursing it for not being able to log in.  She called someone, expressed frustration at being placed on hold, and then got increasingly angry as she was still unable to log in.

It wasn’t at all interesting, and was in fact distracting and annoying for someone who was himself trying to get some work done. Of course, I became much more interested when I heard her say “No, I’m able to sign into the laptop. I just can’t get signed into the training.”

Suddenly I perked up a bit. She was taking a training course, or more specifically, she was trying to and hitting roadblocks.

Once she was able to get access to the training, she quieted down considerably. In fact, over the next hour she yawned repeatedly, sat with her hand holding up her head, and often sighed out of what appeared to be boredom. The only other sound was the click of her mouse, every 30 seconds or so… the slow, prodding cadence of the Next button.

At one point there was a grunt of frustration. She searched through her purse until she found and pulled out a pair of headphones which she then plugged into the laptop to hear the sound of a video.

I wasn’t sneaking views at her screen. It was, in fact, facing away from me. I only know she was watching a video because a minute or so later she groaned at the screen and asked “Do I have to watch this video again?”

After about 90 minutes, she seemed to be getting frustrated again and made another call.  I assume in this case she was contacting a co-worker, because she asked for “help with a stupid question on that training course they’re making us take.”

That’s the chant of the satisfied learner right there.

The call continued,  and she got even more frustrated when she learned that should would not get credit if she went straight to the final test without going through all of the modules.

Towards the end of the call she asked if she could stop now and finish tomorrow. She was very concerned about this question as, and I’m quoting here, “I don’t wanna have to go through all this crap again.”

As I sat there I couldn’t help but think of the designers who created the course she was taking. How would they react to seeing what I observed? I know plenty of instructional designers, and not a single one would say they seek out creating that type of experience.  And yet I fear this user’s experience is far from unique.

I found myself wanting to learn more about her experience beyond just my observations, and decided to see if she’d mind me asking a few questions.

I greeted her and introduced myself. I’ll call her Sally, because I didn’t ask if I could use her name for this post.

I explained that I couldn’t help but notice the frustration she was having with the course she was taking. She responded with “Oh please… don’t get me started.” I explained that I really would love to hear her thoughts on it, and would like to buy her a drink in exchange for a 10 minute chat. Sally asked why I wanted to learn about it, and I explained that designing learning programs is part of my job.

What happened next was very enlightening. Sally leaned back a bit in her chair, as if there was a subconscious need to distance herself from me in some way. She wasn’t rude, but it was a noticeable reaction to learning what I do. It reminded me very much of a similar experience shared by Cammy Bean.

There are lots of careers that, justified or not, have built into them some negative connotations that convey a certain amount of mistrust of an individual just by association.  Lawyers and Politicians are two examples that immediately come to mind. Are instructional designers, whose role should be building experiences that help people achieve greater heights, really falling into the same sort of preconceptions? I certainly hope not.

Sally agreed to chat, and for the price of one Caramel Macchiato, she shared with me more of her thoughts about this course and training in her organization in general.

Let’s just say the feedback wasn’t overwhelmingly positive. In hindsight, the most insightful parts of the conversation were not the questions I asked her, but the questions she asked me.

“Maybe you can explain to me why I have to take the same ?#@*&%! course year after year.”

That just reeks of “Compliance” doesn’t it? There are plenty of regulations that organizations need to follow, but that doesn’t provide an excuse for delivering a poor learning experience.

Sally also expressed frustration with annual changes. Every year the organization updates the course for whatever has changed, and because knowing the changes is important, they update the test to include those things. She understands all of that, but asks “I know they want us to take the whole course, but wouldn’t it be easier to also tell us what’s changed so we can be on the lookout for it?”

Yes Sally. Yes it would.

“I get that we have to do this, but why do they gotta make doing it so hard?”

At first I misinterpreted this question, assuming it was content-related. It wasn’t about content; she was talking about the process.

As an example, she shared her log-in frustration. She allocated about 2 hours to do this training. However, the first 20 minutes she sat in Starbucks were spent trying to gain access to the course (presumably via an LMS). Her password was not working, and she received attitude from the person she called for not remembering it. And, as she put it, “We don’t learn anything from these, and everybody knows that. They just want to know that it’s done.”

Again, she’s correct. Somewhere along the line ‘design’ has been lost. We don’t design learning experiences or performance support without first matching it against ‘our process’.

We’re so used to having courses and learning management systems as part of the learning structure that we see all solutions through that lens.  We also tend to forget LMSs and many other tools we use exist for the benefit of training departments, not for the benefit of learners. They may be necessary for some organizations – a point in itself that can be debated – but instructional designers need to stop pushing learning through ‘our process’ and start finding ways to support learning through ‘their process’: the work itself.

While I didn’t get everything I had planned for that afternoon at Starbucks done, it was productive in ways I had not expected. I think instructional designers should jump at the chance to talk with people about their experiences in learning programs. Many of us do that today, reaching out to people that experience programs we design.

My conversation with Sally was different. Because we did not work for the same organization, and I wasn’t talking about a specific program or even one I designed, the feedback seemed more genuine and raw. There was really no reason for her to filter herself, so she didn’t.

She doesn’t feel like she learns from the programs, hates the process, and just wants to finish so she can move on.

I find it hard to believe she’s unique.


, , , ,

27 Responses to What Do People *Really* Think About That Course You’ve Designed?

  1. wiredless January 22, 2013 at 3:16 pm #

    Interesting read. The problem can be one of design. But I think some of Sally’s frustrations come from flaws in the development process. It doesn’t look like the course producer did a pilot with a sample of people from the target population of learners.

    Another issue is where Sally chose to access the course: Starbucks. There are tons of distractions: tingling butt from a too hard chair (two hours!), visual distractions from people coming and going not to mention the aromas wafting over her. These are issues that make me as a designer cringe.

    One area in which leaves me scratching my head is the part about learning management systems (LMS). I don’t have a problem with them. Reading about Sally’s experience reminded me of a shipwrecked person, someone cast adrift on some nameless sea, carving a notch on some piece of wood to mark the passing of each day. There has to be some way to mark a learner’s passage through a course.

    Anyway.. my take.

    • David Kelly January 23, 2013 at 10:01 am #

      Thanks for your comment. The value of Learning Management Systems can be debated  I do agree that they do a great job of packaging content into a course with a linear flow that learners can easily follow. For me personally, I’m not sure how much value I actually place on that. 
      To go with your example, I don’t see Sally as adrift in this situation. She knew most of the content already, and really just wanted to know what was new for the current year. Still, she was forced to go through an entire curriculum, and then expected to find the needle of change within the haystack of content.
      I think learners need less spoon-fed content, especially when that learner is experienced in her job already, as was the case with Sally.

      • Eboni Dubose January 28, 2013 at 10:31 am #

        Thank you for the post Dave.  I agree that this very much seems like having the ability to test out. Tom Kuhlman mentioned this in this his posts about compliance training.

        If Sally can pass the initial test, she doesn’t need to take the full course. Seeing a short course that detailed the NEW changes to the course would of course be mandatory and gets at her point of “tell me about the NEW changes”. Also, I agree that this Sally’s scenario is probably very common.  Talking with learners to get relevant, juicy content for the course and piloting the course are all critical parts of development but many times gets skimped on

        I really enjoyed reading the 2012 report completed by Kineo and elearning Age https://kineo.com/nz/elearning-reports/learning-insights-report-2012.html
        Your impromptu interview really captured the heart of what this report is talking about moving toward a new learning architecture so we don’t have frustrated employees. 

        • David Kelly January 28, 2013 at 11:56 am #

          I’m glad you found value in the post Eboni. Thanks so much for sharing the additional resources. I look forward to exploring them.

  2. Cnews January 23, 2013 at 11:55 am #

    this is a wake -up call for all of us in learning,. Any course or resource I’m
    involved with needs to be compelling for the learner. You’ve reminded us
    that  sometimes what we produce as an industry falls well short of that

    • David Kelly January 23, 2013 at 2:36 pm #

      Unfortunately I think we – as an industry – do fall short. Some may point part of the blame on the expectations and constraints put on designers by orgs who focus on completions when it comes to learning, but I think that’s a cop out. No one can force us to accept a bar lower than we want to strive for.

  3. Kelly Meeker January 23, 2013 at 1:01 pm #

    This is really painful to hear with some great nuggets of advice within. Thanks for sharing.

    • David Kelly January 23, 2013 at 2:33 pm #

      Thanks Kelly. It was painful to hear, and at the same time, I was somewhat encouraged. “Sally” wanted a better experience, so that’s one less barrier. Now we just need to upgrade what we deliver.

  4. Judy Unrein (@jkunrein) January 23, 2013 at 2:39 pm #

    Dave, I think this situation is FAR from unique and it deeply concerns me. We need to do much better. I was thinking about just this kind of the thing when I wrote this on the ASTD Learning Technology blog: http://www.astd.org/Publications/Blogs/Learning-Technologies-Blog/2013/01/What-Unintended-Messages-Do-Your-Learning-Solutions-Send

    Care to discuss what messages you think “Sally” was receiving through this experience?

  5. Alex Kluge January 23, 2013 at 4:37 pm #

     An interesting article. I wonder what
    the course author thought about the quality of the user experience.
    As you mention, this sounds a lot like corporate compliance training
    with a number of constraints, for example the company is unlikely to
    produce differing content for new and returning students.

    I even suspect a negative feedback
    loop where the student is required to take the course and has low
    expectations for the quality of the material, management looks at it
    as a requirement to be met with the minimum expenditure of resources
    and effort. The designer and developer will be discouraged from
    producing quality engaging content, which in turn lowers the student
    expectations even more. It is no wonder that this environment did not
    produce a high quality product.

    • David Kelly January 23, 2013 at 4:44 pm #

      That loop is exactly the problem Alex. At some point someone needs to make a change. And it will come. The question is if an organization’s learning function changes themselves, or has change forced upon them. 

      I’d choose the former.

  6. Justin Brusino January 24, 2013 at 10:08 am #

    GREAT post, Dave. It’s interesting to take a step outside the training ‘bubble’ and see someone’s unfiltered response to a course and to a field in general. I’ve never thought of trainers and IDs as a maligned group, but it’s true that more people are having experiences like this than positive ones.

    • David Kelly January 27, 2013 at 7:07 pm #

      I don’t normally feel that IDs are maligned, but interactions like this make me think that the only reason we’re not is because there’s not widespread understanding of who is creating the courses people hate to take.

  7. Tom McDonald January 26, 2013 at 11:00 am #

    Thanks for this.

    Unfortunately it’s all too common that sustained, individual learning, as well as the individual learners are not part of the instructional design process.

    We know that this one to many approach is a waste of time and resources for all involved.

    individual sustained, learning, transfer and application?  I think (know) not.

    • David Kelly January 27, 2013 at 7:04 pm #

      Thanks Tom. I think there are ways to leverage technology to scale learning effectively  but actual good examples of it happening are few and far between.

  8. Victor Antonio January 26, 2013 at 11:15 am #

    David, I truly enjoy these real-world encounters…they can be so much more educational in so many ways. Great article and great storytelling 🙂

    • David Kelly January 27, 2013 at 7:03 pm #

      Glad you liked the post Victor. I’m hoping the more stories like this are shared, the more awareness will spread about us needing to raise the bar.

  9. Liza Loop January 27, 2013 at 5:10 pm #

    You observations provide a good argument for calling the process ‘e-teaching’ not ‘e-learning’. Your informant was complaining that very little learning was taking place.

    • Judy Unrein (@jkunrein) January 27, 2013 at 6:22 pm #

      …or designing for learning, rather than for completion.

    • David Kelly January 27, 2013 at 7:02 pm #

      Honestly I think what “Sally” encountered was nothing more than e-information, perhaps interactive information at best. Sadly, it’s not rare.

  10. David Kelly January 27, 2013 at 7:09 pm #

    Sally received very little, if anything from the experience. She was annoyed more than anything else, and sadly her objective was the same of the ID’s, and the same as the org: Completion.

  11. Douglas Smith January 29, 2013 at 3:49 pm #

    So much frustration can be avoided when we allow learners to jump right to the test. If they already know the answers, why waste their time? And if they don’t know some of the answers, the test will tell then what they need to learn so they can focus on that…

  12. Ann Rollins January 30, 2013 at 9:09 am #

    I think this would be a perfect situation to incorporate a pre-test.  If users have completed the compliance course in the past- gone through the content, and passed the assessment, why make them take it again?  Pre-test them on the existing, and not new, current content to make sure its fresh.  If they pass, they move onto a a mini-module with the update. 

    This approach makes course management more difficult on our end (2 versions per re-up period, per compliance course).  That course management, which we all know equals hours, equals dollars, which upper management frowns upon in many cases. 

    Put numbers to it – 500 employees not taking a full hour of training, but say, a 10 minute pretest and a 15 minute mini dodule saves 45 minutes per employee – those numbers add up (in this case, 200 hours of productivity returned), and more than pay for the investment in aligning the right learning to the right learner.

  13. Nash Alissa February 1, 2013 at 11:50 am #

    I really enjoy design, especially when given latitude to try innovative and proven solutions. However, I’ve been doing ID work as a contractor for the past few years, and have consistently run into situations where training departments are staffed and run by people with no training experience whatsoever. They have no clue what they are doing, and they don’t realize it.

    At this rate, I’m afraid people will have so many negative and non-productive training experiences that it will be an uphill battle to repair the perceptions of training departments when/if corporations make a commitment to restaff with actual training professionals.

  14. Srujan February 21, 2013 at 1:41 am #

    Liked this real world story and we should accept it. It’s a wake up call. Eager to hear more  from u!


  1. ¿Qué se dice en la Web? (Link 16/03/13) | Bean-Thinking.com - March 16, 2013

    […] o los parecerían interesantes y desafiantes? David Kelly tuvo la oportunidad de descubrirlo y en este post nos habla sobre su […]

  2. ¿Qué se dice en la Web? (Links 16/03/13) | Bean-Thinking.com - March 16, 2013

    […] o los parecerían interesantes y desafiantes? David Kelly tuvo la oportunidad de descubrirlo y en este post nos habla sobre su […]

Leave a Reply