Ola Jachtorowicz

For T127, Teaching and Learning Lab Practicum, my team’s project is to investigate the use of simulations in Higher Education. To do this, we decided to develop and pilot a simulation using a platform that faculty has indicated interest in: Viewpoint.

In Spring of 2022, two previous students – Sabina Zacco and Alessandra Lopez – had already completed the initial ideation and design stages of the project, including a literature review, expert interviews, empathy map, storyboard, and initial prototype.

It was important to my team to get feedback throughout the development process in order to rapidly iterate on our prototype.

In our readings for Week 5 on Learner-Centered Design, I appreciated this passage from Dave Thomsen:

“When you’re working at breakneck speeds with tight deadlines, taking time out to gather feedback from users can feel like a luxury that’s easy to put off. But there’s no substitute for the nuance and depth of insight …It doesn’t have to be formal or lengthy. In fact, rough prototypes often spark richer conversations than fully realized designs because participants are less likely to be concerned about offending the people in the room. Interactive prototypes allow for less directed feedback. Ask participants to verbalize their thought process as they use it. Try not to correct the participant or defend your prototype and answer their questions with questions. ‘What does this button do?’ ‘Well, what would you like it to do?’”

Dave Thomsen, Why Human-Centered Design Matters, WIRED, December 2013

My team did indeed set a tight timeline that had a us working at breakneck speeds.

Prof. Bill Wisser reserved the entire class time on Tuesday, November 1st for piloting our simulation with our colleagues in T127.

We scheduled our user test a week before our pilot: October 25. This only gave us less than a month to get our first prototype (and evaluation instruments) ready for user testing, and then only a few days to apply our learnings and iterate.

• September 29, 2022

  First team meeting with objectives to create project charter and set timelines and responsibilities

• October 14, 2022

  First iteration of prototype (MVP) and prework reviewed internally

• October 25, 2022

  Live user test of second iteration of prototype with 5 participants (now in Viewpoint)

• October 28, 2022

  Pre-work materials emailed to T127 participants (third iteration finalized)

• November 1, 2022

  T127 Simulation Pilot with 20 participants

Despite the short timeline, we knew user testing was a necessity, not a luxury! We couldn’t risk having a simulation-breaking error by having the first time we ran our simulation be during our precious class time.

Our process was simply to run the simulation exactly as we had developed it for our pilot: we even ran the simulation in our T127 classroom during scheduled class time since the course was asynchronous for two weeks.

We recruited participants through HGSE social media groups on WhatsApp and Slack. We offered candy as an incentive but were really relying on participants’ intrinsic motivation to learn more about learning design. Our project manager, Majo Brito-Paez, was in charge of recruiting and communicating with participants.

Thanks to Majo’s efforts, we were successfully able to recruit the 5 participants we needed.

I’ve used the Prototyping Report Card from the IDEO Design Kit that was also part of our Week 5 Learner-Centered Design reading to describe the prototype user test, learnings, and iterations. More details on our learnings and subsequent iteration follows.

---

What We Learned

• Only 1 out of out 5 participants completed the pre-work we sent out. Some participants said that they hadn’t noticed the pre-work, others that they were aware of it but had not had time to review the materials on such short notice.
• Participants were very engaged and had fun. After the simulation, there was another class scheduled to use the room. All 5 participants lingered in the hallway for approx. 15 additional minutes past scheduled time because they wanted to keep discussing the scenario and their actions. I also noted several moments of laughter and smiling during the simulation. However, despite moments of levity, participants took their roles seriously. The participant playing the Course Lead reported that the meeting “felt very genuine.” My notes include the following observations:
  • [Faculty Member] great at being defensive
  • [Learning Experience Designer] great at trying to reassure professor
  • [Course Lead] great tying it back to objectives
  • [Course Lead] good at trying to mediate
  • [Project Manager] good job trying to assign tasks
  • [Learning Experience Designer] great suggesting to send examples
• The central point of tension meant to elicit learning about respect, trust, and relationship-building emerged marvelously! The participant who role-played the faculty member reported “for me it was about respect” during the debrief and likened the kick-off meeting to “managing the talent” during her previous career filming documentaries. All the participants mentioned trust and relationship building during the debrief.
• Several areas needed clarity. These were identified either through observation of participants completely missing the information or by participants verbally expressing the need for clarification:
  • log-in link, definitions of TLL and kickoff meeting, private beliefs, ownership of design process, deadline, prioritization and consistency of resource placement

Iterations

• Streamline pre-work. We made several changes to make the pre-work easier for learners to digest and to get participants into Viewpoint faster where they could begin reviewing their roles
  • Moved the “Simulation Norms” into Viewpoint platform rather than a separate document
  • Removed watching external demo video and replaced by suggesting the “Dashboard tour” within Viewpoint
  • Added screenshots and direct link to Viewpoint platform (in addition to link sent in invite email)
  This reduced the pre-work steps from 5 to 3.

• Create a “TL;DR” (too long, didn’t read) summary email with priority role information and resources. We called these “Reminder to Self” emails since they were sent from the role to the role. They were scheduled to be sent right before the start of the Kick-Off meeting.

• Prioritize resources for each role and link in multiple places rather than just in the Resources section. The prioritized resources were linked in both the newly created “Reminder to Self” emails as well as the role descriptions.
• Add clarifying language about TLL and Kick-Off meeting (to pre-work including pre-survey) and about private beliefs (in Viewpoint). One of the limitations of the Viewpoint platform is that is not very flexible. We could not rename the “Private Beliefs” to be something like “Internal Motivations.” Instead, we simply clarified that “While these beliefs are private at the start, think of these as personal beliefs and utilize them in the meeting as you see fit.”
• Emphasize deadline (course launch next semester on February 1st) in more places including all role descriptions.

---

Rapid prototyping goes against some of my perfectionist tendencies, but I have really come to appreciate its power during my time at HGSE. I wish I had had one more more round of iteration to go through before the pilot! I’m grateful to my colleagues who volunteered their time and insights. They allowed us to center our very busy and time-strapped learners by making design choices towards pre-work that is as painless as possible.