VitalSource Technologies, an EdTech company, wants to better understand how professors prepare resources and content for their courses in order to strengthen product strategy.
I was the lead researcher in interviewing instructors and leading synthesis workshops.
To comply with my non-disclosure agreement, I have omitted and obfuscated confidential information in this case study. All information in this case study is my own and does not necessarily reflect the views of VitalSource.
The Challenge
Early on, I was already challenged with the fact that VitalSource did not have any research infrastructure. I was the pioneer on a new research frontier for the company and a lot of the heavy lifting had to be done before even starting the interviews.
I met with stakeholders to frame our project and get clarity on the goals and scope before breaking down the process for research. Leadership was fundamentally excited about funding me with any resources I needed to expedite the project. Our goal was to create artifacts that would help leadership pivot and make product decisions.
The fundamental question: Is there a need in the instructor space for new products? If so, how much effort should we invest in these products?
Framing the Challenge
What are we are trying to understand?
Document the process and tools utilized by instructors in preparation of course material.
Frame the objective as a design question:
How do instructors source and customize their course content over time?
What's the impact we are trying to have?
Improve our products to better help instructors prepare for their courses and help drive student success.
Hypothesis
We wanted to make an educated guess based on our prior research and experience with the field of education, in order to have a basis for the research.
Timeline
Secondary Research - 1 week
Conduct online researching to further understand the landscape of instructors in 2019. Part of the research involves looking at various tools used by instructors and conducting informal netnography research by joining instructor online communities and forums.
Screener Surveys - 1 week
Development of screener surveys and testing them to make sure we are collecting diversified data.
Interviews - 4 weeks
Schedule interviews on Calendly for both onsite and remote locations.
Synthesis and Artifacts - 1 week
Synthesize research and create artifacts to guide product decisions.
Secondary Research
One of the challenges was understanding what kind of tools were used by instructors to manage and teach their courses. There are so many tools available for instructors that I can't possibly capture them all. However, I do know that all institutions require a learning management system and there are only a handful of these systems. I was able to focus on the top LMS integrations in 2019 and narrow down the top most used LMS: Canvas and Blackboard. Studying these two systems will help me ascertain the pain points or limitations of instructors using these systems.
State of Education
Research revealed that the education landscape is really diverse. While most of the money exists in the public sector, the per capita of the spending tells a different story. There is more spending per student in the private sector by almost double the amount. Similarly, there are double the amount of private institutions. While my research isn't focused on this type of comparison, this data can help point me in a better direction. I can hypothesize that the extreme cases of education can be found in public sectors, where potentially classes contain more students with less resources. I can diversify my data collection to ensure I've captured these anomalies.
New Instructors
In order to further ensure the data collection wasn't biased, I wanted to interview instructors based on their teaching position. An instructor's role and responsibility varies based on their institution, but we can generalize based on some research. Instructor's leading active roles in research tend to teach less, while those starting out in instructor or assistant roles may be required to teach more hours.
First time instructors, in fields other than education, lacked the experience and knowledge to plan a course. They were experts in their own discipline, but no experience educating others.
Expertise Without Pedagogy
``My husband just finished his Master's and his life goal is to be a full-time community college instructor in Computer Science. He got a job almost immediately as an adjunct teaching Java 101. He commented to me the other day, 'I'm going to be teaching this class without any sort of preparation or clue what I'm doing!'``
Normalcy of Not Knowing
``I was in a very similar situation. No formal teaching training at all, and expected to get straight in teaching advanced subjects from day 1. I think this is fairly normal in higher education (at least in the UK).``
State of Students
Student mental health issues have been rising in academia due to various reasons. My research revealed that less resilient and needy students have shaped the landscape for instructors in that they are expected to do more handholding, lower their academic standards, and not challenge students too much. Younger instructors feel pressured to accede to student wishes lest they get low teacher ratings from their students. Students email about trivial things and expect prompt replies.
**This data point was collected in 2019 before COVID-19.
Screening
I teamed up with marketing and utilized our marketing HubSpot account to send out emails to our internal list of instructors. We sent out screener surveys to thousands of instructors in order to select ~15 - 20 professors to interview for one hour long sessions. We asked them questions that would help us identify professors who were worth interviewing based on the following criteria:
We are aiming for the subset of instructors that teach more hours. In order to diversify the data collection, we want to caste a wide net and get instructors that teach online, hybrid, and physical classrooms. The instructors must also be from different institutions with a range of subject matter, age, and teaching experience.
Interview Script
The stakeholders and I brainstormed some topics they wanted to touch on with the professors. I worked on the usability script based off the topics. We identified 13 professors from the screener survey to interview in person. I was able to go to location for 11 of them and interviewed 2 of them remotely via Zoom video conference.
Sessions were recorded and recordings were transcribed for further synthesis.
Figure 1.4 Images of institutions visited during research.
Synthesis
The company has never held an affinity workshop of this scale. I sent out an email asking for assistance from colleagues who were interested in learning about affinity mapping. The initial workshop I held was centered around project background and how to consolidate ideas into Post-It notes. Volunteers were given a homework assignment to read the introductory pamphlet and translate the attached instructor transcript into notes.
Affinity Mapping
We split everyone up into teams and asked them to work together and sort the notes into groups. They were asked to name those groups. Finally, using the sorted groups, teams were asked to come up with insights they gathered from their readings.
We reconvened at the end of the workshop to create the master affinity map of all the notes and share our insights. Since this was the first time we had held an affinity mapping workshop of this scale, I thought it was important to spend some time in retrospective exploring ideas of how to improve on this workshop.
Personas
The research allowed me to create archetypes centered around real people in the field. I summarized all the instructors we interviewed into four types of people. This will allow stakeholders to understand instructor preferences and empathize with their needs.
Journey Map
I compiled the research and created an instructor's journey from preparation before the semester to adaptation of the course after the semester ends. The journey was separated into new courses and existing courses and the work and pain points that instructors experience over time.
Research Report
We were able to synthesize 11 major insights from the instructors. Each insight delves deeper into how each archetype deals with the individual concerns that arise from teaching.
Instructors Rely Heavily on Textbook
We know that instructors tend to rely on a textbook to provide homework and resources for the students. Students have been vocal about needing some format to reference in order to study and learn.
Current Events
Current events from the news and reliable sources serve as a medium to apply knowledge learned in the classroom. Instructors use this as a way to keep learning interesting and relevant.
Online Credible Sources and Social Media
Instructors aren't afraid of using social media as a way to gather resources for the classroom, as long as they also vet the information before presenting it to their students. Other means of gathering resources for the classroom involving perusing credible sources like Google Scholar and online academic journals.
Retrospective
Blockers
I initially started by reaching out to professors directly from my own pool of contacts. I had reached out to marketing for a list of professors that signed up to receive marketing email from us, but for some reason was met with friction. Stakeholders were definitely trying to be helpful with resources, but there were definitely road blocks due to this being our first big research project.
Synthesis
Stakeholders and I differ in terms of how we wanted to approach the synthesis process. I wanted everyone to meet together and help formulate one affinity map. The stakeholders wanted separate groups of redundant data. This was a lack of understanding on how to complete research synthesis from the stakeholder side. In retrospect, I should have been more adamant in my perspective as to optimize the synthesis plan. Ultimately, we were still able to reach our goal, but the process was definitely slower.
Insights
Stakeholders were using research insights to pivot and validate product goals. Overall, we wanted to know where we can integrate or enhance our products to improve the instructor teaching experience.
For example, in our Bookshelf reader application, stakeholders were considering a current events section in the student view. This was validated by the research insight that instructors utilize current events from various sources to apply their teaching.
