How regular phone interviews with users gets us excellent insights

Simple is an app for hospitals to manage their patients with hypertension. The Simple software is built by a team in Bangalore, Delhi, and New York. Since deploying in October 2018, the app is being used by healthcare workers in over 400 public health facilities in the states of Punjab and Maharashtra, India as part of the India Hypertension Control Initiative. Many of the public hospitals where Simple is used are in remote towns and villages.

As the product team, we care deeply about how the app is faring on the field:

• What are the most pressing issues faced by the users?
• Is it making their work easier and better?
• Are there important user needs that we haven’t thought of?

Without regular interactions with the people who use our software, we can’t know the answers to these questions.

It might sound basic, but regularly scheduled phone calls bring us closer to our users and help us identify patterns of concerns among our users. In fact, the phone calls are one of our most effective ways to keep our fingers on the pulse of Simple in the field.

Here are the key aspects of our interview process that get us helpful and actionable insights:

Create a safe space quickly

If we want to gain authentic feedback from users, the first step is to create a safe space for them. We focus on creating a safe space quickly. This leads to the conversations being very effective — we learn a lot in a very short time.

Within the first one minute of the call, we establish that we intend to listen to them. We pay attention to their words, tone and underlying emotions — and we ask relevant follow-up questions. We ‘reflect’ what we hear to confirm our understanding of the users’ experience and to help them feel heard.

We listen in a way that makes it clear that the person we are speaking with is more important than the software we are building. We are interested in the details of their everyday work that they talk about. We are interested as they share about the issues that they face — both related to the app and to the broader context of their work.

This enables the users to share honest feedback with us — both positive and negative — and that is really valuable.

Keep the interviews short

Our users are busy people with demanding jobs. A phone interview does not need to be exhaustive. A short interview (we do 10–15 minute phone calls) will help you narrow down your questions to the key issues. Short interviews also mean faster synthesis afterwards, so your regular cadence of interviews won’t become a burden to you and your product team.

Structure the interview

The interview is structured to ensure that it flows well, is not rushed, and that we cover all the questions that we have planned for. Here is what the structure looks like:

1. Introduction

In this we introduce ourselves, explain the purpose of the interview, and check that they have ten minutes to speak with us.

2. Intial question about the interviewee

We ask them about their role and their primary responsibilities at the hospital that they work at.

3. Feature usage questions

We ask them about their use of three primary features of the app. We learn about how they use these features, and if they are working well for them.

4. Satisfaction rating

We ask the user to rate the app between 1 and 5. We ask them to support their rating with reasons. We ask them to share what they like about the app and issues that they are facing related to the app.

5. Conclusion

We ask if there’s anything else related to the app that they’d like to share. We ask them to share any further issues on the support group. We thank them for their time.

The key is to be listening intently to the users throughout, while ensuring that the interview progresses according to the structure. Additionally, we want to keep the length of these interviews to be less than 15 minutes because 1) healthcare workers have busy work lives and 2) to keep our telephonic interviews process lean and inexpensive.

Here is the interview guide that we use.

Keep a regular cadence

Me and my product team are busy. I bet yours is too! The only way we keep our interviews on the rails is by scheduling them ahead of time for the whole year. Block off your calendar for a couple of days on a consistent schedule, explicitly for conducting and synthesising your phone interviews.

It’s key to maintain a regular cadence — we call users every 3 weeks. We ask a consistent set of questions to be able to track the progress of the performance of the app over time. Each time, we select one of the districts where Simple is deployed and speak with 5–6 active users of Simple from the district — this helps us find patterns.

Ask the same (or similar) questions every time

We ask a consistent set of questions every time. These help us evaluate the performance of the most important features of the app over time, and also across locations as we continue to scale.

For example, back in January 2019, the patient search feature was causing the most issues for users in Punjab. Since then, the Simple team has made several improvements to patient search including introducing a scannable patient ID. In interviews conducted in October 2019, we received extremely positive feedback on the improvements. 5 of 5 users of the district shared that they were using the new features to find patients and they no longer faced issues in finding the right patients in the app.

Through asking a consistent set of questions over this year, we have been able to track the progress of the key features of the app.

Stay in tune with the product team

The focus of the phone interviews is to get feedback and findings that are both current and relevant. This helps us move fast as a team.

When we release a new feature or a big change, we include a relevant question in our next interviews. Learning whether the feature is working well and how the feature is being used helps us make decisions on improving the product.

When we released the “Recent Patients” feature (a list of the recently visited patients that is displayed on the home screen of the app), we spoke with users to understand how they use the feature and if they use it at all. We learned that it was being used in a variety of ways and some that we had not anticipated while designing the feature. This understanding helps us iterate on and refine our designs.

Later, during our field visits, we observed the use of the feature and further strengthened our understanding of it.

Collate satisfaction rating along with the supporting 'why'

We ask each user to give the app a rating between 1 to 5. We ask them to support their rating with reasons. This question is asked towards the end of the interview to ensure a comfortable space has been established.

We share these ratings with the team every time we conduct the interviews. Additionally, every few months, we do a collation of these ratings. See the consolidated ratings from June 2019.

Users who gave the app a rating of 3 or 4 gave it because they were facing issues while using the overdue list feature or while looking up patients. Users who gave a rating of 5 stated that the app worked well work for them, and it had simplified as well as enhanced the quality of their work.

These satisfaction ratings are a metric that we want to continually improve.

Phone interviews should be a part of a larger effort to learn from users

Phone interviews complement our other customer support and user research efforts. Issues reported to customer support are initiated by users — in the case of phone interviews, we are proactively reaching out to users. This means that sometimes we are a step ahead of finding issues in the field. Secondly, it helps us reach out to people who otherwise may be uncomfortable or may be too busy to post concerns proactively. When we visit the field (which we do often), we can validate the findings from the phone interviews.

Over the past year we have reached out to 50+ users across all of the 7 districts that Simple is deployed in. Phone interviews have established a continual feedback loop between the product team and the users of Simple. They are a foundation for our product development process because they keep us in close touch with our users over time.

Through conducting these interviews we’ve learnt that if you create the right environment and with a strong plan, you can learn a lot from a short 15-minute phone conversation. If you maintain a regular cadence and collate the findings on a regular basis, the interviews will provide excellent insight and direction for your product.

Thanks

Thanks to Padmini Ray Murray and Daniel Burka for editing this article, and to Anand Rama Krishnan for the photography.

Thanks to Daniel Burka, Dhruv Saxena, Akshay Verma, Pragati Mehrotra and Mahima Chandak for their valuable inputs to the phone interview process over the last year.