Welcome back to your Product Management for Beginners Course! It's great tos ee you again for tutorial 3!
Have you had some time to think about your idea? 🤔 Maybe you even looked at some of the competitors out there; how did they end up solving your problem? It's always really interesting to see what others are doing.
Last time, we talked about problem/solution thinking, and highlighted the importance of solving the right problem and building "the right thing". Working out what to build is one of the most important jobs of a product manager. As PM, you'll be the one who prioritizes what gets built first, and what may never see the light of day— so your decisions can make or break a product!
Knowing what to build requires a mix of hard and soft skills. Some of it is intuition, a lot of it comes from experience, but it all starts with research. That brings us neatly to your next topic. In this tutorial, we'll take a closer look at two important methods that we can use to get a clearer idea of what "the right thing" to build is.
We’ll introduce:
Let’s dive in.
An introduction to product analytics
If you have a product that's already in use, analytics is a fantastic way to find areas of the product that are not working well. It's a powerful tool for identifying issues and getting an overall idea of how users are behaving. It's not the best at revealing what exactly is wrong, though—it just shows you where.
A product manager in software doesn't need a PhD in maths, but a solid understanding of statistics is becoming increasingly useful and often required. Don't worry! Analytics is actually quite fun once you've developed a bit of an understanding, and that's what we're here for.
Let's take a look at an example together.
Let's pretend for a moment that you're working for a company that offers a wine recommendation app 🍷 Every user who installs the app is asked a couple of questions in the beginning (during "onboarding") about their preferences. The team has implemented proper tracking throughout the app that allows us to analyze user behaviour.
Take a look at the graph below:
We call this a "funnel" analysis. Every time a user views a screen, the app sends an event to our analytics that says "someone looked at me!". Our analytics team has set up this graph for us to analyze. It shows how many users view each screen of the onboarding process. As you can tell, 100% of users who open the app view the first screen (duh!), but it's also clear that from step to step, the number of users dwindles. Your goal as a product manager is to get as many people successfully started in the app as possible.
It’s normal that users drop off along the way. Maybe they heard about the app, but they're not really that into wine… or they ran out of patience, or they were on the bus when they started and their stop came up…
The reasons why people lose interest are varied, but we can always expect some people to drop off from step to step during onboarding. However, if you look again, you might notice there are two steps where the drop-off is more pronounced:
- The step from "brands" to "red, white, rosé?"
- The step from "sign up" to "special offer"
Each of these has a drop-off of more than 15% of our new users. That's a lot of users who never get to experience the real app.
The drop-off from signup to special offer is fairly natural. Signing up is a big step and requires several interactions (typing out your email address, thinking of a password, agreeing to terms). These types of screens will frequently see a big drop-off—we're not too worried about this one.
But why are people dropping off after seeing the "brands" screen? Here is where the limitations of analytics start to show. While it's great at telling us where to look, it won't be able to tell us what's causing a problem.
For that, we would have to look at the screen and try to come up with some theories of our own. Is it too complex? Hard to understand? Does it cause users anxiety in some way?
So you've pinpointed the location of a problem, but what if you're having trouble coming up with a convincing theory to explain why? Or you don't have any analytics implemented at your company yet?
In that case, it might be time to run a test or conduct an interview!
An introduction to user tests and user interviews
User tests and interviews are great for investigating a specific part of an app, or verifying that a suspicion or theory that we have actually matches with what users feel or experience.
If you have an existing app, a great way to find out what works and what doesn't is to invite a few users and observe them while using the app.
If you don't have a working app (or even a prototype) yet, then you can simply conduct a user interview! Even if an app exists, interviews are great at identifying problems that you might not have thought of in the context of the app before. Maybe your recipe app could offer ingredient delivery so the user doesn’t have to go shopping. Maybe your TV guide app could automatically create calendar entries for the user’s favorite shows so they don’t miss them 💡
For a user interview, you'll want to prepare some questions that you’d like to answer. They can be broad questions, like "What problems do people experience when selecting wine?" or very specific, such as "Do people wish they could compare prices when they're in the supermarket?".
Now, sometimes asking these questions outright will not get you the best answers, so you should think about how you can get unbiased and open answers. For example, in order to get people to tell you about their problems when shopping for wine, it might be useful to start out as follows: “Think back to the last time you bought wine. Can you walk me through what you did?" You might then follow up with: "Were you happy with the wine you bought?"
Here are some good rules of thumb when conducting an interview:
- Ask open-ended questions (e.g. "How was the wine" instead of "Was the wine good?")
- Avoid "leading" or "suggestive" questions (e.g. "Have you ever experienced any difficulties when selecting wine?" instead of "Don't you find it difficult to select wine?" )
- When someone asks you a question about your thoughts, try not to give too much information; instead, try to deflect the question (e.g. User: "Is that how it's supposed to be?" Interviewer: "What do you think?")
- Listen, and allow people time to think. Don't try to be helpful by supplying the end of a sentence when your interview partner is looking for words! They might end up saying something that surprises you, so you don’t want to influence their response.
Most of all: Show that you care 😊 Your users are taking the time to help you out, so be engaged and make sure you're well prepared. If you'd like a deeper dive into the world of user interviews, we recommend watching the following video by CareerFoundry graduate and Senior Product Designer Maureen Herben.
🪄 A quick note on AI: In the previous tutorial we introduced the role of AI in product management, and shared some helpful tools that a PM can integrate into their processes. While there are AI technologies available to help with market research and user testing, user interviews is one area where human interaction really makes a difference. Speaking to real users and conducting detailed interviews will provide you with valuable and nuanced insights about your product in a way that AI can’t replicate. After all, it doesn't have the context or emotions a real human has when interacting with your product.
So that was a long tutorial! But working out what to work on is one of the critical skills you'll need to develop as a product manager. Let's wrap up with another short task.
🧐 Your task
- Think back to your idea from the last tutorial. Do you still have your problems and solutions in mind? Now think of 2-4 questions that you could ask to find out what issues others are experiencing. Are they the same issues, or are different people experiencing different issues?
- If you'd like some help putting the questions together, try using ChatGPT or Bard to generate ideas and frame the questions appropriately.
⚠️ A word of caution ⚠️ ChatGPT, Bard, and similar AI tools are only there to guide you. Review all AI-generated deliverables with a critical eye—they’re not always factually accurate, and AI is sadly notorious for perpetuating bias. Use AI to spark your own creativity and speed up certain tasks, but never to the detriment of ethical, inclusive, human-centric design. Always evaluate the output from the AI carefully before you implement it in your work. - Once you've got your questions ready, find a friend or family member ready for a short interview and ask them your questions! Make sure you show them you appreciate their help.
- Jot down key points that come up in their feedback. What do they like about your product? Do they offer any ideas for improvement? With user research in tow, we'll start transforming your idea into a digital product in the next tutorial!
Coming up
We hope you’ve enjoyed the tutorial. Next up, we'll talk about communicating plans and creating roadmaps. Looking forward to it!
