That's awesome, so you've given some best practices and some guidelines. For example, bring your product designer to customer meetings, bring them to leadership meetings. What are some don'ts? The other side of the anti patterns that you can share the quotes. >> The quotes that I have quote, I think that those might be the anti patterns we were talking about here. So for example, please don't say, "visualize my solutions." I think that's a major motivation killer to great, really motivated, passionate UX designer. You wanted to allow them to apply the process and be a little bit more patient and it really by hearing your solution. But also other people's solutions, all the designers' solution have come up with the best solution and that will be my suggestion. And the number two thing I think I heard often: they say, "yeah, everybody can do the design." Yeah, it is true like everybody can have their own opinions and also ideas. But UX design is, just please acknowledge, they have special trainings also and they have just like PMs. PMs have special training, really have unique strengths and this may be some overlap between PM the UX designer but UX designer has their own unique strengths and as well please also acknowledged that right. So sometimes folks say, "so you guys can create make things prettier." Yeah, that is true. That's part of our job, right? You deliver high fidelity visual fidelity, mockups or prototypes. And to get like buy in from stakeholders or from other folks. But that's not the only thing we do. We can start from emphasize with the customer to understand what they need, what are the problems, challenges and to define the problem like what what problem we actually solving. And then we wanted to really take a step back, not to jump to the solution immediately, to ideate, like I say, what are the possible solutions out there? And then that's the process like we collecting the team's wisdom and so you kind of go broad and collect ideas and then you wanted to narrow down again to really narrow down to the solution. You think it's the best at that given the customer needs, the technical feasibility or time strength, whatever strength is out there, and that's the prototype you're going to get. And also, yeah, please wait before implementing. Maybe you should still also get customer feedback. That's what we call formative testing. Because even though you apply the whole process, but then later you forgot to test and you spend expensive engineering time to build the product and realize, "My goodness, actually, customers don't need it." So it's better to wait maybe one or two weeks to get formative feedback instead of jumping, writing to implement. >> Those are definitely great suggestions, and so for a product manager looking to conduct customer research, what are some best practices or guidelines that you can share with him or her? >> So for customer research, there are three different points of time, in the project life cycle, we generate conduct research at the beginning of the project which when you might have some assumptions about the customers about the functionality where you wanted to do. But it is very important to really conduct general research being totally open minded and willing to get challenged by the real data and should not come going to select, okay, I have this assumption. I wanted to look for data to support my assumption and you don't want to do that. You want to be totally open minded and maybe get data which is different from your previous assumption. You should be willing to revisit and give up your previous assumption that accept the new data. And then I talked about formative research earlier, which is when you have a design concept or prototype, make sure you take some time to do some formatted quick usability studies and to make sure it's a good solution before implement them. And then in the end, after you release it, sometimes I'm like, yeah, this feature is released. I'm done, [LAUGH] let's say right. Yes, you can celebrate. But after that, maybe it is really good to go continuously get the customer feedback. So like, what do you think about this released version? Anything additional would you like to have? What's missing, right? What are the things working? Well, we should keep doing the same direction and things like that and that will help you to continuously improve the product you have launched. >> Thank you for walking us through the process of customer research and how to conduct customer studies. And so at the end of the day, a lot of PMS will think about and we've covered in the course. What are the success metrics or what are some good KPIs? So can you share with us maybe you've looked at option metrics previously or customer satisfaction? How did designers think about that and how would you encourage product managers to also think about those success metrics? >> Yeah, so for the previous products I have worked on, we generally apply we will think about the usability metrics and a customer satisfaction metrics in addition to the general product adoption, because that's actually a key indicator of how how customers are satisfied and how they're going to keep using your product. Or maybe you realize if your usability score is low, customer satisfaction ratings low and that's indicator you should look into that. Back to my very first point of the pyramid were user needs, maybe from functionality, reliability needs are met, but they're not usable and delightful yet and achieve in order to get the razor bar on customer experience in your product. I think that's a great mechanism to keep you on track. Always know what's going on instead of just a few number of how many customers are using it. >> I understand, that's awesome. Thank you so much, Minmin. And thank you for being on this AWIP AWS Coursera course. >> Thank you.
