To recap what we have learned so far, formulating the ML problem involves three aspects. Number one, choosing the input features. Think of all the factors that affect the decision, and make sure that all these factors are in a place where you can join them. Make sure there are no silos that prevent you from gaining access to that data to train the machine learning model. It's also important to know what the schema is. Sometimes, as it's figuring out how many people are in the store that will involve training a machine learning model to create the feature. Number two, figuring out how to get a label. You can use historical data if you have it, or you can build a labeling system to collect data in order to prepare for machine learning, or you can use a labeling service. It depends on the problem and it depends on the business context. Number three, choosing the objective correctly. As with the example of one-year deposits versus 10-year deposits, you have to choose a problem for which you can get the input features and the corresponding label. But the objective can be tricky to get right in some cases. So, let's do an activity to learn the trade-offs in picking the right objective for a product recommendation model. In the activity, you will have to match up an objective that the model is optimizing for and the resulting outcome. Let's say you're running a retail website, where you're selling lots of different items. You want to recommend products. The issue is that to train the model you need to tell it which recommendations are good and which recommendations are not good. Let's say you decide that if you recommend a product and the user clicks on the product, then it's a good recommendation. In other words, your objective is to maximize clicks. What will happen? The machine learning model will then optimize product recommendations to show the kinds of things that people click on. The model will tend to show a lot of novelty items. Things that get a lot of clicks, but that people don't really buy. If your objective is clicks, your model may promote novelty items. Now try the other three. By surprise here, I mean things that they'll be surprised by because it's not the kind of thing that this customer has bought before, but you think that they would like. In a recommendation system for an online retail store. If you optimize for clicks, you will get click-bait. Novelty items that people will click on, but never buy. If you optimize a purchase likelihood, you might end up recommending only popular items like consumer staples. This is the stuff that people buy a lot of, but there were going to buy those anyway. Your recommendation system is not that useful, is it? If you optimize for surprise, that is items that the user might not have otherwise bought, your conversion rate might be too low. They will like it if they buy it, but not everyone likes trying new things, and if your conversion rate is low, then what's the point? If you optimize for high ratings and purchased items, your revenue might suffer because consumer staples might never be rated and some will not get recommended. There is a difference between popular in the sense that things are rated highly and popular in the sense that people buy them a lot. The hardest part of a recommendation system is not the algorithm because the algorithm is standard. It's not the data because you probably have purchase transactions data already. In this case of a recommendation system, one of the hardest parts is getting the objective right. In reality, the objective that most retailers pick will be a weighted combination of all these factors. The retailers will typically tweak the relative weights as the company strategy changes. There is no right answer, it's all a trade-off. So, make sure to discuss what the right objective is with your business stakeholders. Sometimes you might use a mix of criteria or constraints such as, in the top five recommendations to a user will always highlight one novelty item, one consumer staple, two already purchased items, and one surprise item. That's a policy, and then you will train several machine learning models, each of which optimizes for a different objective.
