How do we get from here to there?
Align Vision to Organization
The very first step in embarking on switching to a machine learning search strategy is a keen understanding of the organization’s Vision and Goals. Sometimes a Strategy can be met entirely with a heuristic-based solution and does not require more complex engagement and talent.
Within an organization, if the Vision revolves around predictability, personalization, and data, I view Machine Learning as a valid strategy to pursue.
In this instance, the vision my organization set was “to create a personalized, uplifting Search experience that anticipates and delivers on the evolving needs of our customers.”
Thus, I aligned my vision to “curate personalized search results that anticipate the evolving needs of every customer“. This ensures that the guiding light behind all work is aligned for a common purpose.
Understand Goals of Organization
Throughout my work experience, organizations can approach this step in a myriad of ways. In smaller companies and startups, I have seen north star goals that typically encompass the one true measure for improvement. In sales organizations, this can be Increase Customers, Increase Traffic, or Increase Sales. In larger organizations, it is a bit more challenging to have a single signal for guiding our strategy.
In Search engines, typically there are two major focus points.
- Behavioral signals
- Customer is engaging in more searches
- Customer is spending more time with search
- Customer is efficiently finding products or answers
- Financial signals
- Share of revenue increases
- Overall revenue increases
- Monetized and alternate revenue increases
For the sake of this document, I will align the search engine to a goal where the Customer is Efficiently finding Products. This will help me organize OKRs for the machine learning initiative.
Understand Customer Problems
Once we understand the company’s goals, we must look at our customer. Every product manager holds special interest in solving our customers biggest needs and challenges. Once we can identify the problems, we can set meaningful strategy that both meets the needs of our Organization and the needs of our Customer.
Customer can also mean various things within an organization. Sometimes customers are simply end users, sometimes they are internal parties, and sometimes they are 3rd parties for which we have an invested relationship.
In the case of Search, particularly in Product search, there are typically 4 major players.
- Consumers
- Merchandisers
- Monetization
- 3rd party vendors and entities
For this example, I will provide a few key problems for each of these customers.
- Consumers are seeing search results that don’t relate to their query term. They are confused by filters and often resort to searching multiple terms before they find the right result.
- Merchandisers are seeing severe and poor category performance on particular areas.
- Monetization is missing query intent signals that help generate the right promotion.
- 3rd party vendors and entities are not seeing themselves reflected in category share.
Setting Strategy & OKRs
I’m making an assumption that whoever is currently reading this understands OKRs. If not, I recommend the Measure What Matters series by Google.
One of my favorite ways to focus my OKRs is by setting core tenants of my strategy. These are areas that are true and must be considered when embarking on a Machine Learning strategy.
- Data quality is key
- Every customer is unique
- Finding product must be effortless
- Inspiration matters
- Technology must be fast, reusable, and dependable
I work with my technical leads, product designers, marketing team, and any other affiliated groups to evangelize and confirm that we all agree with the Vision and Strategy tenants. Once these core values are well understood, we can then approach OKRs together as a team.
I firmly believe designers, technical leads, and other internal stakeholders will be involved in OKR setting. OKR setting is always a collaborative process and will depend on technical blockers, design pursuits, financial constraints, and many other areas. Holding these tenants in mind, along with our vision statement, and our customer problems, I have outlined some OKRs that can be set for machine learning.
Objective: Reach meaningful improvement in query intent.
Improve Consumer and Monetization experiences
- Key Result: Increase click-through rate by 5%
- Key Result: Reduce search conversion cart fall out by 5%
- Key Result: Increase customer engagement with dynamic filters by 5% in our highest searched category
- Key Result: Create a service to dynamically position monetized products per customer preferences
Objective: Use our data to delight our customer
Merchandisers and 3rd party customers have more trust in the generated results.
- Key Result: Reduce count of results on top 100 queries by 10%
- Key Result: Improve search conversion in the weakest 5 performing categories by 5%
- Key Result: Naturally boost larger pack sizes in top 5 categories
Having Objectives and Key Results mapped out helps us prioritize the best potential solutions to achieve our goals for the organization.
Roadmap
And voila, here we are! The roadmap. I prefer to jointly discuss, groom, and tease out the roadmap in a high level with my technical leads and designers. When we settle on a natural order of things, or an MVP approach to hit our OKRs and Vision, we start to refine and tease out the finer details.
Here is an example of a roadmap for an organization converting from a Heuristics-based solution to a Machine Learning based solution. In the beginning, data preparation and resource planning is very important. I recommend reading The Unreasonable Importance of Data Preparation as a great resource for understanding why data preparation might take up a significant amount of time in the early stages of implementing Machine Learning.
