AI Product Manager

Good text sharing

Basic guide to managing AI products

In this article, when I use the term AI Product Management (APM), I mean AI and ML (technically more accurate). I believe that AI PM is a key role that requires specific skills, judgment and experience, which is critical to the success of AI products and programs.

As a successful practice of APM and the organizer of AP Meetup, I want to share the useful resources, best practices and techniques I have encountered, and learn from my experience. The principles and tips here are useful to project managers, software managers, and any role you make for a technical team decision. I won't spend time talking about the basics of artificial intelligence, because I think you already have this background. However, if you want to learn about ML or if you want to learn more about PM roles, there will be references.

The role/name of product management is relatively new-I would say ~25 years. AI product management focuses on using AI, deep learning and/or machine learning to enhance, improve, create and shape products. AI Product Management (APM) is certainly a recent role.

AI Product Manager is a subset of Internet Product Manager
AI Product Manager is a subset of Internet Product Manager

A recent survey of global business leaders showed that 70% of people have begun to implement artificial intelligence plans. With the surge of artificial intelligence in business, it is easy to see the application of B2C and B2B products and services: Google Search / Photos / Translate , Alexa, Amazon Recommendations, Stitch Fix. Nest, Tesla Autopilot.Machine learning (and data) is the common denominator of all of these.Of course, there are more products supported by ML behind the scenes.

There is still a huge gap between the right organization of artificial intelligence and those organizations that are struggling to adopt artificial intelligence (which is the majority). My POV is APM's key role in the success of the AI ​​program. AsMarty Cagan saidBehind every successful product is the product manager responsible for its success.

Behind every great artificial intelligence product is a product manager or leader who leads the vision, helps the data, promotes the work of the technical team, listens to the customer's opinions, sympathizes and drives the adoption of business metrics. And growth.

Let's start with the table. Excellent AI product management and leadership (yes, this is two different levels!) should include principles and best practices for technical product management and leadership.

AI Product Manager Role
AI Product Manager Role

For each circle in the Venn diagram, you have specific requirements. For example, the user/customer experience is important for all digital products, but may be different for B2B AI / ML products.

AsThis HBR articlePoint outAs such, the life cycle of the so-called "data products" reflects standard product development: identifying opportunities to address core user needs, building an initial version, affecting iterations. Of course, there are many more APMs, because in addition to the complexity of the data, you have to deal with the complex and iterative nature of AI models and processes. This paper suggests prioritizing cross-data collaboration and assessment and data product opportunities, with a focus on long-term goals.

"There is a lack of established workflows for product managers who build AI applications for enterprises" – Andrew Ng

What are the basic skills and qualities of APM?

#1 Solid data and modeling techniques-should have practical experience or working knowledge of data and models

#2 Communication-Product Manager is "the glue that brings together all the various functions and roles of companies in different languages"-Ken Norton, GV

"PM's job is not just hard skills, but also other skills-persuasion, negotiation, storytelling, visual setting and communication."-Anonymous

#3 Complete tasks: challenge, promote and deliver – for example, one of the key tasks of APM is to create special test sets that capture product attributes.

"A good product manager clearly defines the goal, "what" (not how) and manages the delivery of "what". – Ben Horowitz Link

3 stages of AI product management

Consider how the three key phases of any product change and how it changes when it comes to AI/ML.

1) Establishment: What to decide? why? Combine data, analyze and judge. Ask the right questions. Have enough depth to filter noise and focus on valuable POC, not scientific experiments or business dreams without data or expansion possibilities. POC becomes more important-it must match the reality of the organization.

The job of the product manager is to discover/create a valuable, usable and feasible product—Marty Cagan

Said ML is not a good candidate (production list). Or decide how many ML to use with other methods, such as rules. For all other methods, prioritize management and management of the pipeline!

Example when not using AI:

  • SeeThis videoIt lists some common scenarios, such as when to get 100% accuracy, when there is not enough data, when there is no quality data, and so on. Note that biased data or use cases may discriminate against a certain group, etc.
  • A recent WSJ article pointed out that Amazon's use of machine learning to mark certain products as "Amazon Select" is flawed.

"Amazon's Choice" Not quality assurance.Amazon does not test these products; it crowns them using an algorithm that considers various factors, including popularity, shipping speed, price, etc.Experts say that sellers have begun to figure out how to manipulate the algorithm.

2) Development: Organizational structure is very important. Understand and coordinate organizational structure, roles-pros and cons. Data literacy, field depth, and geek reputation are all important. To involve the right SMEs at different stages, APM is the glue that keeps all AI elements together! The speed and frequency of improvement are important! Agile and smart. POC. Series MVP: Lightweight model, purchase/borrow data

3) Commercialization: How will you生产product? Do you have the right people, processes and tools? How will you continuously monitor performance and improve your product?

Key points of AI PM:

  1. Believe in the unreasonable validity (and critical importance) of the data. Make sure you haveCorrect useThe correct data. Invest in acquisition and maintenance of strategic data sets – they are often sources of competitive differentiation, not models. As Andrew Ng said:The data set is the new (AI) wireframe!

PM’s job is to create special test sets that capture product attributes – (AI Podcast and Spotify product manager from Lex Fridman)

  1. To do an effective translation-on the one hand is the technical depth (Sisk calls it Nerd Cred), on the other hand you need to simplify and remove the jargon-Figure #2 Atlas. The current AI/ML wave is relatively new, subject to hype, fast-paced changes, bottlenecks of high-quality talent, and chaos caused by the rush of many vendors and tools. Become a leader able to bridge the business of technology, ML-IT, finance-the gap of all others!
  2. Management expectations and stakeholders. The uncertainty of AI products is high. The probability of failure is higher than that of ordinary software projects)-It takes effort to truly understand how good the model is. Always remember to consider the stakeholders and remember the key question: How do they affect them? Series MVP: Lightweight model, purchase/borrow data, reduce domain name, hand = curation
  3. Sympathy for customers/users-view privacy with empathy-explore new solutions such as differential privacy-what to adopt? User/human-centered error measurement framework
  4. Reduce risk and increase trust – Some organizations may have dedicated staff to take risks, but most organizations do not. As APM, you are the front door guardian of prejudice, fairness and privacy issues. APM should also promote interpretability. Ensure your trust in the product!
  5. Propagate cautiously-the intelligence of the algorithm and the value of intelligence it brings, how will you measure it, such as false positives. How do you measure value? How will it change behavior? Think about the first few steps-for example, too many swipes?
  6. Process / life cycle / design is very important, engineer KAIZEN from the beginning. Identify key business indicators (KPIs) and how they translate into modeling metrics. Follow agile development (Especially suitableFor AI-based products and systems, it's very important to allow users to share real-time feedback with users in the early stages of the development cycle to help test, improve and improve AI functionality in the product. development team. For AI-driven applications, the testing process must be fundamentally changed. Optimization algorithms require many users to test them to ensure they are suitable for many different scenarios. "
The role of PM in data and modeling - credit
The role of PM in data and modeling – credit

In addition to the main content, there are some tips and suggestions I have collected from different sources and my own experience. Here are some tips and suggestions for APM:

  • You need a different approach to AI strategy and planning-more preliminary work and data exploration to identify and review opportunities (data exploration required)
  • Don't underestimate the importance of a shared vision, understanding and effective communication. AI plans are most effective when multiple teams work in coordination (to reduce friction), develop a common understanding and work through different subcultures. Bring different roles together, learn from each other, and let data technicians act as "evangelists"...provide original access
  • Too much data can be overwhelming and make the initiative look too complicated-display rather than tell-data products have their own unique twists-data management and governance have been around for a while, but must deal with legacy assets and 3V New data
  • The ongoing cycle-how will the product progress over time? Up or down? Who is monitoring the data being collected and monetizing it? How to improve things over time? For example: result data. Evaluation? At what rate does the product improve organically from the data collection? Today, products with exciting indicators may be worth keeping. The speed and nature of iteration – where did it come from? customer feedback? automatic? How will you test and deploy variants of the model?
  • Product leadership description-style, product stage, company, product environment, well-built team-management, process, context switching (from 10K foot view to 2 inch view back and forth)

The last thought:

"Success depends on the company's ability to cultivate a "strong digital backbone", a mature ability to integrate technology and talent into business processes, and continuous learning close to users and the market. "- Irving Berger, WSJ

Notes and references:

== Resources about manager ML ==

** Gen PM Resources**

This article is transferred from medium,Original address

Chinese (Simplified) ZH-CN English EN