Presentation "AI Product Manager" at the Digital Product School (on 10/22/2020) from Datentreiber.
Content:
• Overview over the AI product innovation cycle
• AI Thinking: ideating and prioritizing the right use cases
• AI Prototyping: testing critical hypotheses with experiments
• AI Engineering: building scalable & user friendly AI applications
• AI Management: maintaining AI solutions with DataOps
• Outlook: how to become an AI product manager (links & more)
1. Martin Szugat @ Digital Product School on 10/22/2020
AI Product Manager
2. Agenda
Overview over the AI product innovation cycle00:15
AI Thinking: ideating and prioritizing the right use cases00:30
AI Prototyping: testing critical hypotheses with experiments00:45
AI Engineering: building scalable & user friendly AI applications01:00
AI Management: maintaining AI solutions with DataOps01:15
Outlook: how to become an AI product manager (links & more)01:30
Q&A01:45
Welcoming: introduction & agenda00:00
4. 4
1996-2008
IT-Consultant, Author and
Software Developer
Study and Research of
Bioinformatics (Data Science)
2001-2008
Managing Director & Shareholder of SnipClip
GmbH (Marketing Agency)
2008-2013
Program Director of the Predictive
Analytics World & Deep Learning World
(Conference Series)
2014-dato
Managing Director & Founder of
Datentreiber GmbH (Consultancy)
2014-dato
Advisory Board for Media & IT
for DDG AG (AI Company Builder)
2020-dato
Martin Szugat
Shareholder of Digitaltreiber GmbH
(Recruitment Agency)
2016-dato
Chief Data Officer & Shareholder
of 42AI GmbH (AI Market Network)
2018-dato
6. Agenda
Welcoming: introduction & agenda00:00
Overview over the AI product innovation cycle00:15
AI Thinking: ideating and prioritizing the right use cases00:30
AI Prototyping: testing critical hypotheses with experiments00:45
AI Engineering: building scalable & user friendly AI applications01:00
AI Management: maintaining AI solutions with DataOps01:15
Outlook: how to become an AI product manager (links & more)01:30
Q&A01:45
7. Agenda
Welcoming: introduction & agenda00:00
AI Thinking: ideating and prioritizing the right use cases00:30
AI Prototyping: testing critical hypotheses with experiments00:45
AI Engineering: building scalable & user friendly AI applications01:00
AI Management: maintaining AI solutions with DataOps01:15
Outlook: how to become an AI product manager (links & more)01:30
Q&A01:45
Overview over the AI product innovation cycle00:15
10. What is an AI (Data + Analytics) Strategy?
Accessible Data
AI
Use Cases
Company’s
Objectives
Data / AI Products with a
Business Case
11. Collection of Analytics
Use Cases
(Problem, Solution, Benefit)
Roadmap for Data-Driven
Business Cases
(Costs, Risks, Profits)
Assumptions
(Analytical, Economical, …)
Learnings
(Data, Business, User, …)
Business Value
(Information → Decision → Action →
Impact → Objective)
Data Sources
(Collection, Acquisition, …)
Data
Thinking
Data
Mining
Data Engineering
Data
Management
Data Strategy
Data Prototypes
Data Product
Data SourcesData AssetsData Product
Innovation Cycle
12. Data
Management
Data Engineering
Data
Mining
Data
Thinking
Data, Model &
Product
Management
Data, Software &
UI Engineering
Data
Mining & User
Experiments
Data & Design
Thinking
2. User Under-
standing
(Desirability)
3. Data Under-
standing
(Feasibility)
1. Business
Under-
standing
(Viability)
2. Modelling &
Visualization
3. Evaluation
1. Data
Exploration &
Preparation
3. Learn
1. Build
2. Measure
3. Monitor
1. Deploy
2. Orchestrate
CRISP-DM
Design
Thinking
Proof of Concept (PoC)?
Proof of Value (PoV)?
Lean
Develop-
ment
DataOps
13. Data, Model &
Product
Management
Operating
Data, Software &
UI Engineering
Engineering
Data & Design
Thinking
Data
Mining & User
Experiments
Designing
Experiment-ing
Data LabData Factory
➔ Exploration to
Learn
➔ Exploitation to
Earn
14. 15
Designing
Experiment-ingEngineering
Operating
Data Strategist, AI Translator, …
Canvas, Mockups, …
Design Thinking, Sprints …
Data Scientist, UX Designer, …
Data Analytics, Modelling, …
CRISP-DM, Kanban, …
Data Steward, Product Manager, …
Monitoring, Audits, …
DataOps, SPC, …
Data Engineer, Developer, …
Cloud, MapReduce, …
Scrum, Lean …
Skills, Tools & Methods
15. 16
Designing
Experiment-ingEngineering
Operating
Data Strategist, AI Translator, …
Canvas, Mockups, …
Design Thinking, Sprints …
Data Scientist, UX Designer, …
Data Analytics, Modelling, …
CRISP-DM, Kanban, …
Data Steward, Product Manager, …
Monitoring, Audits, …
DataOps, SPC, …
Data Engineer, Developer, …
Cloud, MapReduce, …
Scrum, Lean …
AI Product Manager
16. Agenda
Welcoming: introduction & agenda00:00
Overview over the AI product innovation cycle00:15
AI Prototyping: testing critical hypotheses with experiments00:45
AI Engineering: building scalable & user friendly AI applications01:00
AI Management: maintaining AI solutions with DataOps01:15
Outlook: how to become an AI product manager (links & more)01:30
Q&A01:45
AI Thinking: ideating and prioritizing the right use cases00:30
17. Holy Grail Use Cases
Everybody wants it. Nobody has it. Some claim to have it.
20. Pet Projects
Bosses are usually furthest away from the actions and thus the
relevant information.
21. Boring is the new Sexy.
Look for use cases that sound boring because they often are very subject-specific.
22. Delegate
Form
Check
Stake-
holders on
board
Business
Plan Check
Proof of Concept
(PoC)
Integration Tests
Proof of Value
(PoV)
Ideas
Use Cases
(Drafts)
Business Cases
(Concepts)
Prototypes
Releases
MVDP
*
Meet-ing
Work-
shop
Designing
Experimenting
Engineering
Operating
Use Case Ideation & Prioritization Process
Count Effort
?
Backlog
* MVP: Minimum Viable (Data) Product
Data (Product
Design)
Sprints
(Agile)
Develop-ment
Sprints
23. From Use Cases to Business Cases
User
Under-
standing
Business
Under-
standing
Data
Under-
standing
Users
Problems
Solutions
Benefits
?
Use
Cases
Costs
Risks
Profits
Business
Cases
?
Object-
ives
Results
Actions
Decisions
?
Diverge Converge Diverge Converge Diverge Converge
Viability Desirability Feasibility
24. 1st Day: Overview of Actual
Status & Outlook on Target
Status.
2nd Day: In-depth Look &
Check into the Details.
25. Martin Szugat & Martijn Baker @ Data Brain Meetup:
➔ https://www.slideshare.net/Datentreiber/presentations
➔ https://www.youtube.com/watch?v=U8EbR2gnl_o
Data Strategy Design:
An Open Source Toolbox &
Method for Data Thinking
26. Agenda
Welcoming: introduction & agenda00:00
Overview over the AI product innovation cycle00:15
AI Thinking: ideating and prioritizing the right use cases00:30
AI Engineering: building scalable & user friendly AI applications01:00
AI Management: maintaining AI solutions with DataOps01:15
Outlook: how to become an AI product manager (links & more)01:30
Q&A01:45
AI Prototyping: testing critical hypotheses with experiments00:45
27. Cross Industry Standard
Process for Data Mining
https://en.wikipedia.org/wiki/Cross_Industry_Sta
ndard_Process_for_Data_Mining
LEARN
BUILD
MEASURE
31. Assum-
ption
A demand forecast of accuracy x% will decrease out
of stock situations by y% and thus save the company
z% euros per year.
12/2020
Martin Szugat 2 Month
Build a simple machine learning model
and test it with n users (demand
planners).
Model performance as RMSE as well
as business performance as OoS delta
rate.
Prediction Performance
RMSE < e.g. current estimation
OoS rate > -10% → Saved costs per year = 1M €
➔ Positive estimated ROI for project
35. Assum-
ption Business performance doesn’t scale
with model performance
10.12.2020
Martin Szugat
A better demand forecast prediction will reduce out of
stock situations.
That even if the RSME is improved by 10% the
OoS rate is only decreased by 2%.
Model performance and business performance
doesn’t scale the same level.
Test other machine learning approaches to improve RSME by
x%.
42. Agenda
Welcoming: introduction & agenda00:00
Overview over the AI product innovation cycle00:15
AI Thinking: ideating and prioritizing the right use cases00:30
AI Prototyping: testing critical hypotheses with experiments00:45
AI Management: maintaining AI solutions with DataOps01:15
Outlook: how to become an AI product manager (links & more)01:30
Q&A01:45
AI Engineering: building scalable & user friendly AI applications01:00
43. PoC Trap
• Data & technology faith
• “Throwing over the fences” phenomena
• “Not thought through to the end” mindset
46. Experiment vs. Test
Source: https://medium.com/data-ops/dataops-is-not-just-devops-for-data-6e03083157b7
Test
Experiment
47. 48
Exploration Stage
Gold Standard Data Sets
Analytics in
Production
Data Lakeland
Validation Stage:
Real World Data Sets
Production Stage
“Real Time” Data Sets
Moni-
toring
Analytics in
Development
Analytics in
Experimentation
Frequent
Exports
Sporadic
Exports
Sandboxes
50. Agenda
Welcoming: introduction & agenda00:00
Overview over the AI product innovation cycle00:15
AI Thinking: ideating and prioritizing the right use cases00:30
AI Prototyping: testing critical hypotheses with experiments00:45
AI Engineering: building scalable & user friendly AI applications01:00
Outlook: how to become an AI product manager (links & more)01:30
Q&A01:45
AI Management: maintaining AI solutions with DataOps01:15
58. Agenda
Welcoming: introduction & agenda00:00
Overview over the AI product innovation cycle00:15
AI Thinking: ideating and prioritizing the right use cases00:30
AI Prototyping: testing critical hypotheses with experiments00:45
AI Engineering: building scalable & user friendly AI applications01:00
AI Management: maintaining AI solutions with DataOps01:15
Q&A01:45
Outlook: how to become an AI product manager (links & more)01:30
59. Non-Linear Data Product Innovation Process (Cycle of Cycles)
Designing Engineering Operating
5Prototypes
20Concepts
3Products
100Ideas
1System
Experiment-
ing
Unit Data Strategy Data Lab Data Factory Data Operations
Back to Backlog Back to Backlog
PoC
PoV
Tests
AI Product Manager
63. AI Product Manager
Analytical
Technical
Business
Design
Thinking
Product Design & Management
DataOps
Scrum / Kanban
Data & Software
Architecture
Data Management &
Governance
Machine Learning
Statistics
CRISP-DM
AI Governance
Business
Analyses
Data Visualization &
Storytelling
Soft Skills: Moderation,
Mediation, Negotiation, ..
CI / CD
DevOps
UI / UX
Lean
Management
64. 65
Further literature
1. Data Strategy & Data Thinking
1. Design thinking for data products
2. Data Strategy: Good Data vs. Bad Data
3. How to Define and Execute Your Data and AI Strategy
4. See next slide
2. Data Science Development Process:
1. Data Science at Roche: From Exploration to Productionization
2. Data Science Development Lifecycle
3. DataOps / ModelOps / AIOps
1. DataOps is NOT Just DevOps for Data
2. The DataOps Cookbook
3. Introducing ModelOps To Operationalize AI
4. Monitoring Machine Learning Models in Production
5. Continuous Delivery for Machine Learning
4. AI Product Management
1. A step-by-step guide to becoming a Data Product Manager
2. Managing Data Science as Products
3. What you need to know about product management for AI
4. Practical Skills for The AI Product Manager
5. Bringing an AI Product to Market
5. Other
1. The New Business of AI (and How It’s Different From
Traditional Software)
2. When is AI not AI?
67. Agenda
Welcoming: introduction & agenda00:00
Overview over the AI product innovation cycle00:15
AI Thinking: ideating and prioritizing the right use cases00:30
AI Prototyping: testing critical hypotheses with experiments00:45
AI Engineering: building scalable & user friendly AI applications01:00
AI Management: maintaining AI solutions with DataOps01:15
Outlook: how to become an AI product manager (links & more)01:30
Q&A01:45
69. datentreiber.deWir treiben Ihr Unternehmen voran.
Web: www.datentreiber.de
Blog: www.datentreiber.de/blog/
Martin Szugat
Geschäftsführer
Telefon: +49 [0]881 12 88 46 53
Email: ms@datentreiber.de