Conductrics interpretable Machine Learning

Agenda
1. Quick overview of Machine learning
2. Complexity is costly
3. Interpretability vs Accuracy
4. AB Testing is NOT Optimization
5. This is hard – which is Good!
6. Get drinks
Conductrics Inc. | Matt Gershoff |
www.conductrics.com | @conductrics

Machine Learning Definition
Statistics
see: Mitchel http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdfConductrics Inc. | Matt Gershoff |

Statistics Computer
Science

Statistics Computer
Science
Machine
Learning

Methods that generate useful
computer programs via
interaction with Data or the
Environment
see: Mitchel http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdf

Example of Machine Learning

Machine Learning Example:

Deep Reinforcement Learning

What are Deep Neural Net?

Start Simple: What is Regression?
Source: Larochelle - Neural Networks 1 - DLSS 2017.pdfConductrics Inc. | Matt Gershoff |

Regression
1) Input Data X1 X2 Xd…x
• Weekend/Weekday
• Mobile/Desktop
• Browser Type
• User Age
• Geo/Census
• Weather
• Tenure/RFM Score

Regression
1) Input Data
2) Output Layer
(The Model)
X1 X2 Xd…
Sf(x)
x

Regression
1) Input Data
2) Output Layer
X1 X2 Xd…
Sf(x)
x
𝑓(𝑥) = 𝑤0 + ෍
𝑑
𝑤 𝑑 ∗ 𝑥 𝑑

Regression
𝑓(𝑥) = 𝒘 𝟎 + ෍
𝑑
𝒘 𝒅 ∗ 𝑥 𝑑
Goal: Learn the weights - the ws

Deep Neural Net?
1) Input Data
2) Hidden Layer
3) Hidden Layer
4) Output Layer
Source: Larochelle - Neural Networks 1 - DLSS 2017.pdfConductrics Inc. | Matt Gershoff |

Deep Neural Net?
Pros:
1 Flexible
2 Accurate/Expressive

Deep Neural Net?
Pros:
1 Flexible
2 Accurate/Expressive
Cons:
1 Complex (brittle?)
2 Expensive
3 Overkill
4 Hard to interpret

IF [Customer]
THEN
[Experiences?]
Online Optimization according to
Conductrics

Online Optimization
Pixel Data Customer Data
• Weekend/Weekday
• Mobile/Desktop
• Browser Type
• User Age
• Geo/Census
• Weather
• Tenure/RFM Score

Four Arguments for Interpretability

Arguments for Interpretability
1. Trust

1. Trust
2. Insights

1. Trust
2. Insights
3. Review/Accountability

1. Trust
2. Insights
3. Review/Accountability
4. Communication

General Data Protection Regulation
1. EU Data Protection Laws

2. Goes in affect May 2018

2. Goes in affect May 2018
3. Article 22

GDPR: Article 22

GDPR: Article 22
Automated Decisions are Contestable
1. What data was used?
2. Explain why a decision was made
to EU citizens.
3. Non discrimination
4. Up to 4% world gross revenues

What are some Options?
1. Use Complex Model and
then try to interpret/explain
2. Use Simple interpretable
model

LIME: Interpreting Complex Models

L Local
I Interpretable
M Model-Agnostic
E Explanations
Lime Interpreting Complex Models
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
(KDD), 2016 Conductrics Inc. | Matt Gershoff |

Lime: Interpreting Complex Models

1.Works for Classification problems (not
regression yet)
2.Approximate Explanations (Lossy)
3.Explanation is not the model that is
actually used – maybe an issue for GDPR?
Lime: Interpreting Complex Models

Don’t Build Complex Model

Atari – Lots of Structure
By Jorge Stolfi (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

Marketing Problems
• High Noise
• Often Low(ish) N

Simple Models
Simple Models have Lower Opportunity Costs

Simple Models
IF World is

Simple Models
IF World is
Rather Than

What Simple Model?
Linear Model
𝑌 = 𝑋𝛽 + 𝜖

Linear Model – the good
1. Online / SGD
What Simple Model?

1. Online / SGD
2. ’s interpretable
What Simple Model?

1. Online / SGD
2. ’s interpretable
3. BLUE (sort of)
What Simple Model?

Linear Model – the Bad
What Simple Model?

𝜷 𝐚𝟎 ⋯ 𝜷 𝐚𝒎
⋮ ⋱ ⋮
𝜷 𝒌𝟎 ⋯ 𝜷 𝒌𝒎
What Simple Model?

𝜷 𝐚𝟎 ⋯ 𝜷 𝐚𝒎
⋮ ⋱ ⋮
𝜷 𝒌𝟎 ⋯ 𝜷 𝒌𝒎
What Simple Model?
Still Not really that interpretable

Decision Tree – Better!
What Simple Model?

Decision Tree – Has nice properties
Use Simple Model?
1)Human Readable for Trust and Insights

Use Simple Model?
2)Easily to Audit for review

Use Simple Model?
2)Easily to Audit for review
3)Loggable
1)Decision policy is represented as rules
2)Compact
3)Each decision can be logged with the exact
policy/rule

Conductrics Approach
Learn with
Online Model
𝑌 = 𝑋𝛽

Conductrics Approach
Learn with
Online Model
Control with
Sparse Tree
𝑌 = 𝑋𝛽

Decision Tree from Model
Use a fact of Regression +
a little algebra

෠𝑌 = ത𝑌 + ෍
𝑗
 𝑗 ∗ 𝑥𝑗 − ഥ𝑥𝑗

• ‘Set’ all targeting to ‘0’
෠𝑌 = ത𝑌 + ෍0

෠𝑌 = ത𝑌
• ‘Set’ all targeting to ‘0’
• Prediction reduces to Mean

1. Start with Simple Means (A,B,C, etc.)

Start Root Node Using unconditional
means

1. Start with Simple Means (A,B, etc.)
2. Collect Data ->Learn Model

3. Play ‘20’ Questions w/ model

4. Add Features to tree

4. Add Features to tree
5. Update Simple Means using this equation
෠𝑌 = ത𝑌 + ෍
𝑗
 𝑗 ∗ 𝑥𝑗 − ഥ𝑥𝑗

Confidential

Confidential
Conductrics Audience Report: Tree

Confidential
Conductrics Audience Report: List

1.Online Learning / Batch Controller
Benefits for Conductrics

2.Degrade gracefully – so shrink back to means

3.Shrinkage/partial pooling on means

3.Shrinkage/partial pooling on means
4.Add arbitrary constraints:
1.Size of leaf nodes
2.Depth of tree
3.Max Leaves

Benefits for Clients
1.Interpretable
2.Easy to manage control logic
3.Auditable/Loggable

1) Importance of Interpretabilty
2) Marketing Problems often Noisy
3) Decision Trees good representation
What did we Do/Learn?

Learn more!
1. www.conductrics.com
2. https://conductrics.com/machine-learning-and-human-interpretability
3. info@conductrics.com

Conductrics interpretable Machine Learning

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Conductrics interpretable Machine Learning

Ähnlich wie Conductrics interpretable Machine Learning (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Conductrics interpretable Machine Learning