Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

of

Conductrics interpretable Machine Learning Slide 1 Conductrics interpretable Machine Learning Slide 2 Conductrics interpretable Machine Learning Slide 3 Conductrics interpretable Machine Learning Slide 4 Conductrics interpretable Machine Learning Slide 5 Conductrics interpretable Machine Learning Slide 6 Conductrics interpretable Machine Learning Slide 7 Conductrics interpretable Machine Learning Slide 8 Conductrics interpretable Machine Learning Slide 9 Conductrics interpretable Machine Learning Slide 10 Conductrics interpretable Machine Learning Slide 11 Conductrics interpretable Machine Learning Slide 12 Conductrics interpretable Machine Learning Slide 13 Conductrics interpretable Machine Learning Slide 14 Conductrics interpretable Machine Learning Slide 15 Conductrics interpretable Machine Learning Slide 16 Conductrics interpretable Machine Learning Slide 17 Conductrics interpretable Machine Learning Slide 18 Conductrics interpretable Machine Learning Slide 19 Conductrics interpretable Machine Learning Slide 20 Conductrics interpretable Machine Learning Slide 21 Conductrics interpretable Machine Learning Slide 22 Conductrics interpretable Machine Learning Slide 23 Conductrics interpretable Machine Learning Slide 24 Conductrics interpretable Machine Learning Slide 25 Conductrics interpretable Machine Learning Slide 26 Conductrics interpretable Machine Learning Slide 27 Conductrics interpretable Machine Learning Slide 28 Conductrics interpretable Machine Learning Slide 29 Conductrics interpretable Machine Learning Slide 30 Conductrics interpretable Machine Learning Slide 31 Conductrics interpretable Machine Learning Slide 32 Conductrics interpretable Machine Learning Slide 33 Conductrics interpretable Machine Learning Slide 34 Conductrics interpretable Machine Learning Slide 35 Conductrics interpretable Machine Learning Slide 36 Conductrics interpretable Machine Learning Slide 37 Conductrics interpretable Machine Learning Slide 38 Conductrics interpretable Machine Learning Slide 39 Conductrics interpretable Machine Learning Slide 40 Conductrics interpretable Machine Learning Slide 41 Conductrics interpretable Machine Learning Slide 42 Conductrics interpretable Machine Learning Slide 43 Conductrics interpretable Machine Learning Slide 44 Conductrics interpretable Machine Learning Slide 45 Conductrics interpretable Machine Learning Slide 46 Conductrics interpretable Machine Learning Slide 47 Conductrics interpretable Machine Learning Slide 48 Conductrics interpretable Machine Learning Slide 49 Conductrics interpretable Machine Learning Slide 50 Conductrics interpretable Machine Learning Slide 51 Conductrics interpretable Machine Learning Slide 52 Conductrics interpretable Machine Learning Slide 53 Conductrics interpretable Machine Learning Slide 54 Conductrics interpretable Machine Learning Slide 55 Conductrics interpretable Machine Learning Slide 56 Conductrics interpretable Machine Learning Slide 57 Conductrics interpretable Machine Learning Slide 58 Conductrics interpretable Machine Learning Slide 59 Conductrics interpretable Machine Learning Slide 60 Conductrics interpretable Machine Learning Slide 61 Conductrics interpretable Machine Learning Slide 62 Conductrics interpretable Machine Learning Slide 63 Conductrics interpretable Machine Learning Slide 64 Conductrics interpretable Machine Learning Slide 65 Conductrics interpretable Machine Learning Slide 66 Conductrics interpretable Machine Learning Slide 67 Conductrics interpretable Machine Learning Slide 68 Conductrics interpretable Machine Learning Slide 69 Conductrics interpretable Machine Learning Slide 70 Conductrics interpretable Machine Learning Slide 71 Conductrics interpretable Machine Learning Slide 72 Conductrics interpretable Machine Learning Slide 73 Conductrics interpretable Machine Learning Slide 74 Conductrics interpretable Machine Learning Slide 75 Conductrics interpretable Machine Learning Slide 76 Conductrics interpretable Machine Learning Slide 77 Conductrics interpretable Machine Learning Slide 78 Conductrics interpretable Machine Learning Slide 79 Conductrics interpretable Machine Learning Slide 80 Conductrics interpretable Machine Learning Slide 81 Conductrics interpretable Machine Learning Slide 82
Upcoming SlideShare
What to Upload to SlideShare
Next

4 Likes

Share

Conductrics interpretable Machine Learning

For many tasks, it makes little difference if these programs are opaque to human introspection. Here, high capacity models, like deep learning, suffer little penalty for representational complexity.
However, for several reasons, marketers tend to be wary about ceding control of their customers’ experiences to black box methods.
This presentation covers Conductrics approach to generating machine learning for marketing optimization that is both machine and human readable.

Conductrics interpretable Machine Learning

  1. 1. Agenda 1. Quick overview of Machine learning 2. Complexity is costly 3. Interpretability vs Accuracy 4. AB Testing is NOT Optimization 5. This is hard – which is Good! 6. Get drinks Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  2. 2. Machine Learning Definition Statistics see: Mitchel http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdfConductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  3. 3. Machine Learning Definition Statistics Computer Science see: Mitchel http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdfConductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  4. 4. Machine Learning Definition Statistics Computer Science Machine Learning see: Mitchel http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdfConductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  5. 5. Machine Learning Definition Methods that generate useful computer programs via interaction with Data or the Environment see: Mitchel http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdf Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  6. 6. Example of Machine Learning Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  7. 7. Machine Learning Example: Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  8. 8. Deep Reinforcement Learning Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  9. 9. What are Deep Neural Net? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  10. 10. Start Simple: What is Regression? Source: Larochelle - Neural Networks 1 - DLSS 2017.pdfConductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  11. 11. Regression 1) Input Data X1 X2 Xd…x • Weekend/Weekday • Mobile/Desktop • Browser Type • User Age • Geo/Census • Weather • Tenure/RFM Score Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  12. 12. Regression 1) Input Data 2) Output Layer (The Model) X1 X2 Xd… Sf(x) x Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  13. 13. Regression 1) Input Data 2) Output Layer X1 X2 Xd… Sf(x) x 𝑓(𝑥) = 𝑤0 + ෍ 𝑑 𝑤 𝑑 ∗ 𝑥 𝑑 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  14. 14. Regression 𝑓(𝑥) = 𝒘 𝟎 + ෍ 𝑑 𝒘 𝒅 ∗ 𝑥 𝑑 Goal: Learn the weights - the ws Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  15. 15. Deep Neural Net? 1) Input Data 2) Hidden Layer 3) Hidden Layer 4) Output Layer Source: Larochelle - Neural Networks 1 - DLSS 2017.pdfConductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  16. 16. Deep Neural Net? 1) Input Data 2) Hidden Layer 3) Hidden Layer 4) Output Layer Source: Larochelle - Neural Networks 1 - DLSS 2017.pdfConductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  17. 17. Deep Neural Net? 1) Input Data 2) Hidden Layer 3) Hidden Layer 4) Output Layer Source: Larochelle - Neural Networks 1 - DLSS 2017.pdfConductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  18. 18. Deep Neural Net? Pros: 1 Flexible 2 Accurate/Expressive Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  19. 19. Deep Neural Net? Pros: 1 Flexible 2 Accurate/Expressive Cons: 1 Complex (brittle?) 2 Expensive 3 Overkill 4 Hard to interpret Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  20. 20. IF [Customer] THEN [Experiences?] Online Optimization according to Conductrics Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  21. 21. IF [Customer] THEN [Experiences?] Online Optimization according to Conductrics Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  22. 22. Online Optimization Pixel Data Customer Data • Weekend/Weekday • Mobile/Desktop • Browser Type • User Age • Geo/Census • Weather • Tenure/RFM Score Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  23. 23. Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  24. 24. Four Arguments for Interpretability Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  25. 25. Arguments for Interpretability 1. Trust Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  26. 26. Arguments for Interpretability 1. Trust 2. Insights Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  27. 27. Arguments for Interpretability 1. Trust 2. Insights 3. Review/Accountability Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  28. 28. Arguments for Interpretability 1. Trust 2. Insights 3. Review/Accountability 4. Communication Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  29. 29. General Data Protection Regulation 1. EU Data Protection Laws Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  30. 30. General Data Protection Regulation 1. EU Data Protection Laws 2. Goes in affect May 2018 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  31. 31. General Data Protection Regulation 1. EU Data Protection Laws 2. Goes in affect May 2018 3. Article 22 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  32. 32. GDPR: Article 22 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  33. 33. GDPR: Article 22 Automated Decisions are Contestable 1. What data was used? 2. Explain why a decision was made to EU citizens. 3. Non discrimination 4. Up to 4% world gross revenues Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  34. 34. What are some Options? 1. Use Complex Model and then try to interpret/explain 2. Use Simple interpretable model Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  35. 35. LIME: Interpreting Complex Models Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  36. 36. L Local I Interpretable M Model-Agnostic E Explanations Lime Interpreting Complex Models ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2016 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  37. 37. Lime: Interpreting Complex Models Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  38. 38. Lime: Interpreting Complex Models Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  39. 39. Lime: Interpreting Complex Models Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  40. 40. 1.Works for Classification problems (not regression yet) 2.Approximate Explanations (Lossy) 3.Explanation is not the model that is actually used – maybe an issue for GDPR? Lime: Interpreting Complex Models Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  41. 41. Don’t Build Complex Model Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  42. 42. Atari – Lots of Structure By Jorge Stolfi (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  43. 43. Marketing Problems By Jorge Stolfi (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons • High Noise • Often Low(ish) N Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  44. 44. Simple Models Simple Models have Lower Opportunity Costs Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  45. 45. Simple Models Simple Models have Lower Opportunity Costs IF World is By Jorge Stolfi (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  46. 46. Simple Models Simple Models have Lower Opportunity Costs IF World is Rather Than By Jorge Stolfi (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  47. 47. What Simple Model? Linear Model 𝑌 = 𝑋𝛽 + 𝜖 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  48. 48. Linear Model – the good 𝑌 = 𝑋𝛽 + 𝜖 1. Online / SGD What Simple Model? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  49. 49. 𝑌 = 𝑋𝛽 + 𝜖 1. Online / SGD 2. ’s interpretable Linear Model – the good What Simple Model? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  50. 50. 𝑌 = 𝑋𝛽 + 𝜖 1. Online / SGD 2. ’s interpretable 3. BLUE (sort of) Linear Model – the good What Simple Model? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  51. 51. Linear Model – the Bad 𝑌 = 𝑋𝛽 + 𝜖 What Simple Model? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  52. 52. Linear Model – the Bad 𝜷 𝐚𝟎 ⋯ 𝜷 𝐚𝒎 ⋮ ⋱ ⋮ 𝜷 𝒌𝟎 ⋯ 𝜷 𝒌𝒎 𝑌 = 𝑋𝛽 + 𝜖 What Simple Model? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  53. 53. Linear Model – the Bad 𝜷 𝐚𝟎 ⋯ 𝜷 𝐚𝒎 ⋮ ⋱ ⋮ 𝜷 𝒌𝟎 ⋯ 𝜷 𝒌𝒎 What Simple Model? Still Not really that interpretable Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  54. 54. Decision Tree – Better! What Simple Model? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  55. 55. Decision Tree – Has nice properties Use Simple Model? 1)Human Readable for Trust and Insights Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  56. 56. Decision Tree – Has nice properties Use Simple Model? 1)Human Readable for Trust and Insights 2)Easily to Audit for review Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  57. 57. Decision Tree – Has nice properties Use Simple Model? 1)Human Readable for Trust and Insights 2)Easily to Audit for review 3)Loggable 1)Decision policy is represented as rules 2)Compact 3)Each decision can be logged with the exact policy/rule Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  58. 58. Conductrics Approach Learn with Online Model 𝑌 = 𝑋𝛽 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  59. 59. Conductrics Approach Learn with Online Model Control with Sparse Tree 𝑌 = 𝑋𝛽 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  60. 60. Decision Tree from Model Use a fact of Regression + a little algebra Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  61. 61. Decision Tree from Model ෠𝑌 = ത𝑌 + ෍ 𝑗  𝑗 ∗ 𝑥𝑗 − ഥ𝑥𝑗 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  62. 62. Decision Tree from Model • ‘Set’ all targeting to ‘0’ ෠𝑌 = ത𝑌 + ෍0 Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  63. 63. Decision Tree from Model ෠𝑌 = ത𝑌 • ‘Set’ all targeting to ‘0’ • Prediction reduces to Mean Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  64. 64. 1. Start with Simple Means (A,B,C, etc.) Decision Tree from Model Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  65. 65. Decision Tree from Model Start Root Node Using unconditional means Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  66. 66. 1. Start with Simple Means (A,B, etc.) 2. Collect Data ->Learn Model Decision Tree from Model Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  67. 67. 1. Start with Simple Means (A,B,C, etc.) 2. Collect Data ->Learn Model 3. Play ‘20’ Questions w/ model Decision Tree from Model Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  68. 68. 1. Start with Simple Means (A,B,C, etc.) 2. Collect Data ->Learn Model 3. Play ‘20’ Questions w/ model 4. Add Features to tree Decision Tree from Model Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  69. 69. 1. Start with Simple Means (A,B,C, etc.) 2. Collect Data ->Learn Model 3. Play ‘20’ Questions w/ model 4. Add Features to tree 5. Update Simple Means using this equation ෠𝑌 = ത𝑌 + ෍ 𝑗  𝑗 ∗ 𝑥𝑗 − ഥ𝑥𝑗 Decision Tree from Model Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  70. 70. Confidential Decision Tree from Model Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  71. 71. Confidential Conductrics Audience Report: Tree Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  72. 72. Confidential Conductrics Audience Report: List Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  73. 73. 1.Online Learning / Batch Controller Benefits for Conductrics Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  74. 74. 1.Online Learning / Batch Controller 2.Degrade gracefully – so shrink back to means Benefits for Conductrics Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  75. 75. 1.Online Learning / Batch Controller 2.Degrade gracefully – so shrink back to means 3.Shrinkage/partial pooling on means Benefits for Conductrics Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  76. 76. 1.Online Learning / Batch Controller 2.Degrade gracefully – so shrink back to means 3.Shrinkage/partial pooling on means 4.Add arbitrary constraints: 1.Size of leaf nodes 2.Depth of tree 3.Max Leaves Benefits for Conductrics Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  77. 77. Benefits for Clients 1.Interpretable 2.Easy to manage control logic 3.Auditable/Loggable Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  78. 78. 1) Importance of Interpretabilty 2) Marketing Problems often Noisy 3) Decision Trees good representation What did we Do/Learn? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  79. 79. 1) Importance of Interpretabilty 2) Marketing Problems often Noisy 3) Decision Trees good representation What did we Do/Learn? Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  80. 80. Learn more! 1. www.conductrics.com 2. https://conductrics.com/machine-learning-and-human-interpretability 3. info@conductrics.com Conductrics Inc. | Matt Gershoff | www.conductrics.com | @conductrics
  • octalxia

    Dec. 19, 2018
  • leenissen

    Sep. 23, 2017
  • MiliIbrulj

    Aug. 21, 2017
  • XiaohuZHU

    Jul. 26, 2017

For many tasks, it makes little difference if these programs are opaque to human introspection. Here, high capacity models, like deep learning, suffer little penalty for representational complexity. However, for several reasons, marketers tend to be wary about ceding control of their customers’ experiences to black box methods. This presentation covers Conductrics approach to generating machine learning for marketing optimization that is both machine and human readable.

Views

Total views

1,558

On Slideshare

0

From embeds

0

Number of embeds

14

Actions

Downloads

0

Shares

0

Comments

0

Likes

4

×