Amazon SageMaker is a fully managed platform for data scientists and developers to build, train and deploy machine learning models in production applications. In this workshop, you will learn how to integrate Amazon SageMaker with other AWS services in order to meet enterprise requirements. Using Amazon S3, Amazon Glue, Amazon KMS, Amazon SageMaker, Amazon CodeStar, Amazon ECR, IAM; we will walkthrough the machine learning lifecycle in an integrated AWS environment and discuss best practices.Attendees must have some familiarities with AWS products as well as a good understanding of machine learning theory. The dataset for the workshop will be provided.

1. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. @dmbanga
AWS re:INVENT
Amazon SageMaker in AWS
D a n R . M b a n g a
B u s i n e s s D e v e l o p m e n t M a n a g e r A I P l a t f o r m s a n d E n g i n e s
M C L 3 4 5
November 29, 2017

2. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank You

3. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Solving Some Of The Hardest Problems In Computer Science
Learning Language Perception Problem
Solving
Reasoning

4. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
The Machine Learning Process
Re-training

5. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Discovery: The Analysts
Re-training
• Help formulate the right
questions
• Domain Knowledge

6. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Integration: The Data Architecture
Retraining
• Build the data platform:
• Amazon S3
• AWS Glue
• Amazon Athena
• Amazon EMR
• Amazon Redshift
Spectrum

7. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Retraining
Modeling: The Data Science
• Builds the ML Models:
• AWS Deep Learning
AMI
• SparkML on Amazon
EMR
• Amazon
SageMaker

8. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Data Visualization &
Analysis
Business Problem –
ML problem framing Data Collection
Data Integration
Data Preparation &
Cleaning
Feature Engineering
Model Training &
Parameter Tuning
Model Evaluation
Are Business
Goals met?
Model Deployment
Monitoring &
Debugging
– Predictions
YesNo
DataAugmentation
Feature
Augmentation
Retraining
Production: The DevOps
• Build Smart Apps
• AWS Lambda
• Amazon S3
• API Gateway
• IoT
• Kinesis
• ECS/ECR
• Mobile Hub
• AWS KMS
• EC2
• Amazon
SageMaker

9. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
A fully managed service that enables data scientists and developers to quickly and easily
build machine-learning based models into production smart applications.
Amazon SageMaker

10. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
 Agile, Reliable, GPU powered, and Productivity Ready Notebook instances for Data Scientist and
Developers
 High-Performance Web-Scale Algorithms Out Of The Box
 Managed Distributed Model Training Service
 Production Ready Model Hosting requiring no engineering
Amazon SageMaker

11. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Amazon SageMaker
Client application
Training code

12. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Trainingdata
Training code Helper code
Client application
Training code
Amazon SageMaker

13. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Client application
Inference code
Training code
Amazon SageMaker

14. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Helper codeInference code
Client application
Inference code
Training code
Amazon SageMaker

15. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Helper codeInference code
Client application
Inference code
Training code
Inference requestInference response
Inference Endpoint
Amazon SageMaker

16. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon ECR
Model Training (on EC2)
Model Hosting (on EC2)
Trainingdata
Modelartifacts
Training code Helper code
Helper codeInference code
GroundTruth
Client application
Inference code
Training code
Inference requestInference response
Inference Endpoint
Amazon SageMaker

17. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon SageMaker
1 2 3 4
I I I I
Notebook Instances 1P Algorithms ML Training Service ML Hosting Service

18. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
1
I
Notebook Instances
Zero Setup For Exploratory Data Analysis
Authoring &
Notebooks
ETL Access to AWS
Database services
Access to S3 Data
Lake
• Recommendations/Personalization
• Fraud Detection
• Forecasting
• Image Classification
• Churn Prediction
• Marketing Email/Campaign Targeting
• Log processing and anomaly detection
• Speech to Text
• More…
“Just add data”

19. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2
I
1P Algorithms
ML Algorithms optimized for speed and Large datasets
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (IM builds the Container)
IM Estimators in
Apache Spark

20. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Managed Distributed Training With Flexibility
Training code
• Matrix Factorization
• Regression
• Principal Component Analysis
• K-Means Clustering
• Gradient Boosted Trees
• And More!
Amazon provided Algorithms
Bring Your Own Script (IM builds the Container)
Bring Your Own Algorithm (You build the Container)
3
I
ML Training Service
Fetch Training data
Save Model Artifacts
Fully
managed –
Secured–
Amazon ECR
Save Inference Image
IM Estimators in
Apache Spark
CPU GPU HPO

21. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there!

22. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there!
Create a Model
ModelName: prod
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

23. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there!
Create versions of a Model
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

24. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
Instance type: c3.4xlarge
MinInstanceCount: 5
MaxInstanceCount: 20
ModelName: prod
VariantName: prodPrimary
VariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there!
Create weighted
ProductionVariants
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

25. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
InstanceType: c3.4xlarge
MinInstanceCount: 5
MaxInstanceCount: 20
ModelName: prod
VariantName: prodPrimary
VariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there!
Create an
EndpointConfiguration
from one or many
ProductionVariant(s)EndpointConfiguration
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

26. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
InstanceType: c3.4xlarge
MinInstanceCount: 5
MaxInstanceCount: 20
ModelName: prod
VariantName: prodPrimary
VariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there! Create an Endpoint from
one EndpointConfiguration
EndpointConfiguration
Inference Endpoint
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

27. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
Amazon ECR
30 50
10 10
InstanceType: c3.4xlarge
MinInstanceCount: 5
MaxInstanceCount: 20
ModelName: prod
VariantName: prodPrimary
VariantWeight: 50
ProductionVariant
Model Artifacts
Inference Image
Model versions
Versions of the same
inference code saved in
inference containers.
Prod is the primary
one, 50% of the traffic
must be served there!
One-Click!
EndpointConfiguration
Inference Endpoint
Amazon Provided Algorithms
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

28. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
4
I
ML Hosting Service
 Auto-Scaling Inference
APIs
 A/B Testing (more to
come)
 Low Latency & High
Throughput
 Bring Your Own Model
 Python SDK
Amazon SageMaker
Easy Model Deployment to Amazon SageMaker

29. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Let’s build <>!

30. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

31. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
SageMaker
Notebooks
Training
Algorithm
SageMaker
Training
Amazon ECR
Code Commit
Code Pipeline
SageMaker
Hosting
Coco dataset http://sm-demo2017.s3-website-us-east-1.amazonaws.com/
AWS
Lambda
API
Gateway
PyTorch on SageMaker Style Transfer App Architecture

32. @dmbanga© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. @dmbanga
THANK YOU!
d m m b a n g a @ a m a z o n . c o m