You have heard how containers are great for running microservices, but running and managing large scale applications with microservices architectures is hard and often requires operating complex container management infrastructure. So what exactly is needed to get microservices to run in production at scale? In this session, we will explore the reasoning and concepts behind microservices and how containers simplify building microservices based applications, and we will walk through a number of patterns used by our customers to run their microservices platforms. We will also dive deep into some of the challenges of running microservices, such as load balancing, service discovery, and secrets management, and we’ll see how Amazon EC2 Container Service (ECS) can help address them. We will also demo how you can easily deploy complex microservices applications using Amazon ECS.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Drew Dennis, Solution Architect, Amazon
July 26, 2017
Deep Dive on
Microservices and Docker

2. What to expect from this session
• Microservices architecture
• Amazon ECS
• Task placement
• The twelve-factor app with Amazon ECS

3. Microservices Architecture

4. What are microservices?
“A software architecture style in which complex
applications are composed of small, independent
processes communicating with each other using
language-agnostic APIs. These services are small, highly
decoupled and focus on doing a small task, facilitating a
modular approach to system-building.” - Wikipedia
https://en.wikipedia.org/wiki/Microservices

5. Monolithic vs. microservices
webserver
.package
Order UI
Order Service
Inventory
Service
Shipping
Service
OrderUI
Inventory
Service
Order
Service
Shipping
Service

6. Characteristics of microservice architectures
Do one
thing wellIndependent
Decentralized
Black box
Polyglot
You build it, you run it

7. Microservices on EC2
Auto Scaling group
Elastic
Load
Balancer
EC2 instance EC2 instance EC2 instance
AWS Elastic
Beanstalk
Database

8. Elastic Beanstalk
Your code
HTTP server
Application server
Language interpreter
Operating system
Host
Elastic Beanstalk configures
each EC2 instance in your
environment with the
components necessary to run
applications for the selected
platform. No more worrying
about logging into instances to
install and configure your
application stack.
Focus on building your
application
Provided by you
Provided and managed by Elastic Beanstalk
Including Docker support…

9. Managed
API Gateway serverless approach
InternetMobile
apps
AWS
Lambda
functions
AWS
API Gateway
cache
Endpoints on
Amazon EC2
Any other
publicly
accessible
endpoint
Amazon
CloudWatch
Amazon
CloudFront
API
Gateway
API Gateway
Other AWS
services
AWS
Lambda
functions

10. Amazon API Gateway
Create, publish, maintain, monitor and
secure RESTful APIs
Powered by our content delivery
network via 60 global edge locations
Provides DDoS protection and
throttling capabilities
Multiple API stages which you define
(e.g., dev, test, prod)
AWS Lambda
Amazon API
Gateway
Amazon EC2
AWS API
On-prem server

11. Why Amazon ECS?
• Fully managed, elastic service—you don’t
need to run anything, and the service
scales as your microservices architecture
grows
• Shared state optimistic scheduling
• Integration with Amazon CloudWatch for
monitoring and logging
• Integration with Code* services for
continuous integration and delivery (CI/CD)

12. Deploying containers on ECS – choose a scheduler
Batch Jobs
ECS task scheduler
Run tasks once
Batch jobs
RunTask (random)
StartTask (placed)
Long-Running Apps
ECS service scheduler
Health management
Scale-up and scale-down
AZ aware
Grouped containers

13. WatchBot
ECS Cluster
SQS
Watcher
Container
Running
Tasks
14
Data Processing
Services
3500
Peak Container
Instances
500 million
Compute Hours
Used in 2016

14. Instacart’s PaaS
Coordinated through RDS database
Same tooling local and cloud
Jobs take desired state from DB, and turn
it into cloud reality

15. • Media encoding experimentation
• Digital watermarking
Netflix

16. Amazon ECS: Under the hood
ALB ALB
AZ 1 AZ 2
user / scheduler
Scheduler
Cluster State Service
Placement Engine
Event Stream

17. Task Placement Examples

18. New placement constraints and attributes
Name Example
AMI ID attribute:ecs.ami-id == ami-eca289fb
Availability Zone attribute:ecs.availability-zone == us-east-1a
Instance Type attribute:ecs.instance-type == t2.small
Distinct Instances type=“distinctInstance”
Custom attribute:stack == prod

19. Anatomy of task placement
Cluster Constraints
Custom Constraints
Placement Strategies
Apply Filter
Satisfy CPU, memory, and port requirements
Filter for location, instance-type, AMI, or custom
attribute constraints
Identify instances that meet spread or binpack
placement strategy
Select final container instances for placement

20. Supported placement strategies
Binpacking Spread Affinity Distinct Instance

21. Placement strategy chaining
Spread tasks across Zones
and Binpack within each Zone

22. New Cluster Query Language

23. Filtering: Match on instance family or type

24. Filtering: Match on availability zone

25. Filtering: Match on multiple expressions

26. Filtering: Match on custom attributes

27. Task Placement Examples

28. g2.2xlarge t2.small g2.2xlarge g2.2xlarge
Placement: Targeting instance type

29. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge
t2.small
t2.small t2.medium
us-east-1aus-east-1d
Placement: Targeting instance type and zone

30. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c
Placement: Availability zone spread

31. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c
Placement: Spread across zone and binpack

32. g2.2xlarge t2.small t2.micro t2.medium
t2.medium t2.small g2.2xlarge t2.small
us-east-1aus-east-1d
g2.2xlarge t2.medium
t2.micro t2.small
us-east-1c
Placement: Affinity and anti-affinity

33. Running a service

34. t2.medium t2.small t2.small
us-east-1aus-east-1d
t2.medium t2.micro t2.small
us-east-1c
Placement: Multiple services on a cluster

35. t2.medium g2.2xlarge t2.micro t2.small
t2.small t2.small g2.2xlarge t2.small
t2.small t2.small
g2.2xlarge t2.small
Placement: Services – distinct instances

36. Console: Getting started with placement

37. Console: Placement templates to get started

38. Console: Customizing placement strategies

39. The Twelve-Factor App
with Amazon ECS
Reference Architectures

40. The Twelve-Factor App
1. Codebase
2. Dependencies
3. Config
4. Backing services
5. Build, release, run
6. Processes
7. Port binding
8. Concurrency
9. Disposability
10. Dev/prod parity
11. Logs
12. Admin processes
https://12factor.net/

41. Reference Architectures

42. Example: Microservice architecture on ECS
Amazon
ECR
Amazon
RDS
Application Load
Balancer
ECS Cluster
ECS Cluster
IAM
Amazon API
Gateway*
Amazon
Route 53
Amazon CloudWatch

43. Automatic service scaling
Publish metrics
Auto Scaling ECS service
Availability
Zone A
Availability
Zone B
TASK A
Add/Remove ECS
tasks
TASK C
TASK BScaling Policies
Amazon
CloudWatch
Amazon ECS
Application
Load Balancer

44. IAM roles for tasks
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
Amazon
DynamoDB
Amazon
S3

45. Continuous deployment
AWS
CodeCommit AWS
CodePipeline
AWS
CodeBuild
Amazon
ECR
Amazon ECS
instance
Spot
Instance
AWS
CloudFormation
1. Commit
Code
2. Trigger
Pipeline
3.Build
Artifact
5.Update
Stack
6. Update Service
4. Push Image

46. Blue-green deployments
TaskTask
Route 53
record set
with
weighted
routing
policy
0%
100%

47. Secrets management
•prod.app1.db-pass
•general.license-code
•prod.app2.user-name
ECS Cluster
EC2 Instance EC2 Instance
TASK A
TASK B
TASK B
EC2 System Manager –
Parameter Store

48. Service Discovery with Route 53 and Application
Load Balancers
Application Load
Balancer
i-aaa i-bbb i-ccc
i-aaa i-bbb i-ccc
oAuth Target Group
8080 8081
Portal Target Group
Weather Target Group
8000 8001 8002
8080
Amazon
Route 53
ECS Cluster
mydomain.com
mydomain.com mydomain.com/weather
mydomain.com/auth

49. Consuming events for service discovery
app1-tst  10.1.0.11
db1-tst  10.1.0.14
app2  10.1.0.16
db2  10.1.0.18
my-app  10.1.0.20
websrv1 10.1.0.1
websrv2 10.1.0.2
websrv3 10.1.0.4
app-dev1 10.1.0.9
app-dev2 10.1.0.5
app-dev3 10.1.0.8
db-dev 10.1.0.19
Amazon Route53 AWS Lambda

50. Governance: AWS Service Catalog
Organizations Developers
Control
Standardization
Governance
Agility
Self-service
Time to market

51. Creates portfolio
Adds constraints and grant access
1
4
5
Administrator
Portfolio
Users
Browse Products
6Launch Products
Creates
product3Authors template2
Product
X
Product
Y
Product
Z
7
Deploys
stacks
Notifications
Notifications
8
8
Governance: AWS Service Catalog

52. Build Learn
Try it out!
Deploy

53. Thank You