The document discusses the evolution of web application architectures from single server deployments to modern cloud-native microservice architectures. It covers how early web apps faced challenges as demand grew that led to adopting caching, load balancing, and the cloud. It then discusses how further scaling issues were addressed through containers, microservices, and patterns like API gateways and circuit breakers. Key lessons are around adopting cloud-native patterns to improve performance, scalability, resilience and developer productivity for applications.
1. Finding Patterns
in the
Clouds
Steve “ardalis” Smith
@ardalis | steve@ardalis.com
ardalis.com | weeklydevtips.com
Design Patterns for Cloud-
Native Applications
2. Please Rate in the App
• AttendeeHub
@ardalis | Finding Patterns in the Clouds
3. More Resources
• Podcast
WeeklyDevTips.com
• Group Mentoring Program
DevBetter.com
• Free Microsoft eBooks
ardalis.com/architecture-ebook
ardalis.com/cloud-native-book
@ardalis | Finding Patterns in the Clouds
4. Let’s take a trip back to the
beginning of today’s web…
@ardalis | Finding Patterns in the Clouds
9. One Web Server To Rule Them All
Client Machine Server
Request
Response
App
Data
(and one Webmaster to run it all – the original Full Stack Developer™)
@ardalis | Finding Patterns in the Clouds
NCSA Mosaic
10. One Web Server To Rule Them All
Client Machine Server
Request
Response
11. Report Card: One Server To Rule Them All
@ardalis | Finding Patterns in the Clouds
Availability
Data Management
Design and Implementation
Messaging Security
Management and Monitoring
Resiliency
Performance and Scalability
14. Availability
How often is the system or service up?
Often expressed as a percentage.
99.99% uptime = 1 minute of downtime per week
99.999% uptime = 26 seconds of downtime per month
@ardalis | Finding Patterns in the Clouds
15. Data
Management
Many more options than in traditional-hosted
single-database apps
Distributed data
Consistency
Synchronization
@ardalis | Finding Patterns in the Clouds
16. Design and
Implementation
Consistency is key
Consider factors like
• Maintenance ease
• Administration
• Development
• Diagnostics
• Cost
@ardalis | Finding Patterns in the Clouds
17. Messaging
How do subsystems communicate?
Direct, synchronous calls?
Asynchronous messaging?
Each option presents challenges.
@ardalis | Finding Patterns in the Clouds
18. Management
and Monitoring
No direct server access to PaaS resources
means other tools are critical.
Cloud resources are more like cattle herds
than pets.
@ardalis | Finding Patterns in the Clouds
19. Performance
and Scalability
How responsive is the system to requests?
How does this responsiveness change with
increased load?
Scaling up
Scaling out
@ardalis | Finding Patterns in the Clouds
20. Resiliency
Can the system gracefully (and automatically)
recover from errors or failures?
Detect failures and replace resources
automatically
@ardalis | Finding Patterns in the Clouds
25. 🙁 Vertical Scaling is
Maxed Out
🙁 Performance is
suffering at times
Remedy:
Move Database to a
separate server
🙂 Performance
improves
Current Assessment:
@ardalis | Finding Patterns in the Clouds
26. 1 Web, 1 DB Server
Client Machine Web Server
Request
Response
App Data
DB Server
@ardalis | Finding Patterns in the Clouds
28. 🙁 Vertical Scaling is
Maxed Out (both
servers) – or at least
there’s no budget for
more right now
🙂 Data has been
optimized with indexes,
etc. No more gains to be
had here.
🙁 Some queries just
hammer the database,
take time, and impact
other queries.
Current Assessment:
@ardalis | Finding Patterns in the Clouds
31. Write-Through Strategy
Data
1. Update the data store 2. Invalidate (or update) its
cache entry
$123
@ardalis | Finding Patterns in the Clouds
Another option is to use a very short cache duration, something I refer to as micro-caching
32. Add Simple Memory Caching
Client Machine Web Server
Request
Response
App Data
DB Server
Cache
@ardalis | Finding Patterns in the Clouds
33. There are only 2 hard things in Computer Science
0. Cache Invalidation
1. Naming Things
2. Off-by-One Errors
@ardalis | Finding Patterns in the Clouds
34. Speaking of Naming Things…
• Use a standard way to generate a cache keys for given scenarios
• Avoid hard-coding keys, especially as local method literals
• Many caching patterns require access to keys from different parts of
your applications (including read vs. write operations)
@ardalis | Finding Patterns in the Clouds
35. 🙂 Performance is
usually (much) better
🙁 Some users still
complain of delays due
to cache misses
🙁 Keeping Cache
up to date is a new
challenge
🙁 Customers may
now see stale data
New Problems…
🙁 New monitoring
required for cache
🙁 New tools to clear
or update cache
required
Hmm, that’s a lot of 🙁
38. Simple Web Farm
Client Machine
Web Server
Request
Response
App
Data
DB Server
Cache
Load Balancer
Web Server
App
Cache
@ardalis | Finding Patterns in the Clouds
39. 🙂Performance and
Scalability improved
🙁 Some users still
complain due to
cache misses
😧 Keeping Multiple
Caches up to date is
a big challenge (in
this model)
New Behavior…
🙁New monitoring
and tools required
for load balancer
🙁 More servers to
manage
🙂 Web servers can
be updated without
taking down the
whole system
@ardalis | Finding Patterns in the Clouds
42. Simple Cloud Architecture
Client Machine
Request
Response
Load Balancer
App Service
Instances
Azure Cache /
Redis Instance
Azure SQL
Database
@ardalis | Finding Patterns in the Clouds
43. 🙂 Scalability
improved
🙁 Some users still
complain due to
cache misses
(consider priming
the cache)
🙂 Cache
synchronization easier
New Behavior…
🙂 No more servers
to manage
🙂 Monitoring tools
built-in to platform
🙂 Web instances
easily managed
without downtime
That ratio of 🙂 to 🙁 is a lot better…@ardalis | Finding Patterns in the Clouds
44. “Let’s build more of these apps”
@ardalis | Finding Patterns in the Clouds
46. More Apps
App Service
Instances
Azure Cache /
Redis Instance
Azure SQL
Database
App A
App Service
Instances
App B
App Service
Instances
App C
Shared Resources
47. Everything’s Great! Except…
Vendor lock-in
Shared database hurts
Shared resources (e.g. data schema) limit app developer agility
We’ll address these in a moment but first…
@ardalis | Finding Patterns in the Clouds
49. Authentication and Identity
These apps require:
• Single Sign-on
• Security – protection from unauthorized use
• This was surely built into the apps before this point, but the pain
becomes apparent now
@ardalis | Finding Patterns in the Clouds
50. Simple Database Managed Identity
App Service
Instances
Azure Cache /
Redis Instance
Azure SQL
Database
App A
App Service
Instances
App B
App Service
Instances
App C
Identity
Tables/Data
Username
Password
Login
Username
Password
Login
Username
Password
Login
YOU get a login screen, and YOU get a login screen
56. Moving from Apps to Microservices
• Apps are very coarse-grained to deploy, scale
• Common functionality duplicated between apps
• Stable parts of apps disrupted by deployment of unstable bits
• Decompose apps into small, independent, cohesive microservices
@ardalis | Finding Patterns in the Clouds
57. App A – An eCommerce Site
Redis
Cache
Azure
SQL
App A
Client apps
Mobile
app
Web
app
@ardalis | Finding Patterns in the Clouds
59. Security Concerns
• Client apps may not need every
microservice feature
• Microservices may have multiple
clients; shouldn’t need to know
security rules of every one
• Should limit feature surface area
specific to client needs
@ardalis | Finding Patterns in the Clouds
61. Using a custom API Gateway Service
Microservice 2
Microservice 1
Client WebApp MVC
container
container
Web API
Web API
ASP.NET Core MVC
container
Microservice 3
container
Web API
Client SPA Web app
JavaScript
Client mobile app
API Gateway
ASP.NET Core
Web API
container
Back end
Traditional Web app
Browser
HTML
HTML
JSON
JSON
62. API Gateway with Azure API Management
Architecture
Client WebApp MVC
ASP.NET Core MVC
container
Client SPA Web app
JavaScript
Client mobile app
Developer portal
API Gateway
Publisher portal
Azure API Management
Microservice 2
Microservice 1
container
container
Web API
Web API
Microservice 3
container
Web API
Back end
64. New Problem – Secure File Access
• Apps control access to media files based on authorized user
• Simple approach of a dumb CDN doesn’t protect actual media URLs
from being accessed by anyone
• Current solution: Web App authenticates user, accesses the file, and
streams it to the end user
@ardalis | Finding Patterns in the Clouds
65. Secure Media File Access
Client Machine
Request
Response
Web App
Request
Response
File/BLOB Store
(not publicly
accessible)
@ardalis | Finding Patterns in the Clouds
66. Concerns
🙁 Load on web app
higher than necessary
🙁 Cost! 💰 May be
paying extra to move
files in/out of web
app
🙁 Greater chance
of downtime with
web app and file
store both required
to stream file
@ardalis | Finding Patterns in the Clouds
67. Valet Key
• Provide direct access to media
files using an access token
• Azure supports Shared Access
Signatures (SAS) for this purpose
• File transfers occur directly
between file store and client
@ardalis | Finding Patterns in the Clouds
69. “I want fast, reliable, always-on services!”
@ardalis | Finding Patterns in the Clouds
70. Retry Pattern
First attempt failed
• Is it likely to be a transient problem? If not, give up.
• Immediately try again (maybe it was just a fluke)
• (wait)
• Try again
• (wait longer)
• Try again
• Give up.
@ardalis | Finding Patterns in the Clouds
71. Retries can overload downstream service
• Imagine typical load is 10 requests per second.
• With a typical “try 3 times then fail” strategy, when the service comes
up it’s immediately seeing 30 requests per second of load!
• Can result in longer time to respond and creation of more resources
than necessary (more hosting $$$)
• Don’t DOS (denial of service) yourself!
@ardalis | Finding Patterns in the Clouds
73. Circuit Breaker States
@ardalis | Finding Patterns in the Clouds
Closed
Open
When failCount > threshold, Open
Half-Open
After [timeout], go to Half-Open
Still not responding…
Reset failCount to zero and Close
Retry on failure; increment failCount
74. Consider a tool like Polly
@ardalis | Finding Patterns in the Clouds
75. Bonus Microservice Anti-Pattern:
Reach-In Reporting
Approach 1
• Microservices access reporting
data directly
• Introduces coupling
• Reduces microservice
independence
• Bypasses microservice logic
Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
76. Bonus Microservice Anti-Pattern:
Reach-In Reporting
Approach 2
• Reporting app hits
microservices directly
• Poor performance
• Data may be too large for
HTTP
• Difficult to perform complex
queries
Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
77. Bonus Microservice Anti-Pattern:
Reach-In Reporting
Approach 3
• Batch job updates reporting db
from microservice dbs
• Same coupling as approach 1.
Changes to microservice db
schemas break batch data job.
Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
78. Bonus Microservice Anti-Pattern:
Reach-In Reporting
Solution
• Async event publication
• Encapsulation and
independence of microservices
is preserved
• Performance is usually
acceptable
Source: Microservices AntiPatterns and Pitfalls by Mark Richards - https://oreil.ly/2J4r67x
79. Key Takeaways
• Cloud architecture abstracts away servers
• Cache Aside pattern is great for performance improvements
• Containers offer improved deployment and scaling options with less
vendor lock-in
• Microservices offer finer-grained control over app functionality
• Federated Identity improves security and user experience
• API Gateways help secure collections of services
• Valet key provides cheaper, faster access to secure media
• Consider Retry but you may need a Circuit-Breaker
• Avoid the Reach In Reporting anti-pattern for your microservices
@ardalis | Finding Patterns in the Clouds
80. More Cloud Design Patterns
https://bit.ly/1T8q2w8
@ardalis | Finding Patterns in the Clouds
81. Thank You!
• Contact me!
twitter.com/ardalis
steve@ardalis.com
• Podcast
WeeklyDevTips.com
• Group Mentoring Program
DevBetter.com
• Free Microsoft eBooks
ardalis.com/architecture-ebook
ardalis.com/cloud-native-book