SpringOne Platform 2016
Speakers: Patricia Anderson; Senior Consultant, Credera. Micah Blalock; Senior Architect, Credera. Jason Goth; Principal Architect, Credera.
A microservice architecture is pattern that is most commonly associated with larger organizations where services and teams are organized around separate business capabilities. In a project our team recently completed, we used a microservice architecture to allow us to organize a large team to develop a large analytics platform at speeds that would not have been possible using a more typical service-oriented architecture.
In this session, we discuss the organizational structure and communication and development strategies and tools to allow teams to work in parallel without drowning in process overhead and coordination costs.
Microservices: Organizing Large Teams for Rapid Delivery
1. Microservices: Organizing a Large
Team for Rapid Delivery
By Jason Goth, Micah Blalock, Patricia Anderson
@jgothtx, @micah_blalock, @patricia_sooner
5. Objectives
• Share lessons learned on a few real world microservices projects:
• Several customer projects
• One internal project
• Topics:
• Organizational structure for rapid delivery
• Design/Development practices to support these organizations
• Communication across multiple teams
5
6. Case Study … Custom Analytics Platform
6
We are here
in the story
2014 2015 2016
AugOct
7. The Problem
• Suite of applications:
• Analytics for hospitals
• Multiple products/users
• Products share core services
(e.g. how much does X cost?
how many of these did I use?)
• Multiple product owners
• Billions of rows of legacy data
• Huge roadmap of features and
functionality (that they’d already
promised…)
7
Lots of Legacy Data
Product
Owner
Product
Owner
Customers
Customers
App
App
App
Shared
Stuff Customers
System
8. They Promised This
8
We are still
here in the
story
2014 2015 2016
AugOct
Product 1 Product 2 Product 3 Product 4 And So On
9. We Decided To Do Something Like This
• Reuse of services ?
• Add applications easily ?
• Scale out forever ?
• Cool factor?
• Multiple teams can deliver products
and features super fast ? Umm, we’ll
come back to that...
9
Clients
Gateway
App 1
Svc 2Svc 1
App 2 App 3 App 4
Svc 3 Svc 4
Service Registry
Configuration Server
Oracle Redis Elastic SearchOracle
Svc N-2Svc N-3 Svc N-1 Svc N
. . .
10. Let’s Use Spring!
10
Clients
Gateway
App 1
Svc 2Svc 1
App 2 App 3 App 4
Svc 3 Svc 4
Oracle Redis Elastic SearchOracle
Svc N-2Svc N-3 Svc N-1 Svc N
. . . Spring
Boot
Spring
Framework
Spring
Security
Spring
Data
Spring
Batch
Spring
AMQP
Spring Cloud
Service Registry
Configuration Server
11. It Worked Great!
• Built the first application in the suite
and the first few services in a few
months
• Spring-* provided a lot of the plumbing
• We had one team (8-10 people) for the
application and the services
• Big success
11
Clients
Gateway
App 1
Svc 2Svc 1
App 2 App 3 App 4
Svc 3 Svc 4
Service Registry
Configuration Server
Oracle Redis Elastic SearchOracle
Svc N-2Svc N-3 Svc N-1 Svc N
. . .Dev
Team
12. This Is Easy … Let’s Do Another One
12
Phase 2 Starts
Start
Big
Success
2014 2015 2016
AugOct AprMar
13. It Worked Ok …
• Second team (8-10 people) brought on
board to build the next application in
the suite
• Did it all in 3 months this time
• Another big(-ish) success
• Teams still did their own thing
• More friction, but manageable
13
Clients
Gateway
App 1
Svc 2Svc 1
App 2 App 3 App 4
Svc 3 Svc 4
Service Registry
Configuration Server
Oracle Redis Elastic SearchOracle
Svc N-2Svc N-3 Svc N-1 Svc N
. . .
14. Let’s Do 2 More Apps … in Only 3 Months
14
“Phase 3”
Starts
Start
Big
Success
2014 2015 2016
AugOct Mar Jun July
Big(-ish)
Success
15. We Just Need To Add Some People, Right?
• Continue trying to scale out as before:
• Add new teams for new features
and services
• Letting teams do their own thing
15
Clients
Gateway
App 1
Svc 2Svc 1
App 2 App 3 App 4
Svc 3 Svc 4
Service Registry
Configuration Server
Oracle Redis Elastic SearchOracle
Svc N-2Svc N-3 Svc N-1 Svc N
. . .
16. Two Months In …
16
Start
Big
Success
2014 2015 2016
AugOct Mar Jun
Big(-ish)
Success
We are here
Sep
19. It Really Looked Like This
• Dependency hell:
• Each feature changed many
services
• Changes cascaded everywhere
• Teams were stepping on each
other
• Versioning helped; but versioning is its
own special hell
19
Clients
Gateway
App 1
Svc 1
App 2 App 3 App 4
Service Registry
Configuration Server
Oracle Redis Elastic SearchOracle
Svc 2 Svc 3 Svc 4
Svc N-2Svc N-3 Svc N-1 Svc N
. . .
20. Team Health Suffering
• Unequal workloads
• Training issues:
• Unfamiliar systems
• Not always the right skillset
• We had sooooo many meetings:
• Many to coordinate with each
other
• Up to 50% of people’s time
20
21. Some Things Outside Our Control
• “Legacy” processes added friction:
• “Architecture Review Board”
• “Change Control Board”
• Shared environments
• QA, Load Testing, etc.
• Customer doesn’t use any cloud
services L
21
23. It can be faster, if you align the solution
and the organization
23
Main Point
of This Talk
24. Alignment
• Conway’s Law is the starting point
• When each team “owns” their own services
• Easy to change
• Coordination costs low
• Sharing exposes stress points, coordination
costs increase:
• Adding features to existing services
• Cascading changes to service contracts
• Coordination costs + partitioning of work =
efficiency of change.
24
SvcSvc Svc
Svc
Svc
Svc
Stress
point
25. Partitioning … Time vs Workers
25
Partitioned
Tasks
Non-partitioned
Tasks
Partitioned Tasks
High Coordination
Time
Workers Workers Workers
Time
Time
Source: The Mythical Man-Month, Frederick P. Brooks, Jr., 1995
26. Better Partitioning
• Refactor to create surface area
• Similar to the open/closed
principle
• Package code by features not layers
• Add packages for new features
• Grow code horizontally not
vertically
26
// This is your clue to refactor
if( purchaseType.equals(“Lab”)) {
savings = getLabSavings()
} else if (purchaseType.equals(“Rx”)) {
savings = getRxSavings()
} else ...
27. Handling Cascading Changes
• Remove semantic coupling
• Shared concepts,data types or domain
entities
• Behavioral dependencies
• The more effects and side-effects a service
has, the tighter the coupling with its
consumers
See: http://www.michaelnygard.com/blog/2015/04/the-perils-
of-semantic-coupling/
27
Oracle Redis Elastic SearchOracle
Gateway
Svc 2
App 2 App 3 App 4
Svc 3 Svc 4
Service Registry
Configuration Server
Svc N-2Svc N-3 Svc N-1 Svc N
. . .
Svc 1
App 1
29. Eliminate Shared Dependencies
• Warning architecture purists … you will not like this…
• Shared dependencies increase coordination costs:
• Some redundancies are ok
• GARY (Go Ahead, Repeat Yourself)
29
30. Example … GARY
30
// Does every service need to include a shared component (JAR) to validate
plan types?
import java.utils.Arrays;
private static String [] PLAN_TYPES= {“HMO”, “PPO”};
public static boolean isValidPlanType(String planType) {
return Arrays.stream(PLAN_TYPES).anyMatch(planType::equals)
}
What if different services accept different types of plans? Are all services using
Java 8?
31. Example … GARY
31
// Service 1
if(planType.equals(“HMO”) ||
planType.equals(“PPO”) {
return true;
}
// Service 2
if(planType.equals(“HMO”) {
return true;
}
• Why not this?
Great discussion here: http://blog.cognitect.com/blog/2016/6/16/the-new-normal-team-scale-autonomy
34. Reorganizing Teams
• Refactored design allows us to
organize around customer needs
(applications/features):
• Single Product Owner
• Few overlapping changes
• Much lower coupling
• Split into smaller teams
(~4 people)
• Focused on the “full stack”…
everyone you need
34
Clients
Gateway
App 1
Svc 2Svc 1
App 2 App 3 App 4
Svc 3 Svc 4
Service Registry
Configuration Server
Oracle Redis Elastic SearchOracle
Svc N-2Svc N-3 Svc N-1 Svc N
. . .
35. Addressing Our Training Problems
• Put some standards in place:
• Just enough of them
• Familiarity when switching between components
• No onerous review/governance process:
o That’s an organizational dependency
o Trust people to follow standards
• Train team on all components and standards:
• Sample code/generators/snippets
• “Real” documentation, not shelf-ware
35
36. Addressing Our Workload Problems
• Warning Scrum Masters ... you will
not like this…
• We shift our resources all the time
• Our standards and consistency
make this easy to do
• Do you want to predict velocity
or do you want to have
velocity?
• Over time it averages out
36
Clients
Gateway
App 1
Svc 2Svc 1
App 2 App 3 App 4
Svc 3 Svc 4
Service Registry
Configuration Server
Oracle Redis Elastic SearchOracle
Svc N-2Svc N-3 Svc N-1 Svc N
37. Addressing our Meeting Problems
• Elect one person per team as “Team Lead”:
• Could be anyone
• Coordinate any cross-team questions
• Facilitate inter-team questions
• External Team Communication:
• Designate someone as the sacrificial
offering to the Legacy Process
Overlords
• You may need more than one person
• Let everyone else work!
37
Legacy Process
Overlord
38. So Where Are We In The Story?
38
Start
Big
Success
2014 2015 2016
AugOct Mar Jun
Big
Success
Sep Feb
Things
Stopped
We are hereLots of refactoring,
documentation,
reorg, etc.
39. We Started Gaining Momentum
• Teams working (mostly) independently
• Meeting frequency greatly reduced
• Not as much cross-team coordination
• Legacy Process Overlords appeased
39
40. And Now We Are Here
40
Start
Big
Success
2014 2015 2016
AugOct Mar Jun
Big(-ish)
Success
Sep Feb
Things
Stopped
Back on
Track
Tons Accomplished
Huge
Success
Jun
Lots of refactoring,
documentation,
reorg, etc.
4 new applications in the suite in just over 4 months
80+ enhancements to existing applications
41. Have a Need for Speed?
41
So, have a need for speed?
42. Summary
• Align solution and the organization
• Improve partitioning
• Reduce coupling and dependencies
• GARY
• Create standalone teams with singular focus
• Have “just enough” standards/process
• Be flexible; its ok to move people around
• Have people responsible for communications
• Others? We’d love to hear from you
42