Spring Boot and Kafka: The New Enterprise Platform

1. © Copyright 2019 Pivotal Software, Inc. All rights Reserved.
James Watters
April 2, 2019
Spring Boot+Kafka: the New
Enterprise Platform

2. Our Journey: Bring continuous delivery
with platform efficiency to enterprises.

3. New
Projects
Waterfall
ITIL
VMs
JavaEE
Batch
Products
Lean
CI/CD
Cloud Native Platforms
Spring Boot
Kafka Streaming Platform
Old

4. “For tech to be a real driver of innovation
and growth, IT needs to reorganize itself
around flexible platforms.” —McKinsey

5. Progress so far...

6. “We are happy to announce
that starting in 2018, Netflix is
also making the transition to
Spring Boot as our core Java
framework, leveraging the
community’s contributions via
Spring Cloud Netflix.”

7. Platform at
T-Mobile
SPEED
➔ Time to Prod: 6 Months to weeks
➔ # of deployments: 10x increase
SCALABILITY
➔ Avg: 300M Transactions a Day
➔ Peak: 1Billion Transactions a Day
STABILITY
➔ 43% reduction in app response time
➔ 83% fewer incidents, fixed 67% faster

8. IBM
IBM
Monolith
App
Cache
Win?
App

9. Full transformation required a
fundamental rethink….

10. “A business is a
series of events
and the reactions
to those events.”
—Jay Kreps, CEO, Confluent

11. “The only reason we don’t
think of events….is that so
far the technology has
trained us to think of data
as a static store.”
—Neha Narkhede

12. We restarted with
events first

13. What we’ve
learned
➔ Events are the language
bridge to the business
➔ This method of
identifying bounded
contexts is a secret to
decoupled architecture
➔ “Tell don’t ask”

14. Completing the Journey
IBM
App
Cache
App
Win? Microservices
Event-driven
Microservices
AppAppApp AppAppApp
AppAppApp

15. A platform built for a new way of thinking
➔ Event + Microservice first
➔ Team autonomy with
platform efficiency
➔ Arbitrary scaling/scope
across enterprise
➔ Turnkey multi-cloud

16. Streaming Data Platform

17. Immediate impacts

18. ➔ Global banking brand building greenfield
core banking and payments with Spring
Streams + Kafka
➔ Focus on microservices velocity required
platform first thinking
➔ Kleppman like view of Kafka: “We count on
Kafka for consistency, strict ordering,
replay, durability and auditability.”
➔ Cross cloud replication
A new continuously
delivered banking
platform

19. ➔ Turning moving packages into streaming
data with RFID, Kafka and Spring Streams
event based microservices
➔ Kafka, Kubernetes and Spring Boot in
every shipping center
➔ Multiple business microservices teams
can layer onto streaming platform to bin
pack last mile services.
➔ Prepared for unanticipated uses cases
Revolutionize our
shipping efficiency with
streaming microservices

20. PKS Managed Clusters
Messaging Middleware
Kafka
Binder Spring Data Repository
Event Driven Microservices
LTL Quote
Service
Scan RFIC
Services
RFID Triggered
Automation
Services

21. ➔ 100,00+ container build out of Spring
Streams, Kafka, key-value store
➔ Durability and consistency are critical
for potential legal actions
➔ Multi-phase stream processing with
Spring Streams leading to real-time
microservices alerting analysts
➔ Cross-cloud replication based on Kafka
➔ Continuously delivery required for real
time apps to improve accuracy and
functionality as project expands
Help secure a
European country?

22. Receiver
App
process queue
Fault tolerant
receiver pairs
staging
and
replication
Apps
Stream
Workers
Data
Enrichment
Stream
Workers
Data
Enrichment
Stream
Workers
Data
Enrichment
process queue
Stream
Workers
Data
Classification
Stream
Workers
Data
Classification
Stream
Workers
Data
Classification
buffer queue
S3 RAW Store
Receiver
App
X.000 Channel
Streams
RDBMS Store
3 DC
KAFKA
Replication
NoSQL
Store
Index
Store

23. ➔ Mainframe and monolithic RDBMS data
teams often the last to move to
continuous delivery
➔ CDC, Event Shunting, patterns
emerging allow streaming data platform
teams to offer mainframe and legacy
RDBMS events to microservices teams
➔ “AirBnB Pattern” growing across
enterprises
➔ Each team can build appropriate
persistence and achieve multi-DC
replication with streaming platform
Let’s empower
pharmacy microservices
developers while
evolving our legacy?

24. PKS Foundation 1
PAS Foundation 1
Event Driven
Microservice
(Producers)
Event Driven
Microservice
(Consumers)
Domain
Driven
Microservice
Legacy
ODS
ODS
Kafka Cluster 1
Broker
MirrorMaker
cluster 1: topics cluster 1: topics
PKS Foundation 2
PAS Foundation 2
Event Driven
Microservice
(Producers)
Event Driven
Microservice
(Consumers)
Domain
Driven
Microservice
ODS
Kafka Cluster 2
Broker
MirrorMaker
cluster 1: topics cluster 1: topics

25. Spring Kafka ++
We are all in…
doubling down on a platform

26. Spring Cloud Stream
deals with the Kafka
scaffolding, so you
don’t have to
The power of Kafka streams
for developers
➔ Rapid on-ramp for Kafka Streams
consumption
➔ Simplifies construction of
Event-Driven Stateful Microservices
➔ Focus on your processing logic not
on configuration
➔ Full support of all Kafka streams
functionality

27. We are all in…
doubling down on a platform
Spring Kafka ++ PCC Kafka Integration Pivotal Function Service
(Knative ++ )
PAS Service Brokers
(Custom today)
PKS Confluent Operator
Testing and Integration
AppTX + Kafka CDC ++

28. Call to action
➔ Tim Berglund and Josh Long talk on
Spring and Kafka
➔ Contribute to GitHub features and issues
for Spring Stream
➔ Come visit us at the booth to talk
streaming microservices

29. Deals with
Configuration &
Infrastructure
Scaffolding
Builds KStream
and/or KTable,
Binds Input
Topics
Assemble
Stream
Business
Logic
Binds
Stream &
Output
Topic
Spring Cloud Stream Developers
Developers 💚Spring Cloud & Spring Cloud 💚 Kafka
Deploy to
Production
on ANY cloud
Spring Cloud Data Flow

30. Spring Framework
Your Logic (in Spring Boot)
Kafka (KStreams, KTables, etc)
Spring for Apache Kafka
Spring Cloud Stream

31. Spring Cloud Stream Application
Kafka
Binder
Application Core
Inputs Outputs

32. Receiver
App
process queue
Fault tolerant
receiver pairs
staging
and
replication
Apps
Stream
Workers
Data
Enrichment
Stream
Workers
Data
Enrichment
Stream
Workers
Data
Enrichment
process queue
Stream
Workers
Data
Classification
Stream
Workers
Data
Classification
Stream
Workers
Data
Classification
buffer queue
S3 RAW Store
Receiver
App
X.000 Channel
Streams
Use Case:
European Security
Agency
Facts:
➔ Thousands of input
streams
➔ No data loss solution
➔ 72h 50TB input buffer
via Kafka
➔ 3 Kafka queue types
150TB per DC site
➔ Replication over 3
regions
➔ Non realtime clients
for case management
➔ Realtime clients,
enrichment and
analytics
RDBMS Store
3 DC
KAFKA
Replication
NoSQL
Store
Index
Store