SlideShare ist ein Scribd-Unternehmen logo
1 von 45
Fear No More:
Embrace Eventual
  Consistency
     Sean Cribbs
     @seancribbs
Watch the video with slide
                         synchronization on InfoQ.com!
                      http://www.infoq.com/presentations
                         /Embrace-Eventual-Consistency

       InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
  Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Presented at QCon San Francisco
                          www.qconsf.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
 - practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
Distributed Systems Experts
Fear No More: Embrace Eventual Consistency
FEAR



       Photo from Wikimedia
ACID vs. BASE
ACID vs. BASE
  • Strong consistency    • Weak consistency
  • Isolation             • Availability first
  • Focus on “commit”     • Best effort
  • Nested transactions   • Approximate answer
  • Conservative          • Aggressive
    (pessimistic)           (optimistic)

   Fox, Gribble, Chawathe, Brewer, Gauthier -
Cluster-Based Scalable Network Services (SOSP97)
ACID           vs.       BASE
  “Inconsistency is       “Being unavailable
the worst thing that       is the worst thing
   could happen.”        that could happen.”
Why BASE / EC?
Why BASE / EC?

• “Omniscience” is expensive and slow.
Why BASE / EC?

• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
Why BASE / EC?

• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
Why BASE / EC?

• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
 “Any sufficiently large system is in a
   constant state of partial failure.”
      Justin Sheehy, Basho CTO
Why BASE / EC?

• “Omniscience” is expensive and slow.
• Availability is often correlated to revenue.
• Failures happen all the time.
• You’re probably doing it already.
Safety &
Liveness
Leslie Lamport
1977
Safety
Safety
• “Bad things
 don’t happen”

• Point-in-time
 identifiable
Safety
• “Bad things      • mutual
 don’t happen”       exclusion

• Point-in-time    • partial
 identifiable        correctness

                   • first-come,
                     first-serve
Liveness
Liveness
• “Good things
 eventually
 happen”

• Always in
 future
Liveness
• “Good things     • starvation
 eventually         freedom
 happen”
                   • termination
• Always in
 future            • guaranteed
                    service
ACID vs. BASE
  • Strong consistency    • Weak consistency
  • Isolation             • Availability first
  • Focus on “commit”     • Best effort
  • Nested transactions   • Approximate answer
  • Conservative          • Aggressive
    (pessimistic)           (optimistic)

   Fox, Gribble, Chawathe, Brewer, Gauthier -
Cluster-Based Scalable Network Services (SOSP97)
Peter Bailis
 Eventual
 consistency is
 not safe
 “...it’s easy to satisfy liveness
 without being useful... If all
 replicas return the value 42 in
 response to every request, the
 system is eventually
 consistent.”

http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/
Liveness of BASE

• Convergence - “eventual delivery”
• Responsiveness - “eventual service”
• Resilience - “eventual recovery”
• Consensus-free - “eventual progress”
Safety of BASE


• Durability - “accepted writes are not lost”
• Integrity - “data is not corrupted”
• Authenticity - “data is not forged”
Real BASE Systems



            Photo from Wikimedia
Domain Name Service
• Federated, hierarchical database
  • How qconsf.com becomes 77.66.16.106
• Layered system with caching
Domain Name Service
• Federated, hierarchical database
  • How qconsf.com becomes 77.66.16.106
• Layered system with caching




                                 Diagrams from Wikimedia
Domain Name Service
• Federated, hierarchical database
  • How qconsf.com becomes 77.66.16.106
• Layered system with caching




                                 Diagrams from Wikimedia
DNS Liveness

• Convergence - caches eventually expire
• Consensus-free - local authority over subtree
  updates
• Responsiveness - intermediaries can cache
  results and reply quicker
• Resilience - authority servers can be replicated/
  load-balanced
DNS Safety



• Authenticity - forgery prevented by DNSSEC
BitTorrent
• Peer-to-peer
  cooperative large-file
  transfer
• Dynamic membership
  and block discovery
  through the “tracker”
  node
• Epidemic effect
                      http://computer.howstuffworks.com/bittorrent2.htm
BitTorrent Liveness

• Convergence - all peers that remain connected
  eventually become seeds
• Resilience - loss of one peer doesn’t impede
  progress
• Responsiveness - closer, faster peers tend to be
  preferred
BitTorrent Safety



• Integrity - each block is checksummed to
  prevent corruption
The Web

• Sparsely-connected graph of hypertext
  documents identified by URIs
• Rich caching semantics: expiration, validation,
  control
• Fluid evolution through uniform interface
• Layered system (federated)
Web: Liveness

• Consensus-free - local documents can be
  changed, moved, removed without coordination
• Convergence - caching semantics prevent
  unbounded staleness, redirection
• Responsiveness - many parties can proxy, cache
• Resilience - failure of one server doesn’t stop
  the system
Web: Safety


• Privacy & Authenticity - HTTPS/SSL/TLS
• Integrity - POST responses don’t pollute caches
Dynamo
                                    1
• Key-value store: distributed,
  replicated, partitioned
                                        2
• Client requests can go to any
  node
• Low-latency at high percentiles       3
• Many clones: Riak, Cassandra,
  Voldemort
Dynamo: Liveness

• Convergence - read-repair, hash-tree
  exchanges, vector-clocks
• Resilience - hinted-handoff, sloppy quorums
• Responsiveness - replication
• Consensus-free - loose coordination, concurrent
  updates
Dynamo: Safety


• Authenticity - won’t serve data you didn’t store
• Durability - confirmed writes are not lost
ACID vs. BASE
CONFLICT

           Photo by Associated Press
SPECTRUM

           Photo by Associated Press
hu g s!



Embrace
Eventual
Consistency

Weitere ähnliche Inhalte

Mehr von C4Media

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoC4Media
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileC4Media
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020C4Media
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsC4Media
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No KeeperC4Media
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like OwnersC4Media
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaC4Media
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideC4Media
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDC4Media
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine LearningC4Media
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at SpeedC4Media
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsC4Media
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsC4Media
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerC4Media
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleC4Media
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeC4Media
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereC4Media
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing ForC4Media
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data EngineeringC4Media
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreC4Media
 

Mehr von C4Media (20)

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy Mobile
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java Applications
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No Keeper
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like Owners
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate Guide
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CD
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine Learning
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at Speed
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep Systems
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.js
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly Compiler
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix Scale
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's Edge
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home Everywhere
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing For
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
 

Kürzlich hochgeladen

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintMahmoud Rabie
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 

Kürzlich hochgeladen (20)

Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Empowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership BlueprintEmpowering Africa's Next Generation: The AI Leadership Blueprint
Empowering Africa's Next Generation: The AI Leadership Blueprint
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 

Fear No More: Embrace Eventual Consistency

  • 1. Fear No More: Embrace Eventual Consistency Sean Cribbs @seancribbs
  • 2. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /Embrace-Eventual-Consistency InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month
  • 3. Presented at QCon San Francisco www.qconsf.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  • 6. FEAR Photo from Wikimedia
  • 8. ACID vs. BASE • Strong consistency • Weak consistency • Isolation • Availability first • Focus on “commit” • Best effort • Nested transactions • Approximate answer • Conservative • Aggressive (pessimistic) (optimistic) Fox, Gribble, Chawathe, Brewer, Gauthier - Cluster-Based Scalable Network Services (SOSP97)
  • 9. ACID vs. BASE “Inconsistency is “Being unavailable the worst thing that is the worst thing could happen.” that could happen.”
  • 10. Why BASE / EC?
  • 11. Why BASE / EC? • “Omniscience” is expensive and slow.
  • 12. Why BASE / EC? • “Omniscience” is expensive and slow. • Availability is often correlated to revenue.
  • 13. Why BASE / EC? • “Omniscience” is expensive and slow. • Availability is often correlated to revenue. • Failures happen all the time.
  • 14. Why BASE / EC? • “Omniscience” is expensive and slow. • Availability is often correlated to revenue. • Failures happen all the time. “Any sufficiently large system is in a constant state of partial failure.” Justin Sheehy, Basho CTO
  • 15. Why BASE / EC? • “Omniscience” is expensive and slow. • Availability is often correlated to revenue. • Failures happen all the time. • You’re probably doing it already.
  • 18. Safety • “Bad things don’t happen” • Point-in-time identifiable
  • 19. Safety • “Bad things • mutual don’t happen” exclusion • Point-in-time • partial identifiable correctness • first-come, first-serve
  • 21. Liveness • “Good things eventually happen” • Always in future
  • 22. Liveness • “Good things • starvation eventually freedom happen” • termination • Always in future • guaranteed service
  • 23. ACID vs. BASE • Strong consistency • Weak consistency • Isolation • Availability first • Focus on “commit” • Best effort • Nested transactions • Approximate answer • Conservative • Aggressive (pessimistic) (optimistic) Fox, Gribble, Chawathe, Brewer, Gauthier - Cluster-Based Scalable Network Services (SOSP97)
  • 24. Peter Bailis Eventual consistency is not safe “...it’s easy to satisfy liveness without being useful... If all replicas return the value 42 in response to every request, the system is eventually consistent.” http://www.bailis.org/blog/safety-and-liveness-eventual-consistency-is-not-safe/
  • 25. Liveness of BASE • Convergence - “eventual delivery” • Responsiveness - “eventual service” • Resilience - “eventual recovery” • Consensus-free - “eventual progress”
  • 26. Safety of BASE • Durability - “accepted writes are not lost” • Integrity - “data is not corrupted” • Authenticity - “data is not forged”
  • 27. Real BASE Systems Photo from Wikimedia
  • 28. Domain Name Service • Federated, hierarchical database • How qconsf.com becomes 77.66.16.106 • Layered system with caching
  • 29. Domain Name Service • Federated, hierarchical database • How qconsf.com becomes 77.66.16.106 • Layered system with caching Diagrams from Wikimedia
  • 30. Domain Name Service • Federated, hierarchical database • How qconsf.com becomes 77.66.16.106 • Layered system with caching Diagrams from Wikimedia
  • 31. DNS Liveness • Convergence - caches eventually expire • Consensus-free - local authority over subtree updates • Responsiveness - intermediaries can cache results and reply quicker • Resilience - authority servers can be replicated/ load-balanced
  • 32. DNS Safety • Authenticity - forgery prevented by DNSSEC
  • 33. BitTorrent • Peer-to-peer cooperative large-file transfer • Dynamic membership and block discovery through the “tracker” node • Epidemic effect http://computer.howstuffworks.com/bittorrent2.htm
  • 34. BitTorrent Liveness • Convergence - all peers that remain connected eventually become seeds • Resilience - loss of one peer doesn’t impede progress • Responsiveness - closer, faster peers tend to be preferred
  • 35. BitTorrent Safety • Integrity - each block is checksummed to prevent corruption
  • 36. The Web • Sparsely-connected graph of hypertext documents identified by URIs • Rich caching semantics: expiration, validation, control • Fluid evolution through uniform interface • Layered system (federated)
  • 37. Web: Liveness • Consensus-free - local documents can be changed, moved, removed without coordination • Convergence - caching semantics prevent unbounded staleness, redirection • Responsiveness - many parties can proxy, cache • Resilience - failure of one server doesn’t stop the system
  • 38. Web: Safety • Privacy & Authenticity - HTTPS/SSL/TLS • Integrity - POST responses don’t pollute caches
  • 39. Dynamo 1 • Key-value store: distributed, replicated, partitioned 2 • Client requests can go to any node • Low-latency at high percentiles 3 • Many clones: Riak, Cassandra, Voldemort
  • 40. Dynamo: Liveness • Convergence - read-repair, hash-tree exchanges, vector-clocks • Resilience - hinted-handoff, sloppy quorums • Responsiveness - replication • Consensus-free - loose coordination, concurrent updates
  • 41. Dynamo: Safety • Authenticity - won’t serve data you didn’t store • Durability - confirmed writes are not lost
  • 43. CONFLICT Photo by Associated Press
  • 44. SPECTRUM Photo by Associated Press