Distributed applications using Hazelcast
•
37 gefällt mir•7,273 views
Do you need to scale your application, share data across cluster, perform massive parallel processing on many JVMs or maybe consider alternative to your favorite NoSQL technology? Hazelcast to the rescue! With Hazelcast distributed development is much easier. This presentation will be useful to those who would like to get acquainted with Hazelcast top features and see some of them in action, e.g. how to cluster application, cache data in it, partition in-memory data, distribute workload onto many servers, take advantage of parallel processing, etc. Presented on JavaDay Kyiv 2014 conference.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (16)
Ähnlich wie Distributed applications using Hazelcast
Ähnlich wie Distributed applications using Hazelcast (20)
Mehr von Taras Matyashovsky
Mehr von Taras Matyashovsky (10)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Distributed applications using Hazelcast
- 1. Distributed applications using Hazelcast This presentation will be useful to those who would like to get acquainted with Hazelcast top features and see some of them in action ! By Taras Matyashovsky Taras Matyashovsky, Software Engineer/Team & Tech Lead, CSM, @tmatyashovsky
- 3. • Software engineer/TL • Worked for outsource companies, product companies and tried myself in startups/ freelancing • 8+ years production Java experience • Fan of Agile methodologies, CSM • Founder of Morning@Lohika tech talks in Lviv Introduction About me
- 4. • This presentation Introduction What?
- 5. • This presentation • covers Hazelcast top features Introduction What?
- 6. • This presentation • covers Hazelcast top features • explains difference between distributed cache and in-memory Introduction data grid What?
- 7. • This presentation What? • covers Hazelcast top features • explains difference between distributed cache and in-memory Introduction data grid • contains live demo sessions
- 8. • This presentation What? • covers Hazelcast top features • explains difference between distributed cache and in-memory Introduction data grid • contains live demo sessions • is not intended for promotion of Hazelcast as the best solution
- 9. • This presentation What? • covers Hazelcast top features • explains difference between distributed cache and in-memory Introduction data grid • contains live demo sessions • is not intended for promotion of Hazelcast as the best solution • draws attention to Hazelcast as it is currently on a rise
- 10. • This presentation What? • covers Hazelcast top features • explains difference between distributed cache and in-memory Introduction data grid • contains live demo sessions • is not intended for promotion of Hazelcast as the best solution • draws attention to Hazelcast as it is currently on a rise • does not describe usage of NoSQL solutions for distributed caching, e.g. Redis, Riak, Cassandra, MongoDB, etc.
- 11. • Hazelcast in a nutshell • Cache fundamentals • Hazelcast as distributed cache • live demo session • Distributed cache vs. in-memory data grid • Hazelcast as in-memory data grid • live demo session • Hazelcast drawbacks • Hazelcast is on a rise Introduction Agenda
- 12. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 13. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 14. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 15. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 16. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 17. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 18. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 19. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 20. In-memory Data Grid Solutions Hazelcast in a Nutshell
- 21. Hazelcast in a Nutshell Hazelcast The leading open source in-memory data grid ! free alternative to proprietary solutions, such as Oracle Coherence, VMWare Pivotal Gemfire and Software AG Terracotta
- 22. Hazelcast in a Nutshell Hazelcast Use Cases
- 23. • scale your application Hazelcast in a Nutshell Hazelcast Use Cases
- 24. • scale your application • share data across cluster Hazelcast in a Nutshell Hazelcast Use Cases
- 25. • scale your application • share data across cluster • partition your data Hazelcast in a Nutshell Hazelcast Use Cases
- 26. • scale your application • share data across cluster • partition your data • balance the load Hazelcast in a Nutshell Hazelcast Use Cases
- 27. • scale your application • share data across cluster • partition your data • balance the load • send/receive messages Hazelcast in a Nutshell Hazelcast Use Cases
- 28. • scale your application • share data across cluster • partition your data • balance the load • send/receive messages • process in parallel on many JVMs, i.e. massively parallel processing (MPP) Hazelcast in a Nutshell Hazelcast Use Cases
- 29. Hazelcast in a Nutshell Hazelcast Features
- 30. • dynamic clustering, backup, discovery, fail-over Hazelcast in a Nutshell Hazelcast Features
- 31. • dynamic clustering, backup, discovery, fail-over • distributed & partitioned map, distributed queue, set, list, lock, semaphore, topic, executor service, etc. Hazelcast in a Nutshell Hazelcast Features
- 32. • dynamic clustering, backup, discovery, fail-over • distributed & partitioned map, distributed queue, set, list, lock, semaphore, topic, executor service, etc. • distributed queries and MapReduce API Hazelcast in a Nutshell Hazelcast Features
- 33. • dynamic clustering, backup, discovery, fail-over • distributed & partitioned map, distributed queue, set, list, lock, semaphore, topic, executor service, etc. • distributed queries and MapReduce API • transaction support, both local and distributed Hazelcast in a Nutshell Hazelcast Features
- 34. • dynamic clustering, backup, discovery, fail-over • distributed & partitioned map, distributed queue, set, list, lock, semaphore, topic, executor service, etc. • distributed queries and MapReduce API • transaction support, both local and distributed • Java client for accessing the cluster remotely Hazelcast in a Nutshell Hazelcast Features
- 35. Cache Fundamentals Cache Fundamentals
- 36. Cache Fundamentals Cache Attributes
- 37. • maximum size, e.g. quantity of entries Cache Fundamentals Cache Attributes
- 38. • maximum size, e.g. quantity of entries • cache algorithm used for invalidation/eviction, e.g.: • least recently used (LRU) • least frequently used (LFU) • FIFO Cache Fundamentals Cache Attributes
- 39. • maximum size, e.g. quantity of entries • cache algorithm used for invalidation/eviction, e.g.: • least recently used (LRU) • least frequently used (LFU) • FIFO • eviction percentage Cache Fundamentals Cache Attributes
- 40. • maximum size, e.g. quantity of entries • cache algorithm used for invalidation/eviction, e.g.: • least recently used (LRU) • least frequently used (LFU) • FIFO • eviction percentage • expiration, e.g.: • time-to-live (TTL) • absolute/relative time-based expiration Cache Fundamentals Cache Attributes
- 41. Cache Fundamentals Cache Access Patterns
- 42. Cache Access Patterns • cache aside • application is responsible for reading and writing from the storage and the cache doesn't interact with the storage at all • the cache is “kept aside” as a faster and more scalable in-memory Cache Fundamentals data store
- 43. Cache Access Patterns • cache aside • application is responsible for reading and writing from the storage and the cache doesn't interact with the storage at all • the cache is “kept aside” as a faster and more scalable in-memory Cache Fundamentals data store • read-through/write-through (refresh-ahead/write-behind) • the application treats cache as the main data store and reads/writes data from/to it • the cache is responsible for reading and writing this data to the database
- 44. Cache Fundamentals Cache Types
- 45. • local cache Cache Fundamentals Cache Types
- 46. • local cache • replicated cache Cache Fundamentals Cache Types
- 47. • local cache • replicated cache • distributed cache Cache Fundamentals Cache Types
- 48. • local cache • replicated cache • distributed cache • remote cache Cache Fundamentals Cache Types
- 49. • local cache • replicated cache • distributed cache • remote cache • near cache Cache Fundamentals Cache Types
- 50. Distributed Cache a cache that partitions its data Cache Fundamentals among all cluster nodes
- 51. Get in Distributed Cache Cache Fundamentals
- 52. Get in Distributed Cache Cache Fundamentals JVM 1
- 53. Get in Distributed Cache Cache Fundamentals Distributed Cache Map JVM 1
- 54. Get in Distributed Cache Cache Fundamentals B A C D Logical Distributed Cache Map JVM 1
- 55. Get in Distributed Cache Cache Fundamentals B A C D Logical A Distributed Cache Map JVM 1 Primary
- 56. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map JVM 1 Primary Backup
- 57. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup
- 58. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup JVM 2
- 59. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup Distributed Cache Map JVM 2
- 60. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical Distributed Cache Map JVM 2
- 61. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical B Primary Distributed Cache Map JVM 2
- 62. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map JVM 2
- 63. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 64. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup JVM 3 A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 65. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup Distributed Cache Map JVM 3 A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 66. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical Distributed Cache Map JVM 3 A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 67. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C Distributed Cache Map JVM 3 Primary A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 68. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 69. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 70. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 JVM 4
- 71. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 Distributed Cache Map JVM 4
- 72. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical Distributed Cache Map JVM 4
- 73. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary Distributed Cache Map JVM 4
- 74. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map JVM 4
- 75. Get in Distributed Cache Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4
- 76. Get in Distributed Cache A Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4
- 77. Get in Distributed Cache A Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A)
- 78. Get in Distributed Cache A Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A)
- 79. Get in Distributed Cache A B Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A)
- 80. Get in Distributed Cache A B Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A) get(B)
- 81. Get in Distributed Cache A B Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A) get(B)
- 82. Get in Distributed Cache A C B Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A) get(B)
- 83. Get in Distributed Cache A C B Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A) get(B) get(C)
- 84. Get in Distributed Cache A C B Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A) get(B) get(C)
- 85. Get in Distributed Cache A C B D Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A) get(B) get(C)
- 86. Get in Distributed Cache A C B D Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A) get(B) get(C) get(D)
- 87. Get in Distributed Cache A C B D Cache Fundamentals B A C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 get(A) get(B) get(C) get(D)
- 88. Put in Distributed Cache Cache Fundamentals
- 89. Put in Distributed Cache Cache Fundamentals C B D Logical C Distributed Cache Map Application JVM 1 Primary Backup
- 90. Put in Distributed Cache Cache Fundamentals C B D Logical C Distributed Cache Map Application JVM 1 Primary Backup C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 91. Put in Distributed Cache Cache Fundamentals C B D Logical C Distributed Cache Map Application JVM 1 Primary Backup C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2
- 92. Put in Distributed Cache Cache Fundamentals C B D Logical C Distributed Cache Map Application JVM 1 Primary Backup C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 C B D Logical D Primary Backup Distributed Cache Map Application JVM 4
- 93. Put in Distributed Cache A Cache Fundamentals C B D Logical C Distributed Cache Map Application JVM 1 Primary Backup C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 C B D Logical D Primary Backup Distributed Cache Map Application JVM 4
- 94. Put in Distributed Cache A put( A) A Cache Fundamentals C B D Logical C Distributed Cache Map Application JVM 1 Primary Backup C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 C B D Logical D Primary Backup Distributed Cache Map Application JVM 4 A
- 95. Put in Distributed Cache A put( A) A Cache Fundamentals C B D Logical C Distributed Cache Map Application JVM 1 Primary Backup C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 A
- 96. Put in Distributed Cache A put( A) A Cache Fundamentals C B D Logical C Distributed Cache Map Application JVM 1 Primary Backup A A A B C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 C D Logical D Primary A Backup Distributed Cache Map Application JVM 4 A
- 97. Failover in Distributed Cache Cache Fundamentals A B C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4
- 98. Failover in Distributed Cache Cache Fundamentals A B C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4
- 99. Failover in Distributed Cache Cache Fundamentals A B C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4
- 100. Failover in Distributed Cache Cache Fundamentals A B C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 B
- 101. Failover in Distributed Cache Cache Fundamentals A B C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary A Backup Distributed Cache Map Application JVM 4 B
- 102. Failover in Distributed Cache Cache Fundamentals A B C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B D B A Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical B Primary D Backup Distributed Cache Map Application JVM 2 A C B D Logical D Primary Backup Distributed Cache Map Application JVM 4
- 103. Failover in Distributed Cache Cache Fundamentals A B C D Logical A C Distributed Cache Map Application JVM 1 Primary Backup A C B D Logical C B D B A Distributed Cache Map Application JVM 3 Primary Backup A C B D Logical D Primary Backup Distributed Cache Map Application JVM 4
- 104. Cache Fundamentals Distributed Cache
- 105. • Pros: • linear performance scalability for reads and writes • fault-tolerant Cache Fundamentals Distributed Cache
- 106. • Pros: • linear performance scalability for reads and writes • fault-tolerant • Cons: • increased latency of reads (due to network round-trip and serialization/deserialization expenses) Cache Fundamentals Distributed Cache
- 107. Distributed Cache Summary Most of distributed caches were built as distributed in-memory key/value stores that supported a simple set of “put” and “get” operations and optionally read-through/write-through behavior Cache Fundamentals for writing and reading values to/from underlying disk-based storage such as an RDBMS
- 108. Hazelcast as Distributed Cache Hazelcast as Distributed Cache
- 109. Hazelcast as Distributed Cache Hazelcast Distributed & Partitioned Map
- 110. • extends the java.util.concurrent.ConcurrentMap, but is designed to be used in a distributed environment Hazelcast as Distributed Cache Hazelcast Distributed & Partitioned Map
- 111. • extends the java.util.concurrent.ConcurrentMap, but is designed to be used in a distributed environment • divided by default in 271 partitions Hazelcast as Distributed Cache Hazelcast Distributed & Partitioned Map
- 112. • extends the java.util.concurrent.ConcurrentMap, but is designed to be used in a distributed environment • divided by default in 271 partitions • partitions are distributed evenly among the members in the cluster Hazelcast as Distributed Cache Hazelcast Distributed & Partitioned Map
- 113. • extends the java.util.concurrent.ConcurrentMap, but is designed to be used in a distributed environment • divided by default in 271 partitions • partitions are distributed evenly among the members in the cluster • partition id = hash(key) % partition count Hazelcast as Distributed Cache Hazelcast Distributed & Partitioned Map
- 114. • extends the java.util.concurrent.ConcurrentMap, but is designed to be used in a distributed environment • divided by default in 271 partitions • partitions are distributed evenly among the members in the cluster • partition id = hash(key) % partition count • scaling out is automatic via addition of new Hazelcast instances Hazelcast as Distributed Cache Hazelcast Distributed & Partitioned Map
- 115. Sample Web Application* • Technologies: • Spring Boot 1.1.7.RELEASE • Hazelcast 3.3.1 • 1 jar dependency • Apache 2 license, i.e. free to use • Postgres 9.3 ! • Application: • HTTP-based API to get/put data from/to cache • HTTP-based API to execute tasks and search data in the cluster • One instance of Hazelcast per application • programmatic configuration • cluster discovery based on TCP/IP cluster network configuraton * Some samples are not optimal and created just to demonstrate usage of existing Hazelcast API Hazelcast as Distributed Cache
- 116. Distributed Cache • Hazelcast configuration • In-memory distributed cache • Write-through distributed cache • Management center Hazelcast as Distributed Cache
- 117. In-Memory Data Grid In-Memory Data Grid
- 118. In-Memory Data Grid In-Memory Data Grid
- 119. In-memory distributed cache plus: In-Memory Data Grid In-Memory Data Grid
- 120. In-memory distributed cache plus: • ability to support co-location of computations with data in a distributed context and move computation to data In-Memory Data Grid In-Memory Data Grid
- 121. In-Memory Data Grid In-memory distributed cache plus: • ability to support co-location of computations with data in a distributed context and move computation to data • distributed massively parallel processing (MPP) based on standard SQL and/or MapReduce, that allows to effectively compute over data stored in-memory In-Memory Data Grid across the cluster
- 122. In-Memory Data Grid In-Memory Data Grid
- 123. In-Memory Data Grid In-Memory Data Grid Distributed Caching
- 124. In-Memory Data Grid In-Memory Data Grid In-Memory Data Grid Distributed Caching
- 125. In-Memory Data Grid In-Memory Data Grid In-Memory Computing/ Database In-Memory Data Grid Distributed Caching
- 126. In-Memory Distributed Cache vs. In-Memory Data Grid In-Memory Data Grid Distributed Cache Data Grid
- 127. In-Memory Distributed Cache vs. In-Memory Data Grid In-Memory Data Grid Distributed Cache In-memory distributed caches were developed in response to a growing need for data high availability Data Grid
- 128. In-Memory Distributed Cache vs. In-Memory Data Grid In-Memory Data Grid Distributed Cache In-memory distributed caches were developed in response to a growing need for data high availability Data Grid In-memory data grids were developed to respond to the growing complexities of data processing
- 129. In-Memory Data Grid Summary In-Memory Data Grid Adding distributed SQL and/or MapReduce based processing required a complete re-thinking of distributed caches, as focus has shifted from pure data management to hybrid data and compute management
- 130. Hazelcast Hazelcast as In-Memory Data Grid as In-Memory Data Grid
- 131. Hazelcast as In-Memory Data Grid Hazelcast as In-Memory Data Grid
- 132. Hazelcast as In-Memory Data Grid • Executor service Hazelcast as In-Memory Data Grid
- 133. Hazelcast as In-Memory Data Grid • Executor service • Entry processor Hazelcast as In-Memory Data Grid
- 134. Hazelcast as In-Memory Data Grid • Executor service • Entry processor • Distributed queries • Criteria API • Distributed SQL Query Hazelcast as In-Memory Data Grid
- 135. Hazelcast as In-Memory Data Grid • Executor service • Entry processor • Distributed queries • Criteria API • Distributed SQL Query • MapReduce API Hazelcast as In-Memory Data Grid
- 136. Hazelcast Executor Service Hazelcast as In-Memory Data Grid
- 137. Hazelcast Executor Service • extends the java.util.concurrent.ExecutorService, but is designed to be used in a distributed environment Hazelcast as In-Memory Data Grid
- 138. Hazelcast Executor Service • extends the java.util.concurrent.ExecutorService, but is designed to be used in a distributed environment • executes tasks implemented as java.util.Runnable or java.util.concurrent.Callable asynchronously Hazelcast as In-Memory Data Grid
- 139. Hazelcast Executor Service • extends the java.util.concurrent.ExecutorService, but is designed to be used in a distributed environment • executes tasks implemented as java.util.Runnable or java.util.concurrent.Callable asynchronously • scaling up via threads pool size Hazelcast as In-Memory Data Grid
- 140. Hazelcast Executor Service • extends the java.util.concurrent.ExecutorService, but is designed to be used in a distributed environment • executes tasks implemented as java.util.Runnable or java.util.concurrent.Callable asynchronously • scaling up via threads pool size • scaling out is automatic via addition of new Hazelcast instances Hazelcast as In-Memory Data Grid
- 141. Hazelcast Executor Service • provides different ways to route tasks • any member • specific member • the member hosting a specific key • all or subset of members • supports future and execution callback Hazelcast as In-Memory Data Grid
- 142. In-Memory Data Grid • Executor service configuration • Simple task • Sum task to sum values in the map • Distributed query Hazelcast as In-Memory Data Grid
- 143. Hazelcast Executor Service Drawbacks Hazelcast as In-Memory Data Grid
- 144. Hazelcast Executor Service Drawbacks • work-queue has no high availability • each member will create local ThreadPoolExecutors with ordinary work-queues that do the real work but not backed up by Hazelcast • if something would happen with that member, all unprocessed work will be lost Hazelcast as In-Memory Data Grid
- 145. Hazelcast Executor Service Drawbacks • work-queue has no high availability • each member will create local ThreadPoolExecutors with ordinary work-queues that do the real work but not backed up by Hazelcast • if something would happen with that member, all unprocessed work will be lost • work-queue is not partitioned • it could be that one member has a lot of unprocessed work, and another is idle Hazelcast as In-Memory Data Grid
- 146. Hazelcast Executor Service Drawbacks • work-queue has no high availability • each member will create local ThreadPoolExecutors with ordinary work-queues that do the real work but not backed up by Hazelcast • if something would happen with that member, all unprocessed work will be lost • work-queue is not partitioned • it could be that one member has a lot of unprocessed work, and another is idle • no customizable load balancing • in future load balancing could be done based on the number of unprocessed tasks, CPU load, memory load etc. Hazelcast as In-Memory Data Grid
- 147. Other Hazelcast Useful Features Other Hazelcast Useful Features
- 148. Other Hazelcast Useful Features • custom (de)serialization mechanism including support for: • Kryo • Jackson Smile • Protobuf • entry processor • locking • distributed topic • SPI (service programming interface) to create own distributed services/structures Other Hazelcast Useful Features
- 149. Hazelcast Drawbacks Hazelcast Drawbacks
- 150. Hazelcast Drawbacks Hazelcast Drawbacks
- 151. • still quite buggy • memory leaks Hazelcast Drawbacks Hazelcast Drawbacks
- 152. • still quite buggy • memory leaks • poor documentation for more complex cases Hazelcast Drawbacks Hazelcast Drawbacks
- 153. • still quite buggy • memory leaks • poor documentation for more complex cases • missing useful features: • update configuration in running cluster • load balancing for executor service • only map is really distributed & partitioned Hazelcast Drawbacks Hazelcast Drawbacks
- 154. • still quite buggy • memory leaks • poor documentation for more complex cases • missing useful features: • update configuration in running cluster • load balancing for executor service • only map is really distributed & partitioned • enterprise edition costs money (about $7k per node), but includes: • elastic memory, i.e. off-heap memory • data security • C# and C++ clients • management center with no limits Hazelcast Drawbacks Hazelcast Drawbacks
- 155. Hazelcast vs. Infinispan Hazelcast vs. Infinispan
- 156. Hazelcast vs. Infinispan Hazelcast vs. Infinispan Infinispan Hazelcast Pros • backed by relatively large company for use in largely distributed environments (JBoss) • been in active use for several years • well-written documentation • a lot of examples of different • easy setup • more performant than Infinispan • simple node/cluster discovery mechanism • relies on only 1 jar to be included on classpath Cons • relies on JGroups that proven to be buggy especially under high load • backed by a startup based in Palo Alto and Turkey
- 157. Hazelcast is on a Rise Hazelcast is on a Rise
- 158. Hazelcast is on a Rise Hazelcast is on a Rise
- 159. • Funding • received $13.5 million in 2 rounds from 2 investors • most recent funding: $11 Million Series B on September 18, 2014 Hazelcast is on a Rise Hazelcast is on a Rise
- 160. • Funding • received $13.5 million in 2 rounds from 2 investors • most recent funding: $11 Million Series B on September 18, 2014 • Greg Luck • creator and founder of Ehcache, the leading open source java cache • former CTO of Terracotta at Software AG • co-specification lead for JCache, the Java Standard for Caching in Spring and Java EE 8 • in January 2014, Greg Luck joined Hazelcast as CTO. On 18th of June, 2014, Hazelcast announced the appointment of Greg Luck as CEO. Hazelcast is on a Rise Hazelcast is on a Rise
- 161. • Funding • received $13.5 million in 2 rounds from 2 investors • most recent funding: $11 Million Series B on September 18, 2014 • Greg Luck • creator and founder of Ehcache, the leading open source java cache • former CTO of Terracotta at Software AG • co-specification lead for JCache, the Java Standard for Caching in Spring and Java EE 8 • in January 2014, Greg Luck joined Hazelcast as CTO. On 18th of June, 2014, Hazelcast announced the appointment of Greg Luck as CEO. • JCache • on 29th of September, 2014, on the opening day of JavaOne Hazelcast announced the release of Hazelcast 3.3.1 JCache, the JCache compatible version of Hazelcast Hazelcast is on a Rise Hazelcast is on a Rise
- 162. Thanks! Questions? Design by Distributed Applications using Hazelcast by Taras Matyashovsky
- 163. References • http://docs.oracle.com/cd/E18686_01/coh.37/e18677/cache_intro.htm • http://coherence.oracle.com/display/COH31UG/Read-Through,+Write-Through,+Refresh-Ahead+and+Write- Behind+Caching • http://blog.tekmindsolutions.com/oracle-coherence-diffrence-between-replicated-cache-vs-partitioneddistributed- References cache/ • http://www.slideshare.net/MaxAlexejev/from-distributed-caches-to-inmemory-data-grids • http://www.slideshare.net/jaxlondon2012/clustering-your-application-with-hazelcast • http://gridgaintech.wordpress.com/2013/10/19/distributed-caching-is-dead-long-live/ • http://hazelcast.com/resources/mastering-hazelcast/ • http://gridgain.com/in-memory-database-vs-in-memory-data-grid-revisited/ • http://hazelcast.com/resources/thinking-distributed-the-hazelcast-way/ • http://www.prleap.com/pr/228862/hazelcast-releases-jcache-implementation-at • http://www.crunchbase.com/organization/hazelcast/funding-rounds • https://github.com/tmatyashovsky/hazelcast-samples/ • https://github.com/tmatyashovsky/hazelcast-samples-javaday-kyiv