Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
What to Upload to SlideShare
Loading in …3
×
1 of 12

Confluent流处理平台之Kafka新技术分享

1

Share

Leo Yuen, Confluent, Senior Partner Solutions Engineer

Confluent Platform 是一个企业级的平台,创新的功能与技术让Kafka变得更完善,

不仅加速应用程序开发和连通性,通过流处理实现转换也简化了大规模的企业运营并满足严格的架构要求。本次分享主要介绍两个新技术,包括分层存储与集群链接。

https://www.meetup.com/Beijing-Kafka-Meetup/events/281812695/

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Confluent流处理平台之Kafka新技术分享

  1. 1. Confluent流处理平台之Kafka新技术分享 Leo Yuen Partner Solutions Engineer
  2. 2. 动态性能和弹性 Dynamic Performance & Elasticity Self-Balancing Clusters | Tiered Storage Flexible DevOps Automation Operator | Ansible GUI-driven Mgmt & Monitoring Control Center | Proactive Support Event Streaming Database ksqlDB Rich Pre-built Ecosystem Connectors | Hub | Schema Registry Multi-language Development Non-Java Clients | REST Proxy Admin REST APIs Global Resilience Multi-region Clusters | Replicator | Cluster Linking Data Compatibility Schema Registry | Schema Validation Enterprise-grade Security RBAC | Secrets | Audit Logs TCO / ROI Revenue / Cost / Risk Impact Complete Engagement Model Efficient Operations at Scale Unrestricted Developer Productivity Production-stage Prerequisites Partnership for Business Success Freedom of Choice Committer-driven Expertise Open Source | Community licensed Fully Managed Cloud Service Self-managed Software Training Partners Enterprise Support Professional Services ARCHITECT OPERATOR DEVELOPER BUYER Apache Kafka
  3. 3. 3 分层存储 Tiered Storage 分层存储通过解耦 Kafka 中的计算层和存 储层,实现无限数据保留 和弹性可扩展性 事件流是存储密集型的,尤其是 随着客户的成熟: ... Micro- service ... SFDC App Splunk ... Device Logs Object Storage Main- frame ... Hadoop Data Stores 3rd Party Apps Custom Apps / Microservices Logs
  4. 4. 4 代理存储可能很昂贵并且与计算 紧密相关: Brokers 分层存储 Tiered Storage 分层存储通过解耦 Kafka 中的计算层和存 储层,实现无限数据保留 和弹性可扩展性
  5. 5. 5 分层存储允许 Kafka 识别两层存 储: Brokers Cost-effective Object Storage Offload old data to object store 分层存储 Tiered Storage 分层存储通过解耦 Kafka 中的计算层和存 储层,实现无限数据保留 和弹性可扩展性
  6. 6. 6 分层存储提供了三个主要优势, 彻底改变了我们客户体验 Kafka 的方式: 无限数据保留 Infinite data retention Reimagine what event streaming apps can do 降低基础设施成本 Reduced infrastructure costs Offload data to cost-effective object storage 平台弹性 Platform elasticity Scale compute and storage independently 分层存储 Tiered Storage 分层存储通过解耦 Kafka 中的计算层和存 储层,实现无限数据保留 和弹性可扩展性
  7. 7. 全球韧性 Dynamic Performance & Elasticity Self-Balancing Clusters | Tiered Storage Flexible DevOps Automation Operator | Ansible GUI-driven Mgmt & Monitoring Control Center | Proactive Support Event Streaming Database ksqlDB Rich Pre-built Ecosystem Connectors | Hub | Schema Registry Multi-language Development Non-Java Clients | REST Proxy Admin REST APIs Global Resilience Multi-region Clusters | Replicator | Cluster Linking Data Compatibility Schema Registry | Schema Validation Enterprise-grade Security RBAC | Secrets | Audit Logs TCO / ROI Revenue / Cost / Risk Impact Complete Engagement Model Efficient Operations at Scale Unrestricted Developer Productivity Production-stage Prerequisites Partnership for Business Success Freedom of Choice Committer-driven Expertise Open Source | Community licensed Fully Managed Cloud Service Self-managed Software Training Partners Enterprise Support Professional Services ARCHITECT OPERATOR DEVELOPER BUYER Apache Kafka
  8. 8. 在独立集群或迁移集群之间共享数 据带来了主要挑战: 1. Requires deploying a separate Connect cluster, adding architectural complexity 2. Offsets are not preserved, so messages are at risk of being skipped or reread 8 1 2 0 1 2 3 4 ... 4 5 6 7 8 ... Topic 1, DC 1: Topic 1, DC 2: DC 1: DC 2: 集群链接 Cluster Linking 集群链接简化了 Kafka 的混合云和多云部署
  9. 9. 9 Cluster Linking 不需要额外的基础 设施并保留偏移量: Migrate clusters to Confluent Cloud 集群链接 Cluster Linking 集群链接简化了 Kafka 的混合云和多云部署
  10. 10. Cluster Linking 还在公共云之间提供了一 个经济高效、安全且高性能的传输层: App App App App App App App App App App App App 高网络成本 复杂的管理 低网络成本 一次移动,多次阅读 Cluster Linking 10 集群链接 Cluster Linking 集群链接简化了 Kafka 的混合云和多云部署
  11. 11. 混合云和多云战略为企业提供了 显着的好处: 消除数据孤岛 Remove data silos and ensure data exists wherever your business needs it Leverage 同类最佳解决方案 best of breed solutions across different public cloud providers Offload data infrastructure to 全托管服务 fully managed services, like Confluent Cloud 避免供应商锁定 Avoid vendor lock-in and utilize the most cost-effective vendors 11 集群链接 Cluster Linking 集群链接简化了 Kafka 的混合云和多云部署
  12. 12. 12 Demo - Clustering Linking

×