The first-ever Pulsar Summit Europe took place on October 6th. More than 20 speakers presented at the summit from regional and global companies, including Databricks, Clever Cloud, Elastic, Flipkart, and Tencent, and open-source communities, such as Delta Lake, Apache Skywalking, and Milvus.
The event was packed with the latest Pulsar project and ecosystem updates, technical deep dives, and exciting use cases from around the world. Whether you are a Pulsar veteran or a newbie, Pulsar Summit has something for you.
OpenShift Commons Paris - Choose Your Own Observability Adventure
Opening Remarks - Pulsar Virtual Summit Europe 2021
1. Pulsar Virtual Summit Europe 2021
Welcome!
Pulsar Virtual Summit
Europe 2021
Carolyn King
VP of Growth, StreamNative
2. Pulsar Virtual Summit Europe 2021
Welcome to the Pulsar Summit Europe 2021!
Carolyn King, Summit Emcee
● 2 Years in Apache Pulsar Community
● VP of Growth at StreamNative
● Yogi, runner, tech enthusiast
● Connect with me on Twitter!
@carolynking22
5. Pulsar Virtual Summit Europe 2021
Program Committee
Matteo Merli
Pulsar PMC Chair;
CTO, StreamNative
Karthik Ramasamy
Head of Streaming,
Databricks
Till Rohrmann
Apache Flink PMC;
Engineering Lead,
Ververica
Ricardo Ferreira
Principal Developer
Advocate, Elastic
Rajan Dhabalia
Principal Software
Engineer, Verizon Media
Addison Higham
Chief Architect and Head
of Cloud Engineering,
StreamNative
Jerry Peng
Principal Software
Engineer, Splunk
7. Pulsar Virtual Summit Europe 2021
Connect with speakers and committee
members in real-time on Slack:
● Join Apache Pulsar on Slack
● Join the #pulsarsummit channel
12. Pulsar Virtual Summit Europe 2021
3 Global Pulsar Summits To-Date:
North America 2020 & 2021 and Asia 2020
110
+
Speaker Sessions
2,100+
Global Attendees
300+
Companies Represented
14. Pulsar Virtual Summit Europe 2021
Upcoming: Pulsar Summit Asia 2021
● November 20th-21st
● Virtual + in-person
● 1-day in-person Pulsar Training
● Save your seat today
● Learn more
15. Pulsar Virtual Summit Europe 2021
Virtual Meetups 2021
Co-Hosts and Community Partners
24. Pulsar Virtual Summit Europe 2021
Pulsar Summit 2021 Keynotes
12:20 PM - 12:40 PM
12:50 PM - 1:10 PM
1:20 PM - 1:50 PM
Apache Pulsar, Supporting the Entire Lifecycle of Streaming Data
Matteo Merli, Apache Pulsar PMC Chair, CTO at StreamNative
Apache Pulsar: A Foundation Backbone for Clever Cloud
Quentin Adam, CEO at Clever Cloud
Pulsar in the Lakehouse: Apache Pulsar™ with Apache Spark™ and
Delta Lake
Ryan Zhu, Apache Spark PMC Member and Committer, RxJava
Committer, Staff Software Engineer at Databricks; Addison
Higham, Chief Architect and Head of Cloud Engineering at
StreamNative
25. Pulsar Virtual Summit Europe 2021
Enjoying a session?
Tweet about it and hashtag:
#Pulsarsummit
#Pulsarsummiteurope2021
26. Pulsar Virtual Summit Europe 2021
Pulsar Summit Tracks
2:00 PM - 2:30 PM
2:30 PM - 3:00 PM
3:00 PM - 3:30 PM
3:30 PM - 4:00 PM
4:00 PM - 4:30 PM
4:30 PM - 5:00 PM
Deep Dive into the Pulsar Binary Protocol
Christophe Bornet, Senior Software Engineer at Datastax
Pulsar Watermarking
Eron Wright, Cloud Engineering Lead at StreamNative
Writing Custom Sink Connectors for Pulsar I/O
Ricardo Ferreira, Principal Developer Advocate at Elastic
Function Mesh - Complex Streaming Jobs Made Simple
on Cloud
Neng Lu, Platform Engineering Lead at StreamNative;
Rui Fu, Software Engineer at StreamNative
Using the JMS 2.0 API with Apache Pulsar
Enrico Olivelli, Apache Pulsar PMC, Datastax Pulsar
Team
No Surprises Geo Replication
Gaurav Ashok, Senior Software Engineer at Flipkart
Pulsar Use Cases
Tracking Apache Pulsar Messages with Apache SkyWalking
Sheng Wu, Apache SkyWalking Founder, Tetrate.io Engineer;
Penghui Li, Apache Pulsar PMC, Engineering Lead at
StreamNative
Log System as Backbone–How We Built the World’s Most
Advanced Vector Database on Pulsar
Xiaofan Luan, Partner and Director of Engineering at Zilliz
Built on Pulsar: A Commercial Consent Management System for
80 Million Citizens
Shivam Arora, Solutions Architect at Softtech; Çetin Yalçın Güleç,
Head of Analytics at Softtech
Application of Apache Pulsar in Tencent Billing and Tencent
Advertising
Mingyu Bao, Senior Engineer at Tencent
Interactive Analytics on Pulsar with Pulsar SQL
Axel Sirota, Lead Machine Learning Engineer at JAMPP
Using the FLiPN Stack for Edge AI (Flink, NiFi, Pulsar)
Timothy Spann, Developer Advocate at StreamNative
Technology Deep Dives
27. Pulsar Virtual Summit Europe 2021
Pulsar Summit Track Moderators
Stage 1: Pulsar Use Cases
Carolyn King
VP of Growth, StreamNative
Stage 2: Technology Deep Dives
Timothy Spann
Developer Advocate, StreamNative
Contact us at events@streamnative.io with any questions!
28. Pulsar Virtual Summit Europe 2021
How to Get Involved
● Join Apache Pulsar on Slack
● Join the #pulsarsummit channel
● Sign up for the StreamNative Newsletter for Apache Pulsar
● Sign up for Pulsar Trainings
29. Pulsar Virtual Summit Europe 2021
Apache Pulsar,
Supporting the Entire Lifecycle
of Streaming Data
Matteo Merli
Pulsar PMC Chair; CTO, StreamNative
Hinweis der Redaktion
Hi, and thank you everyone for joining us today. My name is Carolyn King and I’m happy to welcome you all to the Pulsar Virtual Summit Europe 2021
As a member of the Pulsar community for the past 2 years I’ve been impressed by the growth and engagement in the community.
I’m excited to be here today and to welcome all of our speakers and attendees. I’ll kick-off today with some event details and recent project and community highlights before handing it over to Apache Pulsar PMC Chair - Matteo Merli - for the first keynote.
We’d like to start by thanking everyone who has put so much time and effort into making this event happen. Beginning with our event hosts - StreamNative and Clever Cloud. It takes a lot of heavy-lifting to put on this type of event, and we’d like to thank these two companies for their support.
We’d also like to give a big thanks to our Community Sponsors. These sponsors went above and beyond to help promote the event and to support the Apache Pulsar Community.
We’d like to acknowledge the time and effort of the Program Committee. This group spent the time to review each submission and to put together this amazing agenda. They helped to ensure today is filled with interesting and informative sessions for each of you. The Program Committee includes: Matteo, Karthik, Till, Addison, Ricardo, Rajan, Addison and Jerry, thank you for your participation.
Last but not least, we’d like to thank each of our speakers. They each bring unique Pulsar experiences and expertise and have amazing sessions planned.
We encourage you to reach out to all of our sponsors, program committee members, and speakers to connect both during and after the summit. You can find contact information on the Pulsar Summit website or reach out via the Pulsar Summit slack channel with any questions.
If you haven’t joined the Pulsar Slack channel, now is the time. You can join the speakers and fellow attendees for real-time discussions on Summit sessions in the #Pulsar Summit Channel.
The Apache Pulsar community and project has witnessed considerable growth over the past few years. Today we’ll look at some highlights.
Interest and engagement in Apache Pulsar has skyrocketed over the past year. In May of this year, the community hit 400 contributors, a major milestone for us. In October, we are on are way to 450.
At last count, Pulsar surpassed 9,700 Github stars, and we expect to hit 10,000 soon. This represents more than 2x growth in the past 2 years.
To put the growth of the Apache Pulsar community in perspective, we compared monthly active contributors to a well-known, adjacent project, Apache Kafka. In 2021 Apache Pulsar surpassed Apache Kafka in the number of monthly active contributors, illustrating the growth , engagement and interest in the Pulsar project.
To-date we’ve held 3 global Pulsar summits, in North America and one in Asia. The Pulsar summits have attracted more than 100 speakers, 2000 attendees, and hundreds of companies.
Some key companies participating and attending have included Splunk, Interable, and Intuit, among others.
We will also host one more summit this year - the Pulsar Summit Asia 2021. This event will be held Nov 20th and 21st. It will be a hybrid event with both in-person talks and online sessions and will include a 1-day in-person Pulsar training. You can find out more on the Pulsar Summit website.
While we are eager to get back to in-person meetups and spend time with everyone, we are happy to see the community connect online. Pulsar Meetup groups have held a number of events already in 2021. Including an Pulsar + Flink event hosted by Uber and StreamNative in March and another event hosted by Splunk in April. Additionally, we’ve been expanding our community partners to help grow awareness of Apache Pulsar, working with partners from Ververica to Scylla to TiDB to Aerospike.
But virtual meetups aren’t the only way to get involved. There are online webinars on Youtube, Pulsar Tutorials from StreamNative academy, and two new Pulsar books - Oreilly’s Mastering Apache Pulsar by Jowanza Josephn and Manning’s Apache Pulsar in Action by David Kjerrumgaard. The first 5 chapters of Mastering Apache Pulsar and the full Apache Pulsar in Action are available for FREE download at StreamNative.io.
Additionally, new this week, free on-demand Pulsar training. For you or for your entire team, sign up today.
Now to some project highlights.
In June of 2021 the community released Pulsar 2.8.0, with exciting new features such as Transactions and the expansions to the ecosystem, making it easier than ever to adopt Pulsar. For more details, you can check out the keynote from Matteo Merli and Sijie Guo from the Pulsar Summit North America 2021.
In September, the community released Pulsar 2.8.1 - which included a number of bug fixes and stability fixes across a wide range of Pulsar features.
Flink Connector - we are excited to announce that the Pulsar connector has been released upstream in Flink 1.14.
with this release, teams can start interacting with Pulsar from Flink applications even easier and be assured that Pulsar connector will continue to see improvements as Flink continues to evolve.
Thanks to engineers from StreamNative and Ververica for the collaboration!
Another standout feature is Kafka-on-Pulsar, known as KoP. KoP was originally open-sourced in early 2020 by StreamNative and OVHCloud. It continues to grow in interest and usage, with more organizations beginning their adoption of KoP to seamlessly allow Pulsar to handle Kafka workloads. Recent releases further improve performance.
There have also been a number of updates that have helped to broaden the connector connector ecosystem:
- Cloud Storage Sink - the cloud storage sink allows for writing data from Pulsar to S3, GCS, etc. With recent releases, further improvements have been made
The Pulsar ecosystem continues to further integrate with AWS with recent releases of sources and sinks to integrate with support for SQS and Lambda being added, now teams can consume data into Pulsar directly from SQS and trigger AWS lambda functions when messages in a Pulsar topic.
And with that, let’s jump in to today’s event.
Pulsar Summit Europe is a one-day event. We will begin the summit with 3 keynotes on the mainstage.
I’ll be hosting Stage 1 and my colleague, Timothy Spann, will be hosting Stage 2. You can drop questions directly in the chat, the Pulsar Summit slack channel, or you can email us at events@ streamnative.io.
Many of the speakers will be available for live Q&A at the end of their sessions. Your Track moderator will let you know. Please add your questions in the chat.