Since its stable release in 2016, Akka Streams is quickly becoming the de facto standard integration layer between various Streaming systems and products. Enterprises like PayPal, Intel, Samsung and Norwegian Cruise Lines see this is a game changer in terms of designing Reactive streaming applications by connecting pipelines of back-pressured asynchronous processing stages. This comes from the Reactive Streams initiative in part, which has been long led by Lightbend and others, allowing multiple streaming libraries to inter-operate between each other in a performant and resilient fashion, providing back-pressure all the way. But perhaps even more so thanks to the various integration drivers that have sprung up in the community and the Akka team—including drivers for Apache Kafka, Apache Cassandra, Streaming HTTP, Websockets and much more. In this webinar for JVM Architects, Konrad Malawski explores the what and why of Reactive integrations, with examples featuring technologies like Akka Streams, Apache Kafka, and Alpakka, a new community project for building Streaming connectors that seeks to “back-pressurize” traditional Apache Camel endpoints. * An overview of Reactive Streams and what it will look like in JDK 9, and the Akka Streams API implementation for Java and Scala. * Introduction to Alpakka, a modern, Reactive version of Apache Camel, and its growing community of Streams connectors (e.g. Akka Streams Kafka, MQTT, AMQP, Streaming HTTP/TCP/FileIO and more). * How Akka Streams and Akka HTTP work with Websockets, HTTP and TCP, with examples in both in Java and Scala.

1. akka streams,
Exploring Reactive Integrations with
Alpakka and Kafka
Konrad `ktoso` Malawski

2. Konrad `ktoso` Malawski
Akka Team,
Reactive Streams TCK,
Persistence, HTTP, Remoting / Cluster

3. Make building powerful concurrent &
distributed applications simple.
Akka is a toolkit and runtime
for building highly concurrent,
distributed, and resilient
message-driven applications
on the JVM

4. Actors – simple & high performance concurrency
Cluster / Remoting – location transparency, resilience
Cluster tools – and more prepackaged patterns
Streams – back-pressured stream processing
Persistence – Event Sourcing
HTTP – complete, fully async and reactive HTTP Server
Official Kafka, Cassandra, DynamoDB integrations, tons
more in the community
Complete Java & Scala APIs for all features
What’s in the toolkit?

6. “Stream”
has many meanings

7. akka streams
Asynchronous back pressured stream processing
Source Sink
Flow

8. akka streams
Asynchronous back pressured stream processing
Source Sink
(possible)
asynchronous
boundaries
Flow

9. akka streams
Asynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
OutOfMemoryError!!
Flow

10. akka streams
Asynchronous back pressured stream processing
Source Sink
10 msg/s 1 msg/s
hand me 3 morehand me 3 more
1 msg/s Flow

11. akka streams
Not only linear streams
Source
SinkFlow
Source
Sink
Flow
Flow

12. And the many meanings it carries.
Reactive

13. The many meanings of Reactive
reactivemanifesto.org

14. The many meanings of Reactive

17. “Not-quite-Reactive-System”
The reason we started researching
into transparent to users flow control.

24. Reactive Streams
Reactive Streams is an initiative to provide a standard for
asynchronous stream processing with non-blocking back
pressure. This encompasses efforts aimed at runtime
environments as well as network protocols
http://www.reactive-streams.org

25. Reactive Streams
A buiding-block of Reactive Systems, not the “entire story”.

26. Reactive Streams
Reactive Streams is an initiative to provide a standard for
asynchronous stream processing with non-blocking back
pressure. This encompasses efforts aimed at runtime
environments as well as network protocols
http://www.reactive-streams.org

27. Part of JDK 9
java.util.concurrent.Flow
http://openjdk.java.net/projects/jdk9/

28. JEP-266 – soon…!
public final class Flow {
private Flow() {} // uninstantiable
@FunctionalInterface
public static interface Publisher<T> {
public void subscribe(Subscriber<? super T> subscriber);
}
public static interface Subscriber<T> {
public void onSubscribe(Subscription subscription);
public void onNext(T item);
public void onError(Throwable throwable);
public void onComplete();
}
public static interface Subscription {
public void request(long n);
public void cancel();
}
public static interface Processor<T,R> extends Subscriber<T>, Publisher<R> {
}
}

29. Reactive Streams
RS Library A RS library B
async
boundary

30. Reactive Streams
RS Library A RS library B
async
boundary
Make building powerful concurrent &
distributed applications simple.

31. The API
Akka Streams
Complete and awesome
Java and Scala APIs
(Just like everything in Akka)

32. Akka Streams in 20 seconds:
Source<Integer, NotUsed> source = null;
Flow<Integer, String, NotUsed> flow =
Flow.<Integer>create().map((Integer n) -> n.toString());
Sink<String, CompletionStage<Done>> sink =
Sink.foreach(str -> System.out.println(str));
RunnableGraph<NotUsed> runnable =
source.via(flow).to(sink);
runnable.run(materializer);

33. Akka Streams in 20 seconds:
CompletionStage<String> firstString =
Source.single(1)
.map(n -> n.toString())
.runWith(Sink.head(), materializer);

34. Source.single(1).map(i -> i.toString).runWith(Sink.head())
// types: _
Source<Int, NotUsed>
Flow<Int, String, NotUsed>
Sink<String, CompletionStage<String>>
Akka Streams in 20 seconds:

35. Source.single(1).map(i -> i.toString).runWith(Sink.head())
// types: _
Source<Int, NotUsed>
Flow<Int, String, NotUsed>
Sink<String, CompletionStage<String>>
Akka Streams in 20 seconds:

36. Materialization
Gears from GeeCON.org,(it’s an awesome conf)

37. What is “materialization” really?

38. What is “materialization” really?

39. What is “materialization” really?

40. Introspection, and more coming…
Highly sophisticated stream introspection capabilities.

41. AlpakkaA community for Streams connectors
http://blog.akka.io/integrations/2016/08/23/intro-alpakka

42. Alpakka – a community for Stream connectors
Threading & Concurrency in Akka Streams Explained (part I)
Mastering GraphStages (part I, Introduction)
Akka Streams Integration, codename Alpakka
A gentle introduction to building Sinks and Sources using GraphStage APIs
(Mastering GraphStages, Part II)
Writing Akka Streams Connectors for existing APIs
Flow control at the boundary of Akka Streams and a data provider
Akka Streams Kafka 0.11

43. Alpakka – a community for Stream connectors
http://developer.lightbend.com/docs/alpakka/current/

44. Alpakka – a community for Stream connectors
Demo

45. Akka Streams & HTTP
streams
& HTTP

46. A core feature not obvious to the untrained eye…!
Akka Streams / HTTP
Quiz time!
TCP is a ______ protocol?

47. A core feature not obvious to the untrained eye…!
Akka Streams / HTTP
Quiz time!
TCP is a STREAMING protocol!

48. Streaming in Akka HTTP
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html
HttpServer as a:
Flow[HttpRequest, HttpResponse]

49. Streaming in Akka HTTP
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html

50. Streaming in Akka HTTP
HttpServer as a:
Flow[HttpRequest, HttpResponse]
HTTP Entity as a:
Source[ByteString, _]
Websocket connection as a:
Flow[ws.Message, ws.Message]
http://doc.akka.io/docs/akka/2.4/scala/stream/stream-customize.html#graphstage-scala
“Framed entity streaming”
http://doc.akka.io/docs/akka/2.4/java/http/routing-dsl/source-streaming-support.html

51. Resource efficiency and dynamic fan-out

52. Resource efficiency and dynamic fan-out

53. It’s turtles buffers all the way down!
xkcd.com

54. Streaming from Akka HTTP

55. Streaming from Akka HTTP

56. Streaming from Akka HTTP (Java)
public static void main(String[] args) {
final ActorSystem system = ActorSystem.create();
final Materializer materializer = ActorMaterializer.create(system);
final Http http = Http.get(system);
final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world"));
final Route tweetsRoute =
path("tweets", () ->
completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json())
);
final Flow<HttpRequest, HttpResponse, NotUsed> handler =
tweetsRoute.flow(system, materializer);
http.bindAndHandle(handler,
ConnectHttp.toHost("localhost", 8080),
materializer
);
System.out.println("Running at http://localhost:8080");
}

57. Streaming from Akka HTTP (Java)
public static void main(String[] args) {
final ActorSystem system = ActorSystem.create();
final Materializer materializer = ActorMaterializer.create(system);
final Http http = Http.get(system);
final Source<Tweet, NotUsed> tweets = Source.repeat(new Tweet("Hello world"));
final Route tweetsRoute =
path("tweets", () ->
completeWithSource(tweets, Jackson.marshaller(), EntityStreamingSupport.json())
);
final Flow<HttpRequest, HttpResponse, NotUsed> handler =
tweetsRoute.flow(system, materializer);
http.bindAndHandle(handler,
ConnectHttp.toHost("localhost", 8080),
materializer
);
System.out.println("Running at http://localhost:8080");
}

58. Streaming from Akka HTTP (Scala)
object Example extends App
with SprayJsonSupport with DefaultJsonProtocol {
import akka.http.scaladsl.server.Directives._
implicit val system = ActorSystem()
implicit val mat = ActorMaterializer()
implicit val jsonRenderingMode = EntityStreamingSupport.json()
implicit val TweetFormat = jsonFormat1(Tweet)
def tweetsStreamRoutes =
path("tweets") {
complete {
Source.repeat(Tweet(""))
}
}
Http().bindAndHandle(tweetsStreamRoutes, "127.0.0.1", 8080)
System.out.println("Running at http://localhost:8080");
}

59. Next steps for Akka
Completely new Akka Remoting (goal: 700.000+ msg/s (!)),
(it is built using Akka Streams, Aeron).
More integrations for Akka Streams stages, project Alpakka.
Reactive Kafka polishing with SoftwareMill, Krzysiek Ciesielski
Akka Typed progressing again, likely towards 3.0.
Akka HTTP 2.0 Proof of Concept in progress.
Collaboration with Reactive Sockets

60. Ready to adopt on prod?

61. Totally, go for it.

62. Akka <3 contributions
Easy to contribute tickets:
https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3Aeasy-to-contribute
https://github.com/akka/akka/issues?q=is%3Aissue+is%3Aopen+label%3A%22nice-to-have+%28low-prio%29%22
Akka Stream Contrib
https://github.com/akka/akka-stream-contrib
Mailing list:
https://groups.google.com/group/akka-user
Public chat rooms:
http://gitter.im/akka/dev developing Akka
http://gitter.im/akka/akka using Akka

63. More resources:
Read more tutorials and deep-dives:
http://blog.akka.io/
https://www.lightbend.com/resources

64. Reactive Platform
Reactive Platform
Reactive Platform

65. Further reading:
Reactive Streams: reactive-streams.org
Akka documentation: akka.io/docs
Free O’Reilly report – bit.ly/why-reactive
Example Sources:
ktoso/akka-streams-alpakka-talk-demos-2016
Contact:
Konrad ktoso@lightbend.com Malawski
http://kto.so / @ktosopl

66. Thanks!
Questions?
@apnylle johan.andren@lightbend.com
@ktosopl konrad.malawski@lightbend.com