Building and operating large-scale streaming applications at King

In this talk we walk you through our operational journey for building and operating large-scale production streaming applications at King. Our Flink streaming jobs consume over 40 billion events every day maintaining over 10 TBs of user state with Flink's strong consistency guarantees. We will focus on the evolution of our in-house stream processing platform, and the different challenges we have faced at different times in it's 1.5 years of production lifeline. Topics we'll cover: - Overview of our platform - Cluster environment, and deployment procedures - Flink job management (configuration, versioning, etc.) - Monitoring and debugging practices - Some Flink features that we find super amazing

Speakers involved