Confluent Releases KSQL, a Streaming SQL Engine for Apache Kafka

Open source, distributed, scalable and real time, KSQL is the easiest way to express continuous, interactive queries in Kafka

SAN FRANCISCO – August 28, 2017 (Kafka Summit SF) – Confluent, provider of the leading streaming platform based on Apache Kafka®, today announced KSQL, an open source streaming SQL engine that enables continuous, interactive queries on Apache Kafka. In today’s event-driven world, businesses need to respond to the continuous streams of data that are the source of truth for everything happening within the company. With KSQL, any developer that knows SQL can leverage real-time data. Unlike other stream processing engines that require complex infrastructure or mastery of various programming languages, KSQL gives the user a familiar syntax in an easy-to-manage-and-build solution, while benefiting from Kafka’s distributed, scalable and reliable development and production history.

Common examples of stream processing might include comparing two or more streams of data to understand anomalies and respond to them in real time or transforming data as it’s ingested to better suit downstream consumers. Stream processing can be used to identify fraud by financial services companies, monitor out-of-bounds system performance metrics, and much more. Other common uses include:

• Anomaly Detection: Pattern recognition and anomaly detection are real-time and event-driven processes ideally suited to running against streams of data. This can be used horizontally across a number of industries and use cases, for example: financial institutions can alert a bank’s users of out-of-the-ordinary transactions, while potentially detecting a security attack.
• Monitoring: Business are able to move from batch systems to real-time notifications and catch issues in real time that can prevent entire system failures. With KSQL, teams can leverage SQL-like queries to solve problems through the use of a widely-known language while errors are happening, rather than waiting until the next day.
• Streaming ETL: Companies often have numerous, disparate applications with data that needs to be integrated, standardized and enriched before it can be used in downstream applications or for analysis and reporting. The traditional solution to this batch-oriented process is called ETL (extract, transform, load). With the onset of large scale streams, this process is moving to a real-time world; however, those usually responsible for this function rarely live in a programming world and prefer working with SQL. KSQL enables this group to move forward from batch-processing ETL to doing it in real-time.

“Until now, stream processing has required complex infrastructure, sophisticated developers and a serious investment. Our mission is to make stream processing easily accessible so anyone can derive insights from their streams of data,” said Neha Narkhede, co-founder and CTO at Confluent. “With KSQL, stream processing on Apache Kafka is available through a familiar SQL-like interface, rather than only to developers who are familiar with Java or Python. It is the first completely interactive, distributed streaming SQL engine for Apache Kafka.”

Supporting Quotes

“With KSQL, now users can query Apache Kafka topics directly without dumping data into intermediate databases or key value stores,” said Shant Hovsepian, CTO at Arcadia Data. “This is really a game changer, as a native visual analytics tool like Arcadia Enterprise can enable users to get real-time insights using KSQL without jumping through hoops to get data landed and staged. We are pleased to partner with Confluent to support this exciting new product.”

“Real-time insights are vital for our business as they directly impact our ability to become a industry-leading digital organisation,” said Alasdair Anderson, EVP at Nordea Bank. “We are pleased that Confluent and Arcadia Data are delivering on this vision, and we look forward to integrating their innovative work within our data strategy.”

KSQL is part of Confluent Open Source and licensed under the Apache 2.0 license. Download the developer preview here: https://github.com/confluentinc/ksql.

Confluent will be attending and sponsoring Kafka Summit, the premier streaming systems event for data engineers and developers that brings the Apache Kafka community together to share best practices, write code and discuss the future of streaming technologies. Visit Confluent at booth #105 to talk all things streaming and learn more about KSQL. To watch the keynotes, visit: https://www.youtube.com/watch?v=ySf7bDBHRUQ.

Resources

• About KSQL: https://www.confluent.io/ksql
• Read the blog post: https://www.confluent.io/blog/ksql-open-source-streaming-sql-for-apache-kafka/
• View the quick start here: https://github.com/confluentinc/ksql/tree/0.1.x/docs/quickstart
• Register for the Online Talk: https://www.confluent.io/online-talk/ksql-streaming-sql-for-apache-kafka/

About Confluent

Confluent, founded by the creators of open source Apache Kafka®, provides the only streaming platform that enables enterprises to maximize the value of data. Confluent Platform empowers leaders in industries such as retail, logistics, manufacturing, financial services, technology and media, to move data from isolated systems into a real-time data pipeline where they can act on it immediately. Backed by Benchmark, Index Ventures and Sequoia, Confluent is based in Palo Alto, California. To learn more, please visit www.confluent.io. Download Confluent Open Source at www.confluent.io/download.

Connect with Confluent

Read our blog: www.confluent.io/blog
Follow us on Twitter: www.twitter.com/confluentInc

© Confluent, Inc. 2014–2017. All rights reserved. Apache, Apache Kafka and Kafka are trademarks of the Apache Software Foundation. All other marks used are the property of their respective owners.