Alpha Support for Spark Structured Streaming Spark 2.0 saw the alpha release of Structured Streaming, a new streaming framework that combines the query planning facilities from #SparkSQL with the potential for exactly once processing of stream messages. Now, with ES-#Hadoop 6.0.0-alpha2, we are proud to present our own #Elasticsearch structured streaming sink for #ApacheSpark, complete with all features you’ve come to love from the connector, as well as backed by an internal HDFS-based commit log to help ensure correct message delivery. All of this is detailed further in our documentation. Please remember though: Both Structured Streaming in Spark and this sink in ES-Hadoop are alpha features! There could still be bugs lurking out there. We advise against rolling this out to production without careful consideration.
https://www.elastic.co/blog/eshadoop-6-0-0-alpha2-released
No comments:
Post a Comment