#Google has teamed up with several other companies to submit its open sourced #Dataflow technology to the #Apache Software Foundation (ASF) as an incubator project, a first for the search giant.
Dataflow is used for defining and executing data processing workflows, including workflows for data ingestion and integration. A data pipeline defined and built with Dataflow's unified model and language-specific SDKs can be executed on several runtimes or execution/processing engines.
This, Google says, relieves the burden of having to rewrite data pipelines in order to use a different engine, such as switching from batch-processing Apache Hadoop MapReduce engine in order to enjoy the superior performance and streaming analytics capabilities of Apache Spark.
https://adtmag.com/articles/2016/01/21/google-dataflow-asf.aspx?m=1
No comments:
Post a Comment