Organizations aren't letting ongoing #Apache #Spark development, functionality holes or issues deter them from ramping up usage of the technology. Find out why.
Spark usage is growing rapidly, even though the data processing engine still has some growing of its own to do.
For example, software vendor Xactly Corp. is using Spark to run a mix of batch and real-time applications. While Spark's fast performance makes it a valuable processing tool, the big data technology still has some rough edges that need to be smoothed out, in the eyes of Ron Rasmussen, the company's CTO and senior vice president of engineering.
"It's not immature," Rasmussen said. "But when you compare it to running transaction-level Oracle, it's not there yet." For example, Xactly has had to troubleshoot idiosyncrasies in memory usage, sometimes turning to support technicians at Hadoop vendor MapR Technologies Inc. for help. And monitoring Spark queries is something of a guessing game for Rasmussen's team. "It's hard to know if something is supposed to be taking that long," he said.
Increasingly, Spark is pushing aside #MapReduce, #Hadoop 's original programming environment and execution engine, for batch-processing uses due to its performance advantages. But Spark doesn't fully measure up even to MapReduce on some types of functionality, said Nitin Kak, a lead software development engineer at online marketing analytics platform vendor Quaero. In working with Spark, Kak has had to manually provision the amount of memory and the number of CPU cores required by processing jobs, things that he said MapReduce can take care of automatically.
No comments:
Post a Comment