Apache Spark grows in popularity as Hadoop-based data lakes fill up

#Apache #Spark grows in popularity as #Hadoop -based data lakes fill up Jessica Twentyman Apache Spark is growing in popularity and finding real-time use across Europe, including in online betting and on railways, and with Hadoop At Sky Betting and Gaming ( #SkyBet ), promotions are the key to bringing in punters keen to place a stake on a winning team or horse, to try their luck at poker, or to go “eyes down” in a game of bingo. A well-timed offer of a free bet, or an invitation to join a “no-lose” promotion that guarantees a payout regardless of whether they win or lose goes a long way with customers, improving the rate at which Sky Bet – which has offices in Rome and Guernsey, as well as mainland Britain – can acquire, engage and retain them. But matching the right offer to the right customer shouldn’t be a game of chance, according to Andy Walton, the company’s head of data.  “We spend a frightening amount each year on free bets and bonuses for our customers, right across our sports betting and gaming products, but especially in gaming. Promotions drive that business, whereas sports betting is more driven by the sporting calendar,” he says. The more accurately that Sky Bet can target promotions, he says, the better the returns it can expect from that spend. Walton and his team look to open-source big data framework Apache Spark for answers. Right now, he says, the company’s legacy promotions systems fall into two camps. On one hand, there are those that offer promotions in more or less real time, but don’t know a lot about individual customers, “so they can’t make precise decisions about who to offer promotions”. On the other, there are those that know a lot about customers, but are based on batch processing “so you can’t offer the promotion in a timely manner”. Walton hopes to bridge that gap by building a real-time, rules-based promotion engine in Apache Spark, running on the company’s Cloudera-based Hadoop infrastructure. This will enable Sky Bet to combine in-depth information about individual customers, such as the customer segment to which they belong, the bets or games they prefer, and their overall value to the company, with real-time activity information that shows what they’re doing on the platform at any particular time, “so we can make some data-rich decisions about what promotion to offer to them right now”.

http://www.computerweekly.com/feature/Apache-Spark-grows-in-popularity-as-Hadoop-based-data-lakes-fill-up