Customer Relationship Management News NewsFactor Sites:       NewsFactor.com     Enterprise Security Today     CRM Daily     Business Report     Sci-Tech Today  
   
Home CRM Systems Customer Service Contact Centers Business Intelligence More Topics...
You are here: Home / Big Data / Can Apache Spark Light Up Hadoop?
Gartner's #1 for endpoint backup
Can Super-Fast Apache Spark Light Up Hadoop?
Can Super-Fast Apache Spark Light Up Hadoop?
By Jennifer LeClaire / CRM Daily Like this on Facebook Tweet this Link thison Linkedin Link this on Google Plus
PUBLISHED:
MAY
30
2014



Call it the Hadoop Swiss Army knife of cluster computing frameworks. The Apache Software Foundation just rolled out Apache Spark v1.0, which it's calling a "super-fast, open-source, large-scale data processing and advanced analytics engine."

That's a mouthful, indeed, but why has the technology been dubbed a Hadoop Swiss Army knife? Because Spark lets developers write apps in Java, Scala or Python with a built-in set of more than 80 high-level operators. Apache claims Spark makes possible programs that can run up to 100 times faster than Apache Hadoop MapReduce in memory.

"Apache Spark is an important big data technology in delivering a high-performance analytics solution for the IT industry and satisfying the fast-growing customer demand," said Michael Greene, vice president and general manager of System Technologies and Optimization at Intel.

Who Does this Target?

Apache Spark aims at groups that need to tap into machine learning, interactive queries, and stream processing. Spark is fully compatible with Hadoop's Distributed File System, HBase, Cassandra, and any Hadoop storage system, so existing data is immediately available in Spark. Spark also promises support for SQL queries, streaming data and complex analytics, including machine learning and graph algorithms, right out of the box.

Patrick Wendell, software engineer at Databricks and Apache Spark 1.0 release manager, explained that the new release not only provides long-term stability for Spark's core APIs, it also offers several new features.

"Spark 1.0 adds a unified submission tool for deploying applications on a local machine, Mesos, YARN, or a dedicated cluster," said Wendell. "We've added a new module, Spark SQL, to provide schema-aware data modeling and SQL language support in Spark. Spark's machine learning library, MLLib, has been enhanced with several new algorithms. Spark's streaming and graph libraries have also seen major updates. Across the board, we've focused on building tools to empower the data scientists, statisticians and engineers who must grapple with large data sets every day."

NASA Is All-In

Originally developed at UC Berkeley AMP Lab, Spark is in use by companies like Alibaba, ClearStory Data, Cloudera, Databricks, IBM, Intel, MapR, Ooyala and Yahoo. Beyond enterprise adoption, Apache Spark is also winning code contributors to the project.

Chris Mattmann, who is an Apache Software Foundation (ASF) director and chief architect in the Instrument and Science Data Systems Section at the National Aeronautics and Space Administration Jet Propulsion Laboratory, said NASA is excited to leverage Spark and its "highly interactive analytic capabilities." He also pointed to the speedups 1.0 offers, and said Spark SQL as helpful to critical projects looking at measurement of snow in the western U.S., as well as on projects related to regional climate modeling and in model evaluation for the National Climate Assessment.

"I'm looking forward to designing Spark-related projects in my software architectures and in my search engines courses at USC as well," said Mattmann, who also is an adjunct associate professor there. "The community is one of our most active at the ASF, and the interest has really peaked and these guys are doing a great job."

Tell Us What You Think
Comment:

Name:

Like Us on FacebookFollow Us on Twitter
TOP STORIES NOW
MAY BE OF INTEREST
Protect 100% of your Data The prevalence of laptops and mobile devices in the enterprise makes corporate data increasingly vulnerable to loss and breach. And yet, workforce productivity is now inextricably linked to mobility. Click here to access the white paper "Top 10 Endpoint Backup Mistakes" to learn more about how to confidently protect data across platforms and devices while also providing features designed to enhance the end user experience.
MORE IN BIG DATA
Product Information and Resources for Technology You Can Use To Boost Your Business

Network Security Spotlight
Dairy Queen Latest Retailer To Report Hack
Dairy Queen is known for its hot fries and sweet treats, but it just made cyber history as the latest victim of a hack attack. The fast food chain said that customer data at some stores may be at risk.
 
Lessons from the JPMorgan Chase Cyberattack
JPMorgan Chase is investigating a likely cyberattack. The banking giant is cooperating with law enforcement, including the FBI, to understand what data hackers may have obtained.
 
Who Is the Hacker Group Lizard Squad?
Are they dangerous or just obnoxious? That’s what many are wondering about the hacker group Lizard Squad, which tweeted out a bomb threat that grounded a flight with a Sony exec aboard.
 

Enterprise Hardware Spotlight
Intel Intros Lightning-Fast PC Processors
Call it extreme. Intel just took the covers off its first-ever eight-core desktop processor, which is aimed at hardcore power users who expect more than the status quo from their computers.
 
HP Previews ProLiant Gen9 Data Center Servers
Because traditional data center and server architectures are “constraints” on businesses, HP is releasing new servers aimed at faster, simpler and more cost-effective delivery of computing services.
 
Apple Set To Release Largest iPad Ever
Tech giant Apple seems to have adopted the mantra “go big or go home.” The company is planning to introduce its largest iPad ever: a 12.9-inch behemoth that will dwarf its largest existing models.
 

Mobile Technology Spotlight
Samsung Maps Its Way with Nokia's 'Here' App for Galaxy Phones
Korean electronics giant Samsung has opted to license Here, Nokia’s mapping app -- formerly known as Nokia Maps -- for its Tizen-powered smart devices and Samsung Gear S wearable.
 
Google Successfully Tests Its Own Delivery Drone
While top technology companies are engaged in an "arms race" to develop drones that can quickly deliver goods to anyone anywhere, Google has revealed it successfully tested its own version.
 
Will iPhone Finally Catch Up with NFC Mobile Payment Ability?
Apple's latest version of the iPhone may have a mobile wallet to pay for purchases with a tap of the phone. The iPhone 6 reportedly is equipped with near-field communication (NFC) technology.
 

Navigation
CRM Daily
Home/Top News | CRM Systems | Customer Service | Contact Centers | Business Intelligence | Sales & Marketing | Customer Data | CRM Press Releases
NewsFactor Network Enterprise I.T. Sites
NewsFactor Technology News | Enterprise Security Today | CRM Daily

NewsFactor Business and Innovation Sites
Sci-Tech Today | NewsFactor Business Report

NewsFactor Services
FreeNewsFeed | Free Newsletters

About NewsFactor Network | How To Contact Us | Article Reprints | Careers @ NewsFactor | Services for PR Pros | Top Tech Wire | How To Advertise

Privacy Policy | Terms of Service
© Copyright 2000-2014 NewsFactor Network. All rights reserved. Article rating technology by Blogowogo. Member of Accuserve Ad Network.