Newsletters
Customer Relationship Management News NewsFactor Sites:       NewsFactor.com     Enterprise Security Today     CRM Daily     Business Report     Sci-Tech Today  
   
Home CRM Systems Customer Service Contact Centers Business Intelligence More Topics...
Customer Service
Cisco UCS Invicta Series flash memory systems
Average Rating:
Rate this article:  
Can Super-Fast Apache Spark Light Up Hadoop?
Can Super-Fast Apache Spark Light Up Hadoop?

By Jennifer LeClaire
May 30, 2014 12:41PM

Bookmark and Share
Apache Spark aims at groups that need to tap into machine learning, interactive queries, and stream processing. Spark is fully compatible with Hadoop's Distributed File System, HBase, Cassandra, and any Hadoop storage system, so existing data is immediately available. Spark also promises support for SQL, streaming data and complex analytics.
 



Call it the Hadoop Swiss Army knife of cluster computing frameworks. The Apache Software Foundation just rolled out Apache Spark v1.0, which it's calling a "super-fast, open-source, large-scale data processing and advanced analytics engine."

That's a mouthful, indeed, but why has the technology been dubbed a Hadoop Swiss Army knife? Because Spark lets developers write apps in Java, Scala or Python with a built-in set of more than 80 high-level operators. Apache claims Spark makes possible programs that can run up to 100 times faster than Apache Hadoop MapReduce in memory.

"Apache Spark is an important big data technology in delivering a high-performance analytics solution for the IT industry and satisfying the fast-growing customer demand," said Michael Greene, vice president and general manager of System Technologies and Optimization at Intel.

Who Does this Target?

Apache Spark aims at groups that need to tap into machine learning, interactive queries, and stream processing. Spark is fully compatible with Hadoop's Distributed File System, HBase, Cassandra, and any Hadoop storage system, so existing data is immediately available in Spark. Spark also promises support for SQL queries, streaming data and complex analytics, including machine learning and graph algorithms, right out of the box.

Patrick Wendell, software engineer at Databricks and Apache Spark 1.0 release manager, explained that the new release not only provides long-term stability for Spark's core APIs, it also offers several new features.

"Spark 1.0 adds a unified submission tool for deploying applications on a local machine, Mesos, YARN, or a dedicated cluster," said Wendell. "We've added a new module, Spark SQL, to provide schema-aware data modeling and SQL language support in Spark. Spark's machine learning library, MLLib, has been enhanced with several new algorithms. Spark's streaming and graph libraries have also seen major updates. Across the board, we've focused on building tools to empower the data scientists, statisticians and engineers who must grapple with large data sets every day."

NASA Is All-In

Originally developed at UC Berkeley AMP Lab, Spark is in use by companies like Alibaba, ClearStory Data, Cloudera, Databricks, IBM, Intel, MapR, Ooyala and Yahoo. Beyond enterprise adoption, Apache Spark is also winning code contributors to the project.

Chris Mattmann, who is an Apache Software Foundation (ASF) director and chief architect in the Instrument and Science Data Systems Section at the National Aeronautics and Space Administration Jet Propulsion Laboratory, said NASA is excited to leverage Spark and its "highly interactive analytic capabilities." He also pointed to the speedups 1.0 offers, and said Spark SQL as helpful to critical projects looking at measurement of snow in the western U.S., as well as on projects related to regional climate modeling and in model evaluation for the National Climate Assessment.

"I'm looking forward to designing Spark-related projects in my software architectures and in my search engines courses at USC as well," said Mattmann, who also is an adjunct associate professor there. "The community is one of our most active at the ASF, and the interest has really peaked and these guys are doing a great job."
 

Tell Us What You Think
Comment:

Name:





 Customer Service
1.   Watson's First Consumer-Facing Gig
2.   Social Media Haters Speak Up
3.   Comcast Customer Service Fiasco
4.   'May I Help You?' Asks Jibo the Robot
5.   Why You Need To Hire a CCO


advertisement
Social Media Haters Speak Up
Survey says, now showing a little love.
Average Rating:
Comcast Customer Service Fiasco
Lessons to be learned for all companies.
Average Rating:
T-Mobile Accused of Bogus Fees
FTC has filed a lawsuit for overcharges.
Average Rating:
Product Information and Resources for Technology You Can Use To Boost Your Business

Network Security Spotlight
Researchers Working To Fix Tor Security Exploit
Developers for the Tor privacy browser are scrambling to fix a bug revealed Monday that researchers say could allow hackers, or government surveillance agencies, to track users online.
 
Wall Street Journal Hacked Again
Hacked again. That’s the story at the Wall Street Journal this week as the newspaper reports that the computer systems housing some of its news graphics were breached. Customers not affected -- yet.
 
Dropbox for Business Beefs Up Security
Dropbox is upping its game for business users. The cloud-based storage and sharing company has rolled out new security, search and other features to boost its appeal for businesses.
 

Enterprise Hardware Spotlight
Watson Gets His First Customer Service Gig
Since appearing on Jeopardy, IBM's Watson supercomputer has been making a living using his super-intelligent knowledge base for business verticals. Now, Watson's been hired for his first customer service job.
 
Tablet Giants Apple and Samsung Feel the Heat
When a company saturates its home market with a once-hot product, expect it to pump up efforts elsewhere. Apple, for its part, is now pushing iPads to big corporations and the enterprise market.
 
Microsoft Makes Design Central to Its Future
Over the last four years, Microsoft has doubled the number of designers it employs, putting a priority on fashioning devices that work around people's lives -- and that are attractive and cool.
 

Mobile Technology Spotlight
T-Mobile Calls 'BS' on AT&T's New Promotion
While Verizon Wireless is moving to throttle bandwidth hogs, a scrappy T-Mobile is taking on the giants with a limited-time promotion it hopes will drive up the churn rates of its wireless rivals.
 
Microsoft Update to Windows Phone 8.1 Already Coming
An update to Windows Phone 8.1 is on the way just weeks after the release of the product itself. Microsoft has begun detailing some of the update features to phone manufacturers.
 
Stanford Researchers Report Battery Breakthrough
Stanford researchers have found a way to use lithium in a battery's anode, a breakthrough that could triple capacity and has been described as the "holy grail of battery science."
 

Navigation
CRM Daily
Home/Top News | CRM Systems | Customer Service | Contact Centers | Business Intelligence | Sales & Marketing | Customer Data | CRM Press Releases
NewsFactor Network Enterprise I.T. Sites
NewsFactor Technology News | Enterprise Security Today | CRM Daily

NewsFactor Business and Innovation Sites
Sci-Tech Today | NewsFactor Business Report

NewsFactor Services
FreeNewsFeed | Free Newsletters

About NewsFactor Network | How To Contact Us | Article Reprints | Careers @ NewsFactor | Services for PR Pros | Top Tech Wire | How To Advertise

Privacy Policy | Terms of Service
© Copyright 2000-2014 NewsFactor Network. All rights reserved. Article rating technology by Blogowogo. Member of Accuserve Ad Network.