Newsletters
Customer Relationship Management News NewsFactor Sites:       NewsFactor.com     Enterprise Security Today     CRM Daily     Business Report     Sci-Tech Today  
   
This ad will display for the next 20 seconds. Click for more information, or
Home CRM Systems Customer Service Contact Centers Business Intelligence More Topics...
UCS Invicta: Integrated Flash
Deploy flash memory technology to
deliver peak workload performance.

Find out more>>
Business Intelligence
Gartner's #1 for endpoint backup
Average Rating:
Rate this article:  
Can Super-Fast Apache Spark Light Up Hadoop?

Can Super-Fast Apache Spark Light Up Hadoop?
By Jennifer LeClaire

Share
Share on Facebook Share on Twitter Share on Linkedin Share on Google Plus

Apache Spark aims at groups that need to tap into machine learning, interactive queries, and stream processing. Spark is fully compatible with Hadoop's Distributed File System, HBase, Cassandra, and any Hadoop storage system, so existing data is immediately available. Spark also promises support for SQL, streaming data and complex analytics.
 


Call it the Hadoop Swiss Army knife of cluster computing frameworks. The Apache Software Foundation just rolled out Apache Spark v1.0, which it's calling a "super-fast, open-source, large-scale data processing and advanced analytics engine."

That's a mouthful, indeed, but why has the technology been dubbed a Hadoop Swiss Army knife? Because Spark lets developers write apps in Java, Scala or Python with a built-in set of more than 80 high-level operators. Apache claims Spark makes possible programs that can run up to 100 times faster than Apache Hadoop MapReduce in memory.

"Apache Spark is an important big data technology in delivering a high-performance analytics solution for the IT industry and satisfying the fast-growing customer demand," said Michael Greene, vice president and general manager of System Technologies and Optimization at Intel.

Who Does this Target?

Apache Spark aims at groups that need to tap into machine learning, interactive queries, and stream processing. Spark is fully compatible with Hadoop's Distributed File System, HBase, Cassandra, and any Hadoop storage system, so existing data is immediately available in Spark. Spark also promises support for SQL queries, streaming data and complex analytics, including machine learning and graph algorithms, right out of the box.

Patrick Wendell, software engineer at Databricks and Apache Spark 1.0 release manager, explained that the new release not only provides long-term stability for Spark's core APIs, it also offers several new features.

"Spark 1.0 adds a unified submission tool for deploying applications on a local machine, Mesos, YARN, or a dedicated cluster," said Wendell. "We've added a new module, Spark SQL, to provide schema-aware data modeling and SQL language support in Spark. Spark's machine learning library, MLLib, has been enhanced with several new algorithms. Spark's streaming and graph libraries have also seen major updates. Across the board, we've focused on building tools to empower the data scientists, statisticians and engineers who must grapple with large data sets every day."

NASA Is All-In

Originally developed at UC Berkeley AMP Lab, Spark is in use by companies like Alibaba, ClearStory Data, Cloudera, Databricks, IBM, Intel, MapR, Ooyala and Yahoo. Beyond enterprise adoption, Apache Spark is also winning code contributors to the project.

Chris Mattmann, who is an Apache Software Foundation (ASF) director and chief architect in the Instrument and Science Data Systems Section at the National Aeronautics and Space Administration Jet Propulsion Laboratory, said NASA is excited to leverage Spark and its "highly interactive analytic capabilities." He also pointed to the speedups 1.0 offers, and said Spark SQL as helpful to critical projects looking at measurement of snow in the western U.S., as well as on projects related to regional climate modeling and in model evaluation for the National Climate Assessment.

"I'm looking forward to designing Spark-related projects in my software architectures and in my search engines courses at USC as well," said Mattmann, who also is an adjunct associate professor there. "The community is one of our most active at the ASF, and the interest has really peaked and these guys are doing a great job."
 

Tell Us What You Think
Comment:

Name:



UCS Invicta: Integrated Flash Why wait for the future? Unlock the potential of your applications and create new business opportunities today with UCS Invicta Series Solid State Systems. Take advantage of the power of flash technology. See how it can help accelerate IT, eliminate data center bottlenecks, and deliver the peak application performance and predictability your users demand. Click here to learn more.


 Business Intelligence
1.   Facebook Offers Cross-Device Analytics
2.   Splunk Cuts Prices, Vows 100% Uptime
3.   IBM Beefs Up Identity Intelligence
4.   Brain Waves: the New Focus Group?
5.   Chief Customer Officers' Clout Grows


advertisement
Facebook Offers Cross-Device Analytics
Marketers can track ad conversions.
Average Rating:
Brain Waves: the New Focus Group?
Customer data gets more scientific.
Average Rating:
IBM Beefs Up Identity Intelligence
To offer biz better security products.
Average Rating:
Product Information and Resources for Technology You Can Use To Boost Your Business

Network Security Spotlight
Cost of Target Data Breach: $148 Million Plus Loss of Trust
The now infamous Target data breach is still costing the company -- and its shareholders -- plenty. In fact, the retailing giant forecast the December 2013 incident cost shareholders $148 million.
 
Aruba Networks Handles Black Hat with Aplomb
It's not an easy job. Aruba Networks' task throughout the Black Hat USA conference in Las Vegas this month was to ensure thousands of attendees could connect without malicious attacks.
 
Chinese Hackers Nab Info on Millions of U.S. Patients
A group of Chinese hackers has stolen the personal information, including names and Social Security numbers, of about 4.5 million patients at hospitals operated by Community Health Systems.
 

Enterprise Hardware Spotlight
Three New Lenovo PCs Aimed at Business Users
Businesses everywhere want computing solutions that do more for less money, and Lenovo has unveiled three new desktop PCs that offer solid computing at a budget-minded price.
 
Aruba Networks Handles Black Hat with Aplomb
It's not an easy job. Aruba Networks' task throughout the Black Hat USA conference in Las Vegas this month was to ensure thousands of attendees could connect without malicious attacks.
 
Compression, Deduplication Come to Violin Concerto 2200
Violin Memory has announced that data deduplication and compression capabilities are now available on its Concerto 2200 solution. Typically, users will experience deduplication rates between 6:1 and 10:1.
 

Mobile Technology Spotlight
Apple Stock Soars Ahead of iPhone 6 Launch
The imminent release of the iPhone 6 -- and maybe even an iWatch -- has sent Apple's stock soaring to new heights. Considering what else the firm could have up its sleeve -- the stratosphere may be the limit.
 
HTC Debuts Windows Phone Version of One M8 Smartphone
HTC is bringing the Windows Phone mobile OS to its flagship One M8 device -- the first time any mainstream flagship smartphone has been offered with a choice of operating systems.
 
Verizon Earns Top Rating in Mobile Network Comparison
A new report says Verizon Wireless was the top-performing U.S. cellphone service provider in the first half of 2014, on a nationwide and state-by-state basis, as well as in metro areas.
 

Navigation
CRM Daily
Home/Top News | CRM Systems | Customer Service | Contact Centers | Business Intelligence | Sales & Marketing | Customer Data | CRM Press Releases
NewsFactor Network Enterprise I.T. Sites
NewsFactor Technology News | Enterprise Security Today | CRM Daily

NewsFactor Business and Innovation Sites
Sci-Tech Today | NewsFactor Business Report

NewsFactor Services
FreeNewsFeed | Free Newsletters

About NewsFactor Network | How To Contact Us | Article Reprints | Careers @ NewsFactor | Services for PR Pros | Top Tech Wire | How To Advertise

Privacy Policy | Terms of Service
© Copyright 2000-2014 NewsFactor Network. All rights reserved. Article rating technology by Blogowogo. Member of Accuserve Ad Network.