Microsoft Azure Data Lake Offers Enhanced Analytics Tools - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

02:05 PM
Connect Directly

Microsoft Azure Data Lake Offers Enhanced Analytics Tools

An expansion of Microsoft's Azure Data Lake will include a new analytics service, and it will be ready for public preview by the end of the year.

10 Cloud Storage Options For Your Personal Files
10 Cloud Storage Options For Your Personal Files
(Click image for larger view and slideshow.)

Microsoft is building on its cloud offerings with an expanded Azure Data Lake, arriving with analytics tools designed to simplify big data, and with a new query language.

We first learned about the Azure Data Lake when Microsoft first announced it at the Build conference back in April. The data repository handles data of any size, type, and speed. It eliminates the complexities of processing and storing data while it makes it easier for businesses to get up and running with analytics.

The Azure Data Lake Store, as it has been renamed, will store structured, semi-structured, and unstructured data without forcing application changes as data scales. Data located in the Data Lake Store can be securely shared. It is also accessible from sensors connected to the Internet of Things.

[Office 2016, Windows 10 in China, and more from Microsoft's last week.]

According to a blog post published Sept. 28, the Azure Data Lake Store supports development of big data solutions through a variety of languages and frameworks. The new store works with the Hadoop Distributed File System (HDFS), so Hadoop tools like Hortonworks, Cloudera, and MapR can get the needed data for processing.

Microsoft also today announced Azure Data Lake Analytics, a cloud-based data processing and analytics service. The tool is built on Apache YARN. It scales instantly according to the power needed for each job. It's also cost-efficient; customers only pay for jobs when those jobs are running.

Azure Data Lake Analytics includes U-SQL, a new and scalable query language built on the same runtime that powers Microsoft's big data systems. With U-SQL, users can process queries to analyze data located in the Azure Data Lake Store, as well as information stored on SQL Servers in Azure, Azure SQL Database, and Azure SQL Data Warehouse.

T.K. "Ranga" Rengarajan, corporate vice president of data platforms at Microsoft, acknowledges how developers and data scientists struggle to successfully use existing technologies for big data.

"Code-based solutions offer great power, but require significant investments to master, while SQL-based tools make it easy to get started but are difficult to extend," he wrote on Microsoft's TechNet blog. "We've faced the same problems inside Microsoft and that's why we introduced U-SQL, a new query language that unifies the ease of use of SQL with the expressive power of C#."

(Image: Microsoft)

(Image: Microsoft)

Both the Azure Data Lake Store and Azure Data Lake Analytics will be available in preview later this year, Microsoft reports.

Microsoft adds the Azure Data Lake is supported by Azure Data Lake Tools for Visual Studio, which have been designed to foster an integrated development environment across the Azure Data Lake. It's also supported by Hadoop ISV applications spanning security, governance, data preparation, and analytics that can be deployed from the Azure Marketplace.

Ready today is HDInsight, the Apache Hadoop-based series included in Azure Data Lake that works with analytics services like Hive, Storm, HBase, and Spark. Managed clusters on Linux are now generally available, Microsoft reports.

Kelly Sheridan is the Staff Editor at Dark Reading, where she focuses on cybersecurity news and analysis. She is a business technology journalist who previously reported for InformationWeek, where she covered Microsoft, and Insurance & Technology, where she covered financial ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
11/2/2015 | 8:24:42 AM
Re: Microsoft is too late. SQL and all other programming languages are over.
Yes, and Microsoft Access 1.0 will eliminate the need for DBAs and programmers in 1992. 

To date, every prediction that some technology will "end the need for specialists" has resulted in massive adoption and massive increase in the need for specialists. 
Charlie Babcock
Charlie Babcock,
User Rank: Author
9/28/2015 | 6:58:18 PM
The new 'open source' Microsoft
Microsoft's adoption of open source code as the basis for some of its analytics initiatives is impressive. And it's fair to say it would fall behind if it weren't adopting pieces like YARN and interfacing to Hadoop systems. Something worth watching: in addition to what it adopts, how much does it give back?
Think Like a Chief Innovation Officer and Get Work Done
Joao-Pierre S. Ruth, Senior Writer,  10/13/2020
10 Trends Accelerating Edge Computing
Cynthia Harvey, Freelance Journalist, InformationWeek,  10/8/2020
Northwestern Mutual CIO: Riding Out the Pandemic
Jessica Davis, Senior Editor, Enterprise Apps,  10/7/2020
White Papers
Register for InformationWeek Newsletters
2020 State of DevOps Report
2020 State of DevOps Report
Download this report today to learn more about the key tools and technologies being utilized, and how organizations deal with the cultural and process changes that DevOps brings. The report also examines the barriers organizations face, as well as the rewards from DevOps including faster application delivery, higher quality products, and quicker recovery from errors in production.
Current Issue
[Special Report] Edge Computing: An IT Platform for the New Enterprise
Edge computing is poised to make a major splash within the next generation of corporate IT architectures. Here's what you need to know!
Flash Poll