Amazon Bulks Up Cloud Instances For Big Data

Shadow IT: 8 Ways To Cope

Amazon Web Services has launched a second generation of dense-storage servers, its D2 series of instances. Storage-optimized servers are designed to give users access to multiple terabytes of data, with the virtual servers and storage tied together with high I/O throughput rates.

The D2s don't necessarily have as much compute power as Amazon's compute-intensive instances, such as the C3s and C4s, nor do they have as much RAM as the memory-intensive R3 series. But they can combine large amounts of both with very large storage resources to take on some of an enterprise's largest data-centric tasks.   

Jeff Barr, Amazon Web Services chief evangelist, said the D2 instances' high sequential I/O rates will let them "chew through your massively parallel processing data warehouse, log processing, and MapReduce jobs. They will also make great hosts for your network file systems and data warehouses," he wrote in a March 31 blog.

There are four members of the D2 family, ranging from the smallest at four virtual CPUs, 30.5 GB of memory and three 2-TB high density disks, to the largest at 36 virtual CPUs, 244 GB of memory, and 24 2-TB high density disks.   

[Amazon will soon disclose its cloud revenues. See Amazon Cloud Business: New Openness, Products, Competition.]

In Amazon's latest round of server instances, a virtual CPU equals one of the two processor threads in an Intel E5-2676 v3 (Haswell) or other modern Xeon core. In the older generation, a virtual CPU amounted to the equivalent of 2007 Xeon running at 1 or 1.2 GHz, and those cores were single-threaded. The Haswell E5-2676 offers two threads running at 2.4 GHz and higher. These second-generation AWS virtual CPUs are delivering more power than their first-generation counterparts.

The four D2 instances, for example, are a follow-on to Amazon's first-generation, high-density storage servers, the Hi1 and HS1 instances. The D2 series is priced at $.69 per hour for d2.xlarge; $1.38 for d2.2xlarge; $2.76 for d2.4xlarge; and $5.52 for d2.8xlarge.

Amazon will continue to offer Hi1s and HS1s for customers who have optimized their applications to those instances, but they start to look pricey compared to the new offering. The HS1 8xlarge was a large server with 16 virtual CPUs representing the earlier Xeon 1 GHz-equivalent cores. The HS1 included 117 GB of memory and 24 2,048 GB hard drives at a price of $4.60 an hour. HS1s are "still fully supported" and retain the same features and functionality, a notice accompanying the D2 announcement said.

When it comes to throughput, the largest D2 instance (d2.8xlarge) can provide up to 3,500 MB per second of data read throughput from disk. It offers 3,100 MB per second of write throughput to disk. The smaller D2 (xlarge) moves data off of disk at 875 MG per second.

When calling data out of Amazon's Elastic Block Store (the temporary storage for running applications), the D2s can supply an optimized throughput of 500 Mbps to 4,000 Mbps, Barr's blog said.

Amazon said D2s will perform best with recent versions of Linux from Suse Enterprise Server 12, Red Hat Enterprise Linux 7.1, Amazon's own Linux AMI 2015.03, or Ubuntu Server 14.04. All four support Persistent Grants needed for optimum disk performance with D2 instances, Barr wrote. Persistent grants are a feature on Linux emerging from the Xen open source hypervisor project.

Amazon's product expansion comes as it prepares to start disclosing revenue and profit/loss for its Amazon Web Services Unit. That financial data should come later this month, for the first quarter of 2015. Its movement toward larger virtual servers demonstrates that big companies are interested in using Amazon infrastructure, and that Amazon sees the enterprise market as attractive. Amazon still offers "micro" servers with one first generation virtual CPU, but in a hierarchy of servers that for the most part goes from small to large, it lists those micro instances last.

Are you exploring your data center strategy in the cloud era? Come discuss this issue with leaders from Boeing and eBay, and explore other strategic IT issues at the InformationWeek Conference. Email InformationWeek editor Chris Murphy with questions or register now and save $200 off Conference passes with discount code CMBLOG. Other speakers include the CIOs of Walmart, AstraZeneca, the NBA, ConocoPhillips, Royal Caribbean, and more.