Feds Face 'Big Data' Storage Challenge - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Software // Information Management
11:25 AM
Connect Directly

Feds Face 'Big Data' Storage Challenge

Federal data centers are filling up with data as terabytes accumulate into petabytes. Agencies must adapt their storage architectures and policies to optimize it all.

InformationWeek Government  Digital Supplement - June 2012
InformationWeek Green
Download the entire June 2012 InformationWeek Government supplement, distributed in an all-digital format as part of our Green Initiative
(Registration required.)

Big Storage

At $78.9 billion, the federal government's proposed IT budget for fiscal year 2013 is 0.7% less than the current budget. That's four years in a row that the federal IT budget has been flat, but there's been no letup in the growth of data or the need to store it. During this same period, the feds' data storage requirements have been growing 30% to 40% a year, gobbling up scarce IT funding.

We hear a lot about the imperative to reduce costs and make the federal government more efficient--data center consolidation and "cloud first" are two prominent examples--but less has been said about a key technology: storage.

But the move to cloud computing and the growing importance of big data and information sharing are creating a sense of urgency around data storage. Networked storage systems such as network-attached storage and storage area networks are part of the answer, but they're not enough.

Most agencies have adopted a tiered storage architecture that involves different technology components for different functions and data sensitivity. This architecture includes various physical media (NAS, SAN, and others), as well as policies and services that govern functions within the storage environment.

Some data typically requires real-time or near-real-time access, which can be accommodated by the architecture. With NAS, data files are stored and accessed using standard file systems and protocols, including Network File System for Linux and Unix clients and Common Internet File System for Windows clients. Agencies may choose to spend more on SANs, where data files are stored on highly available devices. Magnetic hard disk drives that store data are typically architected in a way to include highly available RAID. Solid-state disk drives, which are extremely costly, are reserved for data that requires very fast access. For budget-strapped agencies, solid state is largely out of reach.

For slower data access, at significantly lower costs, magnetic tape drives in robotic libraries can handle archived data or data that's protected as part of a backup strategy. They're also used to archive files no longer needed in operational environments but that must be retained in accordance with the federal electronic records retention policy.

Your archiving requirements should govern the movement of data files from fast-access storage devices to slow-access devices. This will help reduce the growth of the faster, but more expensive, storage systems. As needed, data files moved to a tape archive can always be copied back to fast-access data storage using a data recovery service.

A backup service copies data to an indexed storage location on slow-access devices such as magnetic tape drives. These copies are stored for data protection purposes. If the files are needed because of accidental deletion or corruption, they can be recovered.

For agencies that need continuous data protection, a replication service copies all changes to data and files from one storage subsystem to a secondary subsystem. This service can provide a zero recovery point objective to ensure that agencies never miss a beat. A recovery service copies files from the archive or the backup storage location to the operational system with fast-access storage. Recovery also enables data logged during replication to be retrieved from the logs and copied to the operational system.

All of these services require storage media and data governance policies. With the booming growth of data across government, anything less than a well-conceived plan will drain an agency's IT budget.

To read the rest of the article,
Download the June 2012 InformationWeek Government supplement

The New Threat Landscape

Our report on federal government cybersecurity is free with registration.

This includes 26 pages of analysis and 15 charts. What you'll find:
  • Rankings of most-serious cyberthreats and top initiatives
  • Federal IT pros' assessments of their agenices preparedness
  • Key areas of investment to bolster defenses
Get This And All Our Reports

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Augmented Analytics Drives Next Wave of AI, Machine Learning, BI
Jessica Davis, Senior Editor, Enterprise Apps,  3/19/2020
How Startup Innovation Can Help Enterprises Face COVID-19
Joao-Pierre S. Ruth, Senior Writer,  3/24/2020
Enterprise Guide to Robotic Process Automation
Cathleen Gagne, Managing Editor, InformationWeek,  3/23/2020
White Papers
Register for InformationWeek Newsletters
Current Issue
IT Careers: Tech Drives Constant Change
Advances in information technology and management concepts mean that IT professionals must update their skill sets, even their career goals on an almost yearly basis. In this IT Trend Report, experts share advice on how IT pros can keep up with this every-changing job market. Read it today!
Flash Poll