Expanding on its existing resale arrangement with Diligent Technologies, Hitachi Data Systems (HDS) is now shipping preconfigured virtual tape libraries combining Diligent's ProtecTEIR software and HDS' AMS modular Fibre Channel disk array. Based on customer feedback that indicated customers would rather buy, and VARS would rather sell, preconfigured systems than server, software, disk storage, and the inevitable professional services separately.The new Virtual Tape Library Appliance line starts with the Model 500M, which uses a dual processor, dual-core server to backup data at 200 MBps on an AMS-500 array with 8 TB to 20 TB of usable disk space. The 15 TB to 30 TB 1000L and 30 TB to 50 TB 1000E use four processors and the beefier AMS1000 array to achieve 300 MBps and 400 MBps performance, respectively.

With a capacity of more than 1 PB of backup data at a 20:1 de-duplication ratio, which is in line with the real world de-duping environments I've worked on, the 50 TB 1000E is surely enterprise scale. Enterprises may have more trouble with the 400 MBps backup rate than capacity.

ProtecTEIR does real-time de-duplication and uses a de-duplication algorithm that includes byte by byte comparisons before declaring two data blocks to be identical, avoiding any possibility, regardless of how unlikely, of a hash collision corrupting data. Unlike some other VTL vendors, ProtecTEIR can't cluster multiple server nodes to access the same de-duplicated data repository.

Hitachi supports VTL to VTL replication by using the storage array's replication functions. While this will replicate de-duplicated data efficiently using bandwidth, administrators will have to manage separate user interfaces and management tools for the array and the VTL server.

Is it time to declare de-duplication mainstream? I think, like a recession, you can only see such inflection points as mainstream acceptance in the rear-view mirror.