Including planning and implementation, OJP spent about $9 million and about 16 months upgrading two key systems -- its grants management system and its grants draw-down system -- from outdated infrastructure hosted in its own offices to a dynamically virtual environment running the latest software out of two DOJ data centers in Rockville and Dallas.
"Now, we have a lot of flexibility," Santa said. "At any given time, if I only need to allocate three servers to a system, I can say, let's go ahead and provide the resources to support a smaller amount of need, and abstract the other resources to support some other kind of need. This is all about enabling agility by abstracting computer infrastructure."
Before the upgrade, the grants management and drawdown systems were running on 6 to 10 year old infrastructure with old applications, and were running into power and space limitations. The issue was especially pressing as Recovery Act money kicked in this year, forcing OJP to manage double the dollar value of grants it handles in a typical given year, Angel Santa, OJP's deputy CIO, said in an interview.
The agency's grants management system went live on the new infrastructure in May, with the grants drawdown system following a little more than a month later. Santa said that, over time, OJP will move the balance of its IT systems (including contracting and procurement systems) to the virtual environment. OJP will also have the systems run in an active-active environment, whereas today one of the data centers where the systems are hosted acts as a backup and testing facility.
As part of the upgrade, OJP went with VMware hypervisors and virtualization management software, added F5 tools for load balancing and tools for live upgrades, installed Oracle RAC and Grid products, upgraded from Oracle 9 to Oracle 11g Release 2, implemented EMC real-time replication products, and installed a new Web application firewall that can do deep packet inspection. Funding for the project came from some money the DOJ had set aside for consolidation as well as Recovery Act funds and the OJP operating budget.
According to Santa, when the planning for the upgrade began, there was discussion as to whether to put the systems in a public cloud environment, but OJP decided that hosting the systems in DOJ data centers enabled greater control and an ability to monitor systems and access at a deeper level than Santa believed was possible with public cloud services. He also raised concerns over data security.