Healthcare // Policy & Regulation
News
3/14/2014
11:36 AM
Connect Directly
LinkedIn
Google+
Twitter
RSS
E-Mail
100%
0%

Cover Oregon Diagnosis: Oracle Overdose

Federal critique of Oregon's Obamacare health insurance exchange finds project overly dependent on Oracle consultants, architecture bloated with Oracle software.

 7 Big Data Solutions Try To Reshape Healthcare
7 Big Data Solutions Try To Reshape Healthcare
(click image for larger view and for slideshow)

In Oregon, the Affordable Care Act became the Oracle Full Employment Act -- meaning it employs Oracle software in every nook of the architecture and relies on Oracle consultants to implement it with minimal oversight, according to a federal assessment of the state's dysfunctional health insurance exchange.

While the federal HealthCare.gov website and many state-based insurance exchanges stumbled at launch in October, and some state sites are still struggling, only Oregon is still without a website where consumers can shop for and enroll in plans. The deadline for open enrollment is March 31, after which consumers who do not have insurance could be subject to penalties. Oregon residents have been forced to apply on paper or go through insurance agents and other intermediaries to get coverage. The only Web applications delivered so far have been for those intermediaries -- and even they have been plagued with errors.

As reported by The Oregonian newspaper, even as Oregon's Congressional representatives and senators from both parties are calling for a federal investigation, one was completed on Feb. 27 at the request of the Centers for Medicare and Medicaid, which gave the state $305 million in grants for the project. The review was conducted by the MITRE Corporation, a federally funded research and development center that provides technical assistance to many government agencies, through the CMS Alliance to Modernize Healthcare. A spokesperson for Cover Oregon, the organization managing the exchange, told The Oregonian that the report's findings are not news and are already being addressed. Oracle declined to comment.

[Does Obamacare jeopardize the security of patients' health data? Read Obamacare Vs. Patient Data Security: Ponemon Research.]

Under a recently announced transition agreement, Oregon paid Oracle $43.5 million it has been withholding for poor performance, enough to get cooperation on transitioning the project to another consulting firm, yet to be named. The state withheld another $25.5 million and reserved its right to litigate and try to recover any or all of the $160 million Oracle has been paid or claims it is still owed, according to The Oregonian.

Extricating the state from dependency on Oracle won't be easy. One of the audit's primary findings was that the system architecture was overloaded with Oracle software. In an abbreviation-packed critique of Cover Oregon (CO) and Oracle Consulting Services (OCS), the report places ultimate blame on CO for failing to have adequate contracting processes in place but also faults Oracle for poor performance and transparency, meaning that the company failed to provide information needed to remedy issues with website and contract performance.

"The CO solution stack is comprised of almost every single component/product in the Oracle product portfolio. While there are other State Based Marketplaces utilizing elements of the Oracle product portfolio, CO has, without doubt, the most complex mix," the federal report read. One of the reasons the Cover Oregon Web applications have been so unstable, according to the report, has been "the management of the many sessions created by Siebel, Oracle Real Application Clusters (RAC), Service Oriented Architecture (SOA) and Web Center that consume a significant amount of CPU and memory." Servers regularly ran out of memory and needed to be rebooted.

Getting the system working is likely to require a significant reexamination -- and simplification -- of the system architecture, according to the report.

While the report focuses on Cover Oregon, one reason for some of this disorganization in the early phases of the project may have been the division of labor between that agency and the Oregon Health Authority, which provided IT project management particularly in the early phases of the planning. An Oracle marketing case study called "Architects of Reform," published in 2012, celebrated the project as a triumph of Enterprise Architecture, a holistic approach to mapping out enterprise systems and the relationships between them. The original plan was to make the Cover Oregon website and back end systems part of a broader redesign of the state's health and welfare systems around a complete suite of Oracle Software.

Even as the state relied on Oracle to put all the pieces together, the CMS report found OCS "lacking in resources with expertise in Oracle Configuration Management (OCM), which has contributed to the delays experienced at the outset of the CO implementation."

The Cover Oregon project is highly dependent on Oracle consultants, "many of whom do not have extensive knowledge and experience in a formal software development life cycle," according to the report. "The lack of any formal SDLC has had a significant contribution to the root cause of the delay in the delivery of the CO solution," it continued.

Cover Oregon has had little visibility into the configuration of the system. "While OCS does utilize SVN for code management, there is no formal tool in place for system build configuration management. Without this insight into the environments, CO faces challenges in being able to manage their own system and/or bringing other vendors on board if a future decision to terminate the OCS contract is made," read the report. In particular, any release featuring changes to Siebel pose problems because, according to the report, Siebel has no code branching (revision control) capabilities. "If there are any problems with a build installation, the system cannot be restored to the previous state in an efficient manner. Rather, the Siebel environment has to be re-built and deployed to the previous version which, in its own right, has not been seen to operate in a consistent manner."

Meanwhile, Cover Oregon's weak contract administration and monitoring has resulted in ongoing confusion and misunderstanding about what OCS was expected to deliver, the authors found, citing the example of performance testing. The state is dependent on OCS for schedules, fixes, and releases, yet "it appears that CO does not have any leverage in their contract to make OCS accountable for missing key deliverables."

Project management, meanwhile, has been spotty, with no overarching dedicated project manager. Even though the project team professes to be following an agile methodology, the federal critique found it should be following a more disciplined approach under the leadership of an experienced scrum master.

Other findings include:

  • Oracle has often refused to turn over requested information about programming changes, system performance, and testing to the state or to other contractors working on the project. The report also faults Cover Oregon for a lack of transparency, saying it has not done a good job of informing CMS on the project's status.
  • The project has been hampered by "inconsistent configuration management across the various system environments," including a tangle of development, test, and production environments. Despite some attempts to reconcile these, the report authors wrote, "There is still a need for additional environments: true development and true integration test. Not having these separate environments affects the developers and testers abilities to conduct thorough and consistent testing across the suite of environments. "
  • Despite the use of Atlassian's JIRA issue tracker, bugs recorded as fixed have tended to reemerge, suggesting problems with how the tool is being used.
  • Cover Oregon is not using Microsoft Project (or any other mainstream project management tool) in order to track schedule, etc. "As such, the overall project tracking process is cumbersome and difficult for data analysis or to trace any history of updates."
  • The project schedule failed to account for overall project activities other than IT, and the technology plan lacked a detailed implementation plan for each release.

The state was already working on a transition plan when the CMS report was published, and the authors said they "highly recommended" the state seek out a "more appropriate" technology partner or partners. To help explore alternatives, Oregon hired to Deloitte, which worked on some of the more successful health insurance exchange projects in Connecticut, Kentucky, Rhode Island, and Washington -- states that have tended to keep their system architecture relatively simple and their ambitions modest.

With so much rework needing to be done, the report notes Cover Oregon "is burning their funding rapidly with the addition of staff hired to assist with the manual processes and has not planned anything beyond the full individual launch Release 1.1." With state officials saying that finding additional money will be "financially challenging," the report adds, "CMS is committed to working with States to ensure that individuals get health insurance coverage which may include additional funding for IT development."

But if that money comes, federal officials want to make sure it is better spent.

Download Healthcare IT in the Obamacare Era, the InformationWeek Healthcare digital issue on the impact of new laws and regulations. Modern technology created the opportunity to restructure the healthcare industry around accountable care organizations, but IT priorities are also being driven by the shift.

David F. Carr oversees InformationWeek's coverage of government and healthcare IT. He previously led coverage of social business and education technologies and continues to contribute in those areas. He is the editor of Social Collaboration for Dummies (Wiley, Oct. 2013) and ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Pabodie
50%
50%
Pabodie,
User Rank: Apprentice
4/28/2014 | 11:07:38 PM
Re: What's your diagnosis of Oracle's role in the Cover Oregon health insurance exchange?
My head is sore from slapping. After, like a lot of folks, I was exposed to his story by John Oliver's show, I fished for a few good articles about the troubles to see what was real and what was babble. I'm stunned to find that the most damning tidbits turn out to be facts. $300 million for a website? Let's just ignore Oregon's population, and the expected user base, and the interfaces that would need to be implemented with Siebel, et al. $300 million? If Oracle's stack really costs that much to implement over the course of 2 years, shame on them. Even $100 million would be utterly outrageous. But for the state to be suckered into this looting? Sounds like we were taken for one of the greatest rides in the history of conswindling. To wit: "Would I have liked to have the Ferrari day one?" [Exec Dir Rocky King] said. "You betcha. Is it a surprise to me? No." Based on other states' experiences, King called Oregon's scaled-down website the right decision. OMGWTFBBQ. Time to start doomsday prepping.
jries921
50%
50%
jries921,
User Rank: Ninja
3/15/2014 | 2:57:33 PM
Money politics?
I remember years ago it came out that the State of California under Governor Gray Davis had more Oracle licenses than it did employees (the number was subsequently substantially reduced).  It was theorized at the time that this was a political thank you to Larry Ellison, but I don't know how much investigation was done.  It's possible that money politics is rearing its ugly head in this case also.  It's clear that an investigation is necessary, but the proper forum is the Oregon Legislative Assembly with some backup from Congress to the extent that federal money was used in the project.


I'm guessing that some changes to procurement practices will come from this.

 
Somedude8
50%
50%
Somedude8,
User Rank: Ninja
3/15/2014 | 12:04:23 PM
Too Much Money
Projects can have too much budget, this might be one. Having poked around a few of the State websites, I don't see how these budgets of the size being put together can be justified. I am not talking about slightly inflated budget, but order of magnitude.

I sense a lot of "the stew really has to be done in 2 hours, so instead of 1 chef in the kitchen, lets put 100".

Maybe I missed some big chunks, but what I have seen of these sites, we are talking 12-15 person team, 4-6 months. These 8 digit budgets are just completely nuts.
David F. Carr
50%
50%
David F. Carr,
User Rank: Author
3/14/2014 | 5:00:28 PM
Re: What's your diagnosis of Oracle's role in the Cover Oregon health insurance exchange?
To me, the most damning thing in the report is that Oracle Consulting Services wasn't particularly good at configuring Oracle products to work with other Oracle products. Yes, you might assume that products from the same vendor would work well together, and that consultants employed by the vendor would be the best experts you could get to configure and integrate them.

Doug makes a good point that the report shouldn't be taken as gospel and may contain its own biases. Still, it's a scorcher
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
3/14/2014 | 4:50:03 PM
Lack of transparency, over sight; shades of Montclair State University
Montclair State University in N.J. had some similar problems as it tried to automate student registration and other campus functions. There's an element of the two sides not being able to talk to each other, combined, I think, with some arrogance toward the customer that brings these disputes and failures about. See  http://www.informationweek.com/applications/oracle-sued-over-alleged-erp-project-cost-overruns/d/d-id/1097907
jds83210
50%
50%
jds83210,
User Rank: Apprentice
3/14/2014 | 2:36:14 PM
Re: What's your diagnosis of Oracle's role in the Cover Oregon health insurance exchange?
If Oracle is running the project, yeah they are going to use Oracle products, everyone knows this.  Oracle products work best with other Oracle products.  This project screems poor project managment, no change controls, poor communication amoung team members, and a lack of leadership.  It sounds like there was no lead systems artichect and is classic too many IT egos getting in the way.  IT egos are solved by good project management who can make these egos work together for the common goal. 
David F. Carr
100%
0%
David F. Carr,
User Rank: Author
3/14/2014 | 1:40:11 PM
Re: What's your diagnosis of Oracle's role in the Cover Oregon health insurance exchange?
The implication seems to be that Oracle took advantage of the opportunity to pack every element of its portfolio into this project, whether it was the best match for the job or not. On the other hand, a friend I discussed this story with made the point that it's Oracle's job to sell as much software as possible -- and the customer's job to know when to say no.
David F. Carr
50%
50%
David F. Carr,
User Rank: Author
3/14/2014 | 1:36:58 PM
Baltimore switching to Connecticut platform
Story of another troubled project: Maryland may dump its health insurance exchange in favor of one developed for Connecticut, says Baltimore Sun http://buff.ly/1kRtk8d 
D. Henschen
IW Pick
100%
0%
D. Henschen,
User Rank: Author
3/14/2014 | 1:33:40 PM
Re: What's your diagnosis of Oracle's role in the Cover Oregon health insurance exchange?
Keep in mind that this is a federally funded report, and we saw the same "blame the vendor" reflexes in Minnesota when it came to that state's dealings with IBM. I'm not at all surprised to see an Oracle-centric product mix in an Oracle-led project. The money line in the federal report is this: "...there are other State Based Marketplaces utilizing elements of the Oracle product portfolios, without doubt, but CO has, without doubt, the most complex mix." What I would want to know is what are the pieces that are proven in this role and which, if any, pieces seem to be ill-suited to the task at hand.

Siebel plays in plenty of high-scale, consumer-facing Web environments, but what's really surprising here is the lack of project and development management capabilities and controls -- if the federally funded analysis is to be trusted. CO is ultimately responsible for making sure that it delivers what's promised. The question I have is did it really vet the technologies and vendors the successful state-based marketplaces have used?
David F. Carr
50%
50%
David F. Carr,
User Rank: Author
3/14/2014 | 12:00:28 PM
What's your diagnosis of Oracle's role in the Cover Oregon health insurance exchange?
I tried to pull out the details most interesting to IT executives and project managers, but I'd be interested in your analysis of what's most significant in the federal report on what went wrong.

The journalists at the Oregonian deserve credit for uncovering this report, the existence of which state officials had been trying to keep quiet. TV station KATU dug up the Oracle case study I cite, mentioned in a January feature on how the state's grand vision complicated the project.
Research: Healthcare IT Priorities
Research: Healthcare IT Priorities
Meeting regulatory requirements barely inched out managing digital patient data as the top priority for our 363 healthcare provider IT pros.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek - July 21, 2014
Our new survey shows fed agencies focusing more on security, as they should, but they're still behind the times with cloud and overall innovation.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
In this special, sponsored radio episode we’ll look at some terms around converged infrastructures and talk about how they’ve been applied in the past. Then we’ll turn to the present to see what’s changing.
Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.