Chiliad Simplifies Big Data Search

Federal intelligence and law enforcement agencies have long used Chiliad's big data analytics platform to uncover information from a variety of data sources and platforms. Now the Herndon, Va.-based software company hopes to expand the reach of its data analysis tools to less technical users across a wider range of enterprises.

Discovery/Alert 7.0, the latest version of Chiliad's big data search tool, retains its core strength: the ability to query a variety of data sources, both structured and unstructured, across multiple locations. What's new is version 7.0's friendlier, Web-based user interface that allows people to use conversational language to query data. It includes several visualization and analytics tools that help non-technical users find information quickly, see a global ranking of relevant concepts, map information on timelines, and do additional data filtering.

"Let's say I'm a medical director and I'm using conversational queries. I'm very educated and intelligent, but I'm not a data scientist," said Rick Brown, senior director of product marketing at Chiliad, during an online demo of Discovery/Alert 7.0 for InformationWeek.

"When you do the query, instead of just getting a list of results with text links, you get various visualizations and analytics tools to help you find what you're looking for," said Brown.

[ Learn how government agencies are finding creative ways to handle huge government data sets. NASA Issues Big Data Challenge. ]

One of Chiliad's goals is to design data analysis tools that require little or no training to perform basic queries. Discovery/Alert 7.0 plays a key role in the company's mission to expand beyond its core base of government and healthcare enterprises.

"We've got nothing against data scientists. They've been trained in Booleans, and they know all these technical languages. We understand these (languages) also. We just don't require them," said Chiliad VP of marketing Ken Rosen.

"People don't want to read 100 documents. The new user interface is optimized to let you view the thematic concepts underlying your query in one view, and to be able to dive in quickly and extract the knowledge," Rosen said.

The new UI is written in HTML 5, CSS, and JavaScript, and runs on a wide variety of mobile devices and desktop systems. "You can use this on an iPad, just like you can on a PC," said Rosen.

Organizations can also create their own customized user interface that sits on top of Chiliad's analytics engine.

Discovery/Alert 7.0 saves queries and automatically monitors all of an enterprise's data sources for updates. It can alert users if it finds new information relevant to a saved query.

From Chiliad's perspective, one key factor that sets Discovery/Alert 7.0 apart from the competition is its ability to perform big data analytics across highly distributed data.

"People are realizing they can't bring all their data together and analyze it," said Rosen. "It's crazy expensive, takes years and millions of dollars, and flat-out doesn't work a lot of the time."

One area that Chiliad and other big data analytics companies are still working on is the ability to search visual information, such as images and video.

"It's a very experimental field right now. No one is particularly good at it," said Rosen. "The idea of knowing the difference between a 747 and a puppy, we're still not that good at that as an industry."

Today, for instance, Chiliad indexes an image's metadata. "In government intelligence or radiology, someone who's an expert in the field annotates a graphic. Often they're doing it verbally and it gets transcribed. These annotations are attached to the document," said Rosen.

"We index all of those annotations. We don't try to interpret the graphic itself. We're dealing with the alphanumeric data that's associated with it," he said.

Many enterprises are building data warehouses to centralize the ever-increasing information flowing through their organizations into useful repositories. This makes good business sense, but it opens up a slew of concerns from a security standpoint. IT professionals can apply many of the same security best practices used with databases, but there are new lessons to be learned, as well. Download our Securing The Data Warehouse report. (Free registration required.)