Intelligence Agency Boosts Search Capability - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Software // Enterprise Applications

Intelligence Agency Boosts Search Capability

Detailed context is a differentiator for Endeca.

The U.S. Defense Intelligence Agency, the foreign military intelligence arm of the Department of Defense, is expanding its use of cutting-edge search technology to help military, defense policy makers, and combat strategists make better-informed decisions.

DIA analysts use Endeca Technologies' Information Access Platform to search 20 sources of intelligence gathered by agents in the field. Weeks from now, the search engine's reach will be extended to intelligence gathered through the interception of radio and other signals, news feeds such as Reuters, and "message traffic" from the State Department, such as e-mail, instant messaging, and file transfers. The State Department represents "our largest authoritative repository of database information about foreign military capabilities," says Lewis Shepherd, chief of requirements and research for the DIA, which has 11,000 military and civilian employees worldwide.

DIA analysts have access to as many as 300 sources of information--individual data feeds, databases, and data from other intelligence agencies. The 20 sources Endeca is able to search represent more than half of all the agency's data, and the agency plans to add more data sources to Endeca's list over the next two years. The agency also uses search technology from Autonomy, Google, and Vivisimo.

Search is an important tool in intelligence work. "Search injects a note of serendipity into the information-finding business," says IDC analyst Sue Feldman. "Exact matching, like in a database application, only gets you what you ask for. In the defense setting, what you're looking for is what you don't know enough to ask for."

Endeca's Information Access Platform
>> MDEX ENGINE Provides interface that sees metadata relationships between pieces of information
>> INFORMATION TRANSFORMATION LAYER Grabs data from disparate sources
>> DATA-DRIVEN APP COMPONENTS Lets developers customize search applications through standard APIs
There's no shortage of search technologies out there. What continues to appeal to the DIA about Endeca is its ability to offer users guided navigation as part of the summaries it delivers. Endeca uses what the company calls a "faceted classification system" to read metadata tags on all the data flowing through its systems and sort query results by any category in any order. The DIA uses XML formatting and metadata-tagging software packages from Attensity and Inxight Software, as well as Lockheed Martin's AeroText and SRA International's NetOwl, to format its data feeds to accommodate Endeca.

Endeca's search results come back to the user in a series of menus. If one were to run a search on the singer Frank Sinatra, for example, the search results might be broken down into a number of categories, including year, record labels, and movies, as opposed to a Google-like listing of all keyword matches. (In Sinatra's case, that's more than 2.7 million matches.) More specific to the DIA, when an analyst searches for information related to a "tank," and the analyst is also searching for information about artillery, Endeca understands then that "tank" refers to a vehicle rather than a container of water. And the search results are prioritized to reflect that.

InformationWeek Download

The DIA started using the Endeca search system early last year, providing its analysts with access to agents' field reports. Thanks to Endeca's ability to place search findings into categories, the DIA not only improved its search capabilities, "we also see we're missing entire areas of information that we need to collect," Shepherd says. The DIA is using its experience with Endeca to help the Office of the Director of National Intelligence improve the navigability of that organization's defense intelligence.

The lessons learned from intelligence failures of the past are hard to ignore. "We take very seri- ously the mandate of the 9/11 Commission and the WMD commission to do vastly better intelligence gathering and analysis," Shepherd says. With so many U.S. troops in the field counting on the DIA, the agency couldn't have picked a better time to improve its search capabilities.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Commentary
Will AI and Machine Learning Break Cloud Architectures?
Lisa Morgan, Freelance Writer,  6/10/2019
Slideshows
9 Steps Toward Ethical AI
Cynthia Harvey, Freelance Journalist, InformationWeek,  5/15/2019
Commentary
Humans' Fascination with Artificial General Intelligence
Guest Commentary, Guest Commentary,  6/6/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
A New World of IT Management in 2019
This IT Trend Report highlights how several years of developments in technology and business strategies have led to a subsequent wave of changes in the role of an IT organization, how CIOs and other IT leaders approach management, in addition to the jobs of many IT professionals up and down the org chart.
Slideshows
Flash Poll