Government // Leadership
News
2/12/2014
09:30 AM
Connect Directly
RSS
E-Mail
50%
50%

DARPA Pursues Deep Web Search Tools

Defense Advanced Research Projects Agency project Memex will shine light on parts of the Web where commercial engines don't search.

6 Cool Apps From Uncle Sam
6 Cool Apps From Uncle Sam
(Click image for larger view and slideshow.)

The Defense Advanced Research Projects Agency (DARPA) wants to develop the next generation of search technologies, particularly those that can help the military and government agencies find and organize publicly available information on the Internet.

To address the challenges of a one-size-fits-all approach used by today's search engines, DARPA has kicked off a program called Memex, short for memory and index. The program aims to "revolutionize the discovery, organization, and presentation of search results" and "extend the reach of current search capabilities," according to DARPA. Memex will focus on three technical areas: domain-specific indexing, domain-specific search, and applications that pertain to the Department of Defense (DOD).

"We're envisioning a new paradigm for search that would tailor indexed content, search results, and interface tools to individual users and specific subject areas, and not the other way around," DARPA program manager Chris White said in a written statement.

[Memex is just one of DARPA's data development projects. Read DARPA Opens Software, Data To Public.]

DARPA believes current search capabilities miss information in the "deep Web" -- the parts of the Internet that aren't indexed by commercial search engines. They also overlook shared content across Web pages. The agency envisions using commodity hardware and an open-source architecture to build this advanced search technology that would be capable of cross-referencing information more quickly and efficiently.

Memex was inspired by a hypothetical device described in "As We May Think," a 1945 article for The Atlantic Monthly written by Vannevar Bush, a World War II-era director of the Office of Scientific Research and Development (OSRD). DARPA said it wants to improve the ideas described in that article, with the creation of domain-specific Web content indexing and search.

(Image: Montage courtesy of DARPA)
(Image: Montage courtesy of DARPA)

Initially, the program will focus on fighting human trafficking, a problem that touches various parts of government, including the military, law enforcement, and intelligence agencies. The commercial sex trade in particular has garnered significant Web presence via forums, chats, advertisements, and job postings. "An index curated for the counter-trafficking domain, along with configurable interfaces for search and analysis, would enable new opportunities to uncover and defeat trafficking enterprises," DARPA said.

Earlier this month, DARPA issued a solicitation for research on the subject. Research proposals, due on April 8, must address approaches that "enable revolutionary advances in science, devices, or systems," the agency said. Procurement contracts and cooperative agreements, but not grants, will be awarded to the winners. DARPA will hold a conference on February 18 to discuss the technical details of Memex with program participants.

Solid state alone can't solve your volume and performance problem. Think scale-out, virtualization, and cloud. Find out more about the 2014 State of Enterprise Storage Survey results in the new issue of InformationWeek Tech Digest.

Elena Malykhina began her career at The Wall Street Journal, and her writing has appeared in various news media outlets, including Scientific American, Newsday, and the Associated Press. For several years, she was the online editor at Brandweek and later Adweek, where she ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
WKash
50%
50%
WKash,
User Rank: Author
2/12/2014 | 10:32:10 PM
Catching up
Given DARPA's annual budget is dwarded by Google's and Microsoft's resources, it should be interesting to see what comes of DARPA's efforts to develop a deeper search tool.

 
Thomas Claburn
50%
50%
Thomas Claburn,
User Rank: Author
2/12/2014 | 7:14:55 PM
the dark web
If DARPA is missing too much of the deep web, someone could always ask the NSA...
2014 US Salary Survey: 10 Stats
2014 US Salary Survey: 10 Stats
InformationWeek surveyed 11,662 IT pros across 30 industries about their pay, benefits, job satisfaction, outsourcing, and more. Some of the results will surprise you.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest September 18, 2014
Enterprise social network success starts and ends with integration. Here's how to finally make collaboration click.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
The weekly wrap-up of the top stories from InformationWeek.com this week.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.