Big Data // Hardware/Architectures
News
10/16/2013
12:58 PM
Connect Directly
Facebook
Google+
RSS
E-Mail
50%
50%

Massively Parallel Processing Finds More Applications

Parallel computing used on massive datasets is popping up everywhere to create everything from artificial brains to real-time video analysis systems.

20 Great Ideas To Steal In 2013
20 Great Ideas To Steal In 2013
(click image for larger view)

The European particle physics laboratory CERN used it to detect the elusive Higgs Boson. Wall Street has used it for years to evaluate oceans of market data in order to make high-frequency trades. And the National Security Agency (NSA), we've recently learned, uses it to pore over billions of our emails and text messages.

But massively parallel processing -- a computing architecture that uses multiple processors or computers calculating in parallel -- has been harnessed in a number of unexpected places, too.

Identifying who is using these novel applications outside of purely scientific settings is, however, tricky. That's because these systems provide "unique insights" that can give their users competitive advantage, Alex Gorbachev, CTO of remote database services and consulting company The Pythian Group told InformationWeekin a phone interview.

[ Can you learn from these companies? Read Big Data Success: 3 Companies Share Secrets. ]

The moment a massively parallel processing solution gains visibility, "people quickly start using it," Gorbachev said.

Real-time OCR

Pythian's own Hadoop and MapReduce application, now under development, is an extension of a home-grown security protocol it uses today, whereby it records the desktops of hundreds of DBAs as they work remotely on client databases.

"It's like the closed-circuit television cameras used in a bank," Gorbachev explained.

Although hundreds of hours of digital recording is useful for after-the-fact forensics, Pythian is now developing an OCR application using Hadoop and Mapreduce that OCRs the screens, producing a searchable, full-text index.

"This makes it very easy to do mining and indexing," Gorbachev said, adding that massive parallelism of the application means it can also detect, in real-time, suspicious text patterns, such as credit cards, social security numbers or other personally identifiable information.

Pythian presenteddetails of its work at the Hadoop Summit earlier this year in San Jose. The company plans to commercialize the OCR capability next year.

DARPA's 'brain-like' RFP

This summer, the U.S. Defense Advanced Research Projects Agency (DARPA) issued an request for proposal(RFP) for technologies related to developing a computer that emulates a human brain, specifically the neocortex, the part of the brain responsible for higher functions such as motor control, language, sensory activity and thought.

The project, which is aimed at developing new approaches for detecting anomalies in large, complex data sets, will depend on neural models of the human neocortex, according to the DARPA researchers.

"The cortical computational model should be fault tolerant to gaps in data, massively parallel, extremely power efficient, and highly scalable. It should also have minimal arithmetic precision requirements, and allow ultra-dense, low power implementations," the request states.

Not surprisingly, DARPA is looking for solutions using massively parallel technologies that can mimic the brain's gift for temporal and spacial recognition, and its ability to solve "extraordinarily difficult recognition problems in real-time."

The $99 Board

For do-it-your-selfers curious about supercomputer-level hardware performance in an affordable package, there's now a $99 board.

Earlier this year, privately held semiconductor company Adaptevacompleted a $900,000 Kickstarter campaign for its "Parallella" board, featuring its own Epiphany microprocessor, which it calls "the world's most energy efficient and scalable multicore processor chip, designed for parallel computing."

The Parallela board consists of a scalable array of simple RISC processors programmable in C/C++, connected together with a fast on-chip network within a single shared-memory architecture.

In early October, Adapteva published the results of a survey of its Kickstarter backers. Interestingly, the survey found a focus on embedded vision, including, it said, "Robotics, Live Video Processing, 3D rendering, Gesture processing, High speed machine vision, [and] Image Recognition."

The single-board, $99 Parallella with a 16-core Epiphany chip will be available in November. The company's Web site indicates pre-orders have been halted temporarily due to "huge demand and backlog."

Comment  | 
Print  | 
More Insights
In A Fever For Big Data
In A Fever For Big Data
Healthcare orgs are relentlessly accumulating data, and a growing array of tools are becoming available to manage it.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - July 22, 2014
Sophisticated attacks demand real-time risk management and continuous monitoring. Here's how federal agencies are meeting that challenge.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
A UBM Tech Radio episode on the changing economics of Flash storage used in data tiering -- sponsored by Dell.
Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.