Massively Parallel Processing Finds More Applications

20 Great Ideas To Steal In 2013

The European particle physics laboratory CERN used it to detect the elusive Higgs Boson. Wall Street has used it for years to evaluate oceans of market data in order to make high-frequency trades. And the National Security Agency (NSA), we've recently learned, uses it to pore over billions of our emails and text messages.

But massively parallel processing -- a computing architecture that uses multiple processors or computers calculating in parallel -- has been harnessed in a number of unexpected places, too.

Identifying who is using these novel applications outside of purely scientific settings is, however, tricky. That's because these systems provide "unique insights" that can give their users competitive advantage, Alex Gorbachev, CTO of remote database services and consulting company The Pythian Group told InformationWeekin a phone interview.

[ Can you learn from these companies? Read Big Data Success: 3 Companies Share Secrets. ]

The moment a massively parallel processing solution gains visibility, "people quickly start using it," Gorbachev said.

Real-time OCR

Pythian's own Hadoop and MapReduce application, now under development, is an extension of a home-grown security protocol it uses today, whereby it records the desktops of hundreds of DBAs as they work remotely on client databases.

"It's like the closed-circuit television cameras used in a bank," Gorbachev explained.

Although hundreds of hours of digital recording is useful for after-the-fact forensics, Pythian is now developing an OCR application using Hadoop and Mapreduce that OCRs the screens, producing a searchable, full-text index.

"This makes it very easy to do mining and indexing," Gorbachev said, adding that massive parallelism of the application means it can also detect, in real-time, suspicious text patterns, such as credit cards, social security numbers or other personally identifiable information.

Pythian presenteddetails of its work at the Hadoop Summit earlier this year in San Jose. The company plans to commercialize the OCR capability next year.

DARPA's 'brain-like' RFP

This summer, the U.S. Defense Advanced Research Projects Agency (DARPA) issued an request for proposal(RFP) for technologies related to developing a computer that emulates a human brain, specifically the neocortex, the part of the brain responsible for higher functions such as motor control, language, sensory activity and thought.

The project, which is aimed at developing new approaches for detecting anomalies in large, complex data sets, will depend on neural models of the human neocortex, according to the DARPA researchers.

"The cortical computational model should be fault tolerant to gaps in data, massively parallel, extremely power efficient, and highly scalable. It should also have minimal arithmetic precision requirements, and allow ultra-dense, low power implementations," the request states.

Not surprisingly, DARPA is looking for solutions using massively parallel technologies that can mimic the brain's gift for temporal and spacial recognition, and its ability to solve "extraordinarily difficult recognition problems in real-time."

The $99 Board

For do-it-your-selfers curious about supercomputer-level hardware performance in an affordable package, there's now a $99 board.

Earlier this year, privately held semiconductor company Adaptevacompleted a $900,000 Kickstarter campaign for its "Parallella" board, featuring its own Epiphany microprocessor, which it calls "the world's most energy efficient and scalable multicore processor chip, designed for parallel computing."

The Parallela board consists of a scalable array of simple RISC processors programmable in C/C++, connected together with a fast on-chip network within a single shared-memory architecture.

In early October, Adapteva published the results of a survey of its Kickstarter backers. Interestingly, the survey found a focus on embedded vision, including, it said, "Robotics, Live Video Processing, 3D rendering, Gesture processing, High speed machine vision, [and] Image Recognition."

The single-board, $99 Parallella with a 16-core Epiphany chip will be available in November. The company's Web site indicates pre-orders have been halted temporarily due to "huge demand and backlog."