Massively Parallel Processing Finds More Applications - InformationWeek
Data Management // Hardware/Architectures
12:58 PM
Connect Directly
Ransomware: Latest Developments & How to Defend Against Them
Nov 01, 2017
Ransomware is one of the fastest growing types of malware, and new breeds that escalate quickly ar ...Read More>>

Massively Parallel Processing Finds More Applications

Parallel computing used on massive datasets is popping up everywhere to create everything from artificial brains to real-time video analysis systems.

20 Great Ideas To Steal In 2013
20 Great Ideas To Steal In 2013
(click image for larger view)

The European particle physics laboratory CERN used it to detect the elusive Higgs Boson. Wall Street has used it for years to evaluate oceans of market data in order to make high-frequency trades. And the National Security Agency (NSA), we've recently learned, uses it to pore over billions of our emails and text messages.

But massively parallel processing -- a computing architecture that uses multiple processors or computers calculating in parallel -- has been harnessed in a number of unexpected places, too.

Identifying who is using these novel applications outside of purely scientific settings is, however, tricky. That's because these systems provide "unique insights" that can give their users competitive advantage, Alex Gorbachev, CTO of remote database services and consulting company The Pythian Group told InformationWeekin a phone interview.

[ Can you learn from these companies? Read Big Data Success: 3 Companies Share Secrets. ]

The moment a massively parallel processing solution gains visibility, "people quickly start using it," Gorbachev said.

Real-time OCR

Pythian's own Hadoop and MapReduce application, now under development, is an extension of a home-grown security protocol it uses today, whereby it records the desktops of hundreds of DBAs as they work remotely on client databases.

"It's like the closed-circuit television cameras used in a bank," Gorbachev explained.

Although hundreds of hours of digital recording is useful for after-the-fact forensics, Pythian is now developing an OCR application using Hadoop and Mapreduce that OCRs the screens, producing a searchable, full-text index.

"This makes it very easy to do mining and indexing," Gorbachev said, adding that massive parallelism of the application means it can also detect, in real-time, suspicious text patterns, such as credit cards, social security numbers or other personally identifiable information.

Pythian presenteddetails of its work at the Hadoop Summit earlier this year in San Jose. The company plans to commercialize the OCR capability next year.

DARPA's 'brain-like' RFP

This summer, the U.S. Defense Advanced Research Projects Agency (DARPA) issued an request for proposal(RFP) for technologies related to developing a computer that emulates a human brain, specifically the neocortex, the part of the brain responsible for higher functions such as motor control, language, sensory activity and thought.

The project, which is aimed at developing new approaches for detecting anomalies in large, complex data sets, will depend on neural models of the human neocortex, according to the DARPA researchers.

"The cortical computational model should be fault tolerant to gaps in data, massively parallel, extremely power efficient, and highly scalable. It should also have minimal arithmetic precision requirements, and allow ultra-dense, low power implementations," the request states.

Not surprisingly, DARPA is looking for solutions using massively parallel technologies that can mimic the brain's gift for temporal and spacial recognition, and its ability to solve "extraordinarily difficult recognition problems in real-time."

The $99 Board

For do-it-your-selfers curious about supercomputer-level hardware performance in an affordable package, there's now a $99 board.

Earlier this year, privately held semiconductor company Adaptevacompleted a $900,000 Kickstarter campaign for its "Parallella" board, featuring its own Epiphany microprocessor, which it calls "the world's most energy efficient and scalable multicore processor chip, designed for parallel computing."

The Parallela board consists of a scalable array of simple RISC processors programmable in C/C++, connected together with a fast on-chip network within a single shared-memory architecture.

In early October, Adapteva published the results of a survey of its Kickstarter backers. Interestingly, the survey found a focus on embedded vision, including, it said, "Robotics, Live Video Processing, 3D rendering, Gesture processing, High speed machine vision, [and] Image Recognition."

The single-board, $99 Parallella with a 16-core Epiphany chip will be available in November. The company's Web site indicates pre-orders have been halted temporarily due to "huge demand and backlog."

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of IT Report
In today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll