Crisis Response: 6 Ways Big Data Can Help
Whether naturally occurring or man-made, crises and disasters bring chaos to the people in their path. Learn how governments, nonprofits, and businesses are using big data and analytics to respond in fast, efficient ways.
![](https://eu-images.contentstack.com/v3/assets/blt69509c9116440be8/bltdeb36ac0c6948ff6/64cb4a0cfdf5ba78b5e5bc2a/1-fireman-100722_1280.jpg?width=700&auto=webp&quality=80&disable=upscale)
Natural disasters, economic upheavals, and illegal activities happen. Governments, law enforcement agencies, non-government organizations (NGOs), and businesses are using big data and analytics to improve their ability to respond to crises in fast, efficient ways.
Organizations are changing the ways they collect data by using parallel processing to accelerate response times, using third-party data to improve the accuracy of insights, developing new algorithms and models to solve problems more effectively, and dispatching data-gathering drones into situations that would be unsafe for humans or animals. In many cases, the reaction times are reduced by an order of magnitude or more, such as from months to weeks, days, or even hours. But there's still a lot of work to do. When disaster strikes, time is of the essence.
"Our goal is to create as many efficiencies as possible to get on cases quicker. The earlier we intervene, the more future cases we're preventing," said Jim Cole in an interview. Cole is the national program manager for the Victim Identification Program, which runs out of the Homeland Security Investigations' Cyber Crimes Center, part of US Immigration and Customs Enforcement.
[See what happens when technology meets forest fires. Read Using Data To Fight Wildfires: An Inside Look.]
When natural disasters occur, big data enables fast and accurate decision-making. When aberrant human behavior is the problem, big data can speed the identification of the victims and the offenders.
"If you're the head of the Environmental Protection Agency and deciding whether you're sending people into [a disaster] area, you want to have all the information available so you can make the right choice. You may be sending people into an unsafe situation," said Ernest Earon, CTO and cofounder of drone and data company PrecisionHawk, in an interview. "When the people on the ground have better information, they can do their jobs faster and safer."
Crises drive headlines, but what happens behind the scenes isn't always as well known. We'll take you through recent crises -- manmade and natural -- to reveal how organizations are attempting to minimize and manage emergency situations. Once you've reviewed these examples, tell us what you think in the comments section below. Have you been involved in a crisis management or emergency situation? Were you able to use any of the tools and techniques highlighted here? What other options have worked for you?
When the US financial market crashed in 2008, the Treasury Department bailed out government sponsored entities (GSEs) Fannie Mae and Freddie Mac to the tune of $187.5 billion. Prior to the crash, a not-so-perfect storm had been forming that exposed the GSEs to more risks than they had anticipated.
"Leading up to the crash, Fannie and Freddie were in the middle of buying some very suspect loans -- lots of low-documentation loans, negative amortizing loans, interest-only loans, and then traditional 30-year and 15-year mortgages. You also had the subprime market taking off, which was really going much further down the credit spectrum than in the past," said Charles Rumfola, former Fannie Mae VP and current senior VP of strategic initiatives at Veros Real Estate Solutions, in an interview. "Once the market crashed, borrowers were not making their payments. Fannie and Freddie couldn't front that much money to pay the security holders, which is why we had a draw from the Treasury."
There were other problems as well. For example, the lenders originating loans had agreed to representation warranties that required them to buy back loans purchased by the GSEs if those loans failed to meet certain qualifications. But the lenders breached the warranties, either because they went out of business or because they lacked the capital to buy back the mortgages.
"Coming out of the crash, the GSEs completely changed their business model. They said, 'We can't rely on this representation-warranty business model, we need to have another view of what we're buying before we actually buy it,'" said Rumfola. "The first thing they did was change their policies and issue policy statements saying, 'We're not going to buy low-documentation loans and negative amortizing loans, and we're going to increase the credit score eligibility.'"
That was only half the battle. Many buyers had refinanced their loans to pay for luxury items and home improvements, which meant there was little or no equity left in their homes. Meanwhile, property appraisals were overinflated by 5% to 20%, which meant that some mortgage principal balances exceeded the associated property values. As a result, Fannie Mae and Freddie Mac were faced with greater-than-anticipated losses. Subsequently, the companies worked with Veros to create a uniform dataset and an appraisal portal for lenders, which has processed approximately 26 million appraisals to date.
Part of ensuring higher data quality was modifying the appraisal form to include important data elements that were not originally included. To help facilitate data uniformity, free-form text boxes were replaced with drop-down menus for things like the quality of a structure and the condition of a property.
"[If we'd had uniform data collection and the appraisal portal pre-2008], we'd still be doing loans with very low credit scores, low-documentation loans, or negative amortizing loans, but you would not have had the severity of losses that the GSEs experienced."
On March 22, 2014, a mudslide roared through the rural community of Oso, Wash., destroying more than 30 homes and taking the lives of 43 residents. Response teams had to move quickly, but the mudslide was still moving and there was a risk of other slides. In addition, the mudslide had dammed a river, flooding the valley. People were missing, but the area was too unsafe for emergency responders. Helicopters and aircraft were unable to survey the area due to the extreme weather conditions. So, a drone was sent in to collect data that could be used for 3D modeling and for comparison against pre-disaster satellite images.
"We created 3D terrain models so we could see where it was safe for the responders to go and where they needed to wait until the mud had settled a bit more. We wanted to make sure the responders were safe and that [the decision-makers] could see the extent of the damage in the area and how they could fix it and clean it up post-disaster," said Ernest Earon, CTO and founder of drone and data company PrecisionHawk.
Also, in February 2015 a drone was used to assess a fire that broke out at Bennett Industrial Landfill in Lockhart, S.C. At the time, asbestos and other potentially harmful chemicals were being released into the air. The Environmental Protection Agency (EPA) needed to evaluate the situation quickly, but it was unable to obtain an accurate volumetric survey of the area due to an active fire. Surveyors were unable to enter the area because it was not safe, so a drone was sent in to gather data. That way, the entire landfill zone could be recreated and compared with the initial surveys to determine the extent of the damage.
"When harmful chemicals are continually being released into the air, it impacts the entire environmental area, including the towns that are nearby. The main thing in situations like this is ensuring the safety of the people on the ground, but you still need to go in and do an assessment," said Earon. "[The EPA was] able to go in and put the fire out and stop the chemicals from leaking from the areas."
The United Nations Office on Drugs and Crime estimates that the amount of money laundered globally in a single year represents 2% to 5% of global gross domestic product (GDP), or $800 billion to $2 trillion. The deeper the money gets into the international banking system, the more difficult it is to identify its origin.
The Anti-Money Laundering Act and similar international regulations require financial institutions to identify suspicious transactions. Front-office transactions often take place on a website or over the phone, making them prone to human error and fraud. In addition, the pull-down menus in the system UIs may include default and junk values, which are intentionally used in suspect transactions.
"A few financial institutions are very advanced. Most of them take a look at six months of data, compare the new transaction against it, and then [based on that] they determine whether it's a suspicious transaction or not. Or, they take the data quality at its face value and do the test again. This particular process, by definition, doesn't give you the complete picture," said Angsuman Dutta, executive VP at data controls and analytics software company Infogix, in an interview.
Pattern identification is important because suspect transactions can go undetected when they are broken down into smaller pieces and spread out across channels, even within the same financial institution.
"Heuristically, they would have been caught using one channel, but because the person is using multiple channels the bank will catch it, but after the fact," said Dutta.
One New York financial institution has 23 applications capable of receiving incoming transactions. It is setting up a system that cross-checks information against Internal Revenue Service, United States Postal Service, and other data sources to "fingerprint" a transaction. If the transaction includes factual discrepancies, it's flagged for investigation.
"When an employee wants to make a fraudulent transaction, she always uses a default value. There will be no check and the transaction will go through the system. There needs to be a data quality check. Otherwise there's no way to detect it," said Dutta.
As a result of the data quality checks, money-laundering transactions are being identified faster than was previously possible. The goal is to collapse time even further to prevent such transactions from occurring in the first place.
On March 11, 2011, a 9.0 magnitude earthquake hit Japan causing a tsunami with 30-foot waves. As a result, 15,890 people died and several nuclear reactors were damaged. Dun & Bradstreet (D&B) stepped in and combined some of its data with third-party data to help local businesses manage the impact of the disaster.
"The connectedness of all the information we have is a blessing and a curse in a situation like this. You have tens of thousands of records of businesses that are in an area where you really don't know whether your data is correct anymore because of something that happened minutes or hours ago," said Anthony Scriffignano, senior VP and chief data scientist at the commercial data provider. "You could turn it all off, but if you did, the people who survive can't do anything."
Remediating the data required considerable problem-solving. For one thing, D&B needed to identify which companies likely were (and were not) affected. Fairly quickly, it constructed a list that included the maximum possible number of companies impacted by radiation or flooding. Next, it had to figure out a way to reconcile the data quickly.
"There are three universes of data in any problem: There's the data you have in hand, the data you can easily discover or go get, and the data you're not going to have access to without some kind of extraordinary measures," said Scriffignano.
D&B had demographic data on the Japanese companies, as well as information about how they linked to parts of other companies in Japan and around the world. It also had trade data and other data indicating how operational the businesses were, how big they were, and how connected they were a minute before the earthquake hit. The additional or discoverable data included satellite imagery, geospatial data, and topographic information which helped identify the locations of the businesses. D&B also had data on roads and infrastructure, as well as crowd-sourced radiation data, that was pretty close to real-time.
"We combined all that data and built algorithms that [answered], for every business, how shook up did it get, how wet, how impacted were the roads and infrastructure, how many cars were in the parking lot, and what happened with the radiation in that area? We build this giant heuristic that could scientifically treat every business the same way, and make a decision about whether they seemed to be in business or not and whether they're connected to another part of the business," said Scriffignano.
D&B could have commercialized the data asset, but instead it made the data publicly available via the Internet for a limited time so it could be used for outreach or business decisions.
In late October 2012, Hurricane Sandy devastated parts of the Caribbean, as well as 24 US states. The storm claimed the lives of 233 people in eight countries, and the assessed damage exceeded $68 billion. In Jamaica, the high winds left 70% of residents without electricity. Flooding in Haiti left 200,000 people homeless. In New York, a storm surge flooded subway lines, tunnels, and streets, causing a power outage that affected businesses and residents in the city and surrounding areas.
"The people most vulnerable before and during an emergency are most vulnerable after recovery, which is why Direct Relief remains connected daily to its partners caring for people who otherwise would not have access and provides them with free medicines and supplies. In doing so, this channel of support enables facilities to be better equipped to respond to the next emergency that arises," wrote Hannah Rael, a communications specialist, in a Direct Relief International (DRI) blog post.
As a first responder, DRI provided medical supplies and services to thousands of people impacted by the hurricane. According to a case study published by software and services company Palantir, DRI used its Gotham data platform to help identify health needs and coordinate response efforts. At the time, thousands of people needed power, basic provisions, and medical supplies. DRI was able to identify the communities that had been impacted the most, as well as community-based organizations that could help with the effort. It also monitored the availability of pharmacies and health clinics, assessed medical inventory supplies, and routed the necessary medical resources.
The data sources included pharmacy locations, weather reports, flood maps, fuel availability charts, and data from New York's 311 information system. With the data, DRI was able to focus and triage its efforts. That process included identifying the best routes for the delivery of supplies.
In fiscal year 2014, the US Immigration and Customs Enforcement (ICE) Homeland Security Investigations (HSI) Cyber Crimes Center identified 1,036 victims of child exploitation, processed more than 5.2 petabytes of data, and arrested more than 2,300 child predators. Yet, the volume of data continues to grow.
"Several decades ago, child exploitation material was largely produced and distributed in print form, so it had to be physically smuggled. With the advent of the Internet, offenders can now create the material and share that material with other like-minded people in an apparently anonymous way," said Jim Cole, the national program manager for the Victim Identification Unit that's part of HSI's Cyber Crime Center. "Even a few years ago, we quickly found ourselves drowning in data from seizures related to child exploitation cases. The amount of data was outpacing our ability to manage it."
To improve its effectiveness, HSI's Cyber Crime Center helped found Project Vic, which is creating an ecosystem of information and data-sharing among domestic and international law enforcement agencies working on crimes against children.
"The primary goal of Project Vic is to identify and rescue more victims. However, several things had to change to contribute to that goal," said Cole. "We had to work with our vendors and NGOs to help us be more efficient. All of the forensic computing vendors except for two have completely gone to open standards and an open data structure, which is Project Vic-compatible, so law enforcement can now move data seamlessly between Project Vic-compliant tools."
Cole and his team use tools that analyze the content of images and video. With big data, the investigators can identify camera makes, models, and serial numbers.
"We can easily see what images were taken with the same camera, or within a volume of data we've seized, if we have images taken with a camera we've seized or a camera associated with the case."
There's also a hash database of known images and videos. Cole and his team are using PhotoDNA, which was donated by Microsoft, for pixel-by-pixel content analysis, because some offenders save images in multiple formats or use binary hash values to slightly modify files in hopes of avoiding detection. The use of PhotoDNA has resulted in a 30% to 40% workflow reduction. However, because PhotoDNA only works on images, it was necessary to find a similar forensic technology for video, which was provided by Friend Media Technology Systems.
"One of the trends we're seeing, given the affordability of storage media, is video. And because the technology is better, capture devices are smaller, and you can capture 1080p video on your iPhone, we're seeing the volume of data and video dramatically increase," he said. "[Now] we can find videos, or pieces of video within a video that we've already seen before, more efficiently."
Before HSI's Cyber Crime Center implemented the new technology, there was a nine-month lag between the seizure of evidence and the completion of a forensic examination. The same amount of work now takes three weeks. In addition, the time it takes from seizure of illegal material to identification of victims has decreased from weeks or months to as little as 24 to 48 hours.
In fiscal year 2014, the US Immigration and Customs Enforcement (ICE) Homeland Security Investigations (HSI) Cyber Crimes Center identified 1,036 victims of child exploitation, processed more than 5.2 petabytes of data, and arrested more than 2,300 child predators. Yet, the volume of data continues to grow.
"Several decades ago, child exploitation material was largely produced and distributed in print form, so it had to be physically smuggled. With the advent of the Internet, offenders can now create the material and share that material with other like-minded people in an apparently anonymous way," said Jim Cole, the national program manager for the Victim Identification Unit that's part of HSI's Cyber Crime Center. "Even a few years ago, we quickly found ourselves drowning in data from seizures related to child exploitation cases. The amount of data was outpacing our ability to manage it."
To improve its effectiveness, HSI's Cyber Crime Center helped found Project Vic, which is creating an ecosystem of information and data-sharing among domestic and international law enforcement agencies working on crimes against children.
"The primary goal of Project Vic is to identify and rescue more victims. However, several things had to change to contribute to that goal," said Cole. "We had to work with our vendors and NGOs to help us be more efficient. All of the forensic computing vendors except for two have completely gone to open standards and an open data structure, which is Project Vic-compatible, so law enforcement can now move data seamlessly between Project Vic-compliant tools."
Cole and his team use tools that analyze the content of images and video. With big data, the investigators can identify camera makes, models, and serial numbers.
"We can easily see what images were taken with the same camera, or within a volume of data we've seized, if we have images taken with a camera we've seized or a camera associated with the case."
There's also a hash database of known images and videos. Cole and his team are using PhotoDNA, which was donated by Microsoft, for pixel-by-pixel content analysis, because some offenders save images in multiple formats or use binary hash values to slightly modify files in hopes of avoiding detection. The use of PhotoDNA has resulted in a 30% to 40% workflow reduction. However, because PhotoDNA only works on images, it was necessary to find a similar forensic technology for video, which was provided by Friend Media Technology Systems.
"One of the trends we're seeing, given the affordability of storage media, is video. And because the technology is better, capture devices are smaller, and you can capture 1080p video on your iPhone, we're seeing the volume of data and video dramatically increase," he said. "[Now] we can find videos, or pieces of video within a video that we've already seen before, more efficiently."
Before HSI's Cyber Crime Center implemented the new technology, there was a nine-month lag between the seizure of evidence and the completion of a forensic examination. The same amount of work now takes three weeks. In addition, the time it takes from seizure of illegal material to identification of victims has decreased from weeks or months to as little as 24 to 48 hours.
-
About the Author(s)
You May Also Like