The National Guard is helping residents affected by flooding from Hurricane Harvey in 2017. … [+]
The National Guard has a lot of work to do: As first responders to wildfires, floods, and other increasingly frequent major disasters, they must be deployed quickly to where they’re needed.
This is one of the most critical challenges we can face in information technology: across many square miles of affected land, which schools, bridges, or neighborhoods are in most urgent need of assistance? Often, human lives are at stake.
Google’s parent company, Alphabet, has a moonshot factory called X that aims to solve some of the world’s biggest and most vexing problems, and X has developed a groundbreaking solution – a solution that uses predictive AI in much the same way that many companies do. How did X do it, and what can all business professionals learn about using machine learning to combat uncertainty and risk?
Problem: Tagging aerial imagery
During and after severe weather events, drones and manned aircraft collect thousands of aerial photos of affected areas. These images can potentially reveal which buildings and other infrastructure were damaged, but only if each image is tagged with exactly which location it depicts. Unfortunately, images typically lack this metadata.
Manually tagging photos significantly delays the National Guard’s response; it typically takes National Guard teams about 12 hours after an incident to complete this task. Unfortunately, this process has historically remained manual, and it’s a difficult task to automate because photos are taken from a variety of altitudes and oblique angles.
But this is exactly the kind of problem X was designed for: the risks are extremely high, and technological breakthroughs are required. X’s effort to tackle this problem and related challenges is called Bellwether, and is described as “the first predictive engine for the Earth and everything on it.”
“We took on this challenge because we realised that if we could solve this, we could drastically reduce response times to climate disasters and increase many times the number of lives saved,” asserts Sarah Russell, who has run Bellwether since founding the company in 2020.
The solution: Machine learning photo matching
The breakthrough is matching real photos with synthetic ones. Bellwether synthesized a database of simulated reference photos that can be used as exemplars. When a real photo matches one in the database, it’s tagged and the system knows exactly where and what it’s a photo of. To synthesize the reference images, X leveraged Google’s extensive, proprietary geospatial resources, which underpin products like Google Earth and Google Maps.
It works: Just a few years after Bellwether was formed and began working on the solution, the National Guard is already piloting it and will be using it this summer during the wildfire season.
The solution allows National Guard team members to instantly scope out the most impacted areas and know where they’re looking. They know which bridges are not functioning. They can ask “show me all hospitals” in their monitored areas. They can take an informed response immediately, eliminating the process delays that have held them back for years.
ML plays just as central a role in moonshots like this as it does in more typical enterprise systems. After all, photo matching is an imprecise process that ML handles well. There are no guaranteed matches, as aerial photos are not matched precisely. Each aerial photo is taken from its own unique distance, zoom, and angle, can be obscured by weather conditions, and the captured landscape is often affected, sometimes with disastrous results.
ML removes much of the uncertainty by assigning a confidence level to each match. When a large number of photos are fed in, enough of them are found to match with high confidence that the system can provide operations personnel with imagery covering nearly all of the affected location, even after filtering out photos that were not a definite match.
This approach is scalable: “Beyond National Guard deployments, our goal is to make this type of service radically easier for a broader group of disaster responders,” Russell says. “This approach can be applied to rescue and recovery responses to a variety of weather-related events, such as heat waves or tornadoes.”
The ubiquity of predictive AI
Whether you have lofty goals or more general enterprise objectives, ML’s core ability to generate confidence levels solves universal operational challenges, regardless of industry: Which customers are likely to buy? Marketing targets those customers. Which transactions are likely to be fraudulent? Banks block those transactions. Which addresses are likely to receive deliveries tomorrow? UPS plans for those addresses.
This familiar paradigm of using ML predictions to operate at scale has a name: predictive AI: a technique that systematically filters out low-confidence cases and takes action on the remaining high-confidence cases.
So how much confidence is enough? It depends. Each project must determine the optimal choice of decision threshold based on its actual needs. For example, the National Guard requires photos that are matched with a very high degree of confidence. In contrast, marketing or fraud detection can target many cases that don’t pan out. It’s an unavoidable part of the numbers game that inevitably plays in these types of operations.
In other words, predictive AI reduces uncertainty by quantifying it. Bellwether is working to extend this incredible approach in other ways to mitigate the damage caused by climate disasters, such as predicting where the most lives can be saved (which areas should be prioritized for evacuation assistance) and predicting environmental accidents before they happen.
“ML has become a new paradigm in geosciences,” says Russell. “For example, until recently in hydrology, flood predictions were primarily done with site-specific models. Now, with ML, we use data from many different locations to develop the best models. We can use flood behavior on the east coast of the United States to predict flooding on the west coast.”
