Ayodeji Odetola and Devon Reed step up to the challenge of artificial intelligence drone racing.
By: Therese Boyd
“Mathematics is the gate and key to science.”—Roger Bacon
Flying a drone—simple, right? Just move the two little sticks on the transmitter and watch the drone move up, down, sideways, clockwise, counterclockwise, keeping the drone within your sight all the time, right? Actually, it’s more complicated than that. You’re working two sticks at once; you have to control speed and direction at the same time to avoid collision and/or injury to people, property, and the drone itself. The Internet is full of horror stories of drones gone wrong. So it’s not that simple. And when you add high speed—for racing—it gets even more complicated.
Peter Shull, associate professor of engineering at Penn State Altoona, first learned of AlphaPilot—a new artificial intelligence (AI) drone racing competition being held by Lockheed Martin and the Drone Racing League (DRL)—when first-year students Ayodeji Odetola and Devon Reed came to him to propose they enter what Lockheed Martin calls “an open innovation challenge, developing artificial intelligence for high-speed racing drones.” The competition is being held in 2019 and again in 2020, with a top prize of $1 million for each year, plus $250,000 to the first team whose drone races and beats a human-piloted drone.
Shull had his reservations about the students tackling such an ambitious venture: “This is a very, sophisticated project. Otherwise, there would not be a $2 million purse,” he says. Knowing both students’ skills and work ethic through their participation in the Engineering Ahead program, Shull gave them the benefit of the doubt. “They came to me and I asked how they were going to do it.” The students were aware of the challenges ahead of them but had already thought out a plan. “They said the first year was a learning time. In fact, they would be satisfied with only making it through the first two hoops. Then, in the second year, they would build on the knowledge gained in the first year.”
Those “hoops” are actually a series of tests, Reed says. “Test one is a video of who we are and why we are qualified. Test two is to develop an algorithm that acts as the AI pilot” in order to operate the drone, which was supplied by the organizers. “Test three is physics in flight.”
The algorithm in test two is what controls the drone in test three. “The drone is supposed to go through a series of random gates of varying shapes and sizes in 3D space,” Odetola explains. But the drone has to do that at a high rate of speed and without anyone guiding it. In other words, the drone has to be able to “see and respond.” The whole idea, he says, “was based on a paper about mammalian visual cortex,” the part of the brain that processes visual information.
The good news is “I was able to make an algorithm that sees the gates, but it takes too long for the drone to respond,” Odetola continues. So now “we have to optimize it. Mathematical optimization is when you’re given a certain function and you have to figure out how to maximize or minimize the problem. We want to find the values for our network that give us the least amount of error.”
“It’s a mathematical problem, using linear algebra and advanced calculus,” Shull says. “It takes the course and gives it mathematical information. You would think all you need to do is use onboard sensors and see where the gates are and head toward them. But there is no time! This is a race. You need to minimize processing time. The approach is essentially only to find the edges, not the actual object and distance. This has similarities to the way we see at night. We detect an object, mostly the edges, but there is little detail.”
An autonomous drone has a lot of potential, says Shull, such as “Amazon wanting to deliver packages. ‘I need to optimize my time but I cannot run into objects.’ That flight path is not going to be prescribed but will be determined in real-time using an on-board AI pilot. That’s a real-world example. We have methods to do this but it’s way too slow.” He also acknowledges that “this could be great for so many different areas, such as looking at agricultural management.” However, “even with its enormous potential, all technology has a dark side. No doubt the military will be interested. When you see clips of the researchers, not a single one ever mentions the dark side. I feel obligated to.”
Unmanned aerial vehicles—what we now know as drones—have been used in military operations since their inception over 100 years ago. It may be difficult to imagine the earliest days when balloons were rigged to drop bombs on the enemy. Their uses have evolved beyond military weaponry or surveillance to include everything from real estate inspections to recreational flying. No matter the use, however, when flying a drone nothing is more important than accuracy. Not only are there trees and buildings and people to be avoided, but drones also may be flying in shared air space. So there is much to be gained by improving existing autonomous drones.
As they expected, Reed and Odetola did not make it past the second test. Their end game, however, is not winning the money. Before they knew about the contest, they were already discussing the development of drones for search-and-rescue work and that is still the objective. Shull says, “They view the contest experience as support for the long-range goal.” In 2020 they will try again.
