Aside from scenic imagery that drones have afforded us in recent years, drone technology is increasingly becoming part and parcel of our everyday. Drones have been deployed in Norway to clean up Oslo’s Fjords and in Rwanda they’re being used to deliver medical supplies across the country. In an effort to further develop the technology, a group of researchers at MIT are integrating VR technology into drone programming to make the autonomous vehicles more adept at flying in a system they are calling “Flight Goggles.”
The team, comprised of researchers from MIT’s Laboratory for Information and Decision Systems, Computer Science and Artificial Intelligence Laboratory, and Sandia National Laboratories, is working to allow autonomous drones to ‘see’ their environment through Virtual Reality (VR) technology. Sertac Karaman, associate professor of aeronautics and astronautics at MIT said to that the system that he and his team are working on can make drones faster, more responsive, and safer to fly over city skies.“We think this is a game-changer in the development of drone technology, for drones that go fast.”
As drones become more prevalent in our city skies, concerns have been raised that the technology isn’t ready to co-exist with humans. And although drones are usually tested far from residential areas, they continue to crash or fall out of the sky. Earlier this year, a drone crashed into a helicopter in the South Carolina.
In order to ensure that drones are safe to operate in skies overhead, they need to undergo intense testing so as to prevent potential collisions or malfunctions during flight. When drones malfunction or collide with objects either in the sky or on the ground, repairing the drones comes at a hefty price – not to mention the damage wrought on other objects in the way.
When drones are tested, huge nets are usually installed in the area to catch them should they fall to the ground. Researchers often install windows or doors which drones learn to fly through or around. The idea behind Flight Goggles being developed by Karaman’s team is to teach drones to better understand the environment around them by deploying VR technology so as to improve their flight track without causing damage to the environment.
Karaman was initially inspired to develop Flight Goggles by competitive drone racing in which human-controlled drones race one another. He said that, in order to enter one of these races, him and his team would need to work on developing a system that makes these races safer and less costly to repair drones that crash or fall during flight.
The more conventional way to test drones – which includes the safety nets, windows, and doors – is intended for drones that don’t fly as fast, Karaman said. It allows these kinds of drones to map their surroundings as they fly overhead. Using this method of testing on drones that fly faster, particularly for drones that participate in competitive drone races, however, isn’t as effective.
Karaman’s team has developed a VR system made of a motion capture system, an image rendering program, and electronics, which the team can use to process and transmit images to drones. Flight Goggles is also coupled with a hangar-like testing arena where drones will fly around as the team projects images of certain obstacles that the drone must fly around.
Drones that undergo this kind of testing can process these images at up to 90 frames per second, which is three times the number human beings can. This is facilitated by a supercomputer lodged onto a 3D printed nylon and carbon-fiber-reinforced drone frame coupled with an inertial measurement unit and a camera, all of which are integrated into the drones.
Over ten flight tests, through which the drone flew through an empty testing arena, it only crashed into the ‘window’ – which was merely a projection – three times out of 361 smooth passes through. After actual windows were erected in the arena, the drone successfully flew through 119 and crashed or required human assistance six times over eight flight tests.
Karaman and his team hope that Flight Goggles will allow drones to ‘see’ on their own, making the autonomous vehicles safer to fly around human beings in the near future. “There are a lot of mind-bending experiments you can do in this whole virtual reality thing,” he said. “Over time, we will showcase all the things you can do.”