Shield AI: ViDAR, V-BAT, and Tactical Infiltration Drones
How much impact do you believe your job has? Is your job saving someone time? Or money? Or... his life? Well, what if you worked on projects that saved people's lives? Such companies exist, in self-driving cars, in healthcare, and in the case of this article... in Autonomous Defense!
This summer, I have interviewed Vibhav Ganesh from Shield AI, a U.S.-based defense technology company that develops autonomous systems for military and government use.
I would write a big paragraph here, but let me instead show you a quick sample from the interview...
Vibhav has played a pivotal role in the company's growth and innovation. With a background in visual inertial odometry and SLAM, he has been at the forefront of developing autonomous systems like the Nova 2 quadcopter.
Let's read his intro to Shield AI and to their core products: the V-BAT and the ViDAR.
There are a lot of things to note about their products: the V-BAT can last 10 hours, which is a technological achievement itself, thanks to V-TOL (vertical takeoff and landing)... ViDAR is also a very interesting product, which stands for Visual Detection And Ranging... and HiveMind (not shown here) is their AI, or as they call it, "The World's Best AI Pilot".
Let me take you to the v-BAT first, as it's the core product, by showing you this LinkedIn post we did together, where Vibhav Ganesh introduces us to Shield AI.
Together, we recorded an exclusive Fragment of The Edgeneer's Land, my community membership experience, in which he takes us through Shield AI. What is autonomous defense? What are the main technologies involved? What is the range of products?
In this post, I'd like to give you a small sample of that interview, highlighting a very interesting moment where Vibhav talked about infiltration drones.
If you'd like to get started, you can receive the emails here.
Inside Shield AI's Tactical Infiltration Drones
Read the transcript
JEREMY: Okay Vibhav. I'd like to start with the quadcopter. What you call Nova 2. Can you give us an overview of how it works?
VIBHAV: Yeah, I'd love to. So just to kind of understand where we're coming from, I'll give a little backstory of Shield, and talk a little bit about how I evolved in it, and then how Shield has evolved over that that time as well.
So the entire existence of Shield, our mission has been to protect serve, members and civilians using intelligent systems. And we do that by providing platforms that are capable of operating at the edge, providing software that allows different kinds of platforms to be resilient to comms and GPS denial and operate in a really, really sticky and dangerous environments.
And we believe the greatest victory requires no war, and we achieve this by equipping the US and its allies with the ability to see and act anywhere at any time.
That started back in 2016/2015 in very niche ConOps, specifically indoor ConOps. There, we're focused on kind of building clearance.
What our founder, Brandon came back from his deployments and saw as kind of lack of technology really servicing the members that were protecting us, and particularly in areas where they were going in kind of blind to buildings, if you can imagine you had, you know, in the Middle East conflicts, there were just these buildings that were there. You had no idea what's happening inside them.
And in order to kind of under make sure the city was safe, you have to go inside and verify that there was no explosives or militants in there. And what they used to do was send people through this because there was no robots or technology capable to do that.
And if you can imagine yourself doing that, it's extremely scary going in blind, not knowing what's going to happen, what's going to be on the other side of that door.
And so what he wanted to create is a system that could do that for the operator, instead of having the person go do that.
And so the quadcopter Nova 1 was born out of that idea of, how do you provide information before you send a person through?
And the goal there wasn't necessarily to build a quadcopter, but it was just the first apple. Just the first application of autonomy in the defense space that was very, very tangible and very easy for us to apply ourselves to. And so we just designed and built a state of the art indoor autonomous surveillance device.
Nova one was ahead of its league in many different areas. One of the things that really stuck out to me from coming from academia, before I was doing a master's in robotics at CMU, and we saw a lot of really cool applications of autonomy there too, but the hardware systems were not that capable. Like the flight time was 3 to 5 minutes. The processing was very slow. You could operate very slowly. Most of these videos you were seeing back then were sped up by 8x or 7x just to make sure they look compelling.
But what Shield had accomplished was real-time exploration at staggering speeds. At one point, we did a comparison of how fast can a quadcopter clear an environment compared to six Navy SEALs, and the quadcopter actually finished in a third of the time compared to those.
JEREMY: Wow! Okay, I see!
VIBHAV: Isn't that crazy, just how fast this thing was operating. And back then we were, you know, at a limited, limited sensor suite. So we had a 2D scan LiDAR, we had a camera, we had some sonars and an Intel Neural Compute Stick. So it was very limited hardware back then, because it's 2017 but was able to actually accomplish this mission.
So as long as there was a window or door for to fly in, a human operator which would enter, it would enter the vicinity and kind of say, this is the building I want to enter. And from then on, it would be fully autonomous, no comps required. It would find an entrance.
Impressive, isn't it? What I really love about it is that it's down to earth. I could see myself assembling a drone kit, adding a camera, a 2D LiDAR, and starting experimenting with Visual SLAM projects to map a room. This is basically what Shield AI did, when they got started. Except that their drone was (1) targetted to a specific client and (2) better than all competition.
There are 2/3 insights I'd like to share with you, from Vibhav:
1) Self-Driving Car > Autonomous Transfer doesn't work as we'd think
Now, here is something important to note:
A lot of what you learn in autonomous robots CANNOT simply be transferred to drones.
I did think that it was a matter of copy and paste. But I understood I got it wrong when making this fragment, especially with Vibhav Ganesh who told me that their drones don't have LiDARs, fly over seas, or over deserts, no man's land zones, between mountains, campaigns, with 3D constraints, and no map!
I thought about it for a minute, and I realized... "Wait, it's absolutely NOT like autonomous cars!"
And in fact, when you start looking into autonomous drone architecture, they absolutely don't look like self-driving car architectures! For example with Shield AI, they have a Control, a Station, RTOS, a ViDAR, but also a Flight Controller powered with frameworks like PX4 and Maven. This is an entire set of libraries to learn.
From there, here is a second insight:
2) Visual SLAM is mostly used
Coming back to the idea that we are NOT like self-driving cars... the other main difference is that we use no map. So without a map, and with just a camera, they have no choice but to use...Visual SLAM!
And Vibhav explains really well what kind of SLAM they're using, how they implement the mapping, even though there is 0 starting point, and so on. Here is a sample of a vSLAM project I've tested with drones:
This is the nitty gritty of Shield AI's work. Once you build a SLAM MAP, you can then feed that map to the Motion Planner, which sends a flight order to the drone. If you'd like more insights on this technology, I highly recommend my vSLAM article.
Jeremy Cohen
Okay, would you like to see some samples?
Shield AI in Action
Let's take a look at 3 samples here:
courtesy of Shield AI
- On the left, you can see drones being launched. These drones cannot do the "vertical takeoff and landing". They are projected to the air by launchers and then fly like a plane.
- In the middle, you can see the tactical quadcopters we discussed. Notice how they use vSLAM at the end of the shot.
- On the right, you can see a mission of the v-BAT searching for a vessel in a sea canal.
This is really cutting-edge, and totally applied what we are building in the autonomous tech space. Alright, time to wrap up!
Summary & Next Steps
- The defense industry is extremely active. Hundreds of companies work on the autonomous generations of drones, anti-missile detectors, infiltration equipment, RADARs, and more...
- Shield AI is an active actor of the defence space, with a range of multiple products, such as the v-BAT, the ViDAR, and HiveMind.
- Shield AI started with NOVA 1, a quadcopter that could infiltrate into buildings, builds maps, surveys, without the need to send humans into the region. This allowed to prevent human losses due to buildings collapsing or being trapped.
- Besides from being safer, infiltration drones are also more efficient. Shield AI tested their drone against 6 NAVY SEALs clearing a building, and it finished in a third of the time.
- The transfer of autonomous car/robot technology to autonomous drone isn't as simple as we'd think. Architectures are different, products are different, regions/environments are differents, and even the inside technologies and algorithms change.
- On the other hand, some technologies really apply well to autonomous drones, such as Visual SLAM.
If you'd like to get started, you can receive the emails here.
