Biggest AI Innovation in the field of Reinforcement Learning Yet- A Fully-Autonomous Fighter Jet System

Artificial Intelligence just made another big leap, this time in a rather peculiar manner by defeating an F-16 Fighter Jet pilot 5-0 in a dogfight. Yes, an AI pilot (an algorithm developed by the US-based company Heron Systems), with a just few months of training over computer simulations destroyed one of the US Air Force’s most seasoned pilots with years of experience on flying F-16 fighter jets, in a simulated fight that lasted for 5 rounds, with 5 perfect wins.

This unique turn of events occurred during the AlphaDogFight Trials— a program organized by the Pentagon-backed Defense Advanced Research Projects Agency (DARPA) to see whether autonomous flying systems could defeat an actual human pilot in a computer-simulated dogfight. And with the phenomenal success the DARPA achieved, this is just getting started.

DARPA aims at building AI systems that are capable of flying fully-functional fighter jets in the real world. While one might argue that this technology already exists- military drones, that is not exactly the case. Human operators control unmanned military drones. 

This, on the other hand, is just an AI algorithm with complete, autonomous control over a fighter jet without the need of any form of human control. For the US Air Force, this means they can make their fighter jets even faster because no human pilot means they don’t have to account for the G-force that a pilot might experience while flying the jets at supersonic speeds.

Now the question is, how did the makers of this AI Pilot at the Heron Systems achieve this feat? How did they manage to train an AI algorithm so powerful that it can beat an experienced human pilot? 

The answer is— Deep Reinforcement Learning. 

What is Reinforcement Learning?

Deep Reinforced Learning, or just RL for short, is a type of semi-supervised machine learning algorithm, in which the agent (the model) is supposed to take actions in a particular simulated environment to maximize the reward that it gets on a correct decision. 

The algorithm works on what is called an award-and-penalty-based system, in which the model is rewarded for every correct decision it makes (and penalized for every incorrect one). And just like in the case of any other machine or deep learning algorithm, this is an iterative process, with the process of rewards and penalties re-occurring several times until we get an agent accurate enough to operate in the environment with minimal (or ideally 0) penalties.

Reinforming learning systems use a rather unique technique to learn as compared to the other supervised or unsupervised algorithms. Here, the algorithm learns and discovers patterns in the data/environment by learning from its own mistakes during each iteration. As a result, there’s no need to set a lot of explicit boundaries for the model. This makes RL perfect for implementation in areas like game AIs, robotics, recommender systems, and even some advanced content-flagging and spam-detection systems.

Reinforcement Learning Revolutionized AI

For the record, some of the biggest milestones ever achieved in the field of Artificial Intelligence are owed to this powerful algorithm. Reinforcement Learning can be said to be one of the primary reasons behind how AI became so popular that it’s now a household name. 

Let us have a look at some of the biggest achievements of the Deep Reinforcement Learning algorithm.

• 1997, AI Beat Chess Grandmaster Garry Kasparov – In 1997, Reinforcement Learning turned heads when IBM’s computer Deep Blue defeated a chess grandmaster and former world chess champion Garry Kasparov in a chess match. This was one of the first major feats for RL and AI.

• 2013, Boston Dynamics’ Atlas was Born – In 2013, US-based robotics and engineering design company Boston Dynamics showcased their state-of-the-art humanoid named Atlas. What was special about Atlas is that it could walk even on rugged terrains without any assistance. Continuous training through reinforcement learning allowed it to attain near-human skills of traversing uneven terrain.

• 2016, DeepMind’s AlphaGo Beat Professional Go Player Lee Sedol – In 2016, DeepMind’s RL-based AI beat Lee Sedol 4-to-1 in a five-match game. Until then, no one had thought it could be possible for a computer, because the amount of permutation and combination of moves in the game of Go makes it impossible to code a traditional computer program for it. In 2017, DeepMind released another version, AlphaGo Zero, that was termed as “The best Go player in the world”.

• 2017, DeepMind Unveiled AI That Can Walk – In 2017, DeepMind unveiled a walking simulator that taught itself to walk just like a human would use the Reinforcement Learning algorithm. This news caused quite some ripples, especially in the robotics industry.

These are just a few achievements and contributions of RL to the AI community over time, with rapid advancements continuously occurring in the field. 

But now, going back to our AI Pilot. What happens next once the world has AI-powered killing machines flying in the air and swimming in the depths of the oceans in the form of military aircraft, battleships, and submarines?

A World Ruled by Evil Sentient AI – A Certain Future or Just a Science Fiction?

The fear of killer-AI ruling the world was planted into our heads by Hollywood screenplay writers, the concept of sentient robots appearing for the first time in a 1920 play in which a race of self-replicating robots revolts against their human masters. Since then, countless novels and the cult-classic movie series Terminator has managed to keep this irrational fear of “Artificial Intelligence taking over the world” alive within the minds of us humans.

But for real. Can this be true? Is it possible that someday, suddenly our AI Pilot decides that humans are an enemy and starts bombing from the sky? 

Well, thankfully, this is just a mere sci-fi fantasy and we won’t be seeing a sentient AI anytime soon. And, you’ll understand the reason behind this not in terms of science fiction but a fact by how AI systems work.

Generally, the creation of an AI system involves 2 major stages– development and deployment. 

The development stage is where the model learns upon the training data. This is more of an experimental stage where a team of Machine Learning Engineers and Data Scientists work on different architectures and optimization techniques to develop a model that best suits their needs. 

The next step is deployment. Now, as the name suggests, in this stage, the model is deployed into production servers. Here, based on its architecture, the model takes either one of the two learning routes – either the training for the model is turned off and it is used for the sole purpose of inference (batch learning) OR the model keeps learning even in deployment stage but its performance is closely monitored and in case of data deterioration or performance loss, the training can be turned off. 

Therefore, let’s say in case of our AI Pilot, if the team of Data Scientists monitoring its performance detect an anomaly where it classifies ally aircraft or residential buildings as targets, they will turn off the model’s training, revert to an older, more stable version, and then work upon making sure that doesn’t happen again, thus “eliminating the risk” of the AI-Pilot turning against humanity.

Future of the AI Pilot

As we saw earlier, DARPA wants to expand this AI-Pilot program from mere simulations to flying real-world fighter jets. But that’s not just it. There’s more to this. 

While this means a great deal in the sense of how Artificial intelligence will revolutionize modern warfare, this also means that the next generation autopilot systems in commercial flights could be powered by something like this. While the existing human pilots flying commercial jets are no doubt very highly skilled, an autopilot system of the likes of the AI Pilot could make air travel much safer for us. 

This technology is still in a very early stage and we won’t be getting a fully autonomous flight system anytime soon, but this technology looks very promising, and there’s no doubt in that.

What you think is the future of AI Pilot systems? Would you trust an AI with a flight that you are traveling in? Let us know in the comments!