As a delivery drone takes off from a location to go make a drop at another predetermined location it must make a number of decisions along the way. The Markov Decision Process or MDP consists of the planning that an autonomous vehicle’s course often involves.  This sequential decision-making framework is also called the Tree of Possibilities.

Just like a tree can branch out along each node into a multiple number of branches, the decisions made by the drone will help it move sequentially from its starting to its ending point. Each action can result in a number of more possibilities. The drone can compute the different number of routes it can take to reach a location even before it takes to the air.

Then it can compute the most optimal route based on another set of variables such as distance, time, and climatic conditions. The final route it decides to fly can also be changed mid flight in case it encounters problems. There is literally a huge number of choices that the drone needs to keep making from the time it takes off till the time it returns.

It is the computer programmer’s job to ensure that the algorithm that he writes will take into account all these variables and allow the drone to reach the right decision each time. As you can well imagine, this is not an easy science project.