A watchful eye on the self-driving tractor

A watchful eye on the self-driving tractor

Peter Christiansen (left) and Mikkel Fly Kragh have trained a computer to register hazards on a field and then send a message to the tractor about the most appropriate speed and direction. In the long term, the same method could also make driverless cars safer. Photo: Lars Kruse.

Researchers have equipped a self-driving tractor with stereo vision and artificial intelligence, so it can very accurately identify foreign objects in the field and thereby significantly increase safety.

At Aarhus University, farming has for a number of years been a learning lab – an ‘experimentarium’ – for specific applications of artificial intelligence. Here the researchers feed supercomputers with data, and train them to recognise patterns so they achieve a power of analysis, interpretation and judgement that can exceed the human ability to solve advanced tasks.

It is concerned with a phenomenon known as the artificial neural network, where advanced algorithms get machines and robots to learn correlations and act autonomously. This is actually a crucial step on the way to efficient farming with self-driving machines.

“Via deep convolutional neural networks, we teach the computer to analyse large amounts of visual information in the form of image data sets, so it eventually becomes capable of recognising visual characteristics such as a person, a house, a cat or a cow,” says PhD Student Peter Christiansen.

The technology has enormous potential, and the researchers can now demonstrate that the method could make driverless tractors safer than their human-controlled counterparts.

Sees 700,000 dots per second
At the experimental stage, the researchers have equipped an ordinary tractor with a stereo camera, a thermal camera, a radar and a LiDAR, and these enable it to carry out 700,000 distance measurements per second and reproduce them in a 3D representation of the surroundings.

In this way, the tractor can identify obstacles in the field and spot specific objects without human intervention.

“The machine itself now interprets visual and geometrical information from the field at a level where even a very perceptive farmer might give up,” says PhD Student Mikkel Fly Kragh.

The researchers working on the project have built further on an existing neural network with an algorithm that can differentiate between 1,000 classes of objects. They have fine-tuned this so it can also identify unspecified objects that do not belong in a field.

“When the tractor drives across the field, it identifies different objects or elements in the image. This way, it registers the normal environment such as grass, trees, sky and bushes, and it activates the neurons in our network in a particular way. The computer recognises these usual neural activations and can therefore react if the activation deviates from the known pattern,” says Peter Christiansen.

Artificial brain finds danger in the field
The tractor can thus not only see that a person is approaching, but can also see whether an unknown object turns up. This is a crucial step as regards making the technology safe,” explains Peter Christiansen.

“What’s special about our scientific work is the algorithm that can take unforeseen objects into account in the field. This could be a situation where a child playing in a snowsuit and balaclava is hidden by the crops, which would be difficult for the computer to recognise in the human category. In this case, the tractor with our system would have a chance to react to an unknown object and prevent running the child over,” he says.

The mathematical work behind the algorithm is quite complicated, and it can get the tractor to spot objects such as people in the field at a greater distance than is possible with existing safety software for autonomous vehicles.