Vehicle Safeguarding
Application




Tractor accidents occur with surprising frequency

The Problem
Agricultural equipment is involved in a significant number of accidents each year, often resulting in serious injuries or death. Most of these accidents are due to operator error, and could be prevented if the operator could be warned about hazards in the vehicle’s path or operating environment.

At the same time, full automation is only a few steps away in agriculture. John Deere has had great success in commercializing AutoTrac, a John Deere developed automatic steering system based on GPS positioning. AutoTrac is currently sold as an operator-assist product, and does not have any obstacle detection capabilities. Adding machine awareness provides safeguarding to a product like AutoTrac, for example, that would be a significant enabler to full vehicle automation.

Any perception system that is used for safeguarding in this domain should have a very high probability of detecting hazards and a low false alarm rate that does not significantly impact the productivity of the machine.

The Solution
The NREC developed a perception system based on multiple sensing modalities (color, infrared and range data) that can be adapted easily to the different environments and operating conditions to which agricultural equipment is exposed.

We have chosen to detect obstacles and hazards based on color and infrared imagery, together with range data from laser range finders. These sensing modalities are complementary and have different failure modes. By fusing the information produced by all the sensors, the robustness of the overall system is significantly improved beyond the capabilities of individual perception sensors.

An important design choice was to embed modern machine learning techniques in several modules of our perception system. This makes it possible to quickly adapt the system to new environments and new types of operations, which is important for the environmental complexity of the agricultural domain.