Semi-supervised learning algorithm to estimate mass flow from sparsely annotated images using a vision system (SLEM)

Hamdan, Muhammad
Journal Title
Journal ISSN
Volume Title
Research Projects
Organizational Units
Journal Issue

Supervised learning is the workhorse for regression and classification tasks, but the standard approach presumes ground truth (label) for every measurement. In real world applications, limitations due to expense or general infeasibility due to the specific application are common. In the context of agriculture applications, yield monitoring is one such example where simple physics-based measurements such as volume or force-impact have been used to quantify mass flow. These measurements incur error due to sensor calibration.

Using a vision system capturing images of a sugarcane elevator, mass of flowing material (bamboo and sugarcane) is accurately predicted from sparsely annotated images by training a deep neural network (DNN) in a semi-supervised fashion using only final load weights. The DNN succeeds in capturing the complex density physics of random stacking of slender rods as part of the mass prediction model, and surpasses older volumetric-based methods for mass prediction. Furthermore, by incorporating knowledge about the system physics through the DNN architecture and penalty terms, improvements in prediction accuracy and stability as well as faster learning are obtained. It is shown that the classic nonlinear regression optimization can be reformulated with an aggregation term with some independence assumptions to achieve this feat. Since the number of images for any given run is too large to fit on typical GPU vRAM, an implementation is shown that compensates for the limited memory but still achieve fast training times. The same approach presented herein could be applied to other applications like yield monitoring on grain combines or other harvesters using vision or other instrumentation and is by no means limited to the sugarcane application. Using a vision system with a relatively lightweight deep neural network we are able to demonstrate the generalizability of presented methods by estimating mass of bamboo with an average error of 4.5% and 5.9% for a select season of sugarcane.

Computer vision, Deep learning, Semi-supervised learning, Sparse data, Video processing