Low-Power Vision Artificial Intelligence in Mobile Devices For Agriculture

Research Reference: nosh/agri-tech-000001

Principal Investigator: Somdip Dey

Other Investigators: Suman Saha

Project Summary

Computer vision based Artificial Intelligence (AI) has matured over the years and is used in many applications including traffic categorization, human rights violation, weather forecasting, object detection and classification. However, such vision based AI is computationally expensive and hence, such approaches either require processing (training) on the cloud (using cloud offloading) or training on a general-purpose computer and then using the AI model for inference only in the mobile platforms. On the other hand, many robotic applications in the agri-food industry are performed on battery-operated mobile platforms. Such robotic applications include using computer vision based AI to detect or counting fruits and vegetables in the fields. However, given the computationally expensive nature of such applications, one of the key motives is to design and implement low-powered vision AI models such that they could be trained/executed on battery-operated mobile platforms. This project is dedicated to designing and implementing such low-power AI models for the agricultural industry.

Potential use in non-academic contexts:

Using Convolutional Neural Networks to automatically detect and classify fruits and vegetables within the nosh app and add them to the stocked list.

Abstract: Fruit and vegetable classification using Convolutional Neural Networks (CNNs) has become a popular application in the agricultural industry, however, to the best of our knowledge no previously recorded study has designed and evaluated such an application on a mobile platform. In this paper, we propose a power-efficient CNN model, FruitVegCNN, to perform classification of fruits and vegetables in a mobile multi-processor system-on-a-chip (MPSoC). We also evaluated the efficacy of FruitVegCNN compared to popular state-of-the-art CNN models in real mobile platforms (Huawei P20 Lite and Samsung Galaxy Note 9) and experimental results show the efficacy and power efficiency of our proposed CNN architecture.

IRON-MAN: An Approach To Perform Temporal Motionless Analysis of Video using CNN in MPSoC

Abstract: This paper proposes a novel human-inspired methodology called IRON-MAN (Integrated RatiONal prediction and Motionless ANalysis) for mobile multi-processor systems-on-chips (MPSoCs). The methodology integrates analysis of the previous image frames of the video to represent the analysis of the current frame in order to perform Temporal Motionless Analysis of the Video (TMAV). This is the first work on TMAV using Convolutional Neural Network (CNN) for scene prediction in MPSoCs. Experimental results show that our methodology outperforms state-of-the-art. We also introduce a metric named, Energy Consumption per Training Image (ECTI) to assess the suitability of using a CNN model in mobile MPSoCs with a focus on energy consumption and lifespan reliability of the device.