Project Summary

Computer vision based Artificial Intelligence (AI) has matured over the years and is used in many applications including traffic categorization, human rights violation, weather forecasting, object detection and classification. However, such vision based AI is computationally expensive and hence, such approaches either require processing (training) on the cloud (using cloud offloading) or training on a general-purpose computer and then using the AI model for inference only in the mobile platforms. On the other hand, many robotic applications in the agri-food industry are performed on battery-operated mobile platforms. Such robotic applications include using computer vision based AI to detect or counting fruits and vegetables in the fields. However, given the computationally expensive nature of such applications, one of the key motives is to design and implement low-powered vision AI models such that they could be trained/executed on battery-operated mobile platforms. This project is dedicated to designing and implementing such low-power AI models for the agricultural industry.

Potential use in non-academic contexts:

Using Convolutional Neural Networks to automatically detect and classify fruits and vegetables within the nosh app and add them to the stocked list.

Key Findings:

Abstract: Fruit and vegetable classification using Convolutional Neural Networks (CNNs) has become a popular application in the agricultural industry, however, to the best of our knowledge no previously recorded study has designed and evaluated such an application on a mobile platform. In this paper, we propose a power-efficient CNN model, FruitVegCNN, to perform classification of fruits and vegetables in a mobile multi-processor system-on-a-chip (MPSoC). We also evaluated the efficacy of FruitVegCNN compared to popular state-of-the-art CNN models in real mobile platforms (Huawei P20 Lite and Samsung Galaxy Note 9) and experimental results show the efficacy and power efficiency of our proposed CNN architecture.

Abstract: This paper proposes a novel human-inspired methodology called IRON-MAN (Integrated RatiONal prediction and Motionless ANalysis) for mobile multi-processor systems-on-chips (MPSoCs). The methodology integrates analysis of the previous image frames of the video to represent the analysis of the current frame in order to perform Temporal Motionless Analysis of the Video (TMAV). This is the first work on TMAV using Convolutional Neural Network (CNN) for scene prediction in MPSoCs. Experimental results show that our methodology outperforms state-of-the-art. We also introduce a metric named, Energy Consumption per Training Image (ECTI) to assess the suitability of using a CNN model in mobile MPSoCs with a focus on energy consumption and lifespan reliability of the device.

Abstract: Side-channel attacks remain a challenge to information flow control and security in mobile edge devices till this date. One such important security flaw could be exploited through temperature side-channel attacks, where heat dissipation and propagation from the processing cores are observed over time in order to deduce security flaws. In this paper, we study how computer vision-based convolutional neural networks (CNNs) could be used to exploit temperature (thermal) side-channel attack on different Linux governors in mobile edge device utilizing multi-processor system-on-chip (MPSoC). We also designed a power- and memory-efficient CNN model that is capable of performing thermal side-channel attack on the MPSoC and can be used by industry practitioners and academics as a benchmark to design methodologies to secure against such an attack in MPSoC.