Thesis: Enhancing Post-Harvest Harumanis Mango Sorting Process using Fourier Descriptor on a Customizable Vision System

Abstract: The use of vision systems in agriculture has been found to be very beneficial and is quickly becoming somewhat of a necessity for a modern farming system. Various agriculture activities from crop monitoring to post-harvest processes like cleaning and sorting, can be improved tremendously with the help of vision systems. An architecture for a fully-customized vision system implementation that can be used to enhance the sorting process of harumanis mango is proposed. The proposed architecture is presented using generic design components that are not dependent on any hardware platform or software library. This makes it very easy to reimplement the system and to customize it to meet any required cost-performance ratio. A hardware-centric approach is used to obtain faster processing time. Using a Field-Programmable Gate Array (FPGA) as additional processing element to handle basic image processing tasks provide more time for the main processing element to focus on high level vision algorithms like object classifications and tracking. A couple of implementations have been tested on different FPGA devices and are found to be more than capable of executing basic image processing tasks at 30 frame-per-second (fps), which is the normal frame rate of most imaging device. This shows that is can easily be inserted in an existing system with minimal modifications, if any. An open-source image processing library written in C (my1imgpro) has been developed and used as the base code for the software side of the proposed architecture. A custom connected-component labelling (CCL) algorithm and a custom border-following algorithm have been developed on top of that library. Both algorithms, along with other software components, have been developed to implement the software components of the proposed architecture. Shaped-based classification of harumanis is also proposed as a new classification method instead of weight and size. Fourier Descriptors (FD) are used as the shape descriptor for harumanis mango classification. Using k-Means Clustering algorithm, all 306 contours that have been successfully detected are classified into 5 categories based on the 24-term FDs generated using centroid distance data. Further analysis shows that the 5 categories can be finalized into 3 fundamental categories (elongated, normal, odd). These can potentially be used as a new standard by which harumanis mango is sorted and graded for commercial market. A fully software-based system using the proposed architecture have been implemented on a desktop computer. The same code have also been successfully compiled and executed on a Raspberry Pi 3 platform, which enables the system to be highly portable. Using live camera feed, both implementations are able to classify all available harumanis mango images into the proposed grades. The processing rate of this implementation implies that the sorting process can be done for a volume of about 15-tonnes per day (assuming 8-hour of daily operation time of a single conveyor line). This indicates that a hardware-centric implementation should have a much better throughput than that.