TRRAFICC: Toolbox for Robust, Real-time, Automated FIsh Classification and Counting

Lead PI: Dipanjan Saha, Northeastern University

Objectives
To: (1) develop and test multi-modal sensing and advanced machine learning tools for fish classification, counting, and preliminary health assessment towards habitat quality estimation, (2) develop an interface on a portable device for regular use by fisheries and marine scientists, (3) enhance the skillset of one graduate and one undergraduate researcher in areas related to
sensor fusion, machine learning, and interface development, (4) foster collaboration among Northeastern, MIT Sea Grant (MITSG), industry, and other partners to address stakeholder-driven needs and development of technologies designed to overcome sustainable fisheries challenges. MITSG will provide non-compensated Advisory Services to fully engage stakeholders.

Methodology
We envision TRRAFICC as a union of three key aspects: (i) sensing, (ii) algorithms, (iii) interface. These aspects, especially (i) and (ii), are significantly coupled, and hence the final design choices will be decided after many iterations. We will begin the first iteration with three modalities of sensing: acoustic, optical, and hyperspectral. The algorithms will sequentially execute fish detection, classification, updating species count and a binary health assessment (healthy vs. not healthy). The first design of the interface will be according to stakeholders\’ needs and preferences. Trial runs and user feedback will be used to improve the interface.

Rationale
Accurate classification and counting of fish species are essential for sustainability, marine ecosystem monitoring, and understanding patterns of fish abundance and behavior in underwater habitats. TRRAFICC will classify and count fish in realtime, in diverse habitats, reliably in the presence of uncertainties, and without human intervention. We will develop an interface for regular use by fisheries and marine scientists without any significant background on machine learning. It will be capable of daily monitoring of a habitat and its fish population, tracking of fish movement from ocean to freshwater and vice versa, improved understanding of fish behavior, and so on.