DFTNet: Deep Fish Tracker with Attention Mechanism in Unconstrained Marine Environments

Multiple fish tracking in unconstrained marine videos is a highly challenging task. Trajectories of fishes convey critical information for the analysis of fish behavior. In this article, we have proposed deep fish tracking network (DFTNet) that incorporates Siamese network for encoding the appearance similarity and attention long short-term memory network to capture the motion similarity across subsequent frames. Finally, intersection-over-union matching score is computed to amalgamate spatial similarity cue in the final score. The proposed framework can provide joint optimization score to maintain the tracklet information encoding appearance, motion, and spatial similarity cues. We perform exhaustive experiments and compare the proposed approach with competing techniques over Fish4knowledge videos and achieve significant average reduction in identification (ID) switches by 60.9%. The source code is made publicly available at https://github.com/hemanth-s17/Deep-Fish-Tracker-Network.