On Edge FPN Reduction in CMOS Image Sensor Using CNN with Attention Mechanism

Obtaining a good quality image from a CMOS Image Sensor (CIS) is always a constraint due to the effect of noise present within the image sensor system. One of the dominant source of noise in CIS with column-parallel readout is Fixed Pattern Noise (FPN) which significantly degrade the image quality. This work implements an architecture for the reduction of vertical FPN called Fixed Pattern Noise Reduction Network (FPNrNet), which uses a Convolutional Neural Network (CNN) with an attention mechanism. The denoising performance of the FPNrNet model is quite similar to that of standard denoising models; however, a significant reduction in model size is observed due to a reduction in the number of parameters. An average Peak Signal-to-Noise Ratio (PSNR) improvement of around 11.3 dB with respect to input noisy image and an average Structural Similarity Index Measure (SSIM) of 0.99 is observed for Pascal VOC 2012 dataset. Further, the model is quantized on different bit precision using the Qkeras library and synthesized using the High-Level Synthesis for Machine Learning (hls4ml) platform to make it hardware friendly so that inference can be performed on resource-constrained edge devices.