Singh, Richa

Generative Adversarial Networks (GANs) can generate hyperrealistic face images of synthetic identities based on a latent understanding of real images from a large training set. Despite their proficiency, the term "synthetic identity"remains ambiguous, and the uniqueness of the faces GANs produce is rarely assessed. Recent studies have found that identities from the training data can unintentionally appear in the faces generated by StyleGAN2, but the cause of this phenomenon is unclear. In this work, we propose a novel framework, SynthProv, that utilizes the improved interpolation ability of StyleGAN2 latent space and employs image composition to analyze leakage. This is the first method that goes beyond detection and traces the source or provenance of constituent identity signals in the generated image. Experiments show that SynthProv succeeds in both detection and provenance tasks using multiple matching strategies. We identify identities from FFHQ and CelebA-HQ training datasets with the highest leakage into the latent space as "leaking reals". Analyzing latent space behavior to evaluate generative model privacy via leakage is an important research direction, as undetected leaking reals pose a significant threat to training data privacy. Our code is available at https://github.com/jaisidhsingh/SynthProv.

The rising global prevalence of skin conditions, some of which can escalate to life-threatening stages if not timely diagnosed and treated, presents a significant healthcare challenge. This issue is particularly acute in remote areas where limited access to healthcare often results in delayed treatment, allowing skin diseases to advance to more critical stages. One of the primary challenges in diagnosing skin diseases is their low inter-class variations, as many exhibit similar visual characteristics, making accurate classification challenging. This research introduces a novel multimodal method for classifying skin lesions, integrating smartphone-captured images with essential clinical and demographic information. This approach mimics the diagnostic process employed by medical professionals. A distinctive aspect of this method is the integration of an auxiliary task focused on super-resolution image prediction. This component plays a crucial role in refining visual details and enhancing feature extraction, leading to improved differentiation between classes and, consequently, elevating the overall effectiveness of the model. The experimental evaluations have been conducted using the PAD-UFES20 dataset, applying various deep-learning architectures. The results of these experiments not only demonstrate the effectiveness of the proposed method but also its potential applicability under-resourced healthcare environments.

This research addresses the challenges of diagnosing chest X-rays (CXRs) at low resolutions, a common limitation in resource-constrained healthcare settings. High-resolution CXR imaging is crucial for identifying small but critical anomalies, such as nodules or opacities. However, when images are downsized for processing in Computer-Aided Diagnosis (CAD) systems, vital spatial details and receptive fields are lost, hampering diagnosis accuracy. To address this, this paper presents the Multilevel Collaborative Attention Knowledge (MLCAK) method. This approach leverages the self-attention mechanism of Vision Transformers (ViT) to transfer critical diagnostic knowledge from high-resolution images to enhance the diagnostic efficacy of low-resolution CXRs. MLCAK incorporates local pathological findings to boost model explainability, enabling more accurate global predictions in a multi-task framework tailored for low-resolution CXR analysis. Our research, utilizing the Vindr CXR dataset, shows a considerable enhancement in the ability to diagnose diseases from low-resolution images (e.g. 28 × 28), suggesting a critical transition from the traditional reliance on high-resolution imaging (e.g. 224 × 224).

Advances in deep learning have enabled the creation of photo-realistic DeepFakes by switching the identity or expression of individuals. Such technology in the wrong hands can seed chaos through blackmail, extortion, and forging false statements of influential individuals. This work proposes a novel approach to detect forged videos by magnifying their temporal inconsistencies. A study is also conducted to understand role of ethnicity bias due to skewed datasets on deepfake detection. A new dataset comprising forged videos of Indian ethnicity individuals is presented to facilitate this study.

During this unprecedented time of the COVID19 pandemic, wearing face masks has become a necessity. While these masks aim to secure an individual from getting infected by any kind of viruses including COVID-19; they significantly obfuscate the identity. The situation becomes even worse when an attacker performs a crime and the place does not have any surveillance cameras. The identification of criminals in such conditions highly depends on the witnesses and generation of sketches based on their description. To the best of our knowledge, in the literature, no work has been performed for matching sketch images with masks. In this research, we have first created the mask sketch face database using more than 50 identities. The sketch images are generated using a different variant of pencils, which can be seen as different sketch artists. The recognition experiments are performed using state-of-the-art face embedding networks including ArcFace and DeepID which show that the recognition performance degrades significantly when the sketch mask images are used for identification. In another set of experiments, it is observed that the recognition algorithm is robust in handling the digital face mask images. However, the ineffectiveness in handling the variations that occurred due to sketches is a serious concern and needs attention.

The solutions proposed in the literature for adversarial robustness are either not effective against the challenging gradient-based attacks or are computationally demanding, such as adversarial training. Adversarial training or network training based data augmentation shows the potential to increase the adversarial robustness. While the training seems compelling, it is not feasible for resource-constrained institutions, especially academia, to train the network from scratch multiple times. The two fold contributions are: (i) providing an effective solution against white-box adversarial attacks via network fine-tuning steps and (ii) observing the role of different optimizers towards robustness. Extensive experiments are performed on a range of databases, including Fashion-MNIST and a subset of ImageNet. It is found that the few steps of network fine-tuning effectively increases the robustness of both shallow and deep architectures. To know other interesting observations, especially regarding the role of the optimizer, refer to the paper.

Due to the large applicability of AI systems in various applications, fairness in model predictions is extremely important to ensure that the systems work equally well for everyone. Biased feature representations might often lead to unfair model predictions. To address the concern, in this research, a novel method, termed as Attention Aware Debiasing (AAD) method, is proposed to learn unbiased feature representations. The proposed method uses an attention mechanism to focus on the features important for the main task while suppressing the features related to the sensitive attributes. This minimizes the model's dependency on the sensitive attribute while performing the main task. Multiple experiments are performed on two publicly available datasets, MORPH and UTKFace, to showcase the effectiveness of the proposed AAD method for bias mitigation. The proposed AAD method enhances the overall model performance and reduces the disparity in model prediction across different subgroups.

Accidental new born swapping, health-care tracking, and child-abduction cases are some of the scenarios where new born face recognition can prove to be extremely useful. With the help of the right biometric system in place, cases of swapping, for instance, can be evaluated much faster. In this chapter, we first discuss the various biometric modalities along with their advantages and limitations. We next discuss the face biometrics in detail and present all the datasets available and existing hand-crafted, learning-based, as well as deep-learning-based techniques which have been proposed for new born face recognition. Finally, we evaluate and compare these techniques. Our comparative analysis shows that the state-of-the-art SSF-CNN technique achieves an average of rank-1 new born accuracy of 82.075 %.

Recently, transformers have become incredibly popular in computer vision and vision-language tasks. This notable rise in their usage can be primarily attributed to the capabilities offered by attention mechanisms and the outstanding ability of transformers to adapt and apply themselves to a variety of tasks and domains. Their versatility and state-of-the-art performance have established them as indispensable tools for a wide array of applications. However, in the constantly changing landscape of machine learning, the assurance of the trustworthiness of transformers holds utmost importance. This paper conducts a thorough examination of vision-language transformers, employing three fundamental principles of responsible AI: Bias, Robustness, and Interpretability. The primary objective of this paper is to delve into the intricacies and complexities associated with the practical use of transformers, with the overarching goal of advancing our comprehension of how to enhance their reliability and accountability.

Deep learning models have demonstrated significant effectiveness in addressing intricate object segmentation and image classification tasks. Nevertheless, their widespread use is impeded by high computational demands, limiting their applicability on resource-constrained devices and in contexts like medical image segmentation. This paper proposes AssistDistil, a semi-knowledge distillation technique designed to facilitate the transfer of knowledge from a larger teacher network to a more compact student model. During the inference process, the student model works in conjunction with the teacher model by condensing the teacher model's latent information into its own latent representation, thereby boosting its representational capacity. The effectiveness of the proposed approach is demonstrated for multiple case studies in medical image segmentation task of eye segmentation, skin lesion segmentation, and chest X-ray segmentation. Experimental results on the IIITD Cataract Surgery, HAM10000, PH2, Shenzhen and Montgomery chest X-ray datasets demonstrate the efficacy of the proposed approach both in terms of accuracy and computational cost. For example, in comparison to the AUNet-based teacher model, the proposed approach achieves a similar mIOU with only 0.5% of the model size. In the future, we plan to explore knowledge distillation approaches to improve the distillation process in case of large model capacity gap between teacher and student networks. With fewer parameters, we intend for the student model to attain performance comparable to that of the teacher model without additional assistance.