Learning Domain Specific Generative Image Representations for Fine Grained Visual Classification and Species Discovery

This research aims to refine image representation learning for fine-grained classification within generalized category discovery (GCD), particularly for biological and environmental monitoring tasks like species diversity estimation, focusing on plankton. GCD involves learning from labeled and unlabeled datasets to classify both known and novel classes.

Traditional self-supervised pre-training methods, though effective, often overlook details crucial for fine-grained classification.

We propose using generative self-supervision, specifically through reconstructing masked image regions, as an alternative. This approach promises to train robust image representations without extensive data augmentation.

Our research will explore pre-trained generative image representations for GCD, developing new methods tailored to domain-specific data using practically sized networks like ResNet18, ResNet50, and ViT-S.

We hypothesize that diffusion image generation networks, with their exceptional image-generating capabilities, contain powerful image features useful for our task.

With access to a vast repository of diverse, unlabeled biological images, we aim to make our methodologies accessible to practitioners, enhancing the applicability of our work in real-world settings.