README.md

---
license: apache-2.0
---

# InsectSAM: Insect Segmentation and Monitoring

![rb-ibdm-banner](https://github.com/martintmv-git/RB-IBDM/assets/101264514/a22f2069-e3c8-4a4f-a314-59cb01b39b66)

## Overview

InsectSAM is a fine-tuned version of Meta AI's `segment-anything` model, optimized for insect segmentation and monitoring in the Netherlands. Designed for use with the [DIOPSIS](https://diopsis.eu) camera systems, algorithms and datasets, it enhances the accuracy of insect biodiversity segmentation from complex backgrounds.

## Purpose

Trained to segment insects against diverse backgrounds, InsectSAM adapts to changing environments, ensuring its long-term utility for the DIOPSIS datasets.

## Model Architecture

Built on the `segment-anything` architecture, InsectSAM is fine-tuned on an insect-specific dataset and integrated with GroundingDINO for improved detail recognition.

## Quick Start

### Prerequisites

- Python
- Hugging Face Transformers
- PyTorch

### Usage

#### Install 
```bash
!pip install --upgrade -q git+https://github.com/huggingface/transformers
!pip install torch
```

#### Load model via 🤗 Transformers
```python
from transformers import AutoProcessor, AutoModelForMaskGeneration

processor = AutoProcessor.from_pretrained("martintomov/InsectSAM")
model = AutoModelForMaskGeneration.from_pretrained("martintomov/InsectSAM")
```

### Notebooks

Explore InsectSAM's capabilities and integration with GroundingDINO through three Jupyter notebooks available on the RB-IBDM GitHub page:
- [**InsectSAM.ipynb**](https://github.com/martintmv-git/RB-IBDM/blob/main/InsectSAM/InsectSAM.ipynb): Training process
- [**InsectSAM_GroundingDINO.ipynb**](https://github.com/martintmv-git/RB-IBDM/blob/main/InsectSAM/InsectSAM_GroundingDINO.ipynb): Enhanced segmentation performance with GroundingDINO
- [**InsectSAM_script.ipynb**](https://github.com/martintmv-git/RB-IBDM/tree/main/Image%20Processing%20Scripts/InsectSAM): Image processing script

GitHub: https://github.com/martintmv-git/RB-IBDM/tree/main/InsectSAM