Skip to content

Implementation of Zero-shot Anomaly Segmentation (ZSAS) Model based on Dynamic Object-aware Tagging (DOT) Prompt

Notifications You must be signed in to change notification settings

sybeam27/DOT-ZSAS

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

DOT Prompt: Dynamic Object-Aware Tagging Prompt for Texture Zero-Shot Anomlay Segmentation

Zero-Shot Anomaly Segmentation by DOT Prompt, a model designed to detect anomalies in images, precisely identify their locations, and restore abnormal images to normal using LLM prompting and zero-shot segmentation techniques. framework

Requirements & Setup

This codebase utilizes Anaconda for managing environmental dependencies. Please follow these steps to set up the environment:

  1. Download Anaconda: Click here to download Anaconda.
  2. Clone the Repository: Clone the repository using the following command.
    git clone https://github.com/sybeam27/DOT-Prompt
  3. Install Requirements:
    • Navigate to the cloned repository:
      cd DOT-ZSAS
    • Create a Conda environment from the provided environment.yaml file:
      conda env create -f environment.yaml
    • Activate the Conda environment:
      conda activate dot_zsas

This will set up the environment required to run the codebase.

Datasets

Below are the details and download links for datasets used in our experiments:

  1. MVTec-AD (Download): The MVTec AD dataset comprises approximately 5,000 images across 15 classes, including texture-related categories such as fabric and wood.
  2. KSDD1 (Download): The KSDD1 dataset includes 347 normal images and 52 abnormal images, specifically for detecting micro-defects on metal surfaces.
  3. MTD (Download): The MTD dataset contains images of magnetic tiles, featuring various types of defects. These datasets provide valuable resources for our experiments and each known for their high-resolution, texture-rich images that are well-suited for texture anomaly segmentation.

Zero-Shot Anomaly Segmentation (ZSAS) TEST

Replace <dataset> with one of the following options: mvtec, ksdd, mtd.

Replace <model> with one of the following options: base, dot_zsas.

python test_zsas.py --dataset <dataset name> --model <model name> 

This command excel our proposed model for zero-shot anomaly segmentation(ZSAS) on the specified dataset using the selected model, with best configurations loaded, running 10 epochs each.

Ablation Study TEST

Ablatin study on MVTec-AD texture dataset.

python test_ablation.py --image True --prompt True --filter True 

Optional arguments

  --gpu                             gpu number
  --dataset                         dataset name
  --model                           model name
  --box_threshold                   GroundingSAM box threshold
  --text_threshold                  GroundingSAM text threshold
  --size_threshold                  Bounding-box size threshold
  --iou_threshold                   IoU threshold
  --random_img_num                  random image extraction number
  --eval_resolution                 Description of evaluation resolution
  --exp_idx                         Description of experiment index
  --version                         Description of evaluation version

Special Thanks to

We extend our gratitude to the authors of the following libraries for generously sharing their source code and dataset:

RAM, Llama3, Grounding DINO, SAM, SAA+ Your contributions are greatly appreciated.

Citation

About

Implementation of Zero-shot Anomaly Segmentation (ZSAS) Model based on Dynamic Object-aware Tagging (DOT) Prompt

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published