Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking β€œSign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding the pipeline for the task explanation and Llm #2190

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
adbca17
Add Task EXPLANATION and the visualization of images with description.
Bepitic Jul 15, 2024
5611ec1
upd dataset task with explanation
Bepitic Jul 15, 2024
8ed23a3
fix tasktype on metrics, depth, cataset, inferencer.
Bepitic Jul 15, 2024
a463b5b
Merge branch 'main' into llm-pipeline
Bepitic Jul 15, 2024
d5baf6b
fix lint on visualization/image
Bepitic Jul 16, 2024
b7c8eaa
Merge branch 'openvinotoolkit:main' into llm-pipeline
Bepitic Jul 18, 2024
5b563d9
Merge branch 'llm-pipeline' of github.com:Bepitic/anomalib into llm-p…
Bepitic Jul 18, 2024
bfd936e
Fix formatting dataset
Bepitic Jul 18, 2024
f541316
fix format data/base/depth
Bepitic Jul 18, 2024
4e392a9
Fix formatting openvino_inferencer
Bepitic Jul 18, 2024
5fc70ba
fix formatting
Bepitic Jul 18, 2024
75099af
Add Explanation to error-msg.
Bepitic Aug 2, 2024
e5040d3
OpenAI - VLM init
Bepitic Aug 3, 2024
86ad803
Add wrapper to run OpenAI
Bepitic Aug 4, 2024
3678f72
add in ppyproject
Bepitic Aug 4, 2024
7413842
Add Test and fix description/title
Bepitic Aug 12, 2024
dc42cbd
Add Readme and fix bug.
Bepitic Aug 13, 2024
5788d22
Update src/anomalib/models/image/openai_vlm/lightning_model.py
Bepitic Aug 13, 2024
e4f6bec
Update src/anomalib/models/image/openai_vlm/__init__.py
Bepitic Aug 13, 2024
5437467
Add fix pipeline bug.
Bepitic Aug 13, 2024
982c9ca
Add test.
Bepitic Aug 13, 2024
642fd26
Merge branch 'OpenAI-VLM' of github.com:Bepitic/anomalib into OpenAI-VLM
Bepitic Aug 13, 2024
b8cacf0
add changes
Bepitic Aug 16, 2024
0929dc9
Add integration test and unit test + skip export.
Bepitic Aug 16, 2024
39cf996
change to LANGUAGE
Bepitic Aug 16, 2024
671693d
Update images in Readme.
Bepitic Aug 17, 2024
224118b
Update src/anomalib/models/image/chatgpt_vision/__init__.py
Bepitic Aug 20, 2024
b703a41
Update src/anomalib/models/image/chatgpt_vision/chatgpt.py
Bepitic Aug 20, 2024
24c5486
Update src/anomalib/models/image/chatgpt_vision/lightning_model.py
Bepitic Aug 20, 2024
68e757e
Update tests/integration/model/test_models.py
Bepitic Aug 20, 2024
86714a1
Update src/anomalib/models/image/chatgpt_vision/lightning_model.py
Bepitic Aug 20, 2024
196d2a3
Update src/anomalib/models/image/chatgpt_vision/lightning_model.py
Bepitic Aug 20, 2024
b7f345a
fix comments
Bepitic Aug 20, 2024
b285d10
remove last file of chatgpt_vision.
Bepitic Aug 20, 2024
a688530
fix tests
Bepitic Aug 20, 2024
0fb5f79
Merge pull request #1 from Bepitic/OpenAI-VLM (GPTVad)
Bepitic Aug 20, 2024
6503543
Merge branch 'main' into llm-pipeline
Bepitic Aug 20, 2024
8e92e5e
Update src/anomalib/models/image/gptvad/chatgpt.py
Bepitic Aug 21, 2024
5ab044d
upd: language -> VISUAL_PROMPTING
Bepitic Aug 21, 2024
3f9ca93
fix visual prompting and model_name
Bepitic Aug 21, 2024
391b4c4
fix GPT for Gpt and the folder of the tests.
Bepitic Aug 21, 2024
ca1a0bb
fix: change import error outside.
Bepitic Aug 21, 2024
022dcb7
fix readme pointing to the right model.
Bepitic Aug 21, 2024
af7b9e9
fix import cycle, and separate usecase by explicit if.
Bepitic Aug 21, 2024
faf334f
upd: add comments to the few shot / zero shot.
Bepitic Aug 21, 2024
3ed8d3f
fix: dataset expected colums
Bepitic Aug 21, 2024
7f454c4
upd: add the same logic of the label on visualize_full.
Bepitic Aug 22, 2024
45bd520
Merge branch 'main' into llm-pipeline
Bepitic Aug 22, 2024
44586d6
Fix in the logic of the code.
Bepitic Aug 22, 2024
7adb835
Merge branch 'llm-pipeline' of github.com:Bepitic/anomalib into llm-p…
Bepitic Aug 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix GPT for Gpt and the folder of the tests.
Signed-off-by: Bepitic <bepitic@gmail.com>
  • Loading branch information
Bepitic committed Aug 21, 2024
commit 391b4c4578cd2e39be403888bf249c89d519e2e5
4 changes: 2 additions & 2 deletions src/anomalib/models/__init__.py
Original file line number Diff line number Diff line change
@@ -24,7 +24,7 @@
Fastflow,
Fre,
Ganomaly,
GPTVad,
GptVad,
Padim,
Patchcore,
ReverseDistillation,
@@ -52,7 +52,7 @@ class UnknownModelError(ModuleNotFoundError):
"Fastflow",
"Fre",
"Ganomaly",
"GPTVad",
"GptVad",
"Padim",
"Patchcore",
"ReverseDistillation",
4 changes: 2 additions & 2 deletions src/anomalib/models/image/__init__.py
Original file line number Diff line number Diff line change
@@ -14,7 +14,7 @@
from .fastflow import Fastflow
from .fre import Fre
from .ganomaly import Ganomaly
from .gptvad import GPTVad
from .gptvad import GptVad
from .padim import Padim
from .patchcore import Patchcore
from .reverse_distillation import ReverseDistillation
@@ -35,7 +35,7 @@
"Fastflow",
"Fre",
"Ganomaly",
"GPTVad",
"GptVad",
"Padim",
"Patchcore",
"ReverseDistillation",
4 changes: 2 additions & 2 deletions src/anomalib/models/image/gptvad/__init__.py
Original file line number Diff line number Diff line change
@@ -3,6 +3,6 @@
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

from .lightning_model import GPTVad
from .lightning_model import GptVad

__all__ = ["GPTVad"]
__all__ = ["GptVad"]
4 changes: 2 additions & 2 deletions src/anomalib/models/image/gptvad/lightning_model.py
Original file line number Diff line number Diff line change
@@ -21,10 +21,10 @@

logger = logging.getLogger(__name__)

__all__ = ["GPTVad"]
__all__ = ["GptVad"]


class GPTVad(AnomalyModule):
class GptVad(AnomalyModule):
"""OpenAI VLM Lightning model using OpenAI's GPT-4 for image anomaly detection.

Args:
6 changes: 3 additions & 3 deletions src/anomalib/models/image/gptvad/readme.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# GPTVad: Zero-/Few-Shot Anomaly Classification
# GptVad: Zero-/Few-Shot Anomaly Classification

This repository contains the implementation of the `OpenAI VLM`, a model designed for zero-shot and few-shot anomaly detection using OpenAI's GPT-4 for image analysis.

@@ -89,10 +89,10 @@ The model returns a response indicating whether an anomaly is detected:
"NO"
```

![GPTVad result no anomaly](/docs/source/images/gptvad/good.png "GPTVad without anomaly result")
![GptVad result no anomaly](/docs/source/images/gptvad/good.png "GptVad without anomaly result")

```plaintext
"YES: Description of the detected anomaly."
```

![GPTVad result with anomaly](/docs/source/images/gptvad/broken.png "GPTVad with Anomaly result")
![GptVad result with anomaly](/docs/source/images/gptvad/broken.png "GptVad with Anomaly result")
8 changes: 4 additions & 4 deletions tests/integration/model/test_models.py
Original file line number Diff line number Diff line change
@@ -145,7 +145,7 @@ def test_export(
dataset_path (Path): Root to dataset from fixture.
project_path (Path): Path to temporary project folder from fixture.
"""
if model_name == "g_p_t_vad":
if model_name == "gpt_vad":
pytest.skip(f"{model_name} can not be exported")

if model_name == "rkde":
@@ -181,8 +181,8 @@ def _get_objects(
tuple[AnomalyModule, AnomalibDataModule, Engine]: Returns the created objects for model, dataset,
and engine
"""
# Mock the GPTWrapper if the model_name is "g_p_t_vad"
if model_name == "g_p_t_vad":
# Mock the GPTWrapper if the model_name is "gpt_vad"
if model_name == "gpt_vad":
os.environ["OPENAI_API_KEY"] = "fake-api-key"
with (
patch("anomalib.models.image.gptvad.chatgpt.GPTWrapper") as mock_gptwrapper,
@@ -196,7 +196,7 @@ def _get_objects(
task_type = TaskType.DETECTION
elif model_name in ("ganomaly", "dfkde"):
task_type = TaskType.CLASSIFICATION
elif model_name in ("g_p_t_vad"):
elif model_name in ("gpt_vad"):
task_type = TaskType.VISUAL_PROMPTING
else:
task_type = TaskType.SEGMENTATION
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Unit tests for GPTVad zero-/few-shot anomaly detection model."""
"""Unit tests for GptVad zero-/few-shot anomaly detection model."""

# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
"""Unit tests for GPTVad OpenAI Api funtions."""
"""Unit tests for GptVad OpenAI Api funtions."""
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

import os