[PIMO] compare to benchmark

[jpcbertoldo]

📝 Description

• implements what was suggested in out of the box comparison with aupimo's benchmark #2356

the idea is to provide a utility to compare to aupimo's paper's benchmark results out of the box

it fetchs the json-formated results from the official repo, then (to be implemented) returns a dataframe where the user's models are compared to them

currently this is partially show cased in the notebook
https://github.com/openvinotoolkit/anomalib/blob/main/notebooks/700_metrics/701e_aupimo_advanced_iv.ipynb

Note 1: i put the code in utils_benchmark.py because putting it in utils.py would cause circular import

Note 2: i left _validate_benchmark_model and _validate_benchmark_dataset
inside utils_benchmark.py (not in _validate.py) on purpose because they are very specific to this module and its functions, it would not be used elsewhere

✨ Changes

Select what type of change your PR is:

• 🐞 Bug fix (non-breaking change which fixes an issue)
• 🔨 Refactor (non-breaking change which refactors the code base)
• 🚀 New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
• 📚 Documentation update
• 🔒 Security update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

• 📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
• 📚 I have made the necessary updates to the documentation (if applicable).
• 🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

For more information about code review checklists, see the Code Review Checklist.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[jpcbertoldo]

i bypassed an issue raised by a pre-commit hook, which idk how to properly solve

bandit...................................................................Failed
- hook id: bandit
- exit code: 1

[main]  INFO    profile include tests: None
[main]  INFO    profile exclude tests: B101
[main]  INFO    cli include tests: None
[main]  INFO    cli exclude tests: None
[main]  INFO    using config: pyproject.toml
[main]  INFO    running on Python 3.10.14
Run started:2024-10-20 15:10:15.798398

Test results:
>> Issue: [B310:blacklist] Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.
   Severity: Medium   Confidence: High
   CWE: CWE-22 (https://cwe.mitre.org/data/definitions/22.html)
   More Info: https://bandit.readthedocs.io/en/0.0.0/blacklists/blacklist_calls.html#b310-urllib-urlopen
   Location: ./src/anomalib/metrics/pimo/utils_benchmark.py:88:9
87          """Download the JSON content from an URL."""
88          with urllib.request.urlopen(url_str) as url:  # noqa: S310
89              return json.load(url)

--------------------------------------------------

Code scanned:
        Total lines of code: 166
        Total lines skipped (#nosec): 0

Run metrics:
        Total issues (by severity):
                Undefined: 0
                Low: 0
                Medium: 1
                High: 0
        Total issues (by confidence):
                Undefined: 0
                Low: 0
                Medium: 0
                High: 1
Files skipped (0):

[samet-akcay]

@jpcbertoldo, bandit complains because this approach is not safe. Instead you could use a safer library such as requests

[jpcbertoldo]

@jpcbertoldo, bandit complains because this approach is not safe. Instead you could use a safer library such as requests

It solved that problem, but now I have another 😅

mypy.....................................................................Failed
- hook id: mypy
- exit code: 1

src/anomalib/metrics/pimo/utils_benchmark.py:17: error: Library stubs not installed for "requests"  [import-untyped]
src/anomalib/metrics/pimo/utils_benchmark.py:17: note: Hint: "python3 -m pip install types-requests"
src/anomalib/metrics/pimo/utils_benchmark.py:17: note: (or run "mypy --install-types" to install all missing stub packages)
src/anomalib/metrics/pimo/utils_benchmark.py:17: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 2 source files)

I could not figure it out (i followed the suggested commands from the error and from that page) but it won't work. Maybe it'll be ok in the CI?

[samet-akcay]

@jpcbertoldo, you need to install the following mypy --install-types

[jpcbertoldo]

@jpcbertoldo, you need to install the following mypy --install-types

I tried already:/