How to convert Model from PyTorch -> ONNX -> TensorFlow -> TFLite ? How to do Model compression/optimization ? #2459

Abhijeet241093 · 2024-12-06T12:50:50Z

Abhijeet241093
Dec 6, 2024

Describe the bug

How to convert Model from PyTorch -> ONNX -> TensorFlow -> TFLite and compare theirs performance (speed and accuracy) ? for patchcore

Model compression/optimization: How to compare four models and provide theirs performance (speed and accuracy) ? patchcore

Dataset

MVTec

Model

PatchCore

Steps to reproduce the behavior

Thank you

OS information

OS information:

OS: [e.g. Ubuntu 20.04]
Python version: [e.g. 3.10.0]
Anomalib version: [e.g. 0.3.6]
PyTorch version: [e.g. 1.9.0]
CUDA/cuDNN version: [e.g. 11.1]
GPU models and configuration: [e.g. 2x GeForce RTX 3090]
Any other relevant information: [e.g. I'm using a custom dataset]

Expected behavior

How to convert Model from PyTorch -> ONNX -> TensorFlow -> TFLite and compare theirs performance (speed and accuracy) ? for patchcore

Model compression/optimization: How to compare four models and provide theirs performance (speed and accuracy) ? patchcore

Screenshots

No response

Pip/GitHub

GitHub

What version/branch did you use?

No response

Configuration YAML

model:
  class_path: anomalib.models.Patchcore
  init_args:
    backbone: wide_resnet50_2
    layers:
      - layer2
      - layer3
    pre_trained: true
    coreset_sampling_ratio: 0.1
    num_neighbors: 9

Logs

model:
  class_path: anomalib.models.Patchcore
  init_args:
    backbone: wide_resnet50_2
    layers:
      - layer2
      - layer3
    pre_trained: true
    coreset_sampling_ratio: 0.1
    num_neighbors: 9

Code of Conduct

I agree to follow this project's Code of Conduct

alexriedel1 · 2024-12-06T14:30:02Z

alexriedel1
Dec 6, 2024

Exporting the model to onnx is trivial and done by:

 engine.export(model=model, export_type="onnx")

What have you tried so far to export the model to tensorflow from this point?

0 replies

Abhijeet241093 · 2024-12-06T14:34:49Z

Abhijeet241093
Dec 6, 2024
Author

This is okey, How about [TensorFlow -> TFLite ?

0 replies

Abhijeet241093 · 2024-12-06T14:36:13Z

Abhijeet241093
Dec 6, 2024
Author

!onnx-tf convert -t tf -i "C:/Users/wood/Desktop/Anamoly _Detection/anomalib/results/Patchcore/MVTec/toothbrush/v0/weights/onnx/weights/onnx/model.onnx" -o "C:/Users/wood/Desktop/Anamoly _Detection/anomalib/results/Patchcore/MVTec/toothbrush/v0/weights/tensorflow/model.pb"


When I try to convert onnx into tensorflow, it return error : 
raceback (most recent call last):
  File "\\?\C:\Users\wood\anaconda3\envs\anomalib_env\Scripts\onnx-tf-script.py", line 33, in <module>
    sys.exit(load_entry_point('onnx-tf', 'console_scripts', 'onnx-tf')())
  File "\\?\C:\Users\wood\anaconda3\envs\anomalib_env\Scripts\onnx-tf-script.py", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "C:\Users\wood\anaconda3\envs\anomalib_env\lib\importlib\metadata\__init__.py", line 171, in load
    module = import_module(match.group('module'))
  File "C:\Users\wood\anaconda3\envs\anomalib_env\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "c:\users\wood\desktop\anamoly _detection\anomalib\onnx-tensorflow\onnx_tf\__init__.py", line 1, in <module>
    from . import backend
  File "c:\users\wood\desktop\anamoly _detection\anomalib\onnx-tensorflow\onnx_tf\backend.py", line 28, in <module>
    from onnx_tf.common.handler_helper import get_all_backend_handlers
  File "c:\users\wood\desktop\anamoly _detection\anomalib\onnx-tensorflow\onnx_tf\common\handler_helper.py", line 3, in <module>
    from onnx_tf.handlers.backend import *  # noqa
  File "c:\users\wood\desktop\anamoly _detection\anomalib\onnx-tensorflow\onnx_tf\handlers\backend\hardmax.py", line 3, in <module>
    import tensorflow_addons as tfa
  File "C:\Users\wood\anaconda3\envs\anomalib_env\lib\site-packages\tensorflow_addons\__init__.py", line 23, in <module>
    from tensorflow_addons import activations
  File "C:\Users\wood\anaconda3\envs\anomalib_env\lib\site-packages\tensorflow_addons\activations\__init__.py", line 17, in <module>
    from tensorflow_addons.activations.gelu import gelu
  File "C:\Users\wood\anaconda3\envs\anomalib_env\lib\site-packages\tensorflow_addons\activations\gelu.py", line 19, in <module>
    from tensorflow_addons.utils.types import TensorLike
  File "C:\Users\wood\anaconda3\envs\anomalib_env\lib\site-packages\tensorflow_addons\utils\types.py", line 29, in <module>
    from keras.src.engine import keras_tensor
ModuleNotFoundError: No module named 'keras.src.engine'

Even after reinstalling !pip uninstall tensorflow keras tensorflow-addons
!pip install tensorflow==2.14.0 keras==2.14.0 tensorflow-addons==0.21.0

It give same error.

0 replies

Abhijeet241093 · 2024-12-06T14:37:36Z

Abhijeet241093
Dec 6, 2024
Author

import onnx
from onnx2keras import onnx_to_keras

# Load the ONNX model from the specified path
onnx_model = onnx.load('C:/Users/wood/Desktop/Anamoly _Detection/anomalib/results/Patchcore/MVTec/toothbrush/v0/weights/onnx/weights/onnx/model.onnx')

# Convert the ONNX model to Keras model
k_model = onnx_to_keras(onnx_model, ['input'], change_ordering=True)

import tensorflow as tf

# Convert the Keras model to a TensorFlow Lite model
converter = tf.lite.TFLiteConverter.from_keras_model(k_model)
tflite_model = converter.convert()

# Save the TensorFlow Lite model to a file
with open('model.tflite', 'wb') as f:
    f.write(tflite_model)

 ---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[2], [line 8](vscode-notebook-cell:?execution_count=2&line=8)
      [5](vscode-notebook-cell:?execution_count=2&line=5) onnx_model = onnx.load('C:/Users/wood/Desktop/Anamoly _Detection/anomalib/results/Patchcore/MVTec/toothbrush/v0/weights/onnx/weights/onnx/model.onnx')
      [7](vscode-notebook-cell:?execution_count=2&line=7) # Convert the ONNX model to Keras model
----> [8](vscode-notebook-cell:?execution_count=2&line=8) k_model = onnx_to_keras(onnx_model, ['input'], change_ordering=True)
     [10](vscode-notebook-cell:?execution_count=2&line=10) import tensorflow as tf
     [12](vscode-notebook-cell:?execution_count=2&line=12) # Convert the Keras model to a TensorFlow Lite model

File c:\Users\wood\anaconda3\envs\anomalib_env\lib\site-packages\onnx2keras\converter.py:175, in onnx_to_keras(onnx_model, input_names, input_shapes, name_policy, verbose, change_ordering)
    [172](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:172)     logger.debug('... found all, continue')
    [174](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:174) keras.backend.set_image_data_format('channels_first')
--> [175](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:175) AVAILABLE_CONVERTERS[node_type](
    [176](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:176)     node,
    [177](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:177)     node_params,
    [178](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:178)     layers,
    [179](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:179)     lambda_funcs,
    [180](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:180)     node_name,
    [181](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:181)     keras_names
    [182](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:182) )
    [183](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:183) if isinstance(keras_names, list):
    [184](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/converter.py:184)     keras_names = keras_names[0]

File c:\Users\wood\anaconda3\envs\anomalib_env\lib\site-packages\onnx2keras\reshape_layers.py:210, in convert_unsqueeze(node, params, layers, lambda_func, node_name, keras_name)
    [207](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/reshape_layers.py:207) logger = logging.getLogger('onnx2keras:unsqueeze')
...
--> [210](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/reshape_layers.py:210)     raise AttributeError('Number of inputs is not equal 1 for unsqueeze layer')
    [212](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/reshape_layers.py:212) if is_numpy(layers[node.input[0]]):
    [213](file:///C:/Users/wood/anaconda3/envs/anomalib_env/lib/site-packages/onnx2keras/reshape_layers.py:213)     logger.debug('Work with numpy types.')

AttributeError: Number of inputs is not equal 1 for unsqueeze layer

0 replies

samet-akcay · 2024-12-06T14:42:20Z

samet-akcay
Dec 6, 2024
Maintainer

@Abhijeet241093, is this a question or bug?
Can you create ONNX model ok?

0 replies

Abhijeet241093 · 2024-12-06T14:43:38Z

Abhijeet241093
Dec 6, 2024
Author

Yes, I able to create onnx model, but I failed to convert into Tensorflow and TFlite model.

0 replies

Abhijeet241093 · 2024-12-06T15:00:09Z

Abhijeet241093
Dec 6, 2024
Author

When I tried to convert into openvino, it return error 👍

Export the model to OpenVINO format

engine = Engine()
engine.export(
model=model,
export_type=ExportType.OPENVINO,
export_root=export_root,
input_size=[640, 640],
ckpt_path=ckpt_path

ModuleNotFoundError Traceback (most recent call last)
Cell In[5], line 16
14 # Export the model to OpenVINO format
15 engine = Engine()
---> 16 engine.export(
17 model=model,
18 export_type=ExportType.OPENVINO,
19 export_root=export_root,
20 input_size=[640, 640],
21 ckpt_path=ckpt_path
22 )

File ~\Desktop\Anamoly _Detection\anomalib\src\anomalib\engine\engine.py:965, in Engine.export(self, model, export_type, export_root, input_size, transform, compression_type, datamodule, metric, ov_args, ckpt_path)
958 exported_model_path = model.to_onnx(
959 export_root=export_root,
960 input_size=input_size,
961 transform=transform,
962 task=self.task,
963 )
964 elif export_type == ExportType.OPENVINO:
--> 965 exported_model_path = model.to_openvino(
966 export_root=export_root,
967 input_size=input_size,
968 transform=transform,
...
--> 250 raise ModuleNotFoundError
252 import openvino as ov
254 with TemporaryDirectory() as onnx_directory:

ModuleNotFoundError:

0 replies

alexriedel1 · 2024-12-06T21:31:56Z

alexriedel1
Dec 6, 2024

It's unrelated to anomalib, but I was sucessful in converting patchcore onnx models to tf and tflite by using this https://github.com/PINTO0309/onnx2tf

Be sure to export your onnx with input shapes. Also try using the dynamo onnx export, it might let you convert some operations for some models that would fail normally.

1 reply

Abhijeet241093 Dec 7, 2024
Author

It done, with !onnx2tf --input_onnx_file_path -ois input:1,3,640,640 -o "". Thank you

Abhijeet241093 · 2024-12-06T21:36:13Z

Abhijeet241093
Dec 6, 2024
Author

Sure, thank you very much.

…

On Sat, Dec 7, 2024, 6:32 AM Alexander Riedel ***@***.***> wrote: It's unrelated to anomalib, but I was sucessful in converting patchcore onnx models to tf and tflite by using this https://github.com/PINTO0309/onnx2tf Be sure to export your onnx with input shapes. Also try using the dynamo onnx export, it might let you convert some operations for some models that would fail normally. — Reply to this email directly, view it on GitHub <#2457 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJJLGGEXPXDJEQ5O7GV4KED2EIJWHAVCNFSM6AAAAABTETJHJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMRUGIYTIMJUGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

0 replies

Abhijeet241093 · 2024-12-06T22:16:41Z

Abhijeet241093
Dec 6, 2024
Author

How to do model compression and optimization ? On Sat, Dec 7, 2024, 6:35 AM AbhiJeeT ***@***.***> wrote:

…

Sure, thank you very much. On Sat, Dec 7, 2024, 6:32 AM Alexander Riedel ***@***.***> wrote: > It's unrelated to anomalib, but I was sucessful in converting patchcore > onnx models to tf and tflite by using this > https://github.com/PINTO0309/onnx2tf > > Be sure to export your onnx with input shapes. Also try using the dynamo > onnx export, it might let you convert some operations for some models that > would fail normally. > > — > Reply to this email directly, view it on GitHub > <#2457 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AJJLGGEXPXDJEQ5O7GV4KED2EIJWHAVCNFSM6AAAAABTETJHJKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMRUGIYTIMJUGM> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

0 replies

samet-akcay · 2024-12-07T06:10:31Z

samet-akcay
Dec 7, 2024
Maintainer

I'm converting this to discussion as it does not seem to be a bug.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to convert Model from PyTorch -> ONNX -> TensorFlow -> TFLite ? How to do Model compression/optimization ? #2459

{{title}}

Replies: 11 comments 1 reply

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

How to convert Model from PyTorch -> ONNX -> TensorFlow -> TFLite ? How to do Model compression/optimization ? #2459

Abhijeet241093 Dec 6, 2024

Describe the bug

Dataset

Model

Steps to reproduce the behavior

OS information

Expected behavior

Screenshots

Pip/GitHub

What version/branch did you use?

Configuration YAML

Logs

Code of Conduct

Replies: 11 comments · 1 reply

alexriedel1 Dec 6, 2024

Abhijeet241093 Dec 6, 2024 Author

Abhijeet241093 Dec 6, 2024 Author

Abhijeet241093 Dec 6, 2024 Author

samet-akcay Dec 6, 2024 Maintainer

Abhijeet241093 Dec 6, 2024 Author

Abhijeet241093 Dec 6, 2024 Author

Export the model to OpenVINO format

engine = Engine() engine.export( model=model, export_type=ExportType.OPENVINO, export_root=export_root, input_size=[640, 640], ckpt_path=ckpt_path

alexriedel1 Dec 6, 2024

Abhijeet241093 Dec 7, 2024 Author

Abhijeet241093 Dec 6, 2024 Author

Abhijeet241093 Dec 6, 2024 Author

samet-akcay Dec 7, 2024 Maintainer

Abhijeet241093
Dec 6, 2024

Replies: 11 comments 1 reply

alexriedel1
Dec 6, 2024

Abhijeet241093
Dec 6, 2024
Author

Abhijeet241093
Dec 6, 2024
Author

Abhijeet241093
Dec 6, 2024
Author

samet-akcay
Dec 6, 2024
Maintainer

Abhijeet241093
Dec 6, 2024
Author

Abhijeet241093
Dec 6, 2024
Author

engine = Engine()
engine.export(
model=model,
export_type=ExportType.OPENVINO,
export_root=export_root,
input_size=[640, 640],
ckpt_path=ckpt_path

alexriedel1
Dec 6, 2024

Abhijeet241093 Dec 7, 2024
Author

Abhijeet241093
Dec 6, 2024
Author

Abhijeet241093
Dec 6, 2024
Author

samet-akcay
Dec 7, 2024
Maintainer