Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to interpret file <_io.BytesIO object at 0x7fabe124e890> as a pickle #267

Open
vincentduan opened this issue Oct 30, 2024 · 1 comment

Comments

@vincentduan
Copy link

I am a beginner with Towhee, and I am using the Towhee Built-in Pipeline to implement a video deduplication function. The source code can be found at here. I ran the following code, but encountered an error. How can I resolve it?
our main environment:

ubuntu 22.04 
towhee 1.1.3
towhee.models 1.1.3
pymilvus 2.4.8
plyvel 1.5.1
from pymilvus import connections, FieldSchema, CollectionSchema, DataType, Collection, utility
from towhee import AutoPipes, AutoConfig
import glob

connections.connect(host=IP, port='19530')

Collection('video_copy_detection').load()

search_conf = AutoConfig.load_config('video_copy_detection')
print(search_conf)
search_conf.collection = 'video_copy_detection'
search_conf.leveldb_path = 'url_vec.db'
search_conf.top_k = 5
search_conf.device = -1
search_pipe = AutoPipes.pipeline('video_copy_detection', search_conf)

path = glob.glob('./VCSL-demo/madongmei/*')

for i in path:
    print(i)
    result = search_pipe(i)
    DataCollection(result).show()

del search_pipe

exception info :

Node-kvstorage/from-leveldb-6 runs failed, error msg: Failed to interpret file <_io.BytesIO object at 0x7f6164737600> as a pickle, Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/numpy/lib/npyio.py", line 441, in load
    return pickle.load(fid, **pickle_kwargs)
EOFError: Ran out of input

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/towhee/runtime/nodes/node.py", line 158, in _call
    return True, self._op(*inputs), None
  File "/home/ours/.towhee/operators/kvstorage/from-leveldb/versions/main/from_leveldb.py", line 22, in __call__
    vals = np.load(vals, allow_pickle=True)
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/numpy/lib/npyio.py", line 443, in load
    raise pickle.UnpicklingError(
_pickle.UnpicklingError: Failed to interpret file <_io.BytesIO object at 0x7f6164737600> as a pickle
, Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/towhee/runtime/nodes/node.py", line 171, in process
    self.process_step()
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/towhee/runtime/nodes/_map.py", line 63, in process_step
    assert succ, msg
AssertionError: Failed to interpret file <_io.BytesIO object at 0x7f6164737600> as a pickle, Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/numpy/lib/npyio.py", line 441, in load
    return pickle.load(fid, **pickle_kwargs)
EOFError: Ran out of input

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/towhee/runtime/nodes/node.py", line 158, in _call
    return True, self._op(*inputs), None
  File "/home/ours/.towhee/operators/kvstorage/from-leveldb/versions/main/from_leveldb.py", line 22, in __call__
    vals = np.load(vals, allow_pickle=True)
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/numpy/lib/npyio.py", line 443, in load
    raise pickle.UnpicklingError(
_pickle.UnpicklingError: Failed to interpret file <_io.BytesIO object at 0x7f6164737600> as a pickle


  File "/home/ours/code/neimeng-python/milvusdb_test/tutorial/tutorial_5.py", line 23, in <module>
    result = search_pipe(i)
RuntimeError: Node-kvstorage/from-leveldb-6 runs failed, error msg: Failed to interpret file <_io.BytesIO object at 0x7f6164737600> as a pickle, Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/numpy/lib/npyio.py", line 441, in load
    return pickle.load(fid, **pickle_kwargs)
EOFError: Ran out of input

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/towhee/runtime/nodes/node.py", line 158, in _call
    return True, self._op(*inputs), None
  File "/home/ours/.towhee/operators/kvstorage/from-leveldb/versions/main/from_leveldb.py", line 22, in __call__
    vals = np.load(vals, allow_pickle=True)
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/numpy/lib/npyio.py", line 443, in load
    raise pickle.UnpicklingError(
_pickle.UnpicklingError: Failed to interpret file <_io.BytesIO object at 0x7f6164737600> as a pickle
, Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/towhee/runtime/nodes/node.py", line 171, in process
    self.process_step()
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/towhee/runtime/nodes/_map.py", line 63, in process_step
    assert succ, msg
AssertionError: Failed to interpret file <_io.BytesIO object at 0x7f6164737600> as a pickle, Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/numpy/lib/npyio.py", line 441, in load
    return pickle.load(fid, **pickle_kwargs)
EOFError: Ran out of input

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/towhee/runtime/nodes/node.py", line 158, in _call
    return True, self._op(*inputs), None
  File "/home/ours/.towhee/operators/kvstorage/from-leveldb/versions/main/from_leveldb.py", line 22, in __call__
    vals = np.load(vals, allow_pickle=True)
  File "/home/ours/anaconda3/envs/ours_py3.10_milvus/lib/python3.10/site-packages/numpy/lib/npyio.py", line 443, in load
    raise pickle.UnpicklingError(
_pickle.UnpicklingError: Failed to interpret file <_io.BytesIO object at 0x7f6164737600> as a pickle
@Liqueur0206
Copy link

do you solve it? I am facing the same problem

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants