Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

q2_K GGUF not recognized #1279

Open
wittypastoral opened this issue Dec 22, 2024 · 3 comments
Open

q2_K GGUF not recognized #1279

wittypastoral opened this issue Dec 22, 2024 · 3 comments

Comments

@wittypastoral
Copy link

Describe the Issue
Trying to run the smallest model (q2_K) from https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF/tree/main on the CPU.
For some reason, the model is not recognized. (output below) The q5_K_M version from the same source works with no problems at all.

Additional Information:

~/llm_stuff$ ./koboldcpp-linux-x64-cuda1210 --model DarkForest-20B-v2.0.q2_K.gguf --host ***.***.***.*** --port 8080 --usecpu
***
Welcome to KoboldCpp - Version 1.76
Attempting to use CPU library.
Initializing dynamic library: koboldcpp_default.so
==========
Namespace(benchmark=None, blasbatchsize=512, blasthreads=2, chatcompletionsadapter='', config=None, contextsize=4096, debugmode=0, flashattention=False, forceversion=0, foreground=False, gpulayers=0, highpriority=False, hordeconfig=None, hordegenlen=0, hordekey='', hordemaxctx=0, hordemodelname='', hordeworkername='', host='10.243.153.96', ignoremissing=False, launch=False, lora=None, mmproj='', model='DarkForest-20B-v2.0.q2_K.gguf', model_param='DarkForest-20B-v2.0.q2_K.gguf', multiuser=1, noavx2=False, noblas=False, nocertify=False, nommap=False, nomodel=False, noshift=False, onready='', password=None, port=8080, port_param=5001, preloadstory='', prompt='', promptlimit=100, quantkv=0, quiet=False, remotetunnel=False, ropeconfig=[0.0, 10000.0], sdclamped=0, sdconfig=None, sdlora='', sdloramult=1.0, sdmodel='', sdquant=False, sdthreads=0, sdvae='', sdvaeauto=False, showgui=False, skiplauncher=False, smartcontext=False, ssl=None, tensor_split=None, threads=2, unpack='', useclblast=None, usecpu=True, usecublas=None, usemlock=False, usevulkan=None, whispermodel='')
==========
Loading model: ****/llm_stuff/DarkForest-20B-v2.0.q2_K.gguf

---
Unidentified Model Encountered: (ver 0)
---
Using Automatic RoPE scaling, Pre-GGUF (scale:1.000, base:30138.9).
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | BLAS = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | 

Unknown Model, cannot load.
Load Text Model OK: False

Could not load text model: ****/llm_stuff/DarkForest-20B-v2.0.q2_K.gguf
@LostRuins
Copy link
Owner

Looks like a bad download. File version 0. Check the filesize - was the download incomplete or interrupted?

@wittypastoral
Copy link
Author

Tried again but a fresh download doesnt work. nothing unusual as far as I can tell about filesize or contents.

@LostRuins
Copy link
Owner

LostRuins commented Dec 23, 2024

It's very likely that file is corrupted then. Looking at the HF file I can confirm that it is indeed corrupted. Clicking on the properties will show you that as well. You should ask the creator to verify and reupload.

https://huggingface.co/TeeZee/DarkForest-20B-v2.0-GGUF/tree/main?show_file_info=DarkForest-20B-v2.0.q2_K.gguf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants