Verified models can be found from EmbeddedLLM IpexLLM model collections
- EmbeddedLLM IpexLLM Model collections: link
| Model | Model Link |
|---|---|
| Phi-3-mini-4k-instruct | link |
| Phi-3-mini-128k-instruct | link |
| Phi-3-medium-4k-instruct | link |
| Phi-3-medium-128k-instruct | link |
Unverified models, but supported by Upstream IpexLLM could be found in the following model collections.
| Model | Model Link |
|---|---|
| LLaMA (such as Vicuna, Guanaco, Koala, Baize, WizardLM, etc.) | |
| LLaMA 2 | link1, link2 |
| LLaMA 3 | link |
| ChatGLM | |
| ChatGLM2 | link |
| ChatGLM3 | link |
| GLM-4 | link |
| Mistral | link |
| Mixtral | link |
| Falcon | link |
| MPT | link |
| Dolly-v1 | link |
| Dolly-v2 | link |
| Replit Code | link |
| RedPajama | link |
| Phoenix | link |
| StarCoder | link |
| Baichuan | link |
| Baichuan2 | link |
| InternLM | link |
| InternLM2 | link |
| Qwen | link |
| Qwen1.5 | link |
| Qwen2 | link |
| Aquila | link |
| Aquila2 | link |
| Phi-1_5 | link |
| Flan-t5 | link |
| CodeLlama | link |
| Skywork | link |
| InternLM-XComposer | link |
| CodeShell | link |
| Yi | link |
| BlueLM | link |
| Mamba | link1, link2 |
| SOLAR | link |
| Phixtral | link |
| RWKV4 | |
| RWKV5 | |
| DeepSeek-MoE | link |
| Ziya-Coding-34B-v1.0 | link |
| Phi-2 | link |
| Phi-3 | link |
| Yuan2 | link |
| Gemma | link1, link2 |
| DeciLM-7B | link |
| Deepseek | link |
| StableLM | link |
| CodeGemma | link |
| Command-R/cohere | link |
| CodeGeeX2 | link |
| MiniCPM | link |
Resources from: https://github.com/intel-analytics/ipex-llm/
We welcome contributions to the verified model list.
- Upgrade
transformers.pip install --upgrade transformers~=4.42.3. - Edit
lib\site-packages\transformers\models\qwen2\modeling_qwen2.py. - Change
from transformers.models.qwen2.modeling_qwen2 import _prepare_4d_causal_attention_masktofrom transformers.modeling_attn_mask_utils import _prepare_4d_causal_attention_mask.
ImportError: cannot import name '_prepare_4d_causal_attention_mask' from 'transformers.models.qwen2.modeling_qwen2' (C:\Users\hpintel\anaconda3\envs\ellmipex\lib\site-packages\transformers\models\qwen2\modeling_qwen2.py)