-
Notifications
You must be signed in to change notification settings - Fork 243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How to use this in continue with Ollama? #94
Comments
I have the same question! Auto complete doesn't seem to be working for some reason. |
Here is my config:
Of course, the model has to be downloaded first. Auto-complete doesn't work for me, too. |
Please give us some time to give you all a best practice. |
you should use base model for autocompleting, but it doesn't work too. |
So, the the rule is to use |
you should use instruct model for chat, it works. And you should use base model for autocompleting,it works for other model such as starcoder, but not for this model |
It's interesting because the readme talks about autocomplete support. I wonder if this is an issue with the continue.dev extension and the way it implements it, rather than the model. |
FIM does seem to work with qwen2.5-coder-base:
(I control-C'd there because the ollama model doesn't seem to be configured to stop on I believe this needs to be added to the continue templates file https://github.com/continuedev/continue/blob/main/core/autocomplete/templates.ts However, there may be a way to configure a template directly with continue https://docs.continue.dev/customize/config (maybe the |
Hey all, FYI I've added support for this model using Ollama recently in https://github.com/twinnydotdev/twinny. Hope it helps someone. All the best. |
Looks like autocomplete for Qwen2.5-coder will be added to |
This is a workaround solution for autocomplete with
Continue {
"models": [
{
"title": "Qwen2.5-Coder-7B-Instruct",
"provider": "openai",
"apiBase": "http://192.168.1.19:8000/v1/",
"model": "Orion-zhen/Qwen2.5-Coder-7B-Instruct-AWQ"
}
],
"tabAutocompleteModel": {
"title": "Qwen2.5-Coder-7B-Instruct",
"provider": "openai",
"apiKey": "None",
"completionOptions": {
"stop": [
"<|endoftext|>",
"\n"
]
},
"apiBase": "http://192.168.1.19:8000/v1/",
"model": "Orion-zhen/Qwen2.5-Coder-7B-Instruct-AWQ"
},
"tabAutocompleteOptions": {
"multilineCompletions": "never",
"template": "You are a helpful assistant.<|fim_prefix|>{{{ prefix }}}<|fim_suffix|>{{{ suffix }}}<|fim_middle|>"
},
"customCommands": [],
"allowAnonymousTelemetry": false,
"docs": []
} Edit: correct |
This seems to work for me. The base model seems to work better from my testing.
|
cool, but how do you get the tabautocompleteoptions? |
I'm not sure what you are referring to, 'tabautocompleteoptions' is available in continue, if not then check if you need to update the extension. see https://docs.continue.dev/customize/deep-dives/autocomplete#tabautocompleteoptions for more info on 'tabautocompleteoptions'. |
I have the same question with ollama.
|
People have also reported that upstream ollama/ollama#7166. Hopefully it will be fixed (if they do, you can just |
I assume that you want autocompletion across multiple lines, but continue do inline code suggestions by default I believe. You can enable it to always do multiline completion by adding this line to the options
(From https://docs.continue.dev/autocomplete/how-to-customize) |
it does not help on this case even I set multilineCompletions to always |
No description provided.
The text was updated successfully, but these errors were encountered: