Implement ramalama run/serve #8
Conversation
| @@ -68,9 +68,9 @@ def run_curl_command(args, filename): | |||
| sys.exit(e.returncode) | |||
|
|
|||
|
|
|||
| def pull_ollama_manifest(ramalama_store, manifests, accept, registry_head, model_tag): | |||
| def pull_ollama_manifest(repos_ollama, manifests, accept, registry_head, model_tag): | |||
There was a problem hiding this comment.
Why not just call it repos. No need to keep repeating _ollama
There was a problem hiding this comment.
My thinking was shortly I will bring in huggingface support again and there will likely be "repos_hf", "repos_ollama" directory is:
~/.local/share/ramalama/repos/ollama
shortly there will be:
~/.local/share/ramalama/repos/huggingface
There was a problem hiding this comment.
I think you should embed the location of the store inside of the functions, and not force it to be different.
Once we have multiple pull_manifests() functions, then we can start to consildate functionality and get as much reuse as possible.
ramalama
Outdated
| command = sys.argv[1] | ||
| if command == "pull" and len(sys.argv) > 2: | ||
| pull_cli(ramalama_store + "/repos/ollama", | ||
| ramalama_store + "/models/ollama", sys.argv[2]) | ||
| pull_cli(ramalama_store, sys.argv[2]) |
There was a problem hiding this comment.
switch command {
case "pull":
case "run":
}
There was a problem hiding this comment.
Do you have commands that handle 2 options?
There was a problem hiding this comment.
Yeah there will be:
ramalama list/ls
as an example that lists all models downloaded and ready to use, it's coming.
Will change to switch, apparently they call it match in python for some reason
There was a problem hiding this comment.
How would we feel about going back to if/else if/else? The python switch equivalent match is only available from Python 3.10 (2021) onwards and macOS build is failing here, I'm guessing the python3 version on macOS is older
There was a problem hiding this comment.
Going back to if, else, etc. fixed it anyway
ericcurtin
force-pushed
the
ramalama-run
branch
6 times, most recently
from
569718d
to
1bd8647
Compare
Now we can run "ramalama run/serve granite-code", if not using container, one must at least build/install llama.cpp. Added huggingface support. Signed-off-by: Eric Curtin <ecurtin@redhat.com>
|
Ok merging, we can cleanup later. |
Now we can run "ramalama run/serve granite-code", if not using
container, one must at least build/install llama.cpp. Added huggingface
support.