Use container right now #15
Conversation
ericcurtin
force-pushed
the
ramalama-run-serve
branch
2 times, most recently
from
fced804
to
d40b962
Compare
ramalama
Outdated
| @@ -342,6 +342,11 @@ def main(args): | |||
| conman = select_container_manager() | |||
| ramalama_store = get_ramalama_store() | |||
|
|
|||
| if conman: | |||
There was a problem hiding this comment.
Ok so we default to running a container for use with the server.
There was a problem hiding this comment.
Sounds good to me, the other problem is we need to add:
pip install "huggingface_hub[cli]==0.24.2"
to the install script but that's no biggie.
There was a problem hiding this comment.
Re-pushed, it now does the re-exec in a container thing for just run/serve and installs the huggingface dependency in install script
|
LGTM, merge when tests pass. |
ericcurtin
force-pushed
the
ramalama-run-serve
branch
2 times, most recently
from
180a93c
to
47c3d24
Compare
| @@ -215,6 +215,7 @@ def list_cli(ramalama_store, args): | |||
|
|
|||
|
|
|||
| funcDict["list"] = list_cli | |||
| funcDict["ls"] = list_cli | |||
There was a problem hiding this comment.
Ollama has ls as an alias to list, so trying to keep with "norms" from other tools, when it's easy at least
ericcurtin
force-pushed
the
ramalama-run-serve
branch
2 times, most recently
from
34635dd
to
2f3cc71
Compare
|
I think it should work this time, just wanted to get a test in to prevent run/serve breakage again If we can get the llama-cpp-python library working well, it might help reduce dependencies on containers. But these LLM environments in general can get complex |
ericcurtin
force-pushed
the
ramalama-run-serve
branch
from
2f3cc71
to
a6c69f6
Compare
ramalama run/serve right now require the container, it has the version of llama.cpp that works. Long-term we may be able to remove this. Signed-off-by: Eric Curtin <ecurtin@redhat.com>
ericcurtin
force-pushed
the
ramalama-run-serve
branch
from
a6c69f6
to
2f708b9
Compare
|
Merging, people might try this out in the next few minutes, would hope they don't install the broken version |
|
We should create a stable branch soon |
ramalama run/serve right now require the container, it has the version of llama.cpp that works.
Long-term we may be able to remove this.