Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Agent immediately tried Git commands without coding #5662

Open
1 task done
avi12 opened this issue Dec 18, 2024 · 2 comments
Open
1 task done

[Bug]: Agent immediately tried Git commands without coding #5662

avi12 opened this issue Dec 18, 2024 · 2 comments
Labels
bug Something isn't working severity:low Minor issues or affecting single user

Comments

@avi12
Copy link

avi12 commented Dec 18, 2024

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Describe the bug and reproduction steps

https://www.all-hands.dev/share?share_id=2ccf1523f77b5bdf9ce6db8f3f9a8e7c62f73b0b21453744fc20882a0c5445b9
LLM: https://huggingface.co/lmstudio-community/DeepSeek-Coder-V2-Lite-Instruct-GGUF

OpenHands Installation

Docker command in README

OpenHands Version

0.15.3

Operating System

WSL on Windows

Logs, Errors, Screenshots, and Additional Context

No response

@avi12 avi12 added the bug Something isn't working label Dec 18, 2024
@enyst
Copy link
Collaborator

enyst commented Dec 18, 2024

The LLM you are using "decides" what to do, not openhands. I don't really know how good for agents is the specific DeepSeek Lite GGUF you're using.

There's a very significant difference between LLMs and how well they work with the code and code history, for an agent to succeed tasks. We currently recommend Sonnet 3.5 (new) for best results.

Please take a look at this discussion for information on LLMs we've evaluated. You may see that DeepSeek is in the best case we found at 7%, compared to Sonnet at over 40% success.

This will change, but as of now it is very relevant for your tests, I'm afraid.

@avi12
Copy link
Author

avi12 commented Dec 18, 2024

Thanks for your information!

@mamoodi mamoodi added the severity:low Minor issues or affecting single user label Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working severity:low Minor issues or affecting single user
Projects
None yet
Development

No branches or pull requests

3 participants