Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(llama.cpp): disable mirostat as default #2911

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Conversation

mudler
Copy link
Owner

@mudler mudler commented Jul 18, 2024

Even if increasing the quality of the output, it has shown to have performance drawbacks to be so noticeable that the confuses users about speed of LocalAI ( see also #2780 ). This changeset disables Mirostat by default (which can be still enabled manually).

Even if increasing the quality of the output, it has shown to have
performance drawbacks to be so noticeable that the confuses users about
speed of LocalAI ( see also
#2780 ).

This changeset disables Mirostat by default (which can
be still enabled manually).

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Copy link

netlify bot commented Jul 18, 2024

Deploy Preview for localai ready!

Name Link
🔨 Latest commit c680622
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/6698f00126e5d40008ebdefd
😎 Deploy Preview https://deploy-preview-2911--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@mudler mudler added bug Something isn't working ux labels Jul 18, 2024
Copy link

netlify bot commented Jul 18, 2024

Deploy Preview for localai ready!

Name Link
🔨 Latest commit 3a1727a
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/66e958ea061f8800087de712
😎 Deploy Preview https://deploy-preview-2911--localai.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@dave-gray101 dave-gray101 self-requested a review July 18, 2024 16:15
@dave-gray101 dave-gray101 enabled auto-merge (squash) July 18, 2024 16:15
Repository owner deleted a comment from localai-bot Jul 18, 2024
@localai-bot
Copy link
Contributor

localai-bot commented Jul 18, 2024

This PR diff shows a change in the backend_config.go file in the core/config package. The file defines the BackendConfig struct which holds configuration options for a backend.

The change is in the line where the defaultMirostat variable is defined. The comment at the beginning of the line indicates that the change is related to an issue on the GitHub repository, namely issue #2780.

Before the change, the defaultMirostat variable was set to 2. After the change, it is set to 0. This change might be a result of resolving the aforementioned issue, which would have required adjusting the default value of the mirostat configuration option.

The mirostat configuration option might be related to the Mirostat algorithm used in the LocalAI system. The change suggests that the default value of mirostat has been set to 0, which means that the Mirostat algorithm might be disabled or not used by default in the system.

Overall, the change seems to be a configuration update to address an issue and possibly disable or change the behavior of the Mirostat algorithm in the LocalAI system.

@dave-gray101
Copy link
Collaborator

the failing test-linux run was acting strange and was unable to be restarted... auto-merge will kick off a fresh cycle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking-change bug Something isn't working ux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants