HuggingFaceEndpoint returning buggy responses and prompt template back #28572
Labels
🤖:bug
Related to a bug, vulnerability, unexpected error with an existing feature
investigate
Flagged for investigation.
Checked other resources
Example Code
Error Message and Stack Trace (if applicable)
As you can see in langsmith, it returned this output.
Description
I'm using HuggingFaceEndpoint for inference to avoid storing model on my local machine, and I've noticed it gives buggy responses quite a few times. I'm using it for a RAG and a lot of times it just returns back the entire base prompt template inside [INST]...[/INST]. And as seen in screenshot attached, it returned "[/INST]" in a loop until max tokens limit reached.
System Info
System Information
Package Information
Optional packages not installed
Other Dependencies
The text was updated successfully, but these errors were encountered: