Mandatory Hugging Face token to use HuggingFaceEndpoint

[JeanBaptiste-dlb]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.llms.huggingface_endpoint import HuggingFaceEndpoint

# with a huggingface/text-generation-inference server running locally

HuggingFaceEndpoint(
   endpoint_url=INFERENCE_SERVER_URL,
   huggingfacehub_api_token=HF_TOKEN, # now mandatory
)
print(model.invoke("test"))

# previously:

model=HuggingFaceTextGenInference(
   endpoint_url=INFERENCE_SERVER_URL,
)
print(model("test"))

Description

I struggle to understand why huggingfacehub_api_token is now mandatory to provide in order to make an instance of the HuggingFaceEndpoint class when the LLM is running in a local Hugging Face TGI deployment.

Previously I was using the now deprecated HuggingFaceTextGenInference, and was never required to give my huggingfacehub_api_token. I would like to return to the previous behavior as I believe there is no need to check my hf_api_token if I am doing a local deployment.

System Info

langchain==0.1.16
langchain-community==0.0.32
langchain-core==0.1.42
langchain-text-splitters==0.0.1

[dosubot[bot]]

The requirement for a huggingfacehub_api_token in the HuggingFaceEndpoint class, even for local deployments, is due to the class's design, which mandates authentication with the HuggingFace Hub. This change reflects a shift towards stricter security and authentication practices, likely in response to evolving requirements for accessing HuggingFace's API services.

For local deployments where authentication might not be necessary, you can bypass the huggingfacehub_api_token requirement by not setting this parameter or the HUGGINGFACEHUB_API_TOKEN environment variable. This approach relies on the assumption that your local setup does not use features requiring authentication to the HuggingFace Hub.

The transition from the deprecated HuggingFaceTextGenInference to the HuggingFaceEndpoint class, with its mandatory API token requirement, aims to enforce secure access to HuggingFace's services. This change is part of a broader move to ensure that access to the platform's capabilities is securely managed, especially as the platform grows and the scope of its use cases expands.

In summary, while the huggingfacehub_api_token is now required for the HuggingFaceEndpoint class to ensure authenticated access to HuggingFace services, for local deployments that do not require HuggingFace Hub features, you can omit this token by not providing it or setting the environment variable. This workaround allows you to continue using local deployments without needing to authenticate with the HuggingFace Hub.

Details

Sources

• Authentication token for HuggingFaceTextGenInference
• libs/community/langchain_community/llms/huggingface_endpoint.py
• libs/community/langchain_community/llms/huggingface_hub.py

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[dbkinghorn]

@dosu
The suggestion above would be fine if it worked. but it doesn't.

Setting HUGGINGFACEHUB_API_TOKEN to a token I copied directly from my HF account still results in
ValidationError: 1 validation error for HuggingFaceEndpoint

Using a TGI server on my own hardware running an LLM that I have verfiled works correctly fails with Langchain v0.1.16.

Any ideas of why the following fails?

In [1]: from langchain_community.llms.huggingface_endpoint import HuggingFaceEndpoint

In [5]: import os
   ...: HUGGINGFACEHUB_API_TOKEN = 'my token copied from my HF account'
   ...: os.environ["HUGGINGFACEHUB_API_TOKEN"] = HUGGINGFACEHUB_API_TOKEN
   ...: 
   ...: 

In [6]: llm = HuggingFaceEndpoint(
   ...:     endpoint_url="http://ai1:8080/",
   ...:     max_new_tokens=512,
   ...:     top_k=10,
   ...:     top_p=0.95,
   ...:     typical_p=0.95,
   ...:     temperature=0.01,
   ...:     repetition_penalty=1.03,
   ...:     huggingfacehub_api_token=HUGGINGFACEHUB_API_TOKEN
   ...: )
   ...: print(llm.invoke("What is Deep Learning?"))
The token has not been saved to the git credentials helper. Pass `add_to_git_credential=True` in this function directly or `--add-to-git-credential` if using via `huggingface-cli` if you want to set the git credential as well.
Token is valid (permission: read).
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[6], line 1
----> 1 llm = HuggingFaceEndpoint(
      2     endpoint_url="http://ai1:8080/",
      3     max_new_tokens=512,
      4     top_k=10,
      5     top_p=0.95,
      6     typical_p=0.95,
      7     temperature=0.01,
      8     repetition_penalty=1.03,
      9     huggingfacehub_api_token=HUGGINGFACEHUB_API_TOKEN
     10 )
     11 print(llm.invoke("What is Deep Learning?"))

File ~/miniconda3/envs/langchain/lib/python3.12/site-packages/pydantic/v1/main.py:341, in BaseModel.__init__(__pydantic_self__, **data)
    339 values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    340 if validation_error:
--> 341     raise validation_error
    342 try:
    343     object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 1 validation error for HuggingFaceEndpoint
__root__
  Could not authenticate with huggingface_hub. Please check your API token. (type=value_error)

[dinabandhu50]

I am also facing the same issue. I have hosted LLM model using huggingface's Text Generation Inference, I am not using HuggingFace's services, why should I pass my huggingfacehub_api_token? Also by omitting the token we get error message

ValidationError: 1 validation error for HuggingFaceEndpoint __root__ Could not authenticate with huggingface_hub. Please check your API token. (type=value_error)

Please remove the mandatory token requirement or provide a bypass mechanism.

Thanks

[Chepko932]

Is this still a thing?

[JeanBaptiste-dlb]

I believe it is solved by using langchain_huggingface