[Bug] "message": "Invalid image." ask chat about uploaded image

[KnightNFS]

📦 Environment

Docker

📌 Version

1.19.2

💻 Operating System

Other Linux

🌐 Browser

Firefox

🐛 Bug Description

After successfully uploaded a image in jpg or png format ( image correctly show in chat window ) , after asking any questions regarding image show that error message :

{
"error": {
"code": "invalid_image",
"type": "invalid_request_error",
"param": null,
"message": "Invalid image."
},
"endpoint": "https://api.openai.com/v1",
"provider": "openai"
}

📷 Recurrence Steps

No response

🚦 Expected Behavior

No response

📝 Additional Information

No response

[lobehubbot]

👀 @KnightNFS

Thank you for raising an issue. We will investigate into the matter and get back to you as soon as possible.
Please make sure you have given us as much context as possible.
非常感谢您提交 issue。我们会尽快调查此事，并尽快回复您。 请确保您已经提供了尽可能多的背景信息。

[dosubot[bot]]

Hey @KnightNFS! 👋 I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this together!

The error message "Invalid image" indicates that the image being sent to the OpenAI API is not being accepted. This could be due to several reasons, such as incorrect encoding, unsupported format, or size restrictions.

Here are a few steps to troubleshoot and resolve this issue:

1. Check Image Encoding:
   Ensure that the image is correctly encoded in Base64 format before being sent to the OpenAI API. The imageUrlToBase64 function is used for this purpose in the lobe-chat framework. Verify that this function is correctly converting the image URL to a Base64 string.

2. Supported Formats:
   Confirm that the image is in a supported format (JPEG or PNG). The lobe-chat framework should handle these formats, but it's good to double-check.

3. Size Restrictions:
   Verify that the image size adheres to the restrictions specified by the OpenAI API. The allowed sizes are '1792x1024', '1024x1024', or '1024x1792' ^[1].

4. Validation Logic:
   Ensure that the validation logic for images is correctly implemented. The buildAnthropicBlock function handles the validation and conversion of image URLs to Base64 format. Make sure this function is working as expected ^[2].

5. Error Handling:
   Review the error handling logic to ensure that any issues with the image are properly caught and logged. The handleOpenAIError function processes errors from the OpenAI API and should provide more details on what might be going wrong ^[3].

By following these steps, you should be able to identify and resolve the issue with the "Invalid image" error. If the problem persists, you may need to add additional logging to capture more details about the image being sent to the API.

To continue talking to Dosu, mention @dosu.

[checkso]

I have the same error, for some days now.

error in container:
Route: [openai] ProviderBizError: {

message: 'Invalid image.',

type: 'invalid_request_error',

param: null,

code: 'invalid_image'

}

[arvinxx]

try to add an env LLM_VISION_IMAGE_USE_BASE64=1. refs: #3888

[KnightNFS]

try to add an env LLM_VISION_IMAGE_USE_BASE64=1. refs: #3888

After set this env , "invalid image" problem solved

lobehub/lobe-chat-database v1.22.3

[checkso]

can confirm, this env solves the issue