Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

自定义添加的工具无法正确调用 #373

Open
liyoung1992 opened this issue Jan 5, 2024 · 1 comment
Open

自定义添加的工具无法正确调用 #373

liyoung1992 opened this issue Jan 5, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@liyoung1992
Copy link

Issue Description / 问题描述

Please provide a detailed description of the error or issue you encountered. / 请详细描述您遇到的错误或问题。
定义 async def contour_detection(img_path: str = '', run_async: bool=False):后,上传图片,提问 do contour detection for this image.You can't use the network, you can only use local registration tools for processing`,一直重复调用contour_detection,且参数没有匹配上。

Steps to Reproduce / 复现步骤

Please provide the specific steps to reproduce the error. / 请提供复现错误的具体步骤。
`from core.register import toolwrapper
from core.exceptions import ToolExecutionError
from config import CONFIG,logger
import os
import random
import cv2
import numpy as np

@toolwrapper()
async def contour_detection(img_path: str = '', run_async: bool=False):
"""
This is a new tool that do contour detection on the image.

Example:
```
in: contour_detection(img_path = "/work/test.png", run_async=True)
Out: {contour_detection_id: 1}# You can use this id to read the output and error later.
```

:param string img_path: The first parameter of the contour_detection function. It is a string.it is original image path of contour detection.
:param bool run_async: Whether to run the command asynchronously, default is False. If True, call this tool again with shell_id to get the final output and error.
:return: The output of the contour_detection function as a json string. contour_detection_result_id is get result id,you can use contour_detection_id to get detail result
:raises ToolExecutionError: If something goes wrong during the execution of the new_tool function.
"""
if os.path.exists(img_path) is False:
    raise ToolExecutionError({'Error': 'image is not exist!'})
    return
# Do something with param1 and param2
# If something goes wrong, raise a ToolExecutionError
# Return the output as a string
image = cv2.imread(img_path)
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

_, binary = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV)

contours, hierarchy = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

contour_image = np.zeros_like(image)

for contour in contours:
    cv2.drawContours(contour_image, [contour], -1, (0, 255, 0), 2)
random_id = random.randint(1000, 10000)
output_path = 'contour_image_' + str(random_id) + ".jpg"
cv2.imwrite(output_path, contour_image)
full_path = os.path.abspath(output_path)

logger.error("call contour_detection")
logger.error(img_path)
logger.error(run_async)
print("call contour_detection")

result = {
    'contour_detection_result_id': random_id
}
return str(result)`

Expected Behavior / 预期行为

Describe the behavior you expected to see. / 请描述您期望的正确行为。

Environment / 环境信息

  • Operating System / 操作系统:
  • Python Version / Python 版本:
  • Other Relevant Information / 其他相关信息:

Error Screenshots or Logs / 错误截图或日志

If possible, please provide relevant screenshots or logs of the error. / 如果可能,请提供相关的错误截图或日志文件。

using Tools:
Command Name:
"contour_detection"
Arguments:
{
img_path:"/work//test.png"
run_async:true
}
Execution Results:{"detail":{"Error":"image is not exist!"}}
Command Status:❌TOOL_CALL_FAILED

Additional Notes / 其他备注

If you have any additional information or notes, please add them here. / 如果有其他补充信息,请在此处添加。
step详细信息:

thoughts:
To select relevant features from an input image, we can use a variety of techniques such as edge detection or corner detection. These methods will help us identify key points within the image.
reasoning:
The goal is to implement a feature selection method that identifies representative regions in an image. Edge detection and corner detection are commonly used techniques because they involve finding areas with high gradient magnitudes or strong curvature, respectively. By applying these algorithms to our images, we should be able to obtain useful information about their structure.
plan:
Use the FileSystemEnv_search tool to find suitable feature selection methods for image processing
criticism:
This approach assumes that one of the available tools (FileSystemEnv_search) can provide functions specifically designed for performing edge detection or corner detection on images without using external resources.
@liyoung1992 liyoung1992 added the bug Something isn't working label Jan 5, 2024
@ChamfersChen
Copy link

请问,按照官方给的README,自定义tool脚本之后,要重新docker compose build,tool才会生效并写入prompt的function么?还是说有更快捷的方式?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants