CreateCompletionAsStream second call with multiple tools

[Nicelittlewood]

Hi everyone,
i've an issue with CreateCompletionAsStream and parallel function.

my code is like that.

With only a function called, the second stream working fine, but if i have two tools called, it respond only withe the first result.

var completion = openAiService.ChatCompletion.CreateCompletionAsStream(new ChatCompletionCreateRequest()
				{
					Messages = m,
					Model = OpenAI.ObjectModels.Models.Gpt_4_turbo,
					Temperature = (float?)0.7,
					Stream = true,
					Tools = new List<ToolDefinition> {
						fn1, fn2, fn3...
					},
				});

List<ChatMessage> msc = m.ToList();
await foreach (var completionResult in completion)
{
	if (completionResult.Successful)
	{
		var choice = completionResult.Choices.First();

		var tc = choice.Message.ToolCalls;

		if (tc != null)
		{
			listTool.AddRange(tc);
		}
		if (choice.FinishReason == "tool_calls")
		{
			IsToolRespond = true;
			foreach (ToolCall toolCall in listTool)
			{
				if (toolCall.FunctionCall != null)
				{
					if (toolCall.FunctionCall.Name == fn1.Name)
					{
					var result = "...."
					msc.Add(ChatMessage.FromAssistant(result));
					}
				}
				
				msc
			}
		}
		else
		{
			yield return choice.Message.Content;
		}
	}
}

if (IsToolRespond)
{
	var completionSecondCall = openAiService.ChatCompletion.CreateCompletionAsStream(new ChatCompletionCreateRequest()
	{
		Messages = msc,
		Model = Models.Gpt_4_turbo,
		Temperature = (float?)0.3,
		Stream = true,
		ResponseFormat = new ResponseFormat { Type = StaticValues.CompletionStatics.ResponseFormat.Text }
	});

	await foreach (var completionResultSc in completionSecondCall)
	{
		if (completionResultSc.Successful)
		{
			var choiceSc = completionResultSc.Choices.First();
			string messageSuccess = choiceSc.Message.Content;

			yield return messageSuccess;
		}
		else
		{
			yield return "ERROR";
		}

	}
}				

i've found this guide on the official site: https://platform.openai.com/docs/guides/function-calling

they wrote to add the function called to the messagges and add a messagge with the result with role "tool"

response = client.chat.completions.create(
        model="gpt-3.5-turbo-0125",
        messages=messages,
        tools=tools,
        tool_choice="auto",  # auto is default, but we'll be explicit
    )
    response_message = response.choices[0].message
    tool_calls = response_message.tool_calls
    # Step 2: check if the model wanted to call a function
    if tool_calls:
        # Step 3: call the function
        # Note: the JSON response may not always be valid; be sure to handle errors
        available_functions = {
            "get_current_weather": get_current_weather,
        }  # only one function in this example, but you can have multiple
        messages.append(response_message)  # extend conversation with assistant's reply
        # Step 4: send the info for each function call and function response to the model
        for tool_call in tool_calls:
            function_name = tool_call.function.name
            function_to_call = available_functions[function_name]
            function_args = json.loads(tool_call.function.arguments)
            function_response = function_to_call(
                location=function_args.get("location"),
                unit=function_args.get("unit"),
            )
            messages.append(
                {
                    "tool_call_id": tool_call.id,
                    "role": "tool",
                    "name": function_name,
                    "content": function_response,
                }
            )  # extend conversation with function response
        second_response = client.chat.completions.create(
            model="gpt-3.5-turbo-0125",
            messages=messages,
        )  # get a new response from the model where it can see the function response
        return second_response

SO i tried to modify my code like the example.

var completion = openAiService.ChatCompletion.CreateCompletionAsStream(new ChatCompletionCreateRequest()
				{
					Messages = m,
					Model = OpenAI.ObjectModels.Models.Gpt_4_turbo,
					Temperature = (float?)0.7,
					Stream = true,
					Tools = new List<ToolDefinition> {
						fn1, fn2, fn3...
					},
				});

List<ChatMessage> msc = m.ToList();
await foreach (var completionResult in completion)
{
	if (completionResult.Successful)
	{
		var choice = completionResult.Choices.First();

		var tc = choice.Message.ToolCalls;

		if (tc != null)
		{
			listTool.AddRange(tc);
		}
		if (choice.FinishReason == "tool_calls")
		{
			IsToolRespond = true;
			msc.Add(choice.Message);
			foreach (ToolCall toolCall in listTool)
			{
				if (toolCall.FunctionCall != null)
				{
					if (toolCall.FunctionCall.Name == fn1.Name)
					{
						var result = "...."
						ChatMessage cm = new ChatMessage();

						cm.Role = "tool";
						cm.ToolCallId = toolCall.Id;
						cm.Content = JsonConvert.SerializeObject(result);
						cm.Name = toolCall.FunctionCall.Name;

						msc.Add(cm);
					}
				}
				
				msc
			}
		}
		else
		{
			yield return choice.Message.Content;
		}
	}
}

if (IsToolRespond)
{
	var completionSecondCall = openAiService.ChatCompletion.CreateCompletionAsStream(new ChatCompletionCreateRequest()
	{
		Messages = msc,
		Model = Models.Gpt_4_turbo,
		Temperature = (float?)0.3,
		Stream = true,
		ResponseFormat = new ResponseFormat { Type = StaticValues.CompletionStatics.ResponseFormat.Text }
	});

	await foreach (var completionResultSc in completionSecondCall)
	{
		if (completionResultSc.Successful)
		{
			var choiceSc = completionResultSc.Choices.First();
			string messageSuccess = choiceSc.Message.Content;

			yield return messageSuccess;
		}
		else
		{
			yield return "ERROR";
		}

	}
}

in this case, the second call return only a empty value.

i cannot understand how to do this stuff!

Any advice?

[SebastianStehle]

Hi,

I had the same problem. The main culprit is the lack of error handling in the streaming endpoint. I have provided a PR for that and also added a test: https://github.com/betalgo/openai/pull/558/files#diff-245fcbf46eb07eda0cbf0d53ba3842fb93c3dc2f7a7c604b118030e73adfaacc

So what you have to do:

1. Add the choice message to the next response, in your case choice.Message.
2. For whatever reason, OpenAI does not set a role here. Therefore you have to change the role to assistant, in your case choice.Message.Role = "assistant";

[kayhantolga]

this should be resolved in 8.2.2