GenAI Showcase App Function Calling Example with NVIDIA llama-3.3-nemotron-super-49b-v1
0
Hello, I'm running into a few problems using the GenAI Showcase App Function Calling Example with the NVIDIA llama-3.3-nemotron-super-49b-v1 (https://build.nvidia.com/nvidia/llama-3_3-nemotron-super-49b-v1). I set the logging REST Consume to "Trace" as per other posts to get an understanding of the error. When I go to 3 "Invoke the model" and select the model I want to use, with the user prompt "What is the status of ticket 1?" I get an error: HTTP/1.1 400 Bad Request {"error":"[{'type': 'string_too_short', 'loc': ('body', 'messages', 2, 'content'), 'msg': 'String should have at least 1 character', 'input': '', 'ctx': {'min_length': 1}}]"} This was after the following exchange: Request content for POST request to https://integrate.api.nvidia.com/v1/chat/completions HTTP/1.1 Response content for POST request to https://integrate.api.nvidia.com/v1/chat/completions (HTTP/1.1 200 OK) Tool_ExecuteMicroflow: Finished calling microflow ChatCompletions_FunctionCalling.Ticket_GetTicketByID_FunctionMicroflow with Context::[Session [e1f83706-c286-43d7-9d5d-0b5ea16769eb]] Request content for POST request to https://integrate.api.nvidia.com/v1/chat/completions HTTP/1.1 Response content for POST request to https://integrate.api.nvidia.com/v1/chat/completions (HTTP/1.1 400 Bad Request) After examining this further and utilising both the NVIDIA model and the OpenAI model to understand the errors, I'm encountering a problem with the interpretation of the OpenAI API standard by the NVIDIA model. I also tested this with OpenAI API using the GPT4o model and everything worked fine. The problem refers to the following JSON entry in line 4 POST request above: {"role":"assistant","content":"","tool_calls":[{"id":"vK5lk895A","type":"function","function":{"name":"RetrieveTicketByIdentifier","arguments":"{\"TicketIdentifier\": \"1\"}"}}]} specifically: "content":"" ChatGPT 4o suggests that: "In the OpenAI API, for a role: "assistant" message that initiates a tool call, you must not send an empty "content": "" field. Even though you're delegating the response to a tool call, the content field must either be omitted entirely or explicitly set to null." When asked why the GPT4o model through the API this work though. So I asked it why. It came back with the following: OpenAI’s API currently tolerates "content": "" in a role: "assistant" message that includes tool_calls, even though it's technically not aligned with the intended OpenAI API specification. This is a form of lenient parsing — OpenAI's infrastructure will: Treat "" as equivalent to no content. Proceed with processing the tool_calls field as the operative instruction. However, this is not guaranteed to be future-proof and is not part of the OpenAI-recommended behavior. I've attached the whole process flow to this post. Trace (line 127 looks to be the problem) Any ideas how I can solve this, as I would like to use the NVIDIA model and not the OpenAI API. Simon
asked
Simon Cottrell
1 answers
0
Hi Simon,
the OpenAI Connector was primarily developed to match the API of OpenAI, thus we cannot promise that it works with other model providers the same way.
We noted your issue to investigate further, but I cannot promise if and when this will be fixed (due to other topics having a higher priority).
There are two ways you can fix it yourself:
Modify the OpenAI connector module (duplicate the module and adjust for your use case). Most importantly, change the java action RequestMapping_ManipulateJson to remove the assistant's content for tool result calls.