Azure OpenAI Chat Model Connector, Agent Prompter

KNIME Extensions
1

Hi All,
I am using the Azure OpenAI Chat Model Connector together with the Agent Prompter node. When I type in the Connector node the OpenAI deployment name for GPT4 turbo (= GPT4turbo) and 91000 for max token, I receive the following error @ the Agent Prompter.
image
I checked GPT4 turbo from a different platform. There I have a context of 127k.

Could anybody please explain to me what I am doing wrong?
Thank you very much in advance.
Armin

1 Like
2

Hello @Armin_Alois,

the error refers to the maximum number of tokens the models output is allowed to have.
Can you check what number is configured in the OpenAI Chat Model Connector under Maximum Response Length (token)?
For GPT4 turbo this value can be at most 4096.
Note that this is different from the input tokens where GPT4 turbo supports 128K tokens.

Best regards,
Adrian

2 Likes
3

Hello Adrian (@nemad),
Thank you very much.
Best, Armin

1 Like
4

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.