I’ve been working with the AI Agent Chat View node and finding it very useful for interacting with agents. However, I’ve run into a limitation regarding testing and evaluation.
The Request: Could we please get a feature to export the conversation history from the Agent Chat View? Ideally, this would be an optional Output Port that outputs the chat history (User Prompts + Agent Responses) as a KNIME data table.
The Use Case: Currently, there is no native way to extract the text generated by the agent within the view. This makes it difficult to perform LLM evaluation and testing. I need to be able to:
-
Batch test different models.
-
Capture the responses to compare quality and hallucinations.
-
Archive the conversation logs for analysis.
Current Workaround: Right now, I have to resort to using Selenium browser automation to scrape the text from the chat interface, which is quite brittle and inconvenient.
Having the chat history available as a data table downstream would significantly improve the workflow for validating AI Agent’s performance.
Thanks!