Is it possible to run Snowpark (https://docs.snowflake.com/en/developer-guide/snowpark/index) in snowflake via KNIME?

Hi,
Is it possible to run a python worksheet using the snowpark library (snowflake, Snowpark API | Snowflake Documentation) in KNIME platform? Thanks in advance.

Regards,
Prashanth

Hello @pras135 ,
welcome to the KNIME community :slight_smile:
We are currently investigating a Snowpark integration (internal ticket AP-19591). The idea is to provide a node similar to the Python Scripting node but which will run the code within Snowpark.
To get a better idea of the requirements, could you share some use cases with us that you would like to solve with such an integration?
Thanks
Tobias

1 Like

Hi @tobias.koetter,
The use case is to do processing of the data in the snowflake environment itself using snowpark and bring the processed table to KNIME.
For example, I can run few operations on the snowflake side before bringing the data to KNIME. There is a requirement of optimizing few steps in the KNIME workflow that is deployed and it was thought that snowpark is one such solution.

Snowflake provides an option to run snowpark queires even in SQL worksheet using “with PYTHON_WORKSHEET as procedure ()”
I was testing the same with the existing node (DQ Query or DB Selector) in KNIME and it cannot run a code using snowpark and gives the below error

SQL compilation error:
Unknown function PYTHON_WORKSHEET

A node specific for snowpark would be great, when will it become available? Thank you.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.