Hi there, since I upgrade from Knime 4.5 to 4.51 I have to deal with very slow data transfer from Python Source to Knime. The python environment was done by Knime and serialization is set to Apache Arrow (default setting). So it takes more than 1 Minute to get a table with 100.000 rows and 21 Columns. And I skipped execution after 10 Minutes for a table of 1e6 rows. Last week, before I upgraded to 4.51 everything went okay. Actually there isn’t much inside of my Python Source: import pandas as pd import pyodbc engine = pyodbc.connect(DSN=“MyServer”, autocommit=True) cmd = “SELECT * FROM myScheme.myTable LIMIT 100000” df = pd.read_sql(cmd,engine) ‘’’ Some Math ‘’’ output_table = df Andreas

Hi @ActionAndi , the KNIME 4.6 release coming in summer will add support for pandas.Timestamp (= timestamp[ns]) in the Python Script (Labs) node :slight_smile: Cheers, Carsten

KNIME Forum Archive

Python Source: Extreme Slow Data Transfer to Knime

KNIME Analytics Platform

← previous page

Home
Categories
Guidelines
Terms of Service
Privacy Policy

Powered by Discourse, best viewed with JavaScript enabled