Hi @VitaliiKaplan,
thanks for sharing your project. I really like the idea!
Since the other thread KNIME to Python exporter - #10 has been closed, I’ll try to continue with the discussion on the data processing here.
@Add94: Thanks for sharing your insights on the data processing backends. Instead of going for DuckDb, I’d recommend going either for Ibis or Polars.
DuckDb might provide a slightly superior performance, but the reason why Python+Pandas became so popular also applies here: Having a clean and easily understandable API to data structures is much more important than leveraging the last 0.1% of computational performance.
Polars API is close to Pandas API, thus everyone from novice to data processing specialist will feel more confident with using Polars than DuckDB. SQL is only used by few data scientists, especially when looking at the typical user base of Knime.
Furthermore I’ve got the feeling that Polars is evolving faster than DuckDB, especially by moving towards distributed computing.
Best regards,
Johannes