I’m doing some heavy processing (multiple GBs .txt files) using the python script tool. I’m noticing some performance difference when running my code in Jupiter then when running it into the Python Script tool. Thing is I need to encapsulate it into a component for the rest of my organization to use. I found some topics stating that java snippet is faster, but I’ve never coded in JavaScript before, and I just can’t find more elaborates tutorials on how to use java snippet tool properly. So, what I wanted to ask is, if is it worth trying to learn some JavaScript to develop my component using java snippet instead of python script, and, if the answer is yes, where can I find some better tutorials on how to use java snippet tool?
Hi @mojito_insano,
Welcome to the KNIME forum! Did you know that the new Python Snippet nodes run significantly faster when you enable the columnar table backend? Here’s a blog post about it: KNIME Columnar Table Backend Boosts Performance | KNIME (Note: the backend and the nodes are now out of labs). If you haven’t yet, I would try that before considering other options.
Regarding Java: beware not to mix up Java and JavaScript. The Java Snippet runs Java, JavaScript is used in our Column Expression and Generic JavaScript View nodes. I do not think that JavaScript is any faster than Python at all and the Java Snippet only works on row-wise operations, your code never has access to the full table. So depending on your use case it might not be an option. If it is, the Java Snippet can be faster than the Python Script because it works directly on the data format that KNIME uses internally. As said above, though: with the new table backend the Python nodes are also pretty fast!
Kind regards,
Alexander
@mojito_insano maybe you can tell use more about what you want to do with the data and what the setup is. You can always just load the CSV file from within the Python Script node without importing it into KNIME.