How to use "knime_jupyter" module in "PySpark Script Source" node

Hello.

I was wondering if I can use “knime_jupyter” module in “Pyspark Script Source” node.
when I used this module in Pyspark node, the error was occured as below.

I’d appreciate it if you could help me.

Thanks,
hhkim

@hhkim I am not sure if this would work in a PySpark node since you would have to make sure the package is in accessible from the Spark environment.

You could try to import the knime_jupyter module.

import knime_jupyter

1 Like

Is there any advantage of calling a jupyter notebook instead of running the code in KNIME?
br

@Daniel_Weikert it depends. A lot of people like Jupyter notebooks because they can combine explorations with developing code. Often they then are faced with the prospect of isolation the ‚productive‘ code from the other stuff in order to bring it into modules or .PY files and so on.

With the use of Tags (and distilling the code into functions/modules) you could in theory have a Jupyter notebook that contains both exploration and productive code and just run the productive tags in KNIME. If this does make any sense very much depends on your way of doing stuff and maybe the state of your project - I personally like using KNIME to develop workflows and then instantly bring them into production (leaving the exploration in another workflow in a workflow group) - there also would be integrated deployment as a similar concept to the tagging in the Jupyter notebook.

As it is so often with KNIME: it can be done. If and how you use it is up to you :slight_smile:

Thanks to all of your replies.

I just was wondering if I can use “knime_jupyter” module in PySpark node.
Becase I can use “knime_jupyter” module in Python script node without any installing or importing mudule.
I think “knime_jupyter” moduleis very useful.
So I want to use this in Pyspark environment.
If I can’t use this module in Pyspark node, then is there any way to load jupyter notebook in Pyspark node?

Thanks,
hhkim

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.