An overview of KNIME based functions to access big data systems - use it on your own big data system (including PySpark) An overview of KNIME based functions to access big data systems Use SQL with Impala/Hive and Spark and also PySpark to access and manipulate data on a big data system. The example is from the classic MS "Northwind" database. THX to J. Thelen for input from SQL lecture --------------- REMEMBER: Spark is about lazy evaluation. That means it will not do anything besides *planning* and preparing the transformations *until* you force it to do something. So the initial load of Spark may take some time (setting up the environment), the next steps might seem super fast (just structuring RDDs and creating -empty- place holders). The moment you want to get data back Spark springs into action and delivers the results.
This is a companion discussion topic for the original entry at https://kni.me/w/3VOz9zHyju79-hMU