Hello people.
I’ve seen some people here asking whether or not it’s possible run workflows using Airflow - a well know orchestration tool - and yes, it is.
Summary:
1. Spin up airflow with Docker
(I used the docker-compose.yml file that you can find in Airflow’s doc, very handy) https://airflow.apache.org/docs/apache-airflow/stable/docker-compose.yaml
2. Edit the docker-compose.yaml file
include 2 folders as volumes for Airflow be able to “see” your files
-
Your workflow (file or directory), or folder with workflows
For example, you could share “C:/knime_workflows” with docker, and assign a directory inside of linux to be: “/opt/knime/knime_workflows”
Docker won’t copy the files, it will just mount on the provided folder -
A folder with KNIME Linux version inside of it.
Remember, docker runs on linux, so you most provide an executable for linux. Download the .tag.gz file you can find on KNIME’s website, and unzip it, like below.
tar -xzf /opt/knime_<knime_ver>.linux.gtk.x86_64.tar.gz -C /opt
Now you have a directory with knime inside of it.
For the sake of demonstration, here is the docker-compose.yml file i’ve used, properly assigning the correct directories.
docker-compose.yml (11.5 KB)
2. Build and spin up the containers
(“docker compose up -d” in the same directory you have the docker-compose file)
3. Test the execution through command line
/opt/knime_4.5.3/knime -consoleLog -application org.knime.product.KNIME_BATCH_APPLICATION -workflowDir=“/opt/knime/knime_workflows/<your_workflow_name_goes_here>”
4. Build your dag
Now comes the Airflow part. I’m not going to explain it since I’m not an Airflow specialist
5.Have fun 
In case you want to see it working:
Demo Video
Thanks!