Automating file upload and data clean-up for PowerBI

KNIME Analytics Platform
1

Hi everyone,

I’ve been assigned a project to create a PowerBI dashboard with automated back-end data to minimize manual input and sourcing, make it adaptable for changes (like adding columns), and ensure it’s fast—im using KNIME for this. Our raw data is in Excel files.

Currently, I’m using list files and loop to automate data uploads (of our fact table) and standardize column headers so data merges automatically. However, it takes 5 seconds to execute the loop and uses a lot of memory.

Any tips on:

  1. Automating file upload and data transformation
  2. Improving model speed
  3. Automating KNIME data refresh to keep PowerBI updated regularly and share my workflow to my teammate so they can maintain or update it regularly while still keeping it connected to PowerBI

I might be leaving soon, so I need to ensure my teammates can maintain the KNIME workflow for the PowerBI dashboard.

This is my current work flow for automating file upload and column rename

image
image1555×338 110 KB

2

Hello @wuhnderess,

So improving the speed of things is definitely more difficult, for you other 2 points (1 and 3) we do offer through the hub a way to automate execution of workflows.

For example, you can run the workflow you have at set times of day and add your other coworkers to it so they can manage how it runs.

It is only under the Team Plan however to automate running of workflows from the hub:

Hope this helps,
TL

1 Like