Sharing Results with "normal" people

KNIME Analytics Platform
1

I would like to share my findings that I obtained with Knime with others. Especially with people that do not have Knime and which are not that tech savvy. Is there a nice way? Or maybe someone can help me overcome the issues.

I would like to produce some simple visualizations (bar chart, line chart, pie chart), share them and also update them over time.

What I tried so far:
a) Export the data to a Google Sheet which I could share via a link. Export works fine using the “Google Sheets Updater”, but all Date, Int and Double fields are written as Strings. Therefore I can not use them to create charts.

b) Create a BIRT Report. There are hundreds of tutorials on how to create a BIRT reports. I tried it, and suceeded in creating the report. I can then look at it in the Browser ( Run → View Report → In Web Browser ) but I can not find out on how to make it accessible to others. How to export it to a Webserver.

c) Using the “normal” Bar Chart / Line Chart nodes in Knime. Nice, but how can people see them without Knime?

d) Export the Charts as static images. This works, but is obviously an ugly solution. It also does not allow for any kind of interactivity, like mose-over tool-tips.

e) Update an Excel file. Again, very ugly.

2

Hi masgo,

At work, most users have a browser on their computers and usually some kind of access to a server. Therefore, two other options are available:

f) Use R or Python nodes in KNIME to generate interactive views (as a web page) or a set of static views (svg, png, pdf). Examples of libraries which can do this are e.g. the vegawidget package in R (with vega-lite and vega as backends) and the Altair library in Python, the latter also having vega and vega-lite as its backend. In R, you can easily generate embedded web pages by using knitr and rmarkdown.

g) Use a Javascript view in KNIME and directly deploy vega or vega-lite in a web page. These two Javascript libraries generate their visualisations based on JSON specifications and the resulting views expose an API to do further things.

3

@Geo thank you for the suggestions. Do you have an example for any of those? Especially on how to get a website which is accessible without Knime. The generic JavaScript View seems to only offer a Web View in combination with the 20.000 € KNIME Server.

The R View (Table) – KNIME Hub Node also looks like it only produces a static image at it’s output.

4

I found this forum entry Google API: Sheets updater saves values as strings - #5 by oole
It shows how to make the Google Sheets Writer write numbers (instead of strings). Maybe this solves my problem.
The downside to the Google Sheets solution is, that I have to configure the visualization in Google sheets, which is not that powerfull, but for the current problem it might be good enough.

5

I have to add a small disclaimer, I work for KNIME :slight_smile:
Our recommendation for sharing your work with others are KNIME DataApps and yes this needs a commercial license for our KNIME Server. You can here read more about them: Empower Business Users to Drive Action with Data Science | KNIME

If you have KNIME Server, every consumer has access to a webportal, with this you can also download the BIRT Report which is generated from your workflow.

With DataApps :slight_smile: in a webbrowser

3 Likes
6

Chapeau! That is an awesome find! I struggled with the same and quickly implemented this fix.
(And also gave the internal ticket (AP-10461) a big plus, we will fix this soonish!)

4 Likes
7

Hi @masgo,

There is unfortunately no free lunch :slight_smile: With R or Python, you can most certainly do all this, however, you will spend time on your side on assimilating the technology. The following gallery demonstrates what can be done with R’s markdown and knitr: Gallery

If you take a look at the source code of the examples, you will notice that they are essentially markdown reports with embedded R code. These reports can be transformed into fully working html pages using the knitr package. If you use R Studio, Rmarkdown and knitr are well integrated therein.

Regarding the interfacing with KNIME, given that you are doing all the processing in the latter:

  • outside of KNIME, the interface will probably be a set of CSV or JSON files deployed somewhere;
  • inside of KNIME using the R nodes, you will indeed be limited to (multiples of) static images (svg, png, etc.).

Kind regards
Geo

8

If we are talking about data visualization then why not use write the KNIME output to a file / database and then use a tool like Tableau ?

9

Hi @masgo,

maybe there is something in the making for you. I’ve dealt with this issue for quite some time and have also talked to people who construct full HTML-Reports “manually” in KNIME.

To overcome this shortcoming I developed a set of nodes which leverage Asciidoctor for the output. Asciidoc is a human-readable markup language comparable to Markdown with a more strict definition and is supported by Asciidoctor, GitHub, Gitlab and a few others. This language allows to gradually add simple text sections of the final document in Asciidoc and finally render it to .html or .pdf (Asciidoctor also supports other formats like epub or Docbook but thats untested by me so far).

Let me give you a sneak preview from my development workflows:

image
image1244×646 39.9 KB

The three Java snippets add building blocks of the document and the GroupBy Node concatenates everything to Asciidoc-Text in the bottom of this post. The two AsciiDocWriter Nodes export the raw text to either an .html-Document or a .pdf (Asciidoctor also supports epub and Docbook but I haven’t tested that so far).

The results are as follows:
Rendered HTML-File
Rendered PDF-File

For a more complex example of whats possible with Asciidoc see this file

That’s a rather short example but it definitively also works for longer documents. I’ve used it to construct a report about published preprints where the data fore each paper has been added in a loop and then a final document has been rendered. We used to run this workflow in a dockerized KNIME triggered by Jenkins and send out the report by email but it should be somewhat straightforward to also publish the generated files to a webserver.

So far the Nodes have only been used internally but I am currently working with Daniel and Philipp from NodePit to release these nodes for the general public. I hope to get out a beta version by the end of the year at the latest. So if that sounds interesting be sure to follow my posts and the NodePit website for further details… :wink:

Cheers,
Lars

And last but not least for reference the raw source text for both outputs:

= Asciidoc Nodes Example
:description: This document is a quick example about what's possible with the AsciiDoctor Nodes for KNIME
:keywords: AsciiDoc, Asciidoctor, pdf, theme, themeing
:page-description: {description}
:page-keywords: {keywords}
:page-layout: docs
:page-javascripts: [view-result]
:experimental:
:table-caption!:
:example-caption!:
:figure-caption!:
:toc: macro
:icon: font
:icon-set: fab
:imagesdir: D:/Temp


The AsciiDoc Nodes are a new way of reporting in KNIME. It is easy to construct reports in multiple blocks and then convert them to an .html or .pdf-Document


It's also easy to integrate images that have been previously rendered to .jpg, .png or .svg!

.Asciidoctor image
[#AsciidoctorLogo,link=https://asciidoctor.org/]
image::Asciidoctor.png[Asciidoctor,318,159]
5 Likes
10

@Iris Knime Server is just waaaay out of our price range. I am not doing anything that could not be done in Excel. The Workflow is basically just join, join, filter, group by, sort and visualization. But I don’t like Excel and I am a fan of open source software.

About the Google Sheets thing. The option is already there in the node, but hidden in the variable section. Just make it a checkbox on the normal node-config page and maybe add a tooltip to explain what it does and you are good to go.

@Geo by now, I worked with dozens of visualization tools. R is very powerful for complex datasets but can get quite ugly. The beauty of Knime is, that you do something … then come back to it after a few months and it’s quite easy to understand what you did and where you need to change something. The ability to look at the results of each node are like an integrated debugger. I can not say the same about R.

@Daniel_Weikert Tableau was not open source the last time I checked.

@PTDataScientist looks promising. Why not use markdown? Seems to be the more widespread language.

11

@masgo

Two main reasons for Asciidoc:

  1. It has one syntax - you don’t have to deal with a plethora of flavors or deal with the least common denominator CommonMark which still not includes tables although the idea ist discussed since 2014 in a (as of now) 190 post long thread. Coincidentally the 189s post from a month ago reads “It surprises me how often people want to reinvent AsciiDoc.” :innocent:
  2. Asciidoctor treats pdf output with the same priority as html, i.e. it has a dedicated processing toolchain comprising extensive theming options. All relevant markdown libraries that I’ve seen so far only render .html directly and rely on things like OpenHTMLtoPDF to convert html to pdf which has new challanges e.g. with pagination.

Nonetheless I already thought of leveraging commonmark-java or flexmark-java in a different set of nodes later on but as for many things this is a matter of time and priorities. :man_shrugging:

1 Like
12

Tableau was just meant as an example. You could use another tool
Tableau Public is free (by the way) if that is your selection criteria. You could use that if you can live with the limitations.
br

13

That’s not really inline with using Google software. Their software is far away from Open Software.
Maybe you mean Freeware. Even then you should have a deeper look on the contract you make with Google

  • A view to the license agreement that you agree to
  • Check what you need to give in order to use the software
  • Consider whether your partners/consumers may use the software without being bound too

Having said this I think there’s a huge defference between EXCEL (not even freeware), Google sheets (freeware but you will pay with data) and KNIME (Freeware or paid by money).

As you said: There’s no free lunch. All of us need to earn money.
At the end of the day it’s typically more than just (direct) costs of software that count:

  • Support
  • Creditability of the company
  • Data security & ISEC concerns
  • Simplicity of the whole system architecture (esp. data interfaces)
  • Availability of (own and external) resources to maintain the solution

just to name a few.

3 Likes
14

I am a fan of open source software. I am also a fan of getting things done. So if Google works nicely, I might use it if there is no other solution.

Since Google sheets turned out to have it’s own limitations in regard to my project (apart from the usual concerns regarding using Google products), I opted to use static images for now.

I am using an SSH connector to write all images directly to the webserver.
By using a SFTP chroot setup I am lowering the risk of saving user+password in the KNIME flow.

Writing the visualized data came with it’s own challenges (e.g., JavaScript Views seem to be broken in batch mode ), but I found a solution for it as well.

I don’t have interactive graphs, but a static website with graphs that update every day is good enough for this project.

Only caveat at the moment is the DB connection. Since the DB is only accessible through an SSH tunnel, I need to have a batch file which first starts the tunnel, runs KNIME and then stops the tunnel again. It would be really nice if KNIME had SSH tunneling built into the DB connectors.

15

I am personally settling with automated BIRT PDF reports distributed by the email node for my KNIME AP only solution. You can also do some “one off” free interactive html graphs and maps though datawrapper.de

Beyond that I use Tableau.

16

@PTDataScientist - Would these new reporting nodes open up more opportunities to build interactive html dashboard style reports? What would the advantages & disadvantages be vs BIRT?

17

If you settle on the HTML export it should be possible to include raw html/js blocks generated elsewhere (e.g. from plotly) although I have not tested that yet. Obviously that doesn’t work with a PDF-export.

Generally I don’t think that the creation of dashboard like reports would be a prime usecase for the nodes. These are more about “written” type of reports although you can also prepare reveal.js slideshows with AsciiDoc (see here for an example with AsciiDoc source).

For dashboards we need to wait for KNIME to also allow building nodes with R like it is now possible with Python. Then creating (at least simple) flexdashboards from an RMarkdown file would be breeze…

1 Like
18

I am personally settling with automated BIRT PDF reports distributed by the email node for my KNIME AP only solution.

@iCFO
How do you export a BIRT report? I can only find “Image to Report” and “Data to Report” nodes, but they do not have an output port. So how do you obtain the input for the email node?

19

@masgo - I had just posted this one on another thread.

You can use a parent workflow for the main processing, then have it run a child workflow that compiles the BIRT report and saves it to your chosen format and file location.

KNIME Community Hub

ipazin/Public – Multiple Reports

contains 3 items

This solution is geared toward running multiple reports by loop, but it is also my go to approach to save / email single reports as well.

1 Like
20

The explanation: Also from other thread…

Right now BIRT is only accessed separately through the menu after a workflow is executed, so this workaround is the only solution that I am aware of at this point…

Hopefully someone can develop a “Report Output” node that allows us to create and further manage BIRT reporting files within a single workflow. Since BIRT can handle looped file outputs through a child workflow, it seems like all of the programming exists except for the return path from BIRT. We just need the workflow to “pass through” BIRT instead of end there. :crossed_fingers:

1 Like