Sharing Results with "normal" people

I would like to share my findings that I obtained with Knime with others. Especially with people that do not have Knime and which are not that tech savvy. Is there a nice way? Or maybe someone can help me overcome the issues.

I would like to produce some simple visualizations (bar chart, line chart, pie chart), share them and also update them over time.

What I tried so far:
a) Export the data to a Google Sheet which I could share via a link. Export works fine using the “Google Sheets Updater”, but all Date, Int and Double fields are written as Strings. Therefore I can not use them to create charts.

b) Create a BIRT Report. There are hundreds of tutorials on how to create a BIRT reports. I tried it, and suceeded in creating the report. I can then look at it in the Browser ( Run → View Report → In Web Browser ) but I can not find out on how to make it accessible to others. How to export it to a Webserver.

c) Using the “normal” Bar Chart / Line Chart nodes in Knime. Nice, but how can people see them without Knime?

d) Export the Charts as static images. This works, but is obviously an ugly solution. It also does not allow for any kind of interactivity, like mose-over tool-tips.

e) Update an Excel file. Again, very ugly.

Hi masgo,

At work, most users have a browser on their computers and usually some kind of access to a server. Therefore, two other options are available:

f) Use R or Python nodes in KNIME to generate interactive views (as a web page) or a set of static views (svg, png, pdf). Examples of libraries which can do this are e.g. the vegawidget package in R (with vega-lite and vega as backends) and the Altair library in Python, the latter also having vega and vega-lite as its backend. In R, you can easily generate embedded web pages by using knitr and rmarkdown.

g) Use a Javascript view in KNIME and directly deploy vega or vega-lite in a web page. These two Javascript libraries generate their visualisations based on JSON specifications and the resulting views expose an API to do further things.

@Geo thank you for the suggestions. Do you have an example for any of those? Especially on how to get a website which is accessible without Knime. The generic JavaScript View seems to only offer a Web View in combination with the 20.000 € KNIME Server.

The R View (Table) – KNIME Hub Node also looks like it only produces a static image at it’s output.

I found this forum entry Google API: Sheets updater saves values as strings - #5 by oole
It shows how to make the Google Sheets Writer write numbers (instead of strings). Maybe this solves my problem.
The downside to the Google Sheets solution is, that I have to configure the visualization in Google sheets, which is not that powerfull, but for the current problem it might be good enough.

I have to add a small disclaimer, I work for KNIME :slight_smile:
Our recommendation for sharing your work with others are KNIME DataApps and yes this needs a commercial license for our KNIME Server. You can here read more about them: Empower Business Users to Drive Action with Data Science | KNIME

If you have KNIME Server, every consumer has access to a webportal, with this you can also download the BIRT Report which is generated from your workflow.

With DataApps :slight_smile: in a webbrowser

3 Likes

Chapeau! That is an awesome find! I struggled with the same and quickly implemented this fix.
(And also gave the internal ticket (AP-10461) a big plus, we will fix this soonish!)

4 Likes

Hi @masgo,

There is unfortunately no free lunch :slight_smile: With R or Python, you can most certainly do all this, however, you will spend time on your side on assimilating the technology. The following gallery demonstrates what can be done with R’s markdown and knitr: Gallery

If you take a look at the source code of the examples, you will notice that they are essentially markdown reports with embedded R code. These reports can be transformed into fully working html pages using the knitr package. If you use R Studio, Rmarkdown and knitr are well integrated therein.

Regarding the interfacing with KNIME, given that you are doing all the processing in the latter:

  • outside of KNIME, the interface will probably be a set of CSV or JSON files deployed somewhere;
  • inside of KNIME using the R nodes, you will indeed be limited to (multiples of) static images (svg, png, etc.).

Kind regards
Geo

If we are talking about data visualization then why not use write the KNIME output to a file / database and then use a tool like Tableau ?

Hi @masgo,

maybe there is something in the making for you. I’ve dealt with this issue for quite some time and have also talked to people who construct full HTML-Reports “manually” in KNIME.

To overcome this shortcoming I developed a set of nodes which leverage Asciidoctor for the output. Asciidoc is a human-readable markup language comparable to Markdown with a more strict definition and is supported by Asciidoctor, GitHub, Gitlab and a few others. This language allows to gradually add simple text sections of the final document in Asciidoc and finally render it to .html or .pdf (Asciidoctor also supports other formats like epub or Docbook but thats untested by me so far).

Let me give you a sneak preview from my development workflows:

The three Java snippets add building blocks of the document and the GroupBy Node concatenates everything to Asciidoc-Text in the bottom of this post. The two AsciiDocWriter Nodes export the raw text to either an .html-Document or a .pdf (Asciidoctor also supports epub and Docbook but I haven’t tested that so far).

The results are as follows:
Rendered HTML-File
Rendered PDF-File

For a more complex example of whats possible with Asciidoc see this file

That’s a rather short example but it definitively also works for longer documents. I’ve used it to construct a report about published preprints where the data fore each paper has been added in a loop and then a final document has been rendered. We used to run this workflow in a dockerized KNIME triggered by Jenkins and send out the report by email but it should be somewhat straightforward to also publish the generated files to a webserver.

So far the Nodes have only been used internally but I am currently working with Daniel and Philipp from NodePit to release these nodes for the general public. I hope to get out a beta version by the end of the year at the latest. So if that sounds interesting be sure to follow my posts and the NodePit website for further details… :wink:

Cheers,
Lars

And last but not least for reference the raw source text for both outputs:

= Asciidoc Nodes Example
:description: This document is a quick example about what's possible with the AsciiDoctor Nodes for KNIME
:keywords: AsciiDoc, Asciidoctor, pdf, theme, themeing
:page-description: {description}
:page-keywords: {keywords}
:page-layout: docs
:page-javascripts: [view-result]
:experimental:
:table-caption!:
:example-caption!:
:figure-caption!:
:toc: macro
:icon: font
:icon-set: fab
:imagesdir: D:/Temp


The AsciiDoc Nodes are a new way of reporting in KNIME. It is easy to construct reports in multiple blocks and then convert them to an .html or .pdf-Document


It's also easy to integrate images that have been previously rendered to .jpg, .png or .svg!

.Asciidoctor image
[#AsciidoctorLogo,link=https://asciidoctor.org/]
image::Asciidoctor.png[Asciidoctor,318,159]
5 Likes

@Iris Knime Server is just waaaay out of our price range. I am not doing anything that could not be done in Excel. The Workflow is basically just join, join, filter, group by, sort and visualization. But I don’t like Excel and I am a fan of open source software.

About the Google Sheets thing. The option is already there in the node, but hidden in the variable section. Just make it a checkbox on the normal node-config page and maybe add a tooltip to explain what it does and you are good to go.

@Geo by now, I worked with dozens of visualization tools. R is very powerful for complex datasets but can get quite ugly. The beauty of Knime is, that you do something … then come back to it after a few months and it’s quite easy to understand what you did and where you need to change something. The ability to look at the results of each node are like an integrated debugger. I can not say the same about R.

@Daniel_Weikert Tableau was not open source the last time I checked.

@PTDataScientist looks promising. Why not use markdown? Seems to be the more widespread language.

@masgo

Two main reasons for Asciidoc:

  1. It has one syntax - you don’t have to deal with a plethora of flavors or deal with the least common denominator CommonMark which still not includes tables although the idea ist discussed since 2014 in a (as of now) 190 post long thread. Coincidentally the 189s post from a month ago reads “It surprises me how often people want to reinvent AsciiDoc.:innocent:
  2. Asciidoctor treats pdf output with the same priority as html, i.e. it has a dedicated processing toolchain comprising extensive theming options. All relevant markdown libraries that I’ve seen so far only render .html directly and rely on things like OpenHTMLtoPDF to convert html to pdf which has new challanges e.g. with pagination.

Nonetheless I already thought of leveraging commonmark-java or flexmark-java in a different set of nodes later on but as for many things this is a matter of time and priorities. :man_shrugging:

1 Like

Tableau was just meant as an example. You could use another tool
Tableau Public is free (by the way) if that is your selection criteria. You could use that if you can live with the limitations.
br

That’s not really inline with using Google software. Their software is far away from Open Software.
Maybe you mean Freeware. Even then you should have a deeper look on the contract you make with Google

  • A view to the license agreement that you agree to
  • Check what you need to give in order to use the software
  • Consider whether your partners/consumers may use the software without being bound too

Having said this I think there’s a huge defference between EXCEL (not even freeware), Google sheets (freeware but you will pay with data) and KNIME (Freeware or paid by money).

As you said: There’s no free lunch. All of us need to earn money.
At the end of the day it’s typically more than just (direct) costs of software that count:

  • Support
  • Creditability of the company
  • Data security & ISEC concerns
  • Simplicity of the whole system architecture (esp. data interfaces)
  • Availability of (own and external) resources to maintain the solution

just to name a few.

1 Like