Suggestions & Improvements

KNIME Analytics Platform
#1

Hi, Below are a few suggestions and improvements to KNIME:

- When using the scrollbars in the workflow pane, they scroll as far as the furthest most node, likewise for the arrows next to the scrollbars. However, when building a workflow, you often get to the far right, and want more space to correctly place nodes in a desired position. Therefore is it possible to make it so that if you keep clicking on the arrows next to the scrollbars that it carries on beyond the last node and into white space which will then make workflow design less hassle. This has been a constant niggle I have had which I keep meaning to post.

- Inside a metanode, is it possible to make it so that the metanode vertical greybars with the arrows on are always a set distance away from the nearest node, so everytime you add a new node, they continue to move this set distance away. They have a habit of randomly placing themselves ontop of a node when you go back into the metanode, and sometimes when you try and move them further to the right, they jump back into the position they were in (i.e. ontop of a node which then makes editing or moving this node tricky).

- The "Delegating Loop Start" and "Delegating Loop End" nodes are great and really powerful, but its highly likely no-one knows what they do or where to find them. Would it make sense to rename them to "Recursive Loop Start" and "Recursive Loop End" and move them into the Flow Control section?

- Addition of "ChemSpider" nodes which accesses the free chemical database at www.chemspider.com. This would be really powerful to get chemical names from smiles (useful reverse process for the recent OSCAR implementation), to get molecular properties of molecules, analytical spectra, CAS numbers, and really helpfully chemical suppliers which is missing from KNIME at the moment.

- A Quickform chemical drawing program would be useful. Using the workflows through a webserver is very limiting without being able to draw the structure. I appreciate smiles can be used, but this is not very user friendly. Additionally, generation of the interactive table as a popup window for the output would be good too when using the webserver for workflows.

- Also for the quickform nodes, it would be nice to have an option in these nodes which enables them to pop up when the workflow is run in the KNIME application, asking for the user to select an option to be picked. i.e. it would be nice to set up the "String Radio Buttons" quickform node, then when a user runs the workflow, a window pops up in which the user selects the desired radio button. I can appreciate that in all cases, the user designing the workflow may or maynot want these windows to pop up, so just a tick box in the node configuration would be useful to turn this feature on or off. The reason I ask for this, is some users just want to press execute on a workflow and editing nodes can be unsettling to them, so having windows pop up for them to make options is a less scary alternative!

- "Export to E-Mail" node would be useful, where the entire table is emailed to the user specified email address in the node configuration. And also a second email node "Cell to E-Mail" node would be useful which takes one column for the email address and one column for the cell contents to be emailed, running this node then sends out as many emails as rows in the table to the different email addresses. This can be useful to update clients in the table of some regularly changing property associated to them.

- An improvement to the "Maths" node would be good which besides having the column list in the node config, would also have the flow variables list too so you can build maths expressions with the available variables. The get around is to use two Maths nodes, one which just has a variable in the expression to make a new column, and then another to take this new column and build the maths expression, but why the need for making it so cumbersome. Also should the Maths node be moved from "Misc" category and put into "Data Manipulation/Row/Other" along with the "Rule Engine" node which is already there.

- "Statistics" node is missing a vital statistic. It shows how many datapoints have missing values, but strangely does not report how many datapoints there are in total. A rather strange omission. Please can this be included.

- Improvements to "HiLite Collector" node would be nice, at the moment its rather unuser friendly. Instead of editing the view prior to node execution, it would be much more user friendly if this view popped up when the node was executed, and the node remains in executing mode until the user completes the changes and closes the window. At this point the node completes and the rest of the workflow continues.

- "Sorter" node should automatically put missing value entries at the end of the sort list. Its most frustrating having them appear at the top of the list.

- "Integer to Double" node would be good. I appreciate this is easily done with the Maths node, but most users dont realise this. It makes it very simple for novice users if both options are together.

- The quick filter box in the node repository pane is still painfully slow. Can it be made so that if the user types in letters here, the addition of another letter by the user overides the searching of the node repository. The trouble is, after every letter is entered, it searches the node repository, this slows everything down when just typing in a 5 or 6 letter word (and also then deleting this word a letter at a time is slow too). It would be better if it waits for 2 seconds before searching the node repository, so it gives the user a chance to enter or delete all the remaining letters.

- A reworking of node repository structure would be good (please please!), particularly around removal of the KNIME Labs and Community Contributions directories and moving them into more appropriate folders, i.e. RDKit/Indigo/CDK/Erlwood/ChemAxon into Chemistry, and new categories in the repository root for Text Processing, Image Processing, Internet Processing (for Palladian and Web Analytics), Scripting (for Perl, Java, Python, Octave, R, MatLab, Groovy), also moving Weka and Statistics into Mining and rename it to Modelling&Mining, and moving Parallel Looping into the Flow Control section. The current node structure must be overwhelming for new users as there are so many nodes and often in non-intuitive locations, I'm forever showing people nodes they dont know about. If they dont know they are there, they dont know to search for them! If nodes in KNIME Labs are still experimental, you could always put a little test tube symbol next the directory name to indicate this, and for Community nodes, you could always add a little symbol of a crowd of people to indicate they are Community based.

 

Thanks,

Simon.

#2

Re item #1 above ("When using the scrollbars in the workflow pane, they scroll as far as the furthest most node") - I have had the same problem and second this request.

Re " "Integer to Double" node would be good" - I second this one as well.  I have had all kinds of problems with converting types, often because Java snippets won't do implicit data type conversions.  I thought rename might do some of this but a recent post indicates that it is a cosmetic change not an underlying change would could be very confusing.

Thanks

#3

Can you tell an example, when an explicit conversion from integer to double is necessary? If a node is implemented correctly, it can use integer columns out of the box if double values are required.

#4

For example, if you want to join two tables together with the Joiner node, and the column for comparison you use is Integer in one table and Double in the other table. You are forced to convert the other Double column to an Integer, in doing so, you risk losing all your decimalisation in the process. It would be better to be able to convert the Integer column to a Double column. Unless the transformation is done, the Joiner node will not proceed otherwise.

Also the Erlwood Matched Pairs Detector using RDKit requires the SAR column to be as Double, use of Integer does not work.

Speaking of decimalisation, one aspect missing from KNIME is the inability to round up/down values to x significant figures or to x decimal places. It can often be unwanted the number of decimal places on show.

Thanks,

Simon.

#5

Hi Simon,

I have also found that certain nodes require Double as input - which can be a bit of a pain, and I agree that for less experienced users certain "convenience function nodes" are a good idea!

I think the decimal place rounding is pretty straightforward with round(x,n) in the Math node.  However, for the s.f. rounding, I agree that this would be nice to have included (maybe also in the Math node?)

As a quick fix, there are numerous ways round this (I'm sure the Java Snippet node would be better - for support of flow variables, etc; but I still resort to python by default!), but the following can be pasted into the JPython Script 1:1 node to round to eg 3 s.f (outputting both a string column, and a new Double column):

n = 3

def round_to_n(x, n):

    if n < 1:

        raise ValueError("number of significant digits must be >= 1")

    return "%.*e" % (n-1, x)

dts = inData0.getDataTableSpec()

inputColumnIndex= dts.findColumnIndex("value")

iterator = inData0.iterator()

while iterator.hasNext():

    row = iterator.next()

    cell = row.getCell(inputColumnIndex)

    val = str(cell)

    new_val = round_to_n(val, n)

    newStringCell = StringCell(new_val)

    newDoubleCell = DoubleCell(float(new_val))

    newRow = AppendedColumnRow(row, [newStringCell, newDoubleCell])

    outContainer.addRowToTable(newRow) 

 

Obviously, the returned columns need to also be set in the Script Output Tab...  The function contained in this script was lifted directly from http://mail.python.org/pipermail/tutor/2004-July/030327.html

 

Kind regards

James

#6

Hi James,

I will add this script to my growing minibook of useful Java/Python scripts! This will be very useful. Long term, I hope the KNIME developers can wrap this up in a node or at least extend the Maths node functionality.

Thanks for pointing out the decimal rounding in the Maths node, that functionality is one that has somehow missed my attention.

Many thanks,

Simon.

#7

Dear all:

Thanks for the feedback. We'll put a node for Integer-to-Double conversion and for Rounding onto the list.

Cheers,
  Bernd
 

#8

Hi,

I think that this is relevent to this dicussion about shifting decimal points.

I had two integers of 477 ad 5463, but I wanted them both to be 0.xxxx so that my final result for the column was 0.477 and 0.5463.

I used this formula based on what I think is going on above in Python to only adjust values greater than one otherwise leave the value the same.

if($DebtRatio$>1, $DebtRatio$ / (pow(10, floor(log($DebtRatio$)+1))), $DebtRatio$)

However, it seems to be rounding it to 3 decimal palces and I don't know how to fix that issue. (Only started using KNIME today)

#9

The table output view of nodes shows only three digits of real numbers. You can change the display by right-clicking on the column header and selecting a different renderer.

#10

Thor,

I hear you, but it's very incovenient to do this on a column-by-column basis everytime you re-compute dozens of double-type columns (especially the ever-popular percentages common in business contexts).

Cheers
E

#11

You can change the default renderer for columns with real values in the preferences :-)

#12

Yes, but that's for all and any nodes, ever - something in-between *would* be nice & user-empowering! :-)

-E

#13

Slightly off topic but is there a portal to look at feature requests and vote on them? Other companies like Salesforce.com use this as a guide to understanding what users want… it doesn’t define the road map but guides some of the choices. the selected ones are also visible to the community (nice and transparent).SFDC_feature_voting

https://ideas.salesforce.com/s/prioritization

2 Likes
#14

Hi @DemandEngineer -

Certainly the developers are keen to know what feature requests the community comes up with, and there are a few KNIMErs who keep a close eye on feature requests (or bugs!) that appear regularly in the forum.

We currently don’t have a system where the community has access to our tickets and can vote on them, but it’s an interesting idea. I don’t think it’s that we’re opposed to it per se, it’s just figuring out the logistics of how such a system might work.

I’ll definitely relay your feedback though. Thanks for chiming in!

1 Like
#15

Pass this along if it helpful:

Love what you all do… I imagine you are doing some text mining of the feature request in some back end system… a package like this might streamline it.

Cheers,
William

1 Like