Hi guys,
i hope to find you well.
This is just a sincere message to the knime’s developers and please read this in a propositive way.
I’ve been using knime from about 5 years and i consider it like a swiss knife, able to do many things in a very clever way. I often use it to rapid prototiping, it is simply unbeateable compared to other tools.
There’s something that’s been frustrating me lately:
-general performance of workflows
-inconsistency in the menus of the various nodes, like a common search bar for a field
-loops with very very poor performance, why?
-the new UI is nice, but too many features present in the old one are missing
-new releases are not QA enough, there are plenty of bugs
I know that the marketing is pushing you on the shiny things like generative AI and stuff like that, but let’s be onest, that’s only is icing on the cake. You should to actually improve the solid background of the tool in several aspect, and then, and only then, invest some worth time into exotical feature that maybe is used by only 1% of your customer. You know what i mean.
Give us solid performance, a proper columnar backend that is working 100%, an UI coherence between nodes: this makes us very happy, all of us
I’m not much of a fan of the trend everywhere to dumb down the UI and even less in an application that by definition needs to have many options and has complexity. It will just reduce the feature set. Plus I don’t really see “non-tech” users starting to use it. It’s not efficient, let experts to their jobs they are best at and let them ask other experts to help them with their tasks. The people the new UI is geared for, simply don’t have the time for this.
On a positive node, it’s a step to get rid of the eclipse baggage maybe? That could make things more performant down the road but it will take years.
@Luca_Italy when I first saw the new flat and minimalist design of KNIME 5.0 I was reminded of my first impression of Office 365 when they switched to a flat design with a lot of large buttons and no more hundreds of menu items. As a hardcore user for many years I could not find a single thing and was frustrated. But after some time I noticed that all the right functions were at the right place and a lot of thought has been given to make the software more usable for new customers.
When I take a look at the nightly builds of 5.2 I get the same feeling that there is a way where the new UI will be really comfortable and advanced settings are still there.
5.1 as of now lacks consistency and stability and I am not sure I would have made the decision to launch - although it is still not recommended for production. The risk is that new users are confused until the 5.2 is fully there (it might be December ?).
I think for KNIME it is a question of growth. Existing users will most likely continue using the software (they still can use the old UI for some time I think). The question is how to lure new users and integrate KNIME with cloud platforms.
Often I see KNIME in an comfortable/uncomfortable middle position. Hardcore coders do not like low-code (takes away the ‘magic’ of import pandas as pd …) and people with few data analytics experience still deem the software too complicated. My take would be that especially with the upcoming advanced (Python) code editor KNIME really could be the platform - but it is not a household name (yet) despite its various advantages.
The greatest success is when actually business teams and analytics people come together and build solutions. But sometimes management lacks the idea and strategy what a platform/solution should be and just use what seems to be hip or what some consultants promote (or just buy from the largest vendor “nobody has ever been fired for choosing IBM”).
So the best anyone in the forum can do is produce solutions to questions and provide useful examples on the hub and hint at bugs and problems (by raising tickets). It does help the KNIME development and help bringing the power of low-code analytics to a wide array of teams and people.
As an Apple user since a lot of years I really love interfaces where everything is in the right place and you can “think at your ideas forgetting the tools”.
Moreover, I’m not a full coder so I really appreciated the KNIME low code approach since the first day.
In my job, data cleaning, nornalizing and analizing of “real life data” coming from hospitals and from clinical and observational studies, KNIME is the perfect tool ( lots of users here now realized that Excel is not good for “everything” and “everyday”… great!).
So far …
I was very impressed about speed of KAP version 4.x running on Silicon processor.
Finally i saw complex flows processing millions of records using ALL the cores of my Mac.
When 5.1 was released ( I had not time to test the 5.0 ) I updated even I was not so happy of the new “different” UI … hoping to switch to the old one on a, may be, new and faster engine.
I was wrong … and I reported some lacks and inconsistency in some posts here.
I undestand the new approach and I believe is the future.
AI in healthcare data processins will be even more a game changer. So thanks a lot to KNIME Team.
In my personal experience as a technician I need to “translate” requests from my colleagues, mostly biologist and physicians, as they are not “coder” (no matter low, full ore none) and unlikely they will be.
I completely agree with @mlauber71, KAP 5.1 is still NOT ready for production even if is a great project and it is a good starting point to test this new approach.
May be (and I’m pretty sure) 5.2 will be far more better but actually I’m not so keen to test any beta version…and wait months doesn’t make me so happy.
I’m actually in the middle position to choose to get back to 4.7.7 but I’m really afraid of what could happen to my everyday flows…
Hope in some fixing shortly and good for everyone in this forum to support the developer team with bugs tracking and suggest improvements to make KNIME better and better.
( e.g I really miss the APPLY button in new UI nodes … and so on )
Hi @Luca_Italy and thank you for your general feedback,
We have been listing all feedbacks to address them in the next versions. Thanks to our active community we have received so many feedbacks so far. If there is a specific problem that you are experiencing, it would be very helpful to provide more information so that we can reproduce and ivestigate it. Sometimes, issues can be resolved by chagning some configurations or settings.
Would you specifically tell us which nodes you think are perfoming slower while the allocated resources are the same? I would be happy to run a test on those nodes to find the potential problem.
Would you name nodes or provide screenshot so I make sure I’ve understood your point correctly?
Have you noticed any change in the performance in the recent version? Which loops?
We are adding more features to the Modern UI which will be available in the next feature releases.
Sorry to hear that you have faced issues. We are addressing bugs and issues reported by the community. We would appreciate if you report the problems you encounter so we can resolve them as soon as possible.
Thank you for raising your concerns! And thanks to everyone who is chiming in. It is great to hear that KNIME is your swiss army knife, and we are very much dedicated to keeping it that way.
@armingrudd already asked detailed questions while I was writing. Thank you Here is a more general statement:
Yes, we have been making some big changes recently – most visible is the switch to a completely redone user interface, but also adding the new columnar table backend under the hood. As we know these are big changes and we know we cannot achieve feature parity from the start, we keep the old solution around such that you can decide which one you want to use.
We are very sorry to hear about your frustration with the current status quo. Let us address the two main points you mention:
Modern UI:
You nicely said “I often use it to rapid prototyping, it is simply unbeatable compared to other tools”. Making workflow building even easier is one of our main goals of the new user interface! This is the reason why there are new useful features like quick-node-insertion, an always-available output table view, easily accessible node execution buttons as well as the AI chatbot that can help when tackling problems outside of one’s field of expertise. And while they might feel different at the beginning, we hope that you’ll grow to like and use them a lot.
With KNIME 5.1 we deliberately made the modern UI the default to be able to gather feedback from a wide range of the community. Probably that was too early. Nevertheless, we did get a lot of constructive feedback and are working on implementing it. Bend points for node connections, pop-out table views etc. are coming to the next release thanks to this feedback.
If by “inconsistency in the menus of various nodes” you mean that some nodes are using the modern UI while others are still showing up in classic style: Please bear with us, we are in a transition period and we are working on it. As KNIME offers such a plethora of nodes, quite a lot even provided by the community, we cannot switch them all to the modern UI at once. While transitioning these dialogs to the modern UI we are also establishing a higher level of consistency. So if there are specific inconsistencies that come to your mind, please let us know.
Performance: We are always working on the performance of KNIME, here are a few of the recent developments
We improved the CSV reader performance for large files by allowing it to read in parallel.
We have speedups for GroupBy and Sorter nodes prepared which will become available with the next release.
The Columnar Backend is a large effort which provides superior performance compared to the default (row-based) backend in a lot of use cases already. But as the community has so many different use cases and set ups, we can only gradually optimize the Columnar Backend for more and more of them.
It sounds like you noticed degrading performance, what exactly did you observe, can you pinpoint it?
There are a lot of improvements in the works for the next KNIME release and we hope that these will reduce your frustration. If you want, you can try them out in the current nightly version. Until then, please help us in making KNIME as useful as possible by giving us detailed feedback whenever you encounter something that feels off
Hi @armingrudd,
i’ll try to answer to you, step by step:
general performance of workflows
the main and biggest point here is the engine under the hood. It would be wonderful if we could use the standard engine to prototype and then switch completely in the streaming one for better performance. When handling millions of rows the performance comparison with Alteryx is simply subpar.
inconsistency in the menus of the various nodes, like a common search bar for a field
take the groupby node for example, within the group tab we have the search for a field, in the manual aggregation one no.
Lag column:
I can find many many examples of that. If we have to work with tables with tens of columns it is very hard to pinpoint a single column. Let us search it, or give us an A-Z sorter in some way, u get the point.
loops with very very poor performance, why?
Everyone is complaining about performance of loops in Knime, it’s not a specific node or type of loop. I don’t know what is the reason.
the new UI is nice, but too many features present in the old one are missing
Ok on that, it should be reasonable that a current feature should be already present in a newer version
new releases are not QA enough, there are plenty of bugs
it’s a fact. Times ago we have to wait a minor release to go production with new version, now we have to wait the next major release to go live with the previous one.
I have to say, without KNIME I would have never even tried to understand biostatistics. Now, after 5 years of using KNIME, I am coding in Python in VS Code, but I don’t replace KNIME with it. I don’t think you have to convince programmers; they will never accept it. On the contrary, biologists, doctors, and all other non-coding enthusiasts love it. This is because it contextualizes programmatic/statistical solutions to real problems in a way that you can explain to others without forcing them to see the abstract code. Besides, coding is slowly dying as AI gets so good at it, and what is left is giving structure to solve problems and integrating different programming languages. This is where KNIME excels. There’s only one thing: I just wish the server weren’t so expensive to use.
I also want to profit to thank you for your workflows. Specially, for the “GPT4All - Create your own LLM Vector Store” absolutely magnificent.
Regarding KNIME Server costs, are you familiar with the recently released Teams Plan on the KNIME Community Hub? It’s a lot more affordable for smaller groups or individuals.
Sorry, I missed this followup question from you. What “server thing” do you mean? KNIME Business Hub is our enterprise replacement for KNIME Server, but the Teams Plan is an option to let you do certain things like versioning, scheduling, and remote execution for much less cost, as described in the link above.
