Should We Slow Down Fancy Updates?

Feedback & Ideas
1

When I started using OpenAI’s API access through Python, I experienced frequent challenges due to their ever-changing API endpoints. These constant updates have been a nightmare for developers. Every few months, new changes alter how we connect to their platform, forcing us to update tools and workflows repeatedly. It’s a frustrating and disruptive cycle, especially for those managing multiple projects.

I recently experienced a similar challenge when I reopened a workflow I developed in KNIME 5.2.0. This workflow included a CNN that took me over three months to perfect. To my surprise, it no longer worked in the latest KNIME version due to broken nodes and inconsistencies. Fixing it would require rewriting and relearning large parts of the workflow—an enormous investment of time and effort for something that previously worked perfectly.

image
image600×622 39 KB

Of course, I could stick to the old KNIME versions where everything worked, but then I’d miss out on all the new features and enhancements. It feels like a no-win situation: I can’t fully embrace the new tools without losing access to reliable workflows from older versions, and I can’t go back without sacrificing progress. Overall, I feel we are undermining the hard work and time that initial KNIME developers invested to create reliable nodes like Keras and Tensorflow nodes for CNNs and image processing.

While innovation and new features are exciting, it’s essential to ensure reliability and backward compatibility. Rapid updates can inadvertently disrupt workflows, leaving users to rebuild complex solutions from scratch. Reliable tools that once performed seamlessly now feel like they are being “dragged along” without sufficient updates to maintain functionality.

I wonder:

  1. Am I alone in this? Have others faced similar issues with broken workflows?

  2. Should KNIME consider focusing on slower, more deliberate updates that ensure compatibility and reliability over time?

KNIME has always been a community-driven platform, and I believe it has the potential to strike a balance between innovation and stability. Building tools we can trust and maintain for the long term will always be more valuable than rushing for new fancy features at the expense of reliability.

Let me know your take on this.

AG.

2

At no point should a new version be able to negatively impact the production environment.
Best business practice is to make backups and ensure, through testing in a dedicated sandbox environment, that production workflows are not impacted by software updates.

I acknowledge that in today’s AI-driven world, requirements change rapidly, but I agree that a stable and reliable open-source tool is preferable to one full of new features but lacking stability and with compatibility issues with previous versions of workflows.

To be used with professional confidence, a tool like KNIME must be credible and cannot fail to function or lose features when updated.

KNIME is an outstanding tool, and to uphold the high standards we have come to expect, this issues cannot be overlooked. Otherwise, the risk of losing users is high.

2 Likes
3

Thanks @hmfa agree.

I feel I should also add some points for clarification.

  1. When I say “fancy updates,” I don’t mean small new features like those in versions 5.3.0 to 5.3.1, but rather major updates like from 5.2 to 5.3.
  2. TensorFlow and Keras problems are also exacerbated by the rivalry between NVIDIA (CUDA) and Apple Silicon.
  3. There are PyTorch packages that can handle CNNs and are much more compatible with Apple Silicon, but we lack native nodes in KNIME. Perhaps KNIME management could foresee future problems like this and react accordingly.
  4. The problem with fast updates primarily stems from the development of LLM coders. While they motivate programmers to do ingenious things using coding assistants like Copilot, they often don’t check compatibility with older, handcrafted solutions. This is the root of the issue in my opinion. It’s good to use LLMs to accelerate innovation, but we can’t rely solely on them. We must prioritize backward compatibility and maintain a balance between rapid development.
2 Likes