Hello KNIME community,
I’d like to propose an enhancement to the transformation tab in data import nodes like CSV and Excel readers. Currently, this tab allows us to select columns, specify data types, and rename columns. What if we could add a column for comments on each row of the transformation?
Here’s why I think this could be valuable:
- Documentation: Comments could provide context about why certain transformations were made, making it easier for team members or your future self to understand the rationale behind data preparation choices.
- Data quality notes: You could flag potential issues or inconsistencies in specific columns, serving as a reminder during later analysis stages.
- Tracking changes: Comments could help track the evolution of your data preparation process, especially useful in iterative workflows.
- Compliance and auditing: In regulated industries, comments could aid in maintaining a clear audit trail of data transformations.
- Knowledge transfer: New team members could more quickly understand the nuances of data import processes by reading these inline comments.
- Debugging aid: Comments could help pinpoint issues when troubleshooting complex data flows.
What do you think? Would this feature be useful in your KNIME workflows? I’m curious to hear others’ perspectives on this idea.
Hi @ekadagami,
the CSV reader has the option to define the comment character, which usually is a hashtag or double slash.
Do you mean that an existing and properly recognized comment should appear as a separate column?
Best
Mike
Hi Mike, thank you for the opportunity to clarify my suggestion.
I was not concerned with the processing of comments within source files such as CSV, but with the possibility of setting comments in the respective lines of the transformation.
I am not quite certain if I understand you correctly maybe because of some sample data or example workflow being missing.
Let’s assume the following CSV
Column 1,Column 2
text1,0
text2,1#My comment
| Column 1 |
Column 2 |
| text1 |
0 |
| text2 |
1#My comment |
What I understand is that you want to split the comment into a separate column but in a more convenient way as such that when a to be added option “Extract Comments into Column” is hit, the results should look like so
| Column 1 |
Column 2 |
Comment |
| text1 |
0 |
|
| text2 |
1 |
My comment |
Correct?
Best
Mike
Thank you for taking the trouble to understand me. I think our misunderstanding lies in the fact that you are working on the content level of a CSV (or Excel, or…) file, whereas my suggestion referred to the structural level.
My suggestion refers exclusively to the option of performing transformations for the naming of columns in terms of name and type. If the user changes something here, I think it would be good if a comment could be added at this point.
I have attached a screenshot of how I imagine this to work.
Got you … so you mean something like a comment added to a cell in Excel like so
Or Google Sheets like so:
Where the later I personally find most attractive as it allows to collaborate with colleagues. That idea of yours is actually quite nice especially considering where Knime is heading to.
However, if a data cell or column is getting changes, either by name, position, data type etc., a reliable implementation would be difficult to accomplish.
Maybe this has to be envisioned like a developer & user relationship. To give another example, Adobe InDesign and InCopy, work well together but InCopy is more for editors (users) to provide feedback to the designers which use InDesign.
So a nice implementation would be to:
- Add a comment to a column or cell
- Knime creates and keeps a data snapshot for consistency reasons but also allows to verify if the comment, i.e. a task to convert from int to double, was resolved
- Once resolve the data snapshot gets deleted
Best
Mike
