data manipulation with non-standard text files

Hello Knime community!  I'm  a newbie to KNIME but I have discovered in just a few months what a powerful tool it is for data analytics, both for experienced programmers as well as novices (like myself).  I also learn so much about KNIME from these posts and it is a great resource.  Recently, I have run into a problem in working with a non-standard unpivotted text file and I would like to reach out to you for some help.

The file contains missing values in one of the columns (where column name = "substance").  The "substance" value is only contained on the first row of the data chunk, but applies to all subsequent empty rows until the next "substance" value is declared (see sample table below).  I would like to create a table with the appropriate substance value assigned to each row in order to pivot the table by "substance" value.  I thought that this could be done by using flow variables and a looping function but I can't figure it out.  Note that the "substances" do not all have the same number rows of data assigned to them, so grouping the data by a fixed number of rows will not work.  Any suggestions or assistance would be much appreciated.

Best wishes and many thanks,

Steven

substance property value
218445 ENTR(1) 218445
  SUBS(1) V9
  SUBS(2) X99
  DIAG(1) 0
  INDX(1) I 6123
  CTFL(1) Z
218315 ENTR(1) 218315
  UKEY(1) 00218315-  -000
  SUBS(1) V9
  SUBS(2) X99
779 ENTR(1) 218315
  SUBS(1) V9



 

You can read the file with the File Reader and let it insert missing value in the substance column. Then you can use the Time Series Missing Value node (from the Time Series extension) and let it replace the missing values with the last non-missing value.

Hello Thor,

Thank you so much for your help here.  Your solution worked perfectly and saved me a huge amount of time, as I spent hours trying to accomplish this with recursive looping and flow variables.  The Time Series Missing Value node is now part of my "personal favorite nodes collection!

Thanks again and best wishes.

Steven