extract table from txt file

[praffulgoyal]

Hi Guys ,

I need some help as i am new KNIME.

I have a txt file similar to below.

I need to extract this data in below format.

Is it possible to do ?

This is sample data . my data is very big thats why i want to use KNIME.

Summary of Data.txt (813 Bytes)

[ipazin]

Hello @praffulgoyal,

and welcome to KNIME Community!

You can try CSV Reader node and tab (\t) as a column delimiter with option Support short data rows checked.

Br,
Ivan

[rfeigel]

Is this the actual format of your data? Its a mess and will require extensive manipulation in order to process it.

[rfeigel]

Try this.
extract table from txt file.knwf (101.7 KB)

[mlauber71]

@praffulgoyal I have taken the liberty to expand the solution by @rfeigel to have a dynamic detection of the start of the second table:

Extract table from txt file - KNIME Forum (91178) – mlauber71

[fe145f9fb2a1f6b]

there are multiple ways to solve this, depending a bit on how alike the sample is to the full data set.

if the 2 example blocks are all, then I would just configure 2 csv readers with hardcoded amount of rows skipped etc.

if this isnt the case, but the structure repeats exactly as is, I would recommend to do it in 2 steps:

first step splitting the file and creating new files for each “block”. and then reading them in

[rfeigel]

@mlauber71 Very nice improvement! Kudos to you.

[praffulgoyal]

Thanks @rfeigel @mlauber71 ..for your efforts in providing me the workflow. It help me understand KNIME more. But like @fe145f9fb2a1f6b mentioned my data is not limited to these 2 tables. It has several repetition like that. May be i need to consider some kind of looping. But you guys has definetly given me idea. I will try myself. Meanwhile if anyone of you can help me in looping then it will be awesome.

Thanks again ..really appreciate your time and efforts.

[rfeigel]

We can only work with the data you provide. If you provide a complete data set we might be able to take another look.

[mlauber71]

@praffulgoyal as @rfeigel has said: it all depends on your description of the challenges you face, like: repeated blocks that have a common identifier.

One way to do it is to identify the end of one block and then iterate over them. You can import the whole CSV first with the Fixed Width File Reader – KNIME Community Hub into one column - then identify the line which holds the block separator and then loop over the lines with a regular CSV reader with start and end line.

File Reader (Complex format) problem

@Dalmatino16 here os one option how to do this. You determine the start and end block and feed that as a flow variable to the File Reader. [image] The settings are dynamic depending on the position of the table [image] You might have to rename the columns though. kn_forum_43962_file_reader_complex_format.knwf (58.1 KB)

Reference Column Filter Node with LIKE operator

@Brotfahrer I had another look at the data and I would like to point you to another systematic approach. Using the underlying structure of the files to mark and split the relevant data. the big blocks seem to be defined by parts marked ECU. With the help of row numbers, you can identify and mark these blocks as “Level1” value pair. You would have to deal with the header values separately and maybe add the file name as constant column. The new LevelB is formed by Blocks that can be identified b…

MEDIUM: “KNIME, Paths and Loops — Automate Everything”