Text file manipulation resp text trimming

Hi,

it's me again.

This time witha a short question regarding the manipulation of a text file.

So I got a few text files to process, which contain information based on more or less the same pattern. I need to trim/cut off the first ten lines in the file (like nine line breaks).

I tried several things, for example to parse the files as a whole string and do something with the 'String Manipulation' Node or had a look at the Document Data Extractor Node in combination with the Flat File Document Parser Node.

But I can't really find a solution on that, even though its rather a basic problem.

If anyone has a quick suggestion or solution or any information about my problem I would be very thankful :)

Cheers,

Manu

Using a Java Snippet node you can simply do something like this:

int position = 0;
for (int i = 0; i < 10; i++) {
	position = c_column1.indexOf('\n', position + 1);
	if (position == -1) {
		break;
	}
}
out_column1 = position != -1 ? c_column1.substring(position + 1) : null;

Philipp

The file reader has an option "skip the first ... lines" You can find it in the advanced settings tab.

The csv reader has this option as well.

Or you read everything using a line reader and afterwards filter the first rows with a row filter.

Best, Iris

Hi Philipp and Iris,

thanks for your answers!

Regarding the Java Snippet, I didn't try it (yet), because I solved it by creating a shell script in linux using the sed command:


DIRS="$HOME/containing_folder*"

for d in $DIRS
do
	echo "Processing $d folder..."
	DIR=$d
	FILES="$d/*"
		for f in $FILES
		do
	  		echo "Processing $f file..."
	  		sed -e '1,10d' -i $f
		done
done

Regarding the file reader option, in case I'd use that node I can only read in one file at a time. Since I have a folder containing subfolders containing the actual 20000 files I can't really make use of that one.

Nevertheless thanks for your answer.

Cheers

Manu

Hi Manu,

of course you can read multiple files with KNIME Analytics Plattform in one run.

Take a look at the following workflow from our example server /011_FlowVarsAndLoops/011007_readMultipleFiles for an example how to set this up.

Best, Iris