How to create continuous Timestamp between files?

Hello, everyone!

I’m going to analyze data with timestamp.
One preprocessing task is to generate continuous Timestamp between files.

Please find the attached two example files.
The last row of file 001 contains a timestamp: 2018/12/09 00:08:34.562
The first row of file 002 contains a timestamp: 2018/12/09 00:10:31.702 (461 Bytes)

What I need is to create dummy data rows with timestamp (20msec pitch) between the upper two files,
following the last row timestamp in file 001 to connect with timestamp of 002.

An example of new 001 (create dummy from the 5th. row):
2018/12/09 00:08:34.502,2.21128017e-02, 292
2018/12/09 00:08:34.522,2.21128017e-02, 292
2018/12/09 00:08:34.542,5.67591749e-02, 292
2018/12/09 00:08:34.562,5.67591749e-02, 292
2018/12/09 00:08:34.582,0, AAA
2018/12/09 00:08:34.602,0, AAA
2018/12/09 00:08:34.622,0, AAA

2018/12/09 00:10:31.682,0, AAA

Is it possible to do this in KNIME?

Many thanks!

Hi @qianyi,

You can do something like this:

You read in the file you need. Extract the first or last row from the TIME column. Convert that to a flow variable which you can feed in the Create Data&Time range node. If you have to read in the files dynamically, then this gets a bit more complicated and you will have to use loops and flow variables.

Hope that helped.

continuous-timestamp.knwf (79.0 KB)

1 Like

Hi @johannes.schweig

Thanks for your reply, it just worked!

I’m still very new to KNIME, and totally don’t know Node [Merge Variables].
This workflow tells me a lot of things. Appreciate it!

Next step I think I have to use loop since actually I need to handle with thousands of files, I will try to do it myself, but maybe need your help again…

This is very advanced stuff that you want to do. Just try and reach out again if you need help.

Hi @johannes.schweig

Since last your response I have tried a lot to create a roop workflow to solve this stuff, but not successful. Finally I tried other way for I don’t have enough time.

But I still need to create a KNIME workflow solution since I have to face with similar problems in future.
I have create 10 csv files for this (attached), can you please advise on how I can do? (3.2 KB)

Many thanks!

Hi @qianyi,

have a look at the archive I uploaded. I wrapped the previous workflow in a loop that takes always two datafiles in pairs (e.g. 1 and 2, then 2 and 3, then 3 and 4…) and computes the range, concatenates that with the first file and in the end adds the last one. This should work.

continuous_Timestamp.knar (63.5 KB)

Hi @johannes.schweig

That’s awful! This contains many advanced technique for me, I’m studying it and try to run my real data. Thank you so much!!