CONTINUOUS WRITING AND LOOKUP on THE SAME FILE

Hi,

Is there any way we can continuously writing a file and simultaneously Lookup the data from it.I am a newbie to Knime but need to achieve it,Kindly Help on the same.

It sounds as task for a database not a file.

In some etl tools we cn achieve this on file as well…Dynamic lookup is the concept where we can append the record and read at same time.but its a very expensive option as file will be in memory always.

With KNIME you could try and use a local H2 database that also could be created in memory. Maybe you could use that. But I am not entirely sure I understand what you are trying to do. And do you have an example of such a procedure from another system/database.

Yes Please find the scenario of other system

There is one source which is continuous(Data is coming continuously coming).
There is one target.
There is a transformation logic where I am using a lookup file(consist of two fields which are FIELD_1(Key) and FIELD_2(needed in my transformation logic))
If Source File Key is mapped to Loopkup File key then I am fetching the FIELD_2 else I am adding the source data to that lookup file,so that next time as soon as
same key data is coming from the source ,It can fetch the last inserted record of same key from the Lookup file and can perform the calculation on the previous loaded data and current data.

Now In Knime Here is the scenario which i am trying to achieve.

I am having two inputs one is the data sheet and other is the formula sheet.
Inside the formula sheet ,I Can have formula like A+B or A-B,Please refer the attached sheet.
Individual value of A,B,C are available in data sheet.Also Some formulae can again be calculated from some already calculated formula
for eg:Taxonomy_id F_6
Now My thought process is to use the data sheet and also to append record in it.
For eg:
For Taxonomy_id F_1 we can use the value of Taxonomy_id A,B directly from the data sheet and as soon as it is calculated I need to append the value of F_1 Taxonomy_id in the same data sheet
For Taxonomy_id F_2 we can use the value of Taxonomy_id C,D directly from the data sheet and as soon as it is calculated I need to append the value of F_2 Taxonomy_id in the same data sheet
So that For Taxonomy_id F_6 I can use both F_1 and F_2.
I need a generic solution(Not limited to this formula sheet)
to decode any kind of formula having the basic four arithmetic operators*(±*/).*It would be really helpful for me.
Final Output is also attached.
DATA_SHEET.xlsx (9.7 KB) FINAL_OUTPUT.xlsx (8.7 KB) FORMULA_SHEET.xlsx (9.0 KB)

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.