Loop not collecting results

KNIME Analytics Platform
1

Hi,

I have a nested loop where I want to combine certain acids with amines without enumerating the complete matrix. However the loop does not collect the combined results, only the iterations. I have feeling it is something trivial but am not seeing it right now…

Any hints appreciated.

Thanks/Evert

Loop_issue.knwf (42.9 KB)

2

Hi Evert,

what should be collected? The output of the cross joiner node?
I would use “loop end”

image
image944×455 40.1 KB

3

Yes, the output of the Cross Joiner node should be collected, but the wat the Loop End node is det up now it does not work.

BW/Evert

4

Could you provide the data? Or some samples?

I think the combination of two Chunk-loops is tricky.

image
image1060×373 50.5 KB

image
image649×524 80.4 KB

Do you really need to work with chunks? I use them only if the data is HUGE

1 Like
5

@evert.homan_scilifelab.se you might want to wrap the inner loop into a component.

image
image2056×1444 309 KB

Loop issue - KNIME Forum (85363).knwf (94.5 KB)

1 Like
6

This is just a prototype, I want to work with list of up to 40K acids and amines when it works. The purpose is to sample the matrix systematically without enumerating the full matrix first.

BW/Evert

7

Thanks Marcus,

Your suggestion gives me the full matrix of 10K combinations, which is not what I want. I want to take the first 5 acids and enumerate the matrix with the first 5 amines, then next 5 acids with next 5 amines, and so on. Hence I expect 20*25 = 500 rows as combined result. Also unclear why a component is needed to make it work but that is OK so long as it works.

BW/Evert

8

@evert.homan_scilifelab.se the first question was that the collection would not work and that was possibly due to the use of Variable loop end instead of Loop end. Also the component might provide additional stability with this somewhat unusual setup.

The next question would be how to plan ahead what you want to do. Maybe you can elaborate more and then we can discuss how to set up such a task efficiently. I might try to take a further look later.

10

Hi,

I managed to get it to work with this Python script, as suggested by Perplexity:

import knime.scripting.io as knio
import pandas as pd

# Convert input tables to pandas
acids_df = knio.input_tables[0].to_pandas()
amines_df = knio.input_tables[1].to_pandas()

# Assuming first column is ID and second column is the acid/amine name
acid_ids = acids_df.iloc[:, 0].tolist()
acid_names = acids_df.iloc[:, 1].tolist()
amine_ids = amines_df.iloc[:, 0].tolist()
amine_names = amines_df.iloc[:, 1].tolist()

def combine_acids_amines(acid_ids, acid_names, amine_ids, amine_names):
    combined = []
    for i in range(0, len(acid_ids), 5):
        acid_id_chunk = acid_ids[i:i+5]
        acid_name_chunk = acid_names[i:i+5]
        amine_id_chunk = amine_ids[i:i+5]
        amine_name_chunk = amine_names[i:i+5]
        
        for aid, aname in zip(acid_id_chunk, acid_name_chunk):
            for mid, mname in zip(amine_id_chunk, amine_name_chunk):
                combined.append([aid, aname, mid, mname])
    return combined

# Create combinations
combinations = combine_acids_amines(acid_ids, acid_names, amine_ids, amine_names)

# Convert to pandas DataFrame
result_df = pd.DataFrame(combinations, columns=['Acid', 'Acid_ID', 'Amine', 'Amine_ID'])

# Output to KNIME
output_table = knio.Table.from_pandas(result_df)
knio.output_tables[0] = output_table

BW/Evert

1 Like