SSH Connector retry loop

Hi there,

I am trying to build a component that will retry an SSH connection (to a SFTP) until it’s working, and more importantly, that will allow me to use the file system outport from the SSH Connector that succeeded outside of the loop.

The standard start of a SFTP workflow is like this:

The SSH Connector logs into the SFTP, lists the files, and if there are, proceeds to read them and then process the data.

Now I have been able to write a retry loop for the SSH Connection, which works well if all I want is a list of the contents of the SFTP:

I have to put the List Files/Folders in the Try/Catch because the SSH Connector doesn’t generate an output that I can replicate for the Fail input. That way, I add an error column that I add to the table containing the Path column, and the Fail Input reads a table with the same columns, and a different value in the error column, that is then used by the Variable Condition Loop End. Then the IF Switch stops the workflow if the Fail input is returned, or proceeds.

Now what I want to do is something like this:

Basically the same except that the working SSH Connector is then used to read the files outside of the component, in the rest of the workflow. This doesn’t work, the Variable Condition Loop End sends a warning but still works, whereas the Componnent Output does not because the file system connection is leaving the loop.

Is there any way to do this? Obviously the point here is to NOT retry another SSH Connection outside the loop, I am doing this because sometimes it will end in a timeout and just needs a retry, and doing it again outside the loop (without any retry) would defeat the purpose.

Thanks a lot :slight_smile:

Have you played around with the connection timeout settings? You could set 0 for infinite as a test.

I’m not sure that’s a good idea, there are 2 main errors we encounter (depending on which SFTP we connect to):

SSH Connector - Execute failed: Session is being closed (SshException)
SSH Connector - Execute failed: Incorrect identification (null characters not allowed) -  at line 1 character #1 after '' (StreamCorruptedException)

In both cases, it looks like the SSH Connector node is sending corrupted login data or the handshake fails, either way the only solution is to reset the node and execute again. It’s probably a Java bug here.

A retry policy option (as there is on the REST nodes) would be the ideal solution but if that even happens someday, that would be a bit too long term.

I don’t think you can add a file system connection port to a loop end, so I can’t think of a way to do this with “proper” loop structure. That being said, you may be able to pull it off by attaching a variable loop end to outgoing flow variable port on the component output node within the component. It is not ideal. Even though only the last iteration of data is actually passed back into the workflow, the improper loop structure causes the prior loops data to build up unreleased java heap space. You can also run a garbage collector to clean that back up.

That is the only workaround that I can personally think of to possibly get a file system connection port onto a loop output… Since it is an improper loop structure, it will be highly version dependent as to whether it is executable or continues to cycle the process endlessly.

I would recommend just widening your loop in the main workflow instead of trying to make it a standalone component. That way you can just end your loop after the file system connection port has been utilized and that port data wouldn’t need to be passed further downstream.

1 Like

Yeah I was getting the feeling that this would be the only way to do this properly. I was reluctant to go that way because implementing it would be a lot more work.

Thanks for your help :slight_smile:

1 Like