I have a problem where I will run Send Email nodes inside of a loop and eventually, the Send Email node will have an error with a timeout.
My goal is to “resume” the workflow upon error here. In order to do this manually, I rerun the Send Email node after its error, right click on the Loop End node, click “step execution”, right click on the Loop End node again, and then resume loop execution.
It is vitally important that I do not start the loop over from the very beginning.
Is this possible in KNIME Server? Any ideas for implementation?
Do you need to send an email in each iteration? if not you could create the email data e.g. subject line, body etc within the loop and then after the loop send all the emails at once.
When I send the multiple emails all at once, it causes a timeout issue.
So, I use a “Wait…” node in conjunction with the Send Email node in a loop, and that works 99% of the time. The problem is when I am sending hundreds of emails on a workflow posted to KNIME Server, I need it to work 100% of the time, without fail.
We actually prefer to send multiple individual emails over one big email, due to the nature of the business. Additionally, the recipient list is highly variable (dependent on the stores selected in the data), so we cannot create a static list for all email recipients.
Just thinking out loud - what about some sort of design with the Try/Catch nodes where you store the addresses of the emails that didn’t get successfully sent? Then you recurse and try again, using only those emails.
I thought about this too – the tricky aspect is how to trigger the loop to “try again”. Because I don’t think we can feed the “catch” portion of the model back to the original looping node?
However, for mass mailing, I would not handle the action of the actual sending the email within Knime, but rather at the SMTP server instead, an internal SMTP server that is.
You can “send” your emails via Knime, but that should simply send them to the internal SMTP server. The only “error” you should expect here is the SMTP server rejecting an invalid email as in the format of the email address is an invalid format. There should not be any timeout with this approach.
The SMTP server should then be connecting to your provider and manage/handle retries of failed messages, etc
EDIT: The timeout you are getting is because in your case, Knime is probably waiting for the actually send. It’s waiting for a response, for an acknowledgement from the recipient’s server before sending the next email. It’s literally sending one at a time also.
Having all emails sent at once is better, because normally the smtp server would send to the same domain (recipient’s email domain) at once instead of switching domains.
Also, you really need to configure your SMTP properly in order not to be considered as a spammer and get banned from the recipients’ email domain. Spammers do not take the time to configure their server usually, that is one way how email servers flag them. You also need to warm up your IP - do not do huge mass mailing on a new IP. Spammers switch IP all the time, because their IPs get banned, meaning they usually get new IPs and send mass emails right away. You need to start sending small batches at the beginning and then increase the batch size. By that, I mean you may start by sending 5000 emails first, wait for a day, send another 5000, wait for a day, send 7000, etc under the same IP.
I forgot to mention, but each email I am sending out is unique – both in the Excel attachment with store data, and in the text in the body of the email. I am using flow variables to custom-tailor each email.
FYI - I realize that I had not considered the most basic solution here – increase the timeout settings for both the connection to SMTP and for the data read in the Send Email node. I will mark this as the tentative solution for now, and if I continue to have issues, I will revisit this thread. Thank you all for your contributions!
