Hi Team,
We are facing Frequent (almost in every jobs) time out problem while executing KNIME workflow from Knime Web Portal and Knime Data Analytics tool into Server (AWS).
We are not facing any such error while executing the same KNIME job into local machine.
Observations:
i. Only able to load small files (<10mb) into database and i.e. too from KNIME webportal only. We are not able to execute any job into server from “Knime Data Analytics tool”.
ii. While executing more than one CSV reader in parallel flow of one workflow - we are facing time out.
iii. We have observed earlier that, even with r5.large instance, we could be able to load more than 600mb file into table for less than 10mins. But now we are using r5.2 large which is recommended by KNIME too, still not able to load the same file.
iv. We checked utilization (CPU, Memory) of server - i.e. very low.
v. Also as we are executing the jobs on AWS server (KNIME, RDS, s3) - so I believe there should not be any N/W impact.
I would like to request you kindly suggest - how can we tune up the KNIME server?
Also find below details for reference,
Source - csv files in S3 (apac mumbai)
Target - MYSQL
EC2 instance - r5.2 large (apac mumbai). This instance is KNIME recommended too.
RDS - MYSQL (apac mumbai)
KNIME Tier - KNIME small on AWS.
Knime Server - 4.10.2
Knime Executor - 4.1.2
Error Description -
CSV Reader 0:81 - ERROR: Execute failed: Can’t access ‘https://primsinputdata.s3.ap-south-1.amazonaws.com/Sandbox/bsak.csv?X-Amz-Algorithm=xxx-xxx-xxx&X-Amz-Date=20200501T064435Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=xxxx%2F20200501%2Fap-south-1%2Fs3%2Faws4_request&X-Amz-Signature=xxxxx’. (Connection timed out (Connection timed out))
DB Loader 0:359 - WARNING: Database connection does not exist. Please reset the corresponding connector node.
Thanks and Regards,
Arka Prava CHandra