Hi all.
I want to ping a download URL in the start of my workflow to check If the file exists. At the moment I am using a Get Request, but the file I’m downloading it quite big, so it takes quite long to execute the Get Request due to it returning a “body”. Is there any way I can set this Get Request to return only the Status (eg 200)?
Alternatively is there any other node I can use to send a ping?
Hi,
you could do a HEAD request, which would (if supported by the server) only transmit the headers and not the body. This is supported e.g. by the Palladian HTTP Retriever:
And if the server does not support HEAD, you can make use of the Maximum file size option and limit this to e.g. 0 so that no body payload is actually transmitted. Then just check for the response status (e.g. 200 vs 404).
Here’s an older thread which gives some more info:
–Philipp
Hi @tiaandp , the solution proposed by @qqilihq is excellent. You can check the http status via a HEAD request or check for the file size.
However, is there any reason why you want to do this? If you eventually will download the file via Knime, then might as well go for the download. If you just want to check if the file “exists”, and will not download it, then it makes sense to do what you are trying to do.
I quoted “exists”, because some files do not physically exist and are dynamically generated on the fly by the server.
Hi @bruno29a the reason for using this is that I read the files later in the flow using an excel reader. If the URL is faulty the file reader node doesn’t fail but gives an configuration warning, which does not allow my try catch error node to work.
Hi @tiaandp , ok, I see your point, so the main reason behind it is that you can’t catch a faulty url as it does not come back as an error.
As an alternative, could you not check for a RowCount after attempting to open/download the file? If RowCount is 0, or even if the file loaded/downloaded does not reflect expected structure, then it means you can’t process the file and implement an error handling based on these conditions.
Hello @tiaandp,
think you should be able to make use of Files/Folders Meta Info or String to Path node to check if a file exists. Take a look here:
Br,
Ivan
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.