I have Knime desktop installed only but moving to server soon. I’m new to Knime and loving it but have a problem. I wish to carry out the retrieval of data from a web URL which contains the query question mark within the address. For instance:
It seems I am unable to achieve this with the file reader or excel reader as the question mark seems to throw an exception. I’ve searched for other solutions but they always seem to include either extensions that I cannot download due to business restrictions or deprecated nodes. Is there a way to pass this kind of URL to a node to gather the CSV using currently active nodes ?
Hi @d_skweez , what configuration are you specifying on the CSV Reader.I’m able to use a CSV Reader to read the following location:
https://people.sc.fsu.edu/~jburkardt/data/csv/cities.csv?x=1
(the ?x=1 is added by me as it wasn’t part of the actual URL, but the reader doesn’t complain)
Actually, given that you say it’s complaining about the “?”, I’m going to make a guess that you have two things wrong.
(1) You are probably specifying Read from as “Local File System” (it needs to be Custom/KNIME URL), and
(2) You haven’t added the “http://” at the start of your URL. A valid URL has to have a valid protocol identifier (e.g. http://, file:// etc), otherwise it doesn’t know how to make the request.
e.g.
If this still did not fix the issue, can you show what you are using (File Reader? CSV Reader? Web Request (GET, POST…)?) and its configuration? Or even share your workflow? Please also share the error message too.
@bruno29a and @takbb. You are right, I am able to use the URL with (?) within the address with CSV reader and in my confusion this probably went un-noticed. However, whilst I set the timeout to a considerable time it doesn’t seem to want to connect/download. Also, the file is in fact a .xlsx file (apologies). Here is the link that works within a browser, and should continue to work even as the file name changes each month given this link is months old already perhaps.
Hi @d_skweez , a CSV file is basically a text file, and is read as a text file, while xlsx files are Excel files and cannot be read as a text file. So you cannot open xlsx files by the CSV Reader. You have to use the Excel Reader instead.
I took to downloading Knime to my personal computer to test it there and it worked as you would expect; without a problem. Therefore it’s likely that the security at the company is preventing access so I’ll need to take up the issue there…