Hi fellow KNIME users,
I have been trying to execute the Palladian Google Address Geocoder in my workflow.
The node was working fine till very recently. However, it has suddenly started generating errors now as shown below - Execute failed: ws.palladian.extraction.location.geocoder.GeocoderException: Encountered HTTP exception for "https://maps.googleapis.com/maps/api/geocode/json?address=50500%2CGOJRA%2CPakistan&sensor=false&key=<my google geocoding api key>".
The error message also captures different row items every time I run the loop and I can see different row items being captured in the error message under address field.
The workflow is available at the below mentioned link -
Hi Philipp @qqilihq ,
I could not find the Debug Log.
I checked my Google Cloud API Usage. i have set a limit of 3000 requests per minute but, i am well within the quota. I have only sent 1359 requests for the full day.
Could you guide me to where the DEBUG log is available and I can check it out?
I am an amateur KNIME user and hence, not aware of the complete details.
Hi Philipp @qqilihq ,
Please find the extract from the DEBUG console.
2023-06-26 20:05:44,104 : ERROR : KNIME-Worker-30-Google Address Geocoder 3:10 : : Node : Google Address Geocoder : 3:10 : Execute failed: ws.palladian.extraction.location.geocoder.GeocoderException: Encountered HTTP exception for "https://maps.googleapis.com/maps/api/geocode/json?address=94119%2CPALU%2CIndonesia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk".
java.lang.IllegalStateException: ws.palladian.extraction.location.geocoder.GeocoderException: Encountered HTTP exception for "https://maps.googleapis.com/maps/api/geocode/json?address=94119%2CPALU%2CIndonesia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk".
at ws.palladian.nodes.extraction.location.CachedGeocoder.lambda$0(CachedGeocoder.java:53)
at java.base/java.util.HashMap.computeIfAbsent(Unknown Source)
at java.base/java.util.Collections$SynchronizedMap.computeIfAbsent(Unknown Source)
at ws.palladian.nodes.extraction.location.CachedGeocoder.geoCode(CachedGeocoder.java:48)
at ws.palladian.nodes.extraction.location.GeocoderNodeModel$2.getCell(GeocoderNodeModel.java:79)
at org.knime.core.data.container.SingleCellFactory.getCells(SingleCellFactory.java:105)
at org.knime.core.data.container.RearrangeColumnsTable.calcNewCellsForRow(RearrangeColumnsTable.java:575)
at org.knime.core.data.container.RearrangeColumnsTable$ConcurrentNewColCalculator.compute(RearrangeColumnsTable.java:794)
at org.knime.core.data.container.RearrangeColumnsTable$ConcurrentNewColCalculator.compute(RearrangeColumnsTable.java:1)
at org.knime.core.util.MultiThreadWorker$ComputationTask$1.call(MultiThreadWorker.java:442)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:367)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:221)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
Caused by: ws.palladian.extraction.location.geocoder.GeocoderException: Encountered HTTP exception for "https://maps.googleapis.com/maps/api/geocode/json?address=94119%2CPALU%2CIndonesia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk".
at ws.palladian.extraction.location.geocoder.GoogleGeocoder.geoCode(GoogleGeocoder.java:69)
at ws.palladian.nodes.extraction.location.CachedGeocoder.lambda$0(CachedGeocoder.java:51)
... 16 more
Caused by: ws.palladian.retrieval.HttpException: Exception org.apache.http.NoHttpResponseException: maps.googleapis.com:443 failed to respond for URL "https://maps.googleapis.com/maps/api/geocode/json?address=94119%2CPALU%2CIndonesia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk": maps.googleapis.com:443 failed to respond
at ws.palladian.retrieval.HttpRetriever.execute(HttpRetriever.java:494)
at ws.palladian.retrieval.HttpRetriever.execute(HttpRetriever.java:302)
at ws.palladian.retrieval.HttpRetriever.httpGet(HttpRetriever.java:208)
at ws.palladian.extraction.location.geocoder.GoogleGeocoder.geoCode(GoogleGeocoder.java:67)
... 17 more
Caused by: org.apache.http.NoHttpResponseException: maps.googleapis.com:443 failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:294)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:257)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:207)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:679)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:481)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:72)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:164)
at org.apache.http.impl.client.DecompressingHttpClient.execute(DecompressingHttpClient.java:143)
at ws.palladian.retrieval.HttpRetriever.execute(HttpRetriever.java:432)
... 20 more
Hi Philipp @qqilihq ,
I changed the chunk loop counter to 1 and tested the workflow. I am sharing the messages from the DEBUG console -
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 doBeforePreExecution
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 has new state: PREEXECUTE
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 doBeforeExecution
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 has new state: EXECUTING
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 Start execute
DEBUG Google Address Geocoder 3:10 Using Table Backend "ColumnarTableBackend".
DEBUG Google Address Geocoder 3:10 Can't run ColumnRearranger on the Columnar Table Backend. Falling back on the old backend.
DEBUG Google Address Geocoder 3:10 Using Table Backend "ColumnarTableBackend".
DEBUG Google Address Geocoder 3:10 Request URL = https://maps.googleapis.com/maps/api/geocode/json?address=331001%2CCHURU%2CIndia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk
DEBUG Google Address Geocoder 3:10 Using proxy: host=proxy-ap.shell.com:8080
DEBUG Google Address Geocoder 3:10 Connection request: [route: {tls}->http://proxy-ap.shell.com:8080->https://maps.googleapis.com:443][total kept alive: 1; route allocated: 1 of 10; total allocated: 1 of 100]
DEBUG Google Address Geocoder 3:10 Connection leased: [id: 5][route: {tls}->http://proxy-ap.shell.com:8080->https://maps.googleapis.com:443][total kept alive: 0; route allocated: 1 of 10; total allocated: 1 of 100]
DEBUG Google Address Geocoder 3:10 Stale connection check
DEBUG Google Address Geocoder 3:10 CookieSpec selected: compatibility
DEBUG Google Address Geocoder 3:10 Auth cache not set in the context
DEBUG Google Address Geocoder 3:10 Target auth state: UNCHALLENGED
DEBUG Google Address Geocoder 3:10 Attempt 1 to execute request
DEBUG Google Address Geocoder 3:10 Sending request: GET /maps/api/geocode/json?address=331001%2CCHURU%2CIndia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk HTTP/1.1
DEBUG Google Address Geocoder 3:10 >> "GET /maps/api/geocode/json?address=331001%2CCHURU%2CIndia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk HTTP/1.1[\r][\n]"
DEBUG Google Address Geocoder 3:10 >> "Accept-Encoding: gzip,deflate[\r][\n]"
DEBUG Google Address Geocoder 3:10 >> "Content-Length: 0[\r][\n]"
DEBUG Google Address Geocoder 3:10 >> "Host: maps.googleapis.com[\r][\n]"
DEBUG Google Address Geocoder 3:10 >> "Connection: Keep-Alive[\r][\n]"
DEBUG Google Address Geocoder 3:10 >> "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36[\r][\n]"
DEBUG Google Address Geocoder 3:10 >> "[\r][\n]"
DEBUG Google Address Geocoder 3:10 >> GET /maps/api/geocode/json?address=331001%2CCHURU%2CIndia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk HTTP/1.1
DEBUG Google Address Geocoder 3:10 >> Accept-Encoding: gzip,deflate
DEBUG Google Address Geocoder 3:10 >> Content-Length: 0
DEBUG Google Address Geocoder 3:10 >> Host: maps.googleapis.com
DEBUG Google Address Geocoder 3:10 >> Connection: Keep-Alive
DEBUG Google Address Geocoder 3:10 >> User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36
DEBUG Google Address Geocoder 3:10 Connection 0.0.0.0:58101<->100.64.1.1:8080 closed
DEBUG Google Address Geocoder 3:10 Closing the connection.
DEBUG Google Address Geocoder 3:10 Connection 0.0.0.0:58101<->100.64.1.1:8080 closed
DEBUG Google Address Geocoder 3:10 Connection 0.0.0.0:58101<->100.64.1.1:8080 shut down
DEBUG Google Address Geocoder 3:10 Connection 0.0.0.0:58101<->100.64.1.1:8080 closed
DEBUG Google Address Geocoder 3:10 Connection released: [id: 5][route: {tls}->http://proxy-ap.shell.com:8080->https://maps.googleapis.com:443][total kept alive: 0; route allocated: 0 of 10; total allocated: 0 of 100]
DEBUG Google Address Geocoder 3:10 Cache stats: requests=1, hits=0
DEBUG Google Address Geocoder 3:10 reset
ERROR Google Address Geocoder 3:10 Execute failed: ws.palladian.extraction.location.geocoder.GeocoderException: Encountered HTTP exception for "https://maps.googleapis.com/maps/api/geocode/json?address=331001%2CCHURU%2CIndia&sensor=false&key=AIzaSyBgLmqzATNurBDIP4jXruKdpLlhKgx3lWk".
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 doBeforePostExecution
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 has new state: POSTEXECUTE
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 doAfterExecute - failure
DEBUG Google Address Geocoder 3:10 Loop End 3:15 has new state: CONFIGURED
DEBUG Google Address Geocoder 3:10 Wait... 3:16 has new state: CONFIGURED
DEBUG Google Address Geocoder 3:10 reset
DEBUG Google Address Geocoder 3:10 clean output ports.
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 has new state: IDLE
DEBUG Google Address Geocoder 3:10 Configure succeeded. (Google Address Geocoder)
DEBUG Google Address Geocoder 3:10 Google Address Geocoder 3:10 has new state: CONFIGURED
The log messages suggest a network issue - cold it be that your connection is somewhat unstable or flaky? As a workaround you could wrap the geocoder node into try/catch nodes to handle errors and resume workflow execution, and even combine it with some looping to retry failing requests.
Hi Philipp @qqilihq ,
Many thanks for the feedback. I have utilized the Try/Catch nodes along with the generic-variable loop condition to create a wrapper around the geocoder node and found it to be working fine.
