Timed out receiving message from renderer: -0.000

mwiegand · January 20, 2023, 7:22pm

I roughly recall we briefly exchanged about erratic render timeouts. I found now one URL where it seems to be replicatable. Interestingly, both Chrome (regular app and WebDirver on OSX) as well as Safari fail but not Firefox.

I was able to isolate it to the http call respectively it’s redirect to https.

There is also no response code which would indicate a WAF. I will share the URL, as customer related, with you via email.

Best
Mike

qqilihq · January 24, 2023, 12:55pm

Hi Mike,

thank you for sending the URL.

I have now tried many times to access it (using different browsers, different local and mobile networks, different devices, different days and time of the day), but for me, it will just just never reply and timeout. I do not even get an initial redirect from HTTP to HTTPS.

And this is why the “timed out” error is thrown (the message is not particularly user friendly, and I have no clue what that -0.000 is supposed to mean to be honest - I’ll investigate this).

If you’re able to load the URL at all - are you accessing this through a VPN, maybe?

Best,
Philipp

mwiegand · January 24, 2023, 1:08pm

Hey Phillip,

thanks for checking this. That issue is really odd. I can eliminate that a VPN causes an issue as I have disabled it.

Sometimes I also get other timeouts for navigation on a website (to a sub-site) which is loaded. the last example shared succeeded after a few retries via Try-Catch. Any idea how I could collect debug information?

Best
Mike

mwiegand · January 28, 2023, 11:24am

Hi Phillip,

I managed to replicate the issue even when executing only in a single thread. Here is some debug info:

2023-01-27 23:18:55,566 : ERROR : KNIME-Worker-186-Take Screenshot 3:988:0:1048:1059:0:898:0:813 :  : Node : Take Screenshot : 3:988:0:1048:1059:0:898:0:813 : Execute failed: timeout: Timed out receiving message from renderer: 10.000
ws.palladian.nodes.selenium.SeleniumNodeExecutionException: timeout: Timed out receiving message from renderer: 10.000
	at ws.palladian.nodes.selenium.AbstractSearchContextMethodNodeModel.execute(AbstractSearchContextMethodNodeModel.java:135)
	at org.knime.core.node.NodeModel.execute(NodeModel.java:758)
	at org.knime.core.node.NodeModel.executeModel(NodeModel.java:549)
	at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1267)
	at org.knime.core.node.Node.execute(Node.java:1041)
	at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:595)
	at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:98)
	at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
	at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
	at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:367)
	at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:221)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
	at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
Caused by: org.openqa.selenium.TimeoutException: timeout: Timed out receiving message from renderer: 10.000
  (Session info: chrome=109.0.5414.119)
  (Driver info: chromedriver=109.0.5414.74 (e7c5703604daa9cc128ccf5a5d3e993513758913-refs/branch-heads/5414@{#1172}),platform=Mac OS X 13.2.0 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 0 milliseconds
Build info: version: 'unknown', revision: 'unknown', time: 'unknown'
System info: host: 'Mikes-MBP.lan', ip: '2a01:c22:d5dd:ab00:8872:7465:277a:7e40%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '13.2', java.version: '17.0.5'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, acceptSslCerts: false, browserConnectionEnabled: false, browserName: chrome, chrome: {chromedriverVersion: 109.0.5414.74 (e7c5703604da..., userDataDir: /var/folders/jg/zcb9yqn90w5...}, cssSelectorsEnabled: true, databaseEnabled: false, goog:chromeOptions: {debuggerAddress: localhost:61574}, handlesAlerts: true, hasTouchScreen: false, javascriptEnabled: true, locationContextEnabled: true, mobileEmulationEnabled: false, nativeEvents: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, proxy: Proxy(), rotatable: false, setWindowRect: true, strictFileInteractability: false, takesHeapSnapshot: true, takesScreenshot: true, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unexpectedAlertBehaviour: ignore, unhandledPromptBehavior: ignore, version: 109.0.5414.119, webStorageEnabled: true, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: ba268ecf3d01d15ebfc3cc6e15da550a
	at jdk.internal.reflect.GeneratedConstructorAccessor355.newInstance(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Unknown Source)
	at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
	at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:214)
	at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:166)
	at org.openqa.selenium.remote.http.JsonHttpResponseCodec.reconstructValue(JsonHttpResponseCodec.java:40)
	at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:80)
	at org.openqa.selenium.remote.http.AbstractHttpResponseCodec.decode(AbstractHttpResponseCodec.java:44)
	at org.openqa.selenium.remote.HttpCommandExecutor.execute(HttpCommandExecutor.java:158)
	at org.openqa.selenium.remote.service.DriverCommandExecutor.execute(DriverCommandExecutor.java:83)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:552)
	at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:609)
	at org.openqa.selenium.remote.RemoteWebDriver.getScreenshotAs(RemoteWebDriver.java:295)
	at jdk.internal.reflect.GeneratedMethodAccessor192.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at ws.palladian.nodes.selenium.SeleniumUtils$SeleniumSyncInvocationHandler.invoke(SeleniumUtils.java:43)
	at ws.palladian.nodes.selenium.SeleniumUtils$1.invoke(SeleniumUtils.java:173)
	at jdk.proxy17/jdk.proxy17.$Proxy50.getScreenshotAs(Unknown Source)
	at ws.palladian.nodes.selenium.screenshot.TakeScreenshotNodeModel.execMethod(TakeScreenshotNodeModel.java:37)
	at ws.palladian.nodes.selenium.AbstractSearchContextMethodNodeModel.execute(AbstractSearchContextMethodNodeModel.java:133)
	... 14 more

I have two suspicions / ideas:

JavaScript errors interfering
DOM not ready or sub-resource not finished loading

The later I once suggested to extend the Selenium Wait node to listen for states like DOM Ready or Complete. Maybe even custom events “bubbling” in the console.

About JS-Errors, I tried to collect them via various methods but my skill level … well, I didn’t had success.

I did some crude experiments with alternative methods to capture a screenshot too but to no avail (meaning, I failed miserably):

Any thoughts on this?

Cheers
Mike

mwiegand · January 29, 2023, 8:30am

Various sources indicate an issue with a deprecated Chrome version and Driver. However, others report, even after updating, that the issue still persists. Though, that issue dates back to 2020 and earlier which indicated the issue is unfortunately still persistent.

https://bugs.chromium.org/p/chromedriver/issues/detail?id=3657

I tried to brute force my way through via a try-catch approach. The workflow ran through the night and the issue only has gotten worse.

I think I will have to try switching to Firefox but that will break the very nice feature of exporting Cookies via Chrome Dev Tools API to retrieve httponly cookies too. Hmm … may I skip screenshots entirely for the moment.

PS: Another Selenium-based option to fetch the console output which I tried was

gist.github.com

https://gist.github.com/customcommander/fefa2e1a921d2646f5257f86a4caa18a

browser_logs-chrome.js

require('chromedriver');

const path = require('path');
const wd = require('selenium-webdriver');
const chrome = require('selenium-webdriver/chrome');

var builder = new wd.Builder();
var options = new chrome.Options();
var prefs = new wd.logging.Preferences();
var driver;

This file has been truncated. show original

output.txt

[ Entry {
    level: Level { name_: 'INFO', value_: 800 },
    message: 'file:///private/tmp/sample/page.html 3:20 "Apple"',
    timestamp: 1495701249904,
    type: '' },
  Entry {
    level: Level { name_: 'INFO', value_: 800 },
    message: 'file:///private/tmp/sample/page.html 4:20 "Pear"',
    timestamp: 1495701249904,
    type: '' },

This file has been truncated. show original

page.html

<html>
    <body>
        <script>
            console.info('Apple');
            console.log('Pear');
            console.warn('Strawberry');
            console.error('Banana');
            throw new Error('Orange');
        </script>
    </body>

This file has been truncated. show original

qqilihq · February 3, 2023, 8:09pm

@mwiegand I think we can rule out the deprecated Driver - these are up-to-date matching current Chrome versions. Just to be sure, you could try running with the bundled “Chromium” browser (and not the system-installed Chrome).

With one of the (confidential) URLs you sent I was able to reproduce the scenario; I have sent additional details via email. Let me know your feedback!

Have a good weekend,
Philipp

system · May 4, 2023, 8:10pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.