Hello everyone,
I have Knime Analytics installed on Windows 10 Pro with Intel(R) Core™ i7-4790K CPU @ 4.00GHz 4.00 GHz and 12GB of RAM and C drive disk space of 237GB (primary drive with Windows installation) and storage drive of 931GB. I have KNIME and other software installed on the non-primary D drive to save diskspace and all my knime-workspace files/folders are in D drive. However, recently as I add more and process more in KNIME, I have been running into low disk issue in the C drive.
It turned out that C:\Users\myusername\AppData\Local\Temp\knime_myprojectname48394 has huge 94GB knime_container_YYYYMMDD_somenumbers.knable file along with many other knime_container files from other projects.
Is there a configuration in Knime Analytics software to change the target location (to D drive) for these knime_container files? If not, is it safe to delete these files at the end of workflow execution or after the workflow had completed with separate workflow? This is preventing me from executing workflows and basically I am not able to use this computer since it runs out of diskspace in C drive completely.
Thank you so much for your time and help. Have a nice day.
hhmm, I updated this settings and reinstalled Knime to put it in D drive… Maybe reinstall it again and see if it actually puts it in D drive, and hoping that \AppData\Local\Temp\ points to D drive or is that always in C drive? I’ve been searching Knime Forum but not much on this topic.
Hello @ScottF,
Thank you sooooo much for the suggestion. Let me update that setting and see. I believe that should address the issue. I’ll report back.
Thank you again, Sir!!!
When I run with a larger data set, it crashes, seems like. Is this related to knime_container file or is the workflow too long or needs to be more efficient? I have not been able to run the job to completion.
Thank you!!!
Log Name: Application
Source: Application Hang
Date: 8/31/2021 9:52:41 AM
Event ID: 1002
Task Category: (101)
Level: Error
Keywords: Classic
User: N/A
Computer: talon116913
Description:
The program knime.exe version 0.0.0.0 stopped interacting with Windows and was closed. To see if more information about the problem is available, check the problem history in the Security and Maintenance control panel.
Process ID: 571c
Start Time: 01d79dc311829df0
Termination Time: 42
Application Path: C:\Program Files\KNIME\knime.exe
Report Id: b3211d62-e5e2-4ace-938a-60af5399e681
Faulting package full name:
Faulting package-relative application ID:
Hang type: Unknown
I normally use Knime Analytics with .bat file via Windows Scheduler to trigger the workflow, but the workflow has been crashing both ways, with KA software open and also via .bat file command line.
Caused by: java.net.SocketTimeoutException: SocketTimeoutException invoking https://www.googleapis.com/pagespeedonline/v5/runPagespeed?url=https%3A%2F%2Fwww.xxx.com%2Fhomes%2Fcalifornia%2Flos-angeles%2Fwest-hills%2Fsterling-at-west-hills-209560&category=ACCESSIBILITY&category=BEST_PRACTICES&category=PERFORMANCE&category=PWA&category=SEO&strategy=MOBILE&key=i removed this: Read timed out
at sun.reflect.GeneratedConstructorAccessor311.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1402)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1386)
at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77)
at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:673)
at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
… 22 more
Caused by: java.net.SocketTimeoutException: Read timed out
at sun.reflect.GeneratedConstructorAccessor311.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1950)
at sun.net.www.protocol.http.HttpURLConnection$10.run(HttpURLConnection.java:1945)
at java.security.AccessController.doPrivileged(Native Method)
at sun.net.www.protocol.http.HttpURLConnection.getChainedException(HttpURLConnection.java:1944)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1514)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:352)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:377)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream$2.run(URLConnectionHTTPConduit.java:373)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.getResponseCode(URLConnectionHTTPConduit.java:373)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.processRetransmit(HTTPConduit.java:1450)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleRetransmits(HTTPConduit.java:1437)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1567)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1373)
… 27 more
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:990)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:948)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at sun.net.www.protocol.http.HttpURLConnection.getHeaderFields(HttpURLConnection.java:3084)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getHeaderFields(HttpsURLConnectionImpl.java:297)
at org.apache.cxf.transport.http.Headers.readFromConnection(Headers.java:281)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.updateCookiesBeforeRetransmit(URLConnectionHTTPConduit.java:337)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleRetransmits(HTTPConduit.java:1435)
… 29 more
2021-08-31 17:05:49,118 : WARN : KNIME-Worker-122-GET Request 0:3938:0:29 : : Node : String to JSON : 0:3938:0:0:0:96 : The selected column (body) is already a JSON column.
2021-08-31 17:05:49,118 : WARN : KNIME-Worker-122-GET Request 0:3938:0:29 : : Node : End IF : 0:3938:0:0:0:93 : The structures of both active input data tables are not compatible.
2021-08-31 17:08:23,035 : WARN : KNIME-Worker-51-Get Request Plus 0:3938 : : Node : Excel Writer : 0:3949 : Errors overwriting node settings with flow variables: Unknown variable “filePathMissedURLs_location”
2021-08-31 17:08:23,184 : WARN : KNIME-Worker-51-Get Request Plus 0:3938 : : Node : Create Folder : 0:4064 : Errors overwriting node settings with flow variables: Unknown variable “JSONfileDateFolder_location”
2021-08-31 17:08:23,209 : WARN : KNIME-Worker-169-Empty Table Switch 0:3944 : : Node : Excel Writer : 0:3949 : Errors overwriting node settings with flow variables: Unknown variable “filePathMissedURLs_location”
2021-08-31 17:08:23,309 : WARN : KNIME-Worker-170-Extract Table Dimension 0:4015 : : Node : Create Folder : 0:4064 : Errors overwriting node settings with flow variables: Unknown variable “JSONfileDateFolder_location”
2021-08-31 17:08:23,317 : ERROR : KNIME-Worker-172-Send Email 0:3954 : : Node : Send Email : 0:3954 : Execute failed: No such variable “FailingNode” of type STRING
org.knime.core.node.InvalidSettingsException: No such variable “FailingNode” of type STRING
at org.knime.base.node.util.sendmail.SendMailConfiguration.send(SendMailConfiguration.java:583)
at org.knime.base.node.util.sendmail.SendMailNodeModel.execute(SendMailNodeModel.java:95)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:576)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1245)
at org.knime.core.node.Node.execute(Node.java:1025)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:558)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:334)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
Caused by: java.util.NoSuchElementException: No such variable “FailingNode” of type STRING
at org.knime.core.node.workflow.FlowObjectStack.peekFlowVariable(FlowObjectStack.java:399)
at org.knime.core.node.NodeModel.peekFlowVariableString(NodeModel.java:1280)
at org.knime.base.util.flowvariable.FlowVariableResolver$FlowVariableEscaper.readString(FlowVariableResolver.java:237)
at org.knime.base.util.flowvariable.FlowVariableResolver.parse(FlowVariableResolver.java:115)
at org.knime.base.util.flowvariable.FlowVariableResolver.parse(FlowVariableResolver.java:73)
at org.knime.base.node.util.sendmail.SendMailConfiguration.send(SendMailConfiguration.java:581)
… 14 more
2021-08-31 17:08:23,350 : WARN : KNIME-Worker-180-Table Row to Variable 0:3939 : : Node : Excel Writer : 0:3949 : Errors overwriting node settings with flow variables: Unknown variable “filePathMissedURLs_location”
2021-08-31 17:08:23,468 : WARN : KNIME-Worker-181-Table Row to Variable 0:4024 : : Node : Create Folder : 0:4064 : Errors overwriting node settings with flow variables: Unknown variable “JSONfileDateFolder_location”
2021-08-31 17:08:23,640 : WARN : KNIME-Worker-169-Table Row to Variable 0:4025 : : Node : Create Folder : 0:4064 : Errors overwriting node settings with flow variables: Unknown variable “JSONfileDateFolder_location”
2021-08-31 17:08:23,774 : WARN : KNIME-Worker-181-Table Row to Variable 0:4068 : : Node : Create Folder : 0:4064 : Errors overwriting node settings with flow variables: Unknown variable “JSONfileDateFolder_location”
2021-08-31 17:08:23,782 : WARN : KNIME-Worker-165-Table Row to Variable 0:4063 : : Node : Create Folder : 0:4064 : Errors overwriting node settings with flow variables: Unknown variable “JSONfileDateFolder_location”
I’m going to “assume” I’m doing too much too fast simultaneously which caused that Application Hang error. I know the GET REQUEST portion completes because I have email notifications throughout and it’s most likely crashing in JSON PATH section ( I have 3 of them simultaneously) and that section takes several hours. I’m now testing more of a waterfall approach to execute each JSON PATH node one at a time. Meantime, if anyone could provide insights or guidance based on the error logs, I would greatly appreciate your time and support. I’ll do my best in attempting to identify and resolve the issue by trial and error.
Thank you so much and hope you all have a wonderful day.
