KNIME 4.7.4 UI Freezes in CSV reader config window

jmbenedetto · June 23, 2023, 11:23pm

Hi! I’m new to Knime and this community. First, thanks a lot for the time you spend helping newbies as myself. Now I’m hoping you can shad some light on a huge productivity issue I’m having.

I have a MacBook Pro M2 Pro with 16Gb of RAM, and I’m running Knime Analytics Platform 4.7.4.

One of my workflows reads csv files stored on a S3 bucket.

When I try to change something on the transformation tab (1 - unselect a column, 2 - change the format from integer to string) the UI Freezes. I can move the configuration window around, but nothing more. Each change I do freezes the UI for something between many seconds and some minutes. If I close the configuration window (saving or canceling) the same happens.

I put the console into debug mode, and this is the result during the freeze:

DEBUG CSV Reader 3:4 A06_kpis_nivel_serviço_v3 3 has new state: CONFIGURED
DEBUG KNIMEApplication$4 Potential deadlock in AWT Event Queue detected. Full thread dump will follow as debug output.
DEBUG KNIMEApplication$4 inkedBlockingQueue.take(Unknown Source)
at java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
at java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base@17.0.5/java.lang.Thread.run(Unknown Source)

“idle-connection-reaper” Id=768 TIMED_WAITING
at java.base@17.0.5/java.lang.Thread.sleep(Native Method)
at software.amazon.awssdk.http.apache.internal.conn.IdleConnectionReaper$ReaperTask.run(IdleConnectionReaper.java:150)
at java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base@17.0.5/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base@17.0.5/java.lang.Thread.run(Unknown Source)

Number of locked synchronizers = 1

java.util.concurrent.ThreadPoolExecutor$Worker@7c08d741

“Worker-23” Id=867 TIMED_WAITING on org.eclipse.core.internal.jobs.WorkerPool@e31e299
at java.base@17.0.5/java.lang.Object.wait(Native Method)

waiting on org.eclipse.core.internal.jobs.WorkerPool@e31e299
at org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:200)
at org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:242)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

“Worker-24” Id=898 TIMED_WAITING on org.eclipse.core.internal.jobs.WorkerPool@e31e299
at java.base@17.0.5/java.lang.Object.wait(Native Method)

waiting on org.eclipse.core.internal.jobs.WorkerPool@e31e299
at org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:200)
at org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:242)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

“KNIME-Worker-11” Id=902 TIMED_WAITING on java.lang.Object@622666b1
at java.base@17.0.5/java.lang.Object.wait(Native Method)

waiting on java.lang.Object@622666b1
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:232)

“Worker-25” Id=918 TIMED_WAITING on org.eclipse.core.internal.jobs.WorkerPool@e31e299
at java.base@17.0.5/java.lang.Object.wait(Native Method)

waiting on org.eclipse.core.internal.jobs.WorkerPool@e31e299
at org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:200)
at org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:242)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

“AWT-Shutdown” Id=927 WAITING on java.lang.Object@3720cbe8
at java.base@17.0.5/java.lang.Object.wait(Native Method)

waiting on java.lang.Object@3720cbe8
at java.base@17.0.5/java.lang.Object.wait(Unknown Source)
at java.desktop@17.0.5/sun.awt.AWTAutoShutdown.run(Unknown Source)
at java.base@17.0.5/java.lang.Thread.run(Unknown Source)

“AWT-EventQueue-0” Id=926 RUNNABLE
at java.base@17.0.5/sun.nio.ch.Net.poll(Native Method)
at java.base@17.0.5/sun.nio.ch.NioSocketImpl.park(Unknown Source)
at java.base@17.0.5/sun.nio.ch.NioSocketImpl.timedRead(Unknown Source)
at java.base@17.0.5/sun.nio.ch.NioSocketImpl.implRead(Unknown Source)
at java.base@17.0.5/sun.nio.ch.NioSocketImpl.read(Unknown Source)
at java.base@17.0.5/sun.nio.ch.NioSocketImpl$1.read(Unknown Source)
at java.base@17.0.5/java.net.Socket$SocketInputStream.read(Unknown Source)
at java.base@17.0.5/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source)
at java.base@17.0.5/sun.security.ssl.SSLSocketInputRecord.readFully(Unknown Source)
at java.base@17.0.5/sun.security.ssl.SSLSocketInputRecord.decodeInputRecord(Unknown Source)
at java.base@17.0.5/sun.security.ssl.SSLSocketInputRecord.decode(Unknown Source)
at java.base@17.0.5/sun.security.ssl.SSLTransport.decode(Unknown Source)
at java.base@17.0.5/sun.security.ssl.SSLSocketImpl.decode(Unknown Source)
at java.base@17.0.5/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source)
at java.base@17.0.5/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.read(SessionInputBufferImpl.java:197)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:176)
at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:198)
at org.apache.http.impl.io.ContentLengthInputStream.close(ContentLengthInputStream.java:101)
at org.apache.http.impl.execchain.ResponseEntityProxy.streamClosed(ResponseEntityProxy.java:142)
at org.apache.http.conn.EofSensorInputStream.checkClose(EofSensorInputStream.java:228)
at org.apache.http.conn.EofSensorInputStream.close(EofSensorInputStream.java:172)
at java.base@17.0.5/java.io.FilterInputStream.close(Unknown Source)
at software.amazon.awssdk.services.s3.checksums.ChecksumValidatingInputStream.close(ChecksumValidatingInputStream.java:163)
at java.base@17.0.5/java.io.FilterInputStream.close(Unknown Source)
at software.amazon.awssdk.core.io.SdkFilterInputStream.close(SdkFilterInputStream.java:83)
at org.knime.filehandling.core.connections.FSInputStream.close(FSInputStream.java:107)
at java.base@17.0.5/java.io.FilterInputStream.close(Unknown Source)
at org.knime.filehandling.core.util.CompressionAwareCountingInputStream.close(CompressionAwareCountingInputStream.java:128)
at org.apache.commons.io.IOUtils.close(IOUtils.java:339)
at org.apache.commons.io.input.ProxyInputStream.close(ProxyInputStream.java:148)
at java.base@17.0.5/sun.nio.cs.StreamDecoder.implClose(Unknown Source)
at java.base@17.0.5/sun.nio.cs.StreamDecoder.close(Unknown Source)

locked java.io.InputStreamReader@69a6dd4a
at java.base@17.0.5/java.io.InputStreamReader.close(Unknown Source)
at org.knime.base.node.io.filehandling.csv.reader.OSIndependentNewLineReader.close(OSIndependentNewLineReader.java:119)
at java.base@17.0.5/java.io.BufferedReader.close(Unknown Source)

locked org.knime.base.node.io.filehandling.csv.reader.OSIndependentNewLineReader@418f49d0
at com.univocity.parsers.common.input.concurrent.ConcurrentCharLoader.stopReading(ConcurrentCharLoader.java:178)
at com.univocity.parsers.common.input.concurrent.ConcurrentCharInputReader.stop(ConcurrentCharInputReader.java:85)
at com.univocity.parsers.common.AbstractParser.stopParsing(AbstractParser.java:446)
at org.knime.base.node.io.filehandling.csv.reader.api.CSVTableReader$CsvRead.close(CSVTableReader.java:282)
at org.knime.filehandling.core.node.table.reader.read.AbstractReadDecorator.close(AbstractReadDecorator.java:102)
at org.knime.filehandling.core.node.table.reader.read.AbstractReadDecorator.close(AbstractReadDecorator.java:102)
at org.knime.filehandling.core.node.table.reader.IndividualTablePreviewRowIterator.close(IndividualTablePreviewRowIterator.java:126)
at org.knime.filehandling.core.node.table.reader.MultiTablePreviewRowIterator.close(MultiTablePreviewRowIterator.java:132)
at org.knime.filehandling.core.node.table.reader.preview.dialog.ObservablePreviewIterator.close(ObservablePreviewIterator.java:105)
at org.knime.core.node.tableview.TableContentModel.setDataTableIntern(TableContentModel.java:390)
at org.knime.core.node.tableview.TableContentModel.lambda$0(TableContentModel.java:342)
at org.knime.core.node.tableview.TableContentModel$$Lambda$2768/0x000000080171e000.run(Unknown Source)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:367)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:221)
at org.knime.core.node.util.ViewUtils.invokeAndWaitInEDT(ViewUtils.java:136)
at org.knime.core.node.tableview.TableContentModel.setDataTable(TableContentModel.java:342)
at org.knime.filehandling.core.node.table.reader.preview.dialog.TableReaderPreviewModel.lambda$1(TableReaderPreviewModel.java:105)
at org.knime.filehandling.core.node.table.reader.preview.dialog.TableReaderPreviewModel$$Lambda$2767/0x0000000801718f68.run(Unknown Source)
at org.knime.core.node.util.ViewUtils$3.run(ViewUtils.java:353)
at java.desktop@17.0.5/java.awt.event.InvocationEvent.dispatch(Unknown Source)
at java.desktop@17.0.5/java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.desktop@17.0.5/java.awt.EventQueue$4.run(Unknown Source)
at java.desktop@17.0.5/java.awt.EventQueue$4.run(Unknown Source)
at java.base@17.0.5/java.security.AccessController.executePrivileged(Unknown Source)
at java.base@17.0.5/java.security.AccessController.doPrivileged(Unknown Source)
at java.base@17.0.5/java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.desktop@17.0.5/java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.desktop@17.0.5/java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.desktop@17.0.5/java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.desktop@17.0.5/java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.desktop@17.0.5/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.desktop@17.0.5/java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.desktop@17.0.5/java.awt.EventDispatchThread.run(Unknown Source)

Number of locked synchronizers = 2

java.util.concurrent.locks.ReentrantLock$NonfairSync@74019abf

java.util.concurrent.locks.ReentrantLock$NonfairSync@25c37960

“unVocity-parsers input reading thread” Id=929 TIMED_WAITING on com.univocity.parsers.common.input.concurrent.ConcurrentCharLoader$1@7ad0bc1f
at java.base@17.0.5/java.lang.Object.wait(Native Method)

waiting on com.univocity.parsers.common.input.concurrent.ConcurrentCharLoader$1@7ad0bc1f
at com.univocity.parsers.common.input.concurrent.FixedInstancePool.allocate(FixedInstancePool.java:71)
at com.univocity.parsers.common.input.concurrent.ConcurrentCharLoader.readBucket(ConcurrentCharLoader.java:73)
at com.univocity.parsers.common.input.concurrent.ConcurrentCharLoader.run(ConcurrentCharLoader.java:91)
at java.base@17.0.5/java.lang.Thread.run(Unknown Source)

The error log shows a “Unhandled event loop exception”. Please see log attached.
Unhandled event loop exception.log (375.7 KB)
"

denisfi · June 25, 2023, 2:27pm

Hi @jmbenedetto ,

MAYBE the problem is the concurrent process to load data from a server outside your network. If it was at your notebook, you won’t have this situation.

Can you try try serialise the process, split it at process to load only the necessary data to make a job and when finished, start another one? You can use the workflow call to it and you can control where the neck can appears more then other and make some corrections at there.

I’m not sure if I made me clear, but I think that split and conquer can help you with some benefits with a lot of thinks;

Tks,

Denis

jmbenedetto · June 27, 2023, 4:01am

Hello, Denis!

Thanks a lot for taking the time to suggest a solution.

I tried to apply your recommendation by simplifying the load. I deleted every csd reader node except for the 1st one. The result is the same.

I observed another thing. I have a numeric field starting with “0”. Knime identifies it as integer. In the transformation tab, if I change it to string, Knime UI will freeze. When it unfreezes, I’ll see in the field with leading zeros in the vizualization pane. It is as if Knime is frozing because it is trying to apply the transformation to the file, before I presse the run.

Does it makes sense to any of you?

Thanks again for your time and knowledge.

Benedetto

denisfi · June 27, 2023, 10:07am

Hi @jmbenedetto,

It cam make sense IF you need to transformt it before any operation.

For the csv reader, you have a tab for transformation. There is a option to set the kind of column you have there that was automatically identified by kmine, but you can force it to use a unic option as default.

IF your files have a column as string (number started with 0) and you don’t need to convert it into integer or other type, set to be a string one. This step will give you less process to load data.

Try it and tell us more about the results ok?

BR,

Denis

jmbenedetto · July 9, 2023, 2:13pm

Hello, Denis.

In fact, this is exactly my use case. I have many numeric fields that I want to keep as string, and do not let knime convert it to integer. However, every time I change the type of 1 field to string on the transformation pane, Knime freezes.

Since my first post, I gathered a little bit of information:
1- just after I change the type, the screen freezes until the moment when the preview panel updates, showing the field and a string and no more as an integer (I can tell because there are many values with leading zeros).
2- the freezes only occurs with AWS S3 connector. GCP GCS works fine.
3 - My workaround is to download a copy of the files, do all the parametrization pointing to the local copy and then just changing the the system connector from local to AWS. As everything else remains unchanged (including file names and structure), it works fine.

Thanks a lot for your help.

system · October 7, 2023, 2:13pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.