Issues with table with 1.6M rows

Hi

I have a table with 1616399 rows. There should be 2 XML columns and an Integer column. 

This table was generated over the weekend. Unfortunately when I try to view the table I get the following exception it the console:

WARN     pool-1-thread-1 KNIMEApplication$3      Potential deadlock in SWT Display thread detected. Full thread dump will follow as debug ouput.

Then I get a thread dump in the log.

 

2016-05-16 10:48:12,648 : WARN  : pool-2-thread-1 : KNIMEApplication$4 :  :  : Potential deadlock in AWT Event Queue detected. Full thread dump will follow as debug ouput.
2016-05-16 10:48:12,649 : DEBUG : pool-2-thread-1 : KNIMEApplication$4 :  :  : "org.eclipse.jface.text.reconciler.MonoReconciler" Id=85 TIMED_WAITING on org.eclipse.jface.text.reconciler.DirtyRegionQueue@14d76ebb
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.jface.text.reconciler.DirtyRegionQueue@14d76ebb
    at org.eclipse.jface.text.reconciler.AbstractReconciler$BackgroundThread.run(AbstractReconciler.java:179)

"Global KNIME Timer" Id=83 WAITING on java.util.TaskQueue@b15c041
    at java.lang.Object.wait(Native Method)
    -  waiting on java.util.TaskQueue@b15c041
    at java.lang.Object.wait(Object.java:502)
    at java.util.TimerThread.mainLoop(Timer.java:526)
    at java.util.TimerThread.run(Timer.java:505)

"OutPortView-Updater-1" Id=81 WAITING on java.awt.EventQueue$1AWTInvocationLock@89f837e
    at java.lang.Object.wait(Native Method)
    -  waiting on java.awt.EventQueue$1AWTInvocationLock@89f837e
    at java.lang.Object.wait(Object.java:502)
    at java.awt.EventQueue.invokeAndWait(EventQueue.java:1313)
    at java.awt.EventQueue.invokeAndWait(EventQueue.java:1294)
    at javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1348)
    at org.knime.core.node.util.ViewUtils.invokeAndWaitInEDT(ViewUtils.java:142)
    at org.knime.core.node.workflow.OutPortView.updateInternal(OutPortView.java:238)
    at org.knime.core.node.workflow.OutPortView.access$1(OutPortView.java:237)
    at org.knime.core.node.workflow.OutPortView$3.run(OutPortView.java:220)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

    Number of locked synchronizers = 1
    - java.util.concurrent.ThreadPoolExecutor$Worker@13eb28f4

"AWT-EventQueue-0" Id=79 RUNNABLE
    at java.io.FileOutputStream.open0(Native Method)
    at java.io.FileOutputStream.open(FileOutputStream.java:270)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
    at org.knime.core.data.container.CopyOnAccessTask.copyEntryToDir(CopyOnAccessTask.java:300)
    at org.knime.core.data.container.CopyOnAccessTask.createBuffer(CopyOnAccessTask.java:205)
    at org.knime.core.data.container.CopyOnAccessTask.createBuffer(CopyOnAccessTask.java:157)
    at org.knime.core.data.container.ContainerTable.ensureBufferOpen(ContainerTable.java:292)
    -  locked org.knime.core.data.container.CopyOnAccessTask@63708d70
    at org.knime.core.data.container.ContainerTable.size(ContainerTable.java:145)
    at org.knime.core.node.BufferedDataTable.size(BufferedDataTable.java:383)
    at org.knime.core.node.tableview.TableContentModel.setDataTableIntern(TableContentModel.java:391)
    at org.knime.core.node.tableview.TableContentModel.access$1(TableContentModel.java:343)
    at org.knime.core.node.tableview.TableContentModel$1.run(TableContentModel.java:316)
    at org.knime.core.node.util.ViewUtils.invokeAndWaitInEDT(ViewUtils.java:134)
    at org.knime.core.node.tableview.TableContentModel.setDataTable(TableContentModel.java:313)
    at org.knime.core.node.tableview.TableContentView.setDataTable(TableContentView.java:310)
    at org.knime.core.node.tableview.TableView.setDataTable(TableView.java:427)
    at org.knime.core.node.workflow.BufferedDataTableView.updateDataTable(BufferedDataTableView.java:93)
    -  locked java.lang.Object@4cfd9513
    at org.knime.core.node.workflow.BufferedDataTableView.<init>(BufferedDataTableView.java:88)
    at org.knime.core.node.BufferedDataTable.getViews(BufferedDataTable.java:923)
    at org.knime.core.node.workflow.OutPortView$4.runWithContext(OutPortView.java:257)
    at org.knime.core.node.workflow.OutPortView$4.run(OutPortView.java:243)
    at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:301)
    at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:756)
    at java.awt.EventQueue.access$500(EventQueue.java:97)
    at java.awt.EventQueue$3.run(EventQueue.java:709)
    at java.awt.EventQueue$3.run(EventQueue.java:703)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:76)
    at java.awt.EventQueue.dispatchEvent(EventQueue.java:726)
    at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
    at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
    at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
    at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
    at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

"AWT-Shutdown" Id=80 WAITING on java.lang.Object@694f4305
    at java.lang.Object.wait(Native Method)
    -  waiting on java.lang.Object@694f4305
    at java.lang.Object.wait(Object.java:502)
    at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:295)
    at java.lang.Thread.run(Thread.java:745)

"Unknown Progress Timer" Id=78 WAITING on java.util.ArrayList@1dbfa921
    at java.lang.Object.wait(Native Method)
    -  waiting on java.util.ArrayList@1dbfa921
    at java.lang.Object.wait(Object.java:502)
    at org.knime.workbench.editor2.figures.ProgressFigure$UnknownProgressTimer.run(ProgressFigure.java:552)

"KNIME Sync Exec Dispatcher-1" Id=77 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7a43aceb
    at sun.misc.Unsafe.park(Native Method)
    -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7a43aceb
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

"[ThreadPool Manager] - Idle Thread" Id=61 WAITING on org.eclipse.equinox.internal.util.impl.tpt.threadpool.Executor@5ab6fbbd
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.equinox.internal.util.impl.tpt.threadpool.Executor@5ab6fbbd
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.equinox.internal.util.impl.tpt.threadpool.Executor.run(Executor.java:106)

"Thread-9" Id=53 TIMED_WAITING
    at java.lang.Thread.sleep(Native Method)
    at org.eclipse.equinox.internal.p2.updatechecker.UpdateChecker$UpdateCheckThread.run(UpdateChecker.java:68)

"Worker-4" Id=52 TIMED_WAITING on org.eclipse.core.internal.jobs.WorkerPool@4972dc85
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.core.internal.jobs.WorkerPool@4972dc85
    at org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:188)
    at org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:220)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:52)

"Worker-2" Id=50 TIMED_WAITING on org.eclipse.core.internal.jobs.WorkerPool@4972dc85
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.core.internal.jobs.WorkerPool@4972dc85
    at org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:188)
    at org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:220)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:52)

"KNIME Progress Timer" Id=49 TIMED_WAITING on java.util.TaskQueue@29e4822b
    at java.lang.Object.wait(Native Method)
    -  waiting on java.util.TaskQueue@29e4822b
    at java.util.TimerThread.mainLoop(Timer.java:552)
    at java.util.TimerThread.run(Timer.java:505)

"ServerSpace Update Timer" Id=48 TIMED_WAITING on java.util.TaskQueue@4ea75708
    at java.lang.Object.wait(Native Method)
    -  waiting on java.util.TaskQueue@4ea75708
    at java.util.TimerThread.mainLoop(Timer.java:552)
    at java.util.TimerThread.run(Timer.java:505)

"EventAdmin Async Event Dispatcher Thread" Id=47 WAITING on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@23a12672
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@23a12672
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:400)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:336)

"pool-4-thread-3" Id=46 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ce84077
    at sun.misc.Unsafe.park(Native Method)
    -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ce84077
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

"pool-4-thread-2" Id=45 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ce84077
    at sun.misc.Unsafe.park(Native Method)
    -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ce84077
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

"pool-4-thread-1" Id=44 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ce84077
    at sun.misc.Unsafe.park(Native Method)
    -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ce84077
    at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
    at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

"Abandoned connection cleanup thread" Id=43 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@7110701b
    at java.lang.Object.wait(Native Method)
    -  waiting on java.lang.ref.ReferenceQueue$Lock@7110701b
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
    at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:40)

"Java indexing" Id=36 WAITING on org.eclipse.jdt.internal.core.search.indexing.IndexManager@57b6626
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.jdt.internal.core.search.indexing.IndexManager@57b6626
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.jdt.internal.core.search.processing.JobManager.run(JobManager.java:382)
    at java.lang.Thread.run(Thread.java:745)

"pool-2-thread-1" Id=32 RUNNABLE
    at sun.management.ThreadImpl.dumpThreads0(Native Method)
    at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:446)
    at org.knime.core.util.GUIDeadlockDetector.createStacktrace(GUIDeadlockDetector.java:162)
    at org.knime.core.util.GUIDeadlockDetector$CheckTask.run(GUIDeadlockDetector.java:145)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

    Number of locked synchronizers = 1
    - java.util.concurrent.ThreadPoolExecutor$Worker@67dc6b48

"pool-1-thread-1" Id=31 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2fc3524d
    at sun.misc.Unsafe.park(Native Method)
    -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2fc3524d
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

"Provisioning Event Dispatcher" Id=30 WAITING on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@1196543a
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@1196543a
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:400)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:336)

"AWT-Windows" Id=26 RUNNABLE (in native)
    at sun.awt.windows.WToolkit.eventLoop(Native Method)
    at sun.awt.windows.WToolkit.run(WToolkit.java:306)
    at java.lang.Thread.run(Thread.java:745)

"Java2D Disposer" Id=24 WAITING on java.lang.ref.ReferenceQueue$Lock@63746ebb
    at java.lang.Object.wait(Native Method)
    -  waiting on java.lang.ref.ReferenceQueue$Lock@63746ebb
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
    at sun.java2d.Disposer.run(Disposer.java:148)
    at java.lang.Thread.run(Thread.java:745)

"Bundle File Closer" Id=23 WAITING on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@3d607ea5
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@3d607ea5
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:400)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:336)

"EMF Reference Cleaner" Id=22 WAITING on java.lang.ref.ReferenceQueue$Lock@884127e
    at java.lang.Object.wait(Native Method)
    -  waiting on java.lang.ref.ReferenceQueue$Lock@884127e
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
    at org.eclipse.emf.common.util.CommonUtil$1ReferenceClearingQueuePollingThread.run(CommonUtil.java:70)

"Worker-JM" Id=18 WAITING on java.util.ArrayList@2b10b0d0
    at java.lang.Object.wait(Native Method)
    -  waiting on java.util.ArrayList@2b10b0d0
    at org.eclipse.core.internal.jobs.InternalWorker.run(InternalWorker.java:59)

"[Timer] - Main Queue Handler" Id=17 TIMED_WAITING on java.lang.Object@6bd102bf
    at java.lang.Object.wait(Native Method)
    -  waiting on java.lang.Object@6bd102bf
    at org.eclipse.equinox.internal.util.impl.tpt.timer.TimerImpl.run(TimerImpl.java:141)
    at java.lang.Thread.run(Thread.java:745)

"Start Level: Equinox Container: a0307309-4b1b-0016-177e-faf0a3fdc525" Id=15 WAITING on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@4198c057
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@4198c057
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:400)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:336)

"Framework Event Dispatcher: Equinox Container: a0307309-4b1b-0016-177e-faf0a3fdc525" Id=14 WAITING on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@557adc36
    at java.lang.Object.wait(Native Method)
    -  waiting on org.eclipse.osgi.framework.eventmgr.EventManager$EventThread@557adc36
    at java.lang.Object.wait(Object.java:502)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:400)
    at org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:336)

"Active Thread: Equinox Container: a0307309-4b1b-0016-177e-faf0a3fdc525" Id=12 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc53305
    at sun.misc.Unsafe.park(Native Method)
    -  waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7dc53305
    at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
    at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
    at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

"Attach Listener" Id=5 RUNNABLE

"Signal Dispatcher" Id=4 RUNNABLE

"Finalizer" Id=3 WAITING on java.lang.ref.ReferenceQueue$Lock@53fd9a5d
    at java.lang.Object.wait(Native Method)
    -  waiting on java.lang.ref.ReferenceQueue$Lock@53fd9a5d
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
    at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
    at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)

"Reference Handler" Id=2 WAITING on java.lang.ref.Reference$Lock@305abe67
    at java.lang.Object.wait(Native Method)
    -  waiting on java.lang.ref.Reference$Lock@305abe67
    at java.lang.Object.wait(Object.java:502)
    at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)

"main" Id=1 RUNNABLE
    at org.eclipse.swt.internal.win32.OS.WaitMessage(Native Method)
    at org.eclipse.swt.widgets.Display.sleep(Display.java:4728)
    at org.eclipse.ui.application.WorkbenchAdvisor.eventLoopIdle(WorkbenchAdvisor.java:368)
    at org.eclipse.ui.internal.Workbench$3.eventLoopIdle(Workbench.java:517)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$4.run(PartRenderingEngine.java:1132)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:337)
    at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1018)
    at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:156)
    at org.eclipse.ui.internal.Workbench$5.run(Workbench.java:654)
    at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:337)
    at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:598)
    at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
    at org.knime.product.rcp.KNIMEApplication.start(KNIMEApplication.java:132)
    at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
    at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
    at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:669)
    at org.eclipse.equinox.launcher.Main.basicRun(Main.java:608)
    at org.eclipse.equinox.launcher.Main.run(Main.java:1515)

The file seems to be about 3.5Gb. 

I tried to filter the first 20 rows of this table, but it seems to have trouble doing that too. 

Am I asking to much or should this have worked? 

Cheers

Sam

 

Assuming it is a memory related issue, did you try to increase the Java Heap Space as described here?

You can also try to play with the Memory Policy of  your critical nodes.

Cheers,
Marco.

Hi Marco

My heap space was at 16g and the heap space monitor hadn't indicated the application was low on memory but it's a good thought. 

Having to keep large tables in memory wouldn't really be a viable option.

Cheers

Sam

Hi Sam,

Few questions:

  • Has the workflow been saved and closed after it completed the computation of that large table?
  • Has it completed to render the table (or execute the filter node) when you wait fairly long (hours)?
  • If not, do you see a java exception further up in the knime.log file?

The stack trace indicates that it's busy extracting the table file, whereby this table also contains blobs (large xml are saved as blob files). Depending on where the workflow is located (network drive?) and how many 'large' xml cells you have that may take very long as each cell is represented as a file and I/O might be a bottleneck:

???

Bernd

PS: The relevant part of the stack trace (and yes, that should be done the event dispatching thread):

"AWT-EventQueue-0" Id=79 RUNNABLE
    at java.io.FileOutputStream.open0(Native Method)
    at java.io.FileOutputStream.open(FileOutputStream.java:270)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
    at java.io.FileOutputStream.<init>(FileOutputStream.java:162)
    at org.knime.core.data.container.CopyOnAccessTask.copyEntryToDir(CopyOnAccessTask.java:300)
    at org.knime.core.data.container.CopyOnAccessTask.createBuffer(CopyOnAccessTask.java:205)
    at org.knime.core.data.container.CopyOnAccessTask.createBuffer(CopyOnAccessTask.java:157)
    at org.knime.core.data.container.ContainerTable.ensureBufferOpen(ContainerTable.java:292)
    -  locked org.knime.core.data.container.CopyOnAccessTask@63708d70
    at org.knime.core.data.container.ContainerTable.size(ContainerTable.java:145)

Hi Bernd 

  • Has the workflow been saved and closed after it completed the computation of that large table?

The first time I tried to view it no, the second yes. 

  • Has it completed to render the table (or execute the filter node) when you wait fairly long (hours)?

I didn't wait that long. I reliased I hadn't removed the original XML cell that I was extracting from (I have a number of XML files containing multiple <reaction> elements. I'm effectively making a lot of copies of the original XML so I updated my workflow to remove this. 

 

Now that I've re-run it with removing the source I saved then did the following:

View table:

I still get

WARN  KNIMEApplication$4              Potential deadlock in AWT Event Queue detected. Full thread dump will follow as debug ouput.

 

But after a few more seconds the table visualises. I'm also able to scroll through the table without problems. 

Some observations

I just closed KNIME (from the X), the process is still running... I ran it from the console the second time with "knime.exe. -consolelog" and it's still got he console up and my CPU usage for the KNIME process is ~20%. It took a few minutes to finish closing down. Is it tidying up after itself and had a lot to do?

Once closing and reloading KNIME the table takes a while visualise but it does, the only problem this causes is I can't open and dialogs or table views while it is trying to do this. 

Though I remain impressed it loaded the 1.6 milion XML rows in less than 5 minutes: )

I tried to avoid this completely using a streamed wrapped node that process my XML and removes it. But the XPATH node doesn't want to run under the streamed executor for some reason. 

Cheers

Sam

Hi Sam,

All clear now. I don't think that we have a serious problem here anywhere -- it's just a lot of complex data structure and slightly inefficient code when it comes to visualize it.

The "Potential deadlock in AWT Event Queue detected" isn't as severe as it sounds. We recently added a tracker to both UI threads (SWT and AWT) that will trigger that message if any UI update takes longer than 30s (I think). It would then print that message but it doesn't do anything else (it doesn't interfere with the rendering nor the processing of any of the nodes).

Why does it so long to close KNIME? I think (again) it's a problem with much data and many files. Large XML cells are backed by blobs, which are stored in separate files. If you then have millions of those the deletion of them takes already a bit (I think even on an SSD).

Thanks Bernd!

I finished converting the the XML into our chemical objects and now have a table of 1.6m 'reaction' objects. This table loads quickly (not that I actually would need to look at it other than checking it processed) so another nail in the complex object causing the slow down (I use a JSON String to serialize this reaction cell).