KNIME 4.7, Geospatial Extension: Joiner Error

Community Extensions
,
1

Joining a table containing a geometry to another table using the Joiner node throws an error

Problem

A workflow (with data) demonstrating the problem can be downloaded from the KNIME hub here.

Screenshot_20221207_133252

Two tables are read by the workflow:

  • The top table is a CSV file containing an extract of statistical data (Source: UK Office of National Statistics)
  • The bottom table is a Shapefile containing geography polygons (Source: UK Open Geography Portal)

The joiner is setup to join the two tables based upon the OA2021 code.

Joining the tables when the geometry column is included in the bottom table causes the joiner to fail.
Removing the geometry table the joiner succeeds.

Desired Outcome

It would be great if we could join tables containing geometry columns.

Additional Information

The console outputs the following error when the top joiner node fails:

ERROR Joiner               3:19       Execute failed: Error while prefetching rows
ERROR Joiner               3:19       Error when attempting to close open closeable.

The Workspace log issues the following error (yikes! the universe is about to explode).

Message:

Memory was leaked by query. Memory leaked: (294912)
Allocator(ArrowColumnReadStore) 0/294912/294912/9223372036854775807 (res/actual/peak/limit)

Exception Stack Trace:

java.lang.IllegalStateException: Memory was leaked by query. Memory leaked: (294912)
Allocator(ArrowColumnReadStore) 0/294912/294912/9223372036854775807 (res/actual/peak/limit)

	at org.apache.arrow.memory.BaseAllocator.close(BaseAllocator.java:437)
	at org.knime.core.columnar.arrow.AbstractArrowBatchReadable.close(AbstractArrowBatchReadable.java:100)
	at org.knime.core.columnar.arrow.ArrowBatchReadStore.close(ArrowBatchReadStore.java:1)
	at org.knime.core.columnar.cache.data.ReadDataReadCache.close(ReadDataReadCache.java:178)
	at org.knime.core.columnar.data.dictencoding.DictEncodedBatchReadable.close(DictEncodedBatchReadable.java:93)
	at org.knime.core.columnar.cache.object.ObjectReadCache.close(ObjectReadCache.java:108)
	at org.knime.core.data.columnar.table.WrappedBatchReadStore.close(WrappedBatchReadStore.java:187)
	at org.knime.core.data.columnar.table.DefaultColumnarBatchReadStore.close(DefaultColumnarBatchReadStore.java:191)
	at org.knime.core.data.columnar.table.ColumnarRowReadTable.close(ColumnarRowReadTable.java:208)
	at org.knime.core.data.columnar.table.AbstractColumnarContainerTable.clear(AbstractColumnarContainerTable.java:201)
	at org.knime.core.data.columnar.table.UnsavedColumnarContainerTable.clear(UnsavedColumnarContainerTable.java:133)
	at org.knime.core.node.BufferedDataTable.clearSingle(BufferedDataTable.java:972)
	at org.knime.core.node.Node.disposeTables(Node.java:1727)
	at org.knime.core.node.Node.cleanOutPorts(Node.java:1691)
	at org.knime.core.node.Node.cleanup(Node.java:1851)
	at org.knime.core.node.workflow.NativeNodeContainer.cleanup(NativeNodeContainer.java:386)
	at org.knime.core.node.workflow.WorkflowManager.cleanup(WorkflowManager.java:9805)
	at org.knime.core.node.workflow.WorkflowManager.removeNode(WorkflowManager.java:1060)
	at org.knime.core.node.workflow.WorkflowManager.removeProject(WorkflowManager.java:796)
	at org.knime.workbench.ui.navigator.ProjectWorkflowMap.remove(ProjectWorkflowMap.java:439)
	at org.knime.workbench.editor2.WorkflowEditor.dispose(WorkflowEditor.java:676)
	at org.eclipse.ui.internal.e4.compatibility.CompatibilityPart.invalidate(CompatibilityPart.java:264)
	at org.eclipse.ui.internal.e4.compatibility.CompatibilityPart.destroy(CompatibilityPart.java:421)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:58)
	at org.eclipse.e4.core.internal.di.InjectorImpl.processAnnotated(InjectorImpl.java:995)
	at org.eclipse.e4.core.internal.di.InjectorImpl.processAnnotated(InjectorImpl.java:960)
	at org.eclipse.e4.core.internal.di.InjectorImpl.uninject(InjectorImpl.java:201)
	at org.eclipse.e4.core.internal.di.Requestor.uninject(Requestor.java:177)
	at org.eclipse.e4.core.internal.contexts.ContextObjectSupplier$ContextInjectionListener.update(ContextObjectSupplier.java:89)
	at org.eclipse.e4.core.internal.contexts.TrackableComputationExt.update(TrackableComputationExt.java:103)
	at org.eclipse.e4.core.internal.contexts.EclipseContext.removeListenersTo(EclipseContext.java:485)
	at org.eclipse.e4.core.contexts.ContextInjectionFactory.uninject(ContextInjectionFactory.java:184)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.safeRemoveGui(PartRenderingEngine.java:947)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$4.run(PartRenderingEngine.java:861)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.removeGui(PartRenderingEngine.java:845)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.subscribeTopicToBeRendered(PartRenderingEngine.java:185)
	at jdk.internal.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:58)
	at org.eclipse.e4.core.di.internal.extensions.EventObjectSupplier$DIEventHandler.handleEvent(EventObjectSupplier.java:92)
	at org.eclipse.equinox.internal.event.EventHandlerWrapper.handleEvent(EventHandlerWrapper.java:205)
	at org.eclipse.equinox.internal.event.EventHandlerTracker.dispatchEvent(EventHandlerTracker.java:203)
	at org.eclipse.equinox.internal.event.EventHandlerTracker.dispatchEvent(EventHandlerTracker.java:1)
	at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:234)
	at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:151)
	at org.eclipse.equinox.internal.event.EventAdminImpl.dispatchEvent(EventAdminImpl.java:133)
	at org.eclipse.equinox.internal.event.EventAdminImpl.sendEvent(EventAdminImpl.java:75)
	at org.eclipse.equinox.internal.event.EventComponent.sendEvent(EventComponent.java:44)
	at org.eclipse.e4.ui.services.internal.events.EventBroker.send(EventBroker.java:55)
	at org.eclipse.e4.ui.internal.workbench.UIEventPublisher.notifyChanged(UIEventPublisher.java:63)
	at org.eclipse.emf.common.notify.impl.BasicNotifierImpl.eNotify(BasicNotifierImpl.java:424)
	at org.eclipse.e4.ui.model.application.ui.impl.UIElementImpl.setToBeRendered(UIElementImpl.java:314)
	at org.eclipse.e4.ui.internal.workbench.PartServiceImpl.hidePart(PartServiceImpl.java:1401)
	at org.eclipse.e4.ui.internal.workbench.PartServiceImpl.hidePart(PartServiceImpl.java:1331)
	at org.eclipse.e4.ui.workbench.renderers.swt.StackRenderer.closePart(StackRenderer.java:1182)
	at org.eclipse.e4.ui.workbench.renderers.swt.StackRenderer$3.close(StackRenderer.java:1042)
	at org.eclipse.swt.custom.CTabFolder.onMouse(CTabFolder.java:1961)
	at org.eclipse.swt.custom.CTabFolder.lambda$0(CTabFolder.java:336)
	at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
	at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4251)
	at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1066)
	at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4068)
	at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3645)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1155)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1046)
	at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155)
	at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:644)
	at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
	at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:551)
	at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:156)
	at org.knime.product.rcp.KNIMEApplication.start(KNIMEApplication.java:191)
	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:402)
	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
	at java.base/java.lang.reflect.Method.invoke(Unknown Source)
	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:596)
	at org.eclipse.equinox.launcher.Main.run(Main.java:1467)

Session data:

eclipse.buildId=unknown
java.version=17.0.5
java.vendor=Eclipse Adoptium
BootLoader constants: OS=win32, ARCH=x86_64, WS=win32, NL=en_GB
Command-line arguments:  -os win32 -ws win32 -arch x86_64 -data file:/D:/workspace/knime/

DiaAzul

2

Hi @DiaAzul,

Thanks for trying out the new GeoSpatial extension and thanks for the detailed feedback! Sorry about the nasty error message, that should not happen. We are looking into it!

Best, Carsten

4 Likes
3

Moved to the new Geospatial subcategory. Thanks for the feedback!

1 Like
4

If anyone needs a workaround. It is possible to:

  1. Convert the geometry column to well-known text (WKT)
  2. Remove the geometry column
  3. Do the join
  4. Convert the WKT back to geometry.

Slow, but works.

Screenshot_20221208_133148

DiaAzul
LinkedIn Profile

2 Likes
5

Thanks so much for the workflow reproducing the error @DiaAzul.

We have solved the error, the fix will be released with KNIME AP 4.7.1 – or you can go ahead and try it out already in the latest nightly build (but use that build with caution :slight_smile: ).

5 Likes
6

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.