Writing excel to sharepoint location results in Execute failed: ("NullPointerException"): null

Hi all,

I’m trying to write an excel file to a sharepoint location and it results in a error:
Execute failed: (“NullPointerException”): null

The workflow first opens the same file, does some changes and updates, and then tries to overwrite the file, but fails. The excel writer node starts working and at 75% it fails with an error. A temporary file is saved in the sharepoint location but i cannot open it.

I have successfully written the file to a local location, so it does not seem to be an issue with the data/table itself, but rather with the sharepoint connection. I also tried writing to a new file on the sharepoint location, but this results in the same error.

The log is showing this:
2021-10-12 11:12:36,788 : ERROR : KNIME-Worker-135-Excel Writer 4:463 : : Node : Excel Writer : 4:463 : Execute failed: (“NullPointerException”): null
java.lang.NullPointerException
at org.knime.ext.sharepoint.filehandling.GraphApiUtil.unwrapIOE(GraphApiUtil.java:109)
at org.knime.ext.sharepoint.filehandling.fs.SharepointSeekableByteChannel.uploadLarge(SharepointSeekableByteChannel.java:173)
at org.knime.ext.sharepoint.filehandling.fs.SharepointSeekableByteChannel.copyToRemote(SharepointSeekableByteChannel.java:112)
at org.knime.ext.sharepoint.filehandling.fs.SharepointSeekableByteChannel.copyToRemote(SharepointSeekableByteChannel.java:1)
at org.knime.filehandling.core.connections.base.TempFileSeekableByteChannel.close(TempFileSeekableByteChannel.java:164)
at org.knime.filehandling.core.connections.FSSeekableByteChannel.close(FSSeekableByteChannel.java:89)
at java.nio.channels.Channels$1.close(Channels.java:178)
at org.knime.filehandling.core.connections.FSOutputStream.close(FSOutputStream.java:104)
at java.io.FilterOutputStream.close(FilterOutputStream.java:159)
at org.knime.ext.poi3.node.io.filehandling.excel.writer.table.ExcelMultiTableWriter.saveWorkbook(ExcelMultiTableWriter.java:158)
at org.knime.ext.poi3.node.io.filehandling.excel.writer.table.ExcelMultiTableWriter.saveFile(ExcelMultiTableWriter.java:147)
at org.knime.ext.poi3.node.io.filehandling.excel.writer.table.ExcelMultiTableWriter.writeTables(ExcelMultiTableWriter.java:126)
at org.knime.ext.poi3.node.io.filehandling.excel.writer.ExcelTableWriterNodeModel.write(ExcelTableWriterNodeModel.java:195)
at org.knime.ext.poi3.node.io.filehandling.excel.writer.ExcelTableWriterNodeModel.execute(ExcelTableWriterNodeModel.java:162)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:576)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1245)
at org.knime.core.node.Node.execute(Node.java:1025)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:558)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:334)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)
Suppressed: java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
at sun.nio.ch.FileChannelImpl.size(FileChannelImpl.java:300)
at org.knime.filehandling.core.connections.base.TempFileSeekableByteChannel.close(TempFileSeekableByteChannel.java:158)
at org.knime.filehandling.core.connections.FSSeekableByteChannel.close(FSSeekableByteChannel.java:89)
at java.nio.channels.Channels$1.close(Channels.java:178)
at org.knime.filehandling.core.connections.FSOutputStream.close(FSOutputStream.java:104)
… 18 more

Does anybody have an idea of what is going wrong?

Regards,
Mark

Hello,

What nodes do you use before the Excel Writer to create a connection to the sharepoint and how are they configured?

Greetings
Torben

1 Like

I’m using the microsoft authentication and the sharepoint online connector node to set up the sharepoint connection. In the authentication node i have selected the read/write option.

Any specific option you are thinking about?

1 Like

Hi @maroelof

looking through the code, there is a bug in our code that handles an error response from Sharepoint. This bug results in this rather non-descriptive NullPointerException (I have opened an internal ticket for the bug). But the NullPointerException only occurs if there was an error response coming from Sharepoint while trying to do a multi-part upload (we use multi-part uploads for files >4MB).

To figure out what the Sharepoint error response is, could you try use the Excel Writer to write a small file (smaller than 4MB) to the same location? For small files we do not use multi-part uploads, and hence we should be able to see the actual Sharepoint error response. I suspect it might be a permissions issue on the Sharepoint side.

Best,
Björn

5 Likes

Hi Björn,

i just successfully saved a small excel file to the sharepoint using the same workflow. So there is no error message to share in this case.

Gr,
Mark

1 Like

I also just saved the file I had issues with onto sharepoint. The only difference is that now i’m working at the office, and i had the issues at home. I will check whether it still does not work at home when i login there.

I am also getting the same null pointer exception error using a similar workflow (MS Authentication + Sharepoint Connector) while writing to an xlsx file.

I have the same issue :frowning: can someone please help with ideas on how to solve it?

Seeing this issue on my end as well. 4.7MB XLSX file writing to SharePoint Online. This is the first time I have had this issue. KNIME v4.4.1.

Hi
maybe you guys share some more information about your connection settings or other configuration which might help to shed some light on that issue
best regards

2 Likes

I am writing large files (~150MB) to SharePoint using the following flow:

Microsoft Authentication >> SharePoint Online Connector >> (standard KNIME data manipulation nodes) >> Excel Writer connected using a file system port to SPO

I don’t get the error every time I run, but close to 50%, and seemingly more when I run from KNIME Server as opposed to my desktop. I am running v4.4.2 on my desktop.

As I wrote above, there is a bug in our code that results in a NullPointerException when Sharepoint rejects the file upload (we are using multi-part uploads for larger file). The NullPointerException only “hides” the true error message coming back from Sharepoint.

I can reproduce this failure by having several concurrent KNIME nodes (eg. Excel Writer) that write to the same file. They fail randomly, depending on timing:

A bugfix for the NPE is now available in our nightly build: Nightly Build | KNIME
We will also release this bugfix with 4.5.2 in the future.
Note that the bugfix just results in a better error message!

If the root cause of the problem are in fact concurrent writes to the same file, then this needs to be fixed in your workflow(s). Theoretically there can also be other processes than KNIME trying to write the file at the same time.

2 Likes

With help from the KNIME team, there is a possible resolution. I changed the read timeout and connection timeout in the SharePoint Online Connection node to 180 seconds and 60 seconds respectively and have not had the error since. I’ll continue testing but so far it looks good.

Screeny637

Note that since the original error message was generic, this may not be the resolution for your issue. After running the latest 4.6 nightly build with the updated SharePoint error message capability, my error was updated/translated from “NULL” to “Execute failed: Read timed out”. This led to the change in the connection node timeout settings.

Thanks again to the KNIME team on this one.

3 Likes

This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.