TransferFile Node hostname not verified

zeru · September 15, 2021, 6:33am

Continuing the discussion from TransferFile Node - hostname not verified error:

Hi, I am still getting hostname not verified error after I import the sharepoint certificate with the DNS name defined. Here is my error log:

ERROR : KNIME-Worker-8785-Transfer Files 135:609 : Node : Transfer Files : 135:609 :
Execute failed: Hostname xxx-my.sharepoint.com not verified:
certificate:
DN: CN=.azureedge.net, O=Microsoft Corporation, L=Redmond, ST=WA, C=US
subjectAltNames: [.azureedge.net, *.media.microsoftstream.com, .origin.mediaservices.windows.net, .streaming.mediaservices.windows.net]
javax.net.ssl.SSLPeerUnverifiedException: Hostname xxx-my.sharepoint.com not verified:
certificate:
DN: CN=.azureedge.net, O=Microsoft Corporation, L=Redmond, ST=WA, C=US
subjectAltNames: [.azureedge.net, *.media.microsoftstream.com, *.origin.mediaservices.windows.net, *.streaming.mediaservices.windows.net]
at okhttp3.internal.connection.RealConnection.connectTls(RealConnection.java:350)
at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:300)
at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:185)
at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:224)
at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java:108)
at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:88)
at okhttp3.internal.connection.Transmitter.newExchange(Transmitter.java:169)
at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:41)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.microsoft.graph.httpcore.RedirectHandler.intercept(RedirectHandler.java:123)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.microsoft.graph.httpcore.RetryHandler.intercept(RetryHandler.java:140)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.microsoft.graph.httpcore.AuthenticationHandler.intercept(AuthenticationHandler.java:31)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.microsoft.graph.httpcore.TelemetryHandler.intercept(TelemetryHandler.java:43)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
at okhttp3.RealCall.execute(RealCall.java:81)
at com.microsoft.graph.http.CoreHttpProvider.sendRequestInternal(CoreHttpProvider.java:398)
at com.microsoft.graph.http.CoreHttpProvider.send(CoreHttpProvider.java:220)
at com.microsoft.graph.http.CoreHttpProvider.send(CoreHttpProvider.java:200)
at com.microsoft.graph.http.BaseStreamRequest.send(BaseStreamRequest.java:88)
at com.microsoft.graph.requests.extensions.DriveItemContentStreamRequest.get(DriveItemContentStreamRequest.java:53)
at org.knime.ext.sharepoint.filehandling.fs.SharepointFileSystemProvider.newInputStreamInternal(SharepointFileSystemProvider.java:194)
at org.knime.ext.sharepoint.filehandling.fs.SharepointFileSystemProvider.newInputStreamInternal(SharepointFileSystemProvider.java:1)
at org.knime.filehandling.core.connections.base.BaseFileSystemProvider.newInputStream(BaseFileSystemProvider.java:282)
at java.nio.file.Files.newInputStream(Files.java:152)
at java.nio.file.CopyMoveHelper.copyToForeignTarget(CopyMoveHelper.java:125)
at java.nio.file.Files.copy(Files.java:1277)
at org.knime.filehandling.utility.nodes.transfer.PathCopier.getOverwriteCopyFunction(PathCopier.java:203)
at org.knime.filehandling.utility.nodes.transfer.PathCopier.lambda$0(PathCopier.java:145)
at org.knime.filehandling.utility.nodes.transfer.PathCopier.copyFiles(PathCopier.java:287)
at org.knime.filehandling.utility.nodes.transfer.PathCopier.copyPath(PathCopier.java:239)
at org.knime.filehandling.utility.nodes.transfer.TransferFilesNodeModel.copy(TransferFilesNodeModel.java:183)
at org.knime.filehandling.utility.nodes.transfer.TransferFilesNodeModel.execute(TransferFilesNodeModel.java:139)
at org.knime.core.node.NodeModel.executeModel(NodeModel.java:576)
at org.knime.core.node.Node.invokeFullyNodeModelExecute(Node.java:1245)
at org.knime.core.node.Node.execute(Node.java:1025)
at org.knime.core.node.workflow.NativeNodeContainer.performExecuteNode(NativeNodeContainer.java:558)
at org.knime.core.node.exec.LocalNodeExecutionJob.mainExecute(LocalNodeExecutionJob.java:95)
at org.knime.core.node.workflow.NodeExecutionJob.internalRun(NodeExecutionJob.java:201)
at org.knime.core.node.workflow.NodeExecutionJob.run(NodeExecutionJob.java:117)
at org.knime.core.util.ThreadUtils$RunnableWithContextImpl.runWithContext(ThreadUtils.java:334)
at org.knime.core.util.ThreadUtils$RunnableWithContext.run(ThreadUtils.java:210)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.knime.core.util.ThreadPool$MyFuture.run(ThreadPool.java:123)
at org.knime.core.util.ThreadPool$Worker.run(ThreadPool.java:246)

zeru · September 15, 2021, 6:36am

Based on the error log, I compared the TLS on my desktop and the server(cause that worklfow can run on my desktop without any issue), and noticed server does not have TLS13 installed, but seems TLS13 does not support windows server…

LukasS · September 15, 2021, 12:59pm

Hi @zero,

indeed, TLS 1.3 is not available on windows (yet). If this should cause the error, I’m afraid you’d have to use a lower TLS Version on the server and configure Sharepoint such that it allows that.

I’m not sure whether your problem is related to that though - the subjectAltNames still look suspicious to me: there is no *.sharepoint.com or the like in there. Thus the requested hostname is not part of the certificate and hence the error. Could you double check that you updated your certificate with the hostname?

Best,
Lukas

zeru · September 16, 2021, 12:23am

Hi, Lukas,

Below is the certificate I imported, I have DNS names includes the SharePoint site. And cause the knime server version is 4.3, so I imported the certificate to <knime-folder>/plugins/org.knime.binary.jre.<..>/jre/lib/security/cacerts

Thanks,
Zeru

LukasS · September 16, 2021, 9:55am

Hi @Zeru,

your cert_sharepoint_sub.pem-certificate looks good to me, and I think something went wrong with the import: if you compare the Subject of your screenshot to the error message they do not conform (e.g. the common name CN=*.sharepoint.com vs CN=.azureedge.net) - so we have to put this certificate in the correct place

The folder is correct - based on your previous screenshot you imported file cert_sh.pem, based on the current screenshot its named cert_sharepoint_sub.pem. Is the cert_sh.pem correct as well?

You can check the content of the keystore e.g. with keytool -list -keystore cacerts or a specific alias with keytool -list -keystore cacerts -alias <sharepoint-alias>. Would something show up if you try keytool -list -keystore cacerts | findstr "sharepoint"? If not, could you retry to import the certificate to the store?

Best, Lukas

zeru · September 17, 2021, 12:36am

Hi, Lukas,

Yes, I have sharepoint_sub imported. The screenshot you saw from the previous post named sharepoint.pem does not have DNS names defined so I removed that from cacerts. I also imported the certificated from SharePoint’s log-in page (sharepoint_login) in case it was redirected to the log-in page.

One thing I am wondering, from the error page, looks like it is looking for CN=.azureedge.net, but I confirmed with our IT, seems like we don’t have that.

Thanks,
Zeru

zeru · September 17, 2021, 12:51am

One more thing, besides import the certificates into KNIME, is there anything I need to set up for the server? (I mean the server mahcine that host KNIME).

Some notes:

I can use Transfer File node to download file from SharePoint at very beginning, but as I used the node more and more often, the errors comes out and it hapens 70% of the time.
I can use SharePoint on Server use browser, and I confirmed with our IT, the server machine is not blocking SharePoint.

Thanks,
Zeru

LukasS · September 17, 2021, 11:50am

Hi @zeru,

hmm, that sounds weird: IMO the sharepoint connection should either be possible or not, and not work only 70% of the time. Maybe its not a certificate problem after all but you run into a sharepoint limit or something like that? Are proxies involved? A few things to try and pinpoint things down:

Could you please restart the KNIME Executor service, reproduce the error and share the executor log file? It lives on the server under <executor-workspace>\.metadata\knime\knime.log.
To check whether it is a cacerts issue (it looks correct to me based on your screenshot), you could try to put your locally working cacerts on the server under <executor-folder>/plugins/org.knime.binary.jre.<..>/jre/lib/security/cacerts (back up the original one!) and see if the same error still pops up. With this we can check whether the cacerts on the server is misbehaving.
You can start the executor GUI on the server (as you would your KNIME AP locally), in a dummy workspace. Would the respective workflow work there?

Thanks for hanging in there!
Best, Lukas

LukasS · September 20, 2021, 2:39pm

(You can also reach out to us directly at support@knime.com instead of sharing the logs here.)

zeru · September 23, 2021, 7:54am

Thanks, Lukas. Email sent.

jeffgullick-knime · December 7, 2021, 1:43pm

@zeru

Just to follow-up on this post, the issue seems to have been that the executors where running as local system accounts, and not windows domain accounts. when trying to access certain resources, the node was being re-directed to a login services since the user running KNIME was not recognized on the domain.

TL;DR - If the OS is windows, try running the executor as a domain user. Certain network services expect windows authentication, and will reject and/or prompt the calling service to login.