we recently got our business hub (I am admin) on AWS and everything is running fine so far, but I still have one big problem:
BigQuery Connection (and therefore queries/getting data) via Google BigQuery Connector.
Locally: Install/unzip Simba drivers, create database connection, no problem.
knime-server (years ago): export/copy database related rows from local epf file onto knime-server epf file and change local driver paths with hardcoded VM paths for the VM where the server was running (not a nice way, but it worked for years).
knime-business hub: I do not see any clear documentation nor any other forum post about this.
As far as I see now: create a customization profile with the bigquery driver/database related rows from a local epf and add (the whole package of jars) into the zipfile to upload via the workflow provided ?(Customization Profile Application – KNIME Community Hub )
But how do I handle the local absolute path to the jar files? What do I replace them with?
Any idea on this? Or is my approach simply wrong?
Does anyone have a solution?
I think I understand your question. I’ll use these abbreviations: (KAP = KNIME Analytics Platform; KBH = KNIME Business Hub; KS = KNIME Server)
Here are the things I wished were clearer from the documentation (I’m also writing this out for “future me”) :
What do I want to keep from the export of the preferences and what do I want to throw away?
It is a little easier to understand the answer to that if you happen to know how to access the Configuration in KAP: On Mac, click on KNIME in the upper left, and then select About KAP. On Windows, go to the Help menu in the upper right, and then select About KAP. From there you can click on Installation details on the lower left and finally the rightmost tab to get to the Configuration screen.
and finally the rightmost tab to get to the Configuration screen:
You want BigQuery, so you can type that into the search bar to show only the info you care about. Those (User) Preferences are what you’re going to keep.
There you can find the steps you need to edit the info from the preferences (either pasted from the Configuration section we just looked at or edited down from the export).
You’ll notice there, that it provides (what I think is) the answer to your question: Instead of the path that you have on KAP or KS, you are using the .zip file as the point of reference. It shows up as ${profile:location} with the subdirectories referenced from there.
At the moment I cannot verify this works because the app doesn’t like my self-signed certificates. But one can use Postman and the API endpoint for the Execution Context to upload the .zip file as a new (or modified) Customization Profile.
thank you for the quick answer.
You got exactly what I mean.
I looked at the knime business hub admin guide, the bigquery guide - but not in the db_extension guide…my first mistake.
It is only written to use a specific JAR file.
Screenshots from the KAP are always nice, but instead of just one JAR file, it is possible to use a whole folder within KAP. Is the same possible on KBH? If yes, that would be great to read in the documentation as well; or in case of bigquery, since it already has it’s own documentation anyway, please put a working example in there where any KBH admin simply needs to adjust their own path’s.
The problem with the simba drivers is: It is not just one .JAR file that we need to make it work, it is many of them, so essentially the whole folder. (please correct me, if just one JAR file is enough (which one?), I cannot see it on your screenshot, because it is cut off on the right side)
Let’s go into detail:
I create a zip file, for example named “hub_pref_incl_bq.zip” and the contents are:
One folder containing all the simba driver files and one epf with these lines:
I have not had time to test, and might not within the next day or so.
My understanding is that the only difference between KAP, KS, KBH is that there is no absolute path for the assets, so instead you use ${profile: location} and the subdirectory/filename. I don’t know that it worked for you with just the directory on the other products, but if it did, I would think it would work the same with KBH. The configuration of my KAP has the directory in that line.
indicates that you need the name of the .jar you’re actually using.
Though you don’t have a dev environment in which to test, you can create a test executor and try it there. You won’t need to restart your main executor until you’ve tested that the respective nodes work in test workflows.
At the moment, none of the custom drivers are working on your KBH, so even if you upload a Customization Profile, apply it, and find it doesn’t work, then you haven’t lost much. You’ll just update the .epf file to point to the .jar for each resource and apply that instead. Then when you are confident on what will work you can do the same on your production executor.
Putting that epf and the folder into a zip file does it:
With only one execution context, I could not really test it, but with the server workflow it is not a problem to apply/detach/delete any custom profiles.
One more thing to mention:
On windows the filepaths usually contain “\\”, that needs to be changed to “/” for any Linux based systems for the KBH.
Thank you for your support, @llepome .
I hope this helps anyone who faces the same problem.
