One idea could be to use an H2 database (which KNIME supports). They come in two flavours. One is operated in-memory which should make access very fast. The downside would be you would have to load all your files into memory every time you fire up your computer. Also, you can store the tables in a single file like SQLite - from what I read H2 databases should be very effective in dealing with a lot of tables. Not sure what will happen when you feed them 10.000.
Also, H2 can have different schemes (Like PUBLIC in the example) that might help you organize your tables into categories. INFORMATION_SCHEMA helps you see and access which datasets you already have so you could use that to organize the access.
Of course, you would have to see what happens when you stress that concept. Under ‘Advanced’ you will find additional settings that might help you tweak the performance.
kn_example_h2_database.knar (50.2 KB)