Getting preferences in multithreaded workflow fails

I have some problems when retrieving preferences in a multithreaded workflow.  I implemented the access to the preference store like in org.knime.workbench.ui.preferences.MainPreferencePage, e.g.

KNIMECorePlugin.getDefault().getPreferenceStore()

Now I have multiple instances of the same node that need to access some values from the preference store.  When running the workflow, often (not reproducible always) the node instances that were started first fail, because they get empty values from the preference store.  The other node instances succeed.

With 2 workarounds the workflow will always succeed:

  1. Switch off multithreading with -Dorg.knime.core.maxThreads=1
  2. Do a while loop, until the fetched preference values are not empty, with a sleep of some 100ms in between

 So I'm wondering if I miss some essential preference store initialization step?

In http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Fguide%2Fruntime_preferences.htm the following is described: "The org.eclipse.core.runtime.preferences package provides infrastructure for storing a plug-in's preferences."

Is this a completely different preferences system or just kind of a different "view"/approach of getting the same information?

Hmm, maybe I found the solution all by myself.  Do I have to derive my plugin from KNIMECorePlugin?  It's currently derived from AbstractUIPlugin.

We've got this working ok by ensuring that the plugin extends AbstractUIPlugin:

public class MyPlugin extends AbstractUIPlugin {
	// The shared instance.
	private static MyPlugin plugin;
	public static final String PLUGIN_ID = "my.plugin.id.or.whatever.yours.is";

	

	/**
	 * The constructor.
	 */
	public MyPlugin() {
		super();
		plugin = this;
	}

	
	@Override
	public void start(final BundleContext context) throws Exception {
		super.start(context);
		plugin = this;
	}

	
	@Override
	public void stop(final BundleContext context) throws Exception {
		super.stop(context);
		plugin = null;
	}

	
	public static MyPlugin getDefault() {
		return plugin;
	}

}

Then a prefereince initilalizer extending AbstractPreferenceInitialiser:

public class MyPreferenceInitializer extends
		AbstractPreferenceInitializer {

	@Override
	public void initializeDefaultPreferences() {
		MyPreferencePage.initializeDefaultPreferences();
	}

}

Then in the preference page class:

public class MyPreferencePage extends FieldEditorPreferencePage
		implements IWorkbenchPreferencePage {

...
...
...
/**
	 * 
	 */
	public static void initializeDefaultPreferences() {
		if (!m_defaultInitialised) {
			m_defaultInitialised = true;
			try {
				MyPlugin plugin = MyPlugin
						.getDefault();
				if (plugin != null) {
					final IPreferenceStore prefStore = plugin
							.getPreferenceStore();
					prefStore.setDefault(PREF_NAME_1,
							DEFAULT_VAL_1);
					prefStore.setDefault(PREF_NAME_2,
							DEFAULT_VAL_2);

					//etc etc etc
				}
			} catch (Exception e) {
				// TODO: handle exception
			}
		}
	}

 

You also need to add org.eclipse.ui.preferencePages and org.eclipse.core.runtime.preferences to the extensions tab of the plugin.xml

Steve

Hmm, still encountering the same problem.  I checked

  • Plugin extends AbstractUIPlugin
    • PLUGIN_ID is set
  • PreferenceInitializer extends AbstractPreferenceInitializer
    • Calls PreferencePage.initializeDefaultPreferences()
  • PreferencePage extends FieldEditorPreferencePage
    • Implements initializeDefaultPreferences()
  • Extensions to
    • org.eclipse.ui.preferencePages
    • org.eclipse.core.runtime.preferences

I looked for some similar code.  I found e.g. org.knime.chem.preferences.ChemTypesPreferencePage, which doesn't call any PreferenceInitializer.  But there is also no org.eclipse.core.runtime.preferences extension defined; maybe not a good example to compare with.

Hi Steve

Now I managed to reproduce this behavior with a dummy node that was generated by the knime node wizard and the "preference page" extension wizard.  What I did:

  1. Open knime sdk 2.11.3
  2. File > New > Other > "Create a new KNIME Node-Extension" named "ThreadedPreferencesExample"
  3. Check box "Include sample code in generated classes"
  4. Change base class of ThreadedPreferencesExampleNodePlugin to AbstractUIPlugin
  5. Remove inPort in ThreadedPreferencesExampleNodeFactory.xml
  6. Change constructor of ThreadedPreferencesExampleNodeModel to super(0, 1)
  7. Double-click file plugin.xml in navigator window
    1. A special plugin editor opens with sub-tabs
    2. Click "Extensions" tab
    3. Click "Add..."
    4. In dialog, click "Extension Wizards"
    5. Double-click "Preference Page" extension template
    6. Click "Finish"
  8. Add member variable "preferences" to ThreadedPreferencesExampleNodeModel:
    private final IPreferenceStore preferences =
        		ThreadedPreferencesExampleNodePlugin.getDefault().getPreferenceStore();
  9. Add the following code at the beginning of the "execute()" method:
    String value = preferences.getString(PreferenceConstants.P_STRING);
    if (value == null || value.isEmpty()) {
            throw new Exception("Could not determine preferences value");
    }

Now the crucial part of the whole story, I think.  In case that determining and setting of the default preferences lasts some time, in the range of some 0.01 - 0.1 seconds, retrieving preference values concurrently in multiple threads may lead to problems!  In PreferenceInitializer.initializeDefaultPreferences(), before setting the defaults, I added a sleep for 500 ms to simulate this extra time to set the defaults:

IPreferenceStore store = ThreadedPreferencesExampleNodePlugin.getDefault().getPreferenceStore();
try {
	Thread.sleep(500);
} catch (InterruptedException e) {
	e.printStackTrace();
}
store.setDefault(PreferenceConstants.P_BOOLEAN, true);
:
:

When I define a workflow with a couple of these nodes, all but one workers get an invalid preference value, and only the last worker runs correctly without exception.  I tried e.g. with 9 nodes, and I get the following output on the console:

ERROR     KNIME-Worker-7 ThreadedPreferencesExample     Execute failed: Could not determine preferences value
ERROR     KNIME-Worker-6 ThreadedPreferencesExample     Execute failed: Could not determine preferences value
ERROR     KNIME-Worker-5 ThreadedPreferencesExample     Execute failed: Could not determine preferences value
ERROR     KNIME-Worker-4 ThreadedPreferencesExample     Execute failed: Could not determine preferences value
ERROR     KNIME-Worker-3 ThreadedPreferencesExample     Execute failed: Could not determine preferences value
ERROR     KNIME-Worker-2 ThreadedPreferencesExample     Execute failed: Could not determine preferences value
ERROR     KNIME-Worker-1 ThreadedPreferencesExample     Execute failed: Could not determine preferences value
ERROR     KNIME-Worker-0 ThreadedPreferencesExample     Execute failed: Could not determine preferences value

 

Can you help me with this issue?  Do you need more input data?  I can attach the whole source bundle of this example, if this helps.

Thanks, Frank

Shouldn't simply synchronizing access to the preference store solve your issue? E.g.

synchronized (preferences) {
	String value = preferences.getString(PreferenceConstants.P_STRING);
	if (value == null || value.isEmpty()) {
		throw new Exception("Could not determine preferences value");
	}
}

 

1 Like

Hi Philipp,

great, you made my day.  Thank you so much.

I also found the function for this kind of "lazy evaluation": org.eclipse.core.internal.preferences.EclipsePreferences.create().  If I get it right, on first call it creates an empty node and initiates loading of the corresponding preferences, and on further calls it returns this new node, no matter whether loading has finished or not.

I assume that there is a good reason, why eclipse returns preference nodes that are still loading.  But if you don't know ...

Finally I decided not to use thread synchronization but instead an initial call for the preference store in the start() method of the plugin's activator.  But one has to take care, a call to getPreferenceStore() is not sufficient, you have to access the preference node, e.g.

getDefault().getPreferenceStore().getString("dummy");

Best, Frank

1 Like