Hello everybody, I am using Knime again and I have two "little" problems:
- The first one is about visualizate some windows of the program. I use Ubuntu 7.1 and Knime 1.3.3. The problem is when I use Arff reader for example, I enter the location of the arff file, and the node is executed correctly, but I would like see the execution (the data) using the option "Data outport 0: Data from Arff" but when the new window appears, It has nothing in it, It is just a completely grey window. Any idea?
- The second one thing is a doubt about something we need to do. We have 1800 files more or less, and we would like using Knime to do something like: Create a model of the first file using neuronal networks for example, and to compare it with the others 1799 files. After this, do the same operation with the second file and so on. Finally, we want a matrix with the outcome of the 1800x1800 comparations executed, Is it possible??? It ought to be as automatic as possible.
- The second one thing is a doubt about something we need to do. We have 1800 files more or less, and we would like using Knime to do something like: Create a model of the first file using neuronal networks for example, and to compare it with the others 1799 files. After this, do the same operation with the second file and so on. Finally, we want a matrix with the outcome of the 1800x1800 comparations executed, Is it possible??? It ought to be as automatic as possible.
Dear snyder,
we are not completely sure what you try to do. To be honest, we had a quick get-together and discussed what you might mean with "compare it [a NN model] with the other files" but then we didn't reach consensus, though we believe you want to use the learnt model to predict the data from the other files? Is that correct?
If so, you could use the batch executor and repeat the execution of the flow with different input settings in each iteration. If I'm correct, you will have a very simple workflow consisting of a two file (or arff) reader nodes, a learner node and a predictor (and possibly a scorer and a csv writer?).
But before talking about complicated bash scripts that launch the batch executor, I'd like to wait for your acknowledgement.
Regards
Bernd
PS: Things like that should be doable without the batch executor once we have the workflow variables available (which we will have in 2.0).
Regarding the ARFF reader problem Could you tell a bit more about what you mean by "completely gray"? In the port view there are several tabs the "DataTable", the "DataTableSpec" and the "DataColumnProperties" tab. If, after the node successfully executed, you look at the "DataTableSpec" tab, is that completely emtpy? Or does it show the columns (attributes) defined in the arff file? If that's empty, it means the node can't even read the arrf header - which is new.
If the column headers are there, and the "DataTable" tab is empty, it couldn't read the data section. There is one known bug with the arff reader Everything after a '@' in a data line is ignored. I.e. if your data in the first column starts with a @ everything will be empty (but I guess it would introduce a missing value cell...).
Anyway, maybe you could also send me your arff file (maybe only containing the first few lines of the data section)? You could send it to peter.ohl at knime.org.
Thanks!
- Peter.
You are right, my explanation was quite bad...but you guess correctly, what I want is use the learnt model, obtain a model and then apply it to the other data to compare the outcomes. My problem was that I did not know if was possible read from differents files in each iteration.
In terms about the grey window problem, I obtain a empty window without tabs (neither DataTable, nor DataTableSpec...). I am using the classical arff file about the weather, so I think is a interface problem.
I gave it a try and attach a bash script which does the job. I noticed that it takes quite long (between 5-10 seconds) to start KNIME in each iteration (it performs the whole eclipse bootstrap procedure...) so applying that flow to 1800 * 1799 different inputs takes about 9000 hours, which is more than a year. (If you like you can do a race between KNIME 1.3.3 (starting the job rightaway) and 2.0 (starting it as soon as it is available) - I bet the latter one wins.)
Anyway: Here is the script that I used:
INPUT="$HOME/temp/snyder/input";
OUTPUT="$HOME/temp/snyder/output";
KNIME="$HOME/temp/knime_1.3.3";
cd $INPUT;
for TRAIN in * ; do
for TEST in * ; do
test $TRAIN = $TEST && continue;
echo "$TRAIN - $TEST";
$KNIME/knime.sh -nosplash\
-application org.knime.product.KNIME_BATCH_APPLICATION\
-workflowDir=$KNIME/workspace/learnerPredictorFlow -nosave\
-option=1,FileURL,"file:$INPUT/$TRAIN",String\
-option=2,FileURL,"file:$INPUT/$TEST",String\
-option=5,filename,"$OUTPUT/${TRAIN}-${TEST}.csv",String
done;
done;
The sample flow has two arff reader nodes (id 1 - train and id 2 - test) and a csv writer (node 5) that writes the output to a new file.
Regarding the output table view: Does it trigger a repaint if you manually resize the window? What window manager do you use?
Do not worry about the scritp, I have already resolved it, but I have had to use another program..
Regarding to the visual problem...It is fixed too I think, I am using gnome with advanced visual effects (compiz), I disabled these effects and the problem has gone, so maybe it was a compatibility problem.
Good to know that compiz causes problems with the UI. There is not much we can do about it, it's apparently a general problem with java/compiz. A google-search brought up interesting posts, some user have reported that compiz crashes entire java applications; others say that installing a package called "libXp" and setting an envrionment variable "export AWT_TOOLKIT=MToolkit" solve the problem, see for instance the java bug report:http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6429775 - we can expect "less problems" in Java 1.6 (which KNIME 2.0 will be based on).