Analyse Keras RNN results

[knimeoutjie]

Dear KNIME Community,

I have trained a Keras LSTM Neural Network and my in and out of sample accuracy is around 70%. I am pretty happy with that figure since I expected lower results.

Question: How do I analyse my results? How do I see which actual training/validation samples predicts lower than average?..and which samples did really badly? In general I want to do a proper analysis and understand what happened for which sample.

I would appreciate any help on this matter,

Thanks,

Leon

[lcollia]

Hi Leon,

I think your question is not related to Knime, so I am not sure you will have a lot of answers here.

However, in your case the first thing is surely to test your model on independent data sets. You can cluster your test set ad see if one cluster is answering better than another.
ofc you can test a lot of differents hyperparameters and try to understand which parameter has the biggest influence.
Search in Google/Qwant for “RNN analysis” and you will find a lot of examples which could give you hints.

Cheers, Lionel

[ScottF]

Hi @knimeoutjie -

This is a bit hard to answer without more information on your use case. Do you have a sample workflow you can provide (or maybe even just screenshots, if your workflow is confidential)?

How is your input structured, and generally what are you trying to achieve? Is an LSTM even the best possible approach to begin with?

Any additional detail or context you can provide would be helpful.

[knimeoutjie]

Thanks Everyone,

Apologies, it was a bit of a stupid question.
I used the Keras Network Executor to do the predictions and then compared that with the real output.

Thanks,
Leon