Keras Network Learner doesn't update model


I’m new to deep learning and I’m trying to get a baby LSTM-example running, but it seems that during learning, the parameters are not updated at all. This is seen by periodic accuracy results in the monitoring window of the node (presumably because of minibatch processing over several epochs).

I’m wondering what is the cause…

My implementation:

  • Network in DL Python Network Creator:
    model = Sequential()
    model.add(LSTM(20, input_shape = (1,20)))
    model.add(Dense(1, activation=‘sigmoid’))
  • Training with DL Keras Network Learner)
  • Input: Table with 500 samples à 20 columns plus label column
  • Keras uses local theano installation

Any help is highly appreciated!.

Hello ice-crunch,

one issue might be your input shape (1,20) because you specify that your sequences have length one with feature dimension 20. In this case an LSTM is probably not the best choice as it can never utilize its inner state.
Try switching to (20,1) which would mean that you have a sequence of length 20 with one feature per timestep.

If this doesn’t help I will need more information, e.g. which versions your Python, Keras and Theano are and in the best case you could also provide me your workflow, so that I can see if I can reproduce the problem on my machine.

Another thing I would try is switching to the TensorFlow backend if possible just to check if Theano could be the cause of the problem.

A final note on the periodic pattern you are witnessing in the progress view: You are completely right, it is caused by iterating over the table without shuffling it between epochs which is especially detrimental in case of small datasets like yours. However, we will add the option to shuffle the data before each epoch in a future release.