I am working on understanding the innerworkings of an LSTM layer and I have a rather simple neural network implemented in Knime. It works just fine, but the issue is the output table has a few headers that I cannot decipher. I went into the node to request additional data after the network has trained and that output is where I am getting these headers. Specifically:
LSTM_<# indicating a specific LSTM layer>/TensorArrayReadV3:0_X
LSTM_<# indicating a specific LSTM layer>/while/Exit_3:0_X
LSTM_<# indicating a specific LSTM layer>/while/Exit_4:0_X
Where x is a number that is dependent on the network itself, but it isn’t important for this issue.
The documentation says the output is just an output table, but I cannot find documentation that indicates what exactly those headers are. I suspect they indicate the hidden state values, cell state values, and perhaps bias values, but I cannot confirm that or tell which is which.
I’ve done a bit of digging and have been unable to find what you’re asking about, so I’ve asked one of our deep learning experts to weigh in. Sorry for the delay here.
No worries! I’m grateful for whatever help I can get. To give a little more information, this is a of the architecture of the network where I am getting the data: . I can post the Excel spreadsheet output as well but I think it’s a bit too large to post. Please let me know if I can provide any more info.
the headers you see in the output are taken from the names of the operations that produce them (I presume in TensorFlow).
If you want to give them other names, you should be able to do so in the dialog of the Keras Network Executor.
The particular outputs you got seem to be the output of the LSTM as well as its two hidden state vectors.
Thanks for the solution @nemad! The only thing I cannot tell is which output associates with which hidden state. Would you happen to know which is which?