Online Course: [L4-DL] Introduction to Deep Learning - Online

Deep learning is used successfully in many data science applications, such as image processing, text processing, and fraud detection. KNIME offers an integration to the Keras libraries for deep learning, combining the codeless ease of use of KNIME Analytics Platform with the extensive coverage of deep learning paradigms by the Keras libraries.

Though codeless, implementing deep learning networks still requires orientation to distinguish between the learning paradigms, the feedforward multilayer architectures, the networks for sequential data, the encoding required for text data, the convolutional layers for image data, and so on.

This course will provide you with the basic orientation in the world of deep learning and the skills to assemble, train, and apply different deep learning modules.

This is an instructor-led course consisting of four, 75 minute online sessions run by two KNIME data scientists. Each session has an exercise for you to complete at home. The course concludes with a 15- to 30-minute wrap up session.

Course Content:

  • Session 1: Classic Neural Networks and Introduction to KNIME Deep Learning Extensions
  • Session 2: Deep Learning Case Studies and Different Design Components
  • Session 3: Recurrent Neural Networks
  • Session 4: Convolutional Neural Networks
  • Session 5: Exercise walk though and final Q&A
3 Likes

Hi,

thank you everyone who attended our deep learning course last week!

Please find below the answers to the open questions from our Q&A session:

  1. Will I get the same results if I use a seed in the Keras Network Learner node?
    The seed in the Keras Network Learner node ensures that if the node is re-executed always the same subsets are used for the different epochs. That doesn’t mean automatically that you will get always the same results, as the there are more random variables used when training a neural network, which influence the trained network, e.g. the initialized weights for each layer.

  2. I have a GPU. How can I check whether or not it is used to train the network?
    To do so you can check the GPU utilisation. On windows this is possible via Performance Tab in the Task Manager. Another option is to use the command nvidia-smi in the console.

Cheers
Kathrin

Hi there,

We are using word2vec embeddings of product desriptions for product classification.

The word embeddings are used as imput parameters for a random forest. We started out with a bag-of-words vector, but word2vec improved the model accuracy significantly.

My reasoning says, that the modell performed better because the input vectors contain more information. The next logical step would be to enrich the vectors with context information to improve model performance even further.

My idea would be, to replace the convolutional layer with a transformer/attention layers (like used in BERT) to create a word embedding with added contextual information.

Do you have any idea how this could be implemented in KNIME?

1 Like

Hi @abodenmiller,

I like the idea!

For this task I would use the TF2 integration of KNIME Analytics Platform and would probably download, maybe fine tune, and apply a pretrained BERT model, optimally pertained on some data similar to your data. A great resource for pre-trained model is the TensorFlow Hub: TensorFlow Hub

These two example workflows show, how a BERT model can be applied using the TF2 integration and some Python code.

Important: In addition to the BERT Model you always need the according preprocessing, also available on the TensorFlow Hub

Cheers
Kathrin

5 Likes

Hi Kathrin,

I was able to get embeddings using the nodes from the first workflow (the Covid Dataset).
I would like to use a different BERT Model, because to me it looks like the Model used in the workflow is trained on Covid text data.
I cannot get it to work with different Models from the tf hub.
Another thing I´m struggling with is how to fine-tune the network with my own data and where to find the names of the different layers so I can choose the right output layer.

I would like to enrich the nlp classification model I talked about earlier with information from images.
The model uses word2vec embeddings as an input for a random forest classifier.
Instead of creating a seperate CNN for image classification, I would like to add embeddings from the product images as additional inputs to the existing classification model.
The reason I would like to try this, is because it is computationally very efficient and with our data, we got similar results to more complex architectures.
Which layers would I need to create a “image2vec” model?

Is Input-Convolution-Dense Layers enough or do I need max pooling and flatten layers as well?
Also the activation fuctions are giving me a small headache. ReLu creates a sparse vector with lot of 0s. I would go with Sigmoid or are there more obvious choices?

I would be grateful for any kind of help.

Cheers