Anomaly Detection

Hi all,

Is it possible to do anomaly detection for my project?
Let me briefly explain the project: I am a bank and have customers doing business with companies. My customers are exporters and send their products to companies scattered all over the world. I have the following information about the individual transactions:
Transactions_ID (unique), transaction date, exporter (my customer), importer, port of origin and port of destination.
My customer ABC Ltd. often concludes transactions with a company called XYZ Ltd. which are carried out via the same port of origin and port of destination. However, it can happen that my customer completes another transaction with the company XYZ Ltd. during the next transaction, where the port of origin remains the same but where the port of destination changes. For example, my customer sends the goods from Germany (port of Origin) to his contractual partner XYZ Ltd. in Taiwan (Port of Destination). In the next transaction, however, he sends the goods to the same contractual partner from Germany to China. My machine learning algorithm should perceive this business as an anomaly and give me a message. But if my customer sends more than 2 times his goods from Germany to China, then my machine learning algorithm should not be able to see this business as an anomaly anymore, i.e. it should learn, so not rule based.

It is a predictive model in the case where my algorithm has to predict whether a transaction has an anomaly or not. I would have to train the model, but is it possible to say that I only give the model the data and then automatically learn it from the data. I have over 300000 records, so transactions, on the basis of which I would like automatic “patterns” or “Cluster” to be created, and then find in these patterns anomalies.

Is this possible in KNIME? If so, is there already a similar example based on which I can continue working?

Thank you very much in advance.

Friendly greetings
canan

Hi there!

Anomaly detection is possible in KNIME for sure and example workflows you can find on KNIME Hub. There are a lot examples of anomaly detection. Check them out.

I can point out Fraud Detection example which can be a good starting point :wink:

To my knowledge this would require to periodically train and deploy your model.

Br,
Ivan

1 Like

Hi Ivan,

thank you for your response, I will have a look on the examples.

What does [quote=“ipazin, post:2, topic:14798”]
To my knowledge this would require to periodically train and deploy your model
[/quote] mean?

Does this mean, that I have to train it manually or is it possible, that my algorithm automatically learns it itself?

Cheers,
Canan

Hi Canan!

If you automate it you do not have to do it manually :smiley:

Br,
Ivan

1 Like

Hi Ivan,

thank you very much. When I am done with my workflow is it possible then to show it to you?

Kind regards,
Canan

Hi Canan!

Sure! Post it here or you can start a new topic and some will review it for sure :wink:

Br,
Ivan

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.