Performance expectation, Future performance of my algorithm

KNIME Analytics Platform
1

Hi everyone!
I’m learning regression and trying regression datasets.

image

I built following simple model

image

Now I need to estimate the future performance of algorithm (Question 1). How do I do that in Knime.?
Also what would be performance expectation (Question 2)? How can I be sure that the estimation is close to true performance(Question 3).?

2

Hi,
Question 1:you can use the scorer node on the data you reserved on the second port to test.
Question 2: The statistics you find as output on the scorer node answers this question.
Question 3: If the data you used to train/test is a random sample of the data you’ll be using your model on in general it should perform similarly. Analyze the results and try to be sure you are not overfitting

1 Like
3

thanks iperez!

I don’t quite get your point. This is what I get after running the workflow.
image

How can I tell the future performance ? by using R^2 ? Did I understand correctly that R^2 is the answer ?

4

Hi. The different statistics should help you.
R² is a measure of the degree of fitness of your model to your data. It measures the percentage of the variation of y explained by your model.
The mean absolute error is the average of the absolute value of the differences between real and forecasted values.

Chapter 3 of https://www.statlearning.com/ explains the theory behind and should help you to interpret the output of the node

2 Likes
5

Sorry for asking more. I don’t have access to this book. Can you please explain in simple terms how can I answer all these by using my simple model?
Thanks!

6

HI. The book is free to download in the link sent . If you check 3.1.3 you’ll have the basis to answer your question.

2 Likes
7

There is no need to use Normalizer Apply after the Normalizer node. The apply is for your test set.

1 Like
8

Sorry. Can you elaborate it a little bit? Thanks

9

Thanks dude. Let me read. I will post my findings.

10

Here’s a recent blog post from our Evangelism team covering much the same ground:

https://towardsdatascience.com/numeric-scoring-metrics-acd3896c5eff

2 Likes
11

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.