Help: Any clearly detailed/explained/outlined time-series tutorials? Price prediction/forecasting examples? Etc?

#1

This is my first time posting on a forum anywhere for anything, so please bare with me if the format of my post isn’t quite right.

I have only started learning how to use the KNIME analytics platform briefly during the past 6-7 months, basically just basic classification stuff, and selected a topic for my masters dissertation pertaining to use of the platform to formulate and exhibit the benefits of using machine learning algorithsm and predictive analytics to observe and forecast Bitcoin and Ethereum price trends.

To my mistake, I didn’t - and don’t currently - have anywhere near even a half-way understanding of how to properly build, execute and interpret the KNIME workflows/method/nodes/etc that I have been reading about annd working with these past months directly in relation to my project setup.

My advisor for my project has been on vacation from May and will be still until the week before I have to turn this in (end of September) and I have no other professionals to turn to (because everyone at my uni is on vacation and not answering emails - even the department head - I tried). I have finally decided to post here in hopes that maybe, albeit late and a last ditch effort, I might be pointed in a more productive direction than that I have been going in myself. I am here to beg the aid of the fine, intelligent KNIME community that I have been silently perusing for information and ideas in the background - until now!

I don’t know a thing about programming or coding (no experience with R, Python, Java/JS, etc.).
I have referenced and tried to test my data and process it with several KNIME examples available within the platform example hub (time series, MLP, GBTs, Neural Nets, etc etc so on) but in the end I don’t think I really understand what I’m doing. I get numbers, scores, stuff, cool. Looks all nice and dandy. Line plots seem to make sense, I guess. But in the end, I keep finding myself thinking “What the heck am I doing? What does this all mean? Why am I/why am I not getting these numbers?”

I am really interested in the whole time-series analysis method but I don’t understand some of it (like the lag columns - what are their purposes? Why do you want duplicates when concatenating in preprocessing?). --> this one specifically 02_Example_for_Predicting_Time_Series
If anyone could expand on that, it would be a huge help for me I think.

In general, classification stuff (Decision trees/ensemble learners) make enough sense to me (like predicting if something is a fruit or a vegetabel based on x data etc.), but numerical value prediction? What? I am really uncomfortable with it.

If anyone would like to take a bleak stab at correcting me or giving me any tips or hints. I can post or email you my paper and/or workflows so far. I am pretty embarassed to have extremely novice work critiqued on a forum such as this, but at this point I would 9000% be willing to withstand any shame I might be put to for my shoddy and unprofsesional work.

If anyone takes the time to read this long-winded and probably confusing query, then thank you already enough for that, since your time is precious. Regardless, I’ll keep plugging away to make something work. Even just linking me whatever examples from anywhere that you think might help open my eyes a little bit more is appreciated.

tl;dr I have no idea what I am doing with predicting/forecasting cryptocurrency price movements using KNIME because of my inexperience. Pleae advise.

Have a great day.
Kind Regards
-Angela S

1 Like

#2

Hi @AngelaMS23

Well you are in some kind of trouble, and now you expect some Harry Potter to do the magic.I hope it will happen to you.

So you want to do “some analysis” on predicting / forecasting cryptocurrency price movements.

  1. Start with a clear research question (explicit and one-dimensional formulated)
  2. Think about the data and how to collect it (derive additional features).
  3. Think about a method to answer your question. Why using time series?

To me your story is like you have to drive a real Formula 1 car and win the race without any real driving experience (and license), except from the hours spent on the PlayStation.

My suggestion is to invest time and look at other approaches and analysis on bitcoin price development that are available on the Internet e.g. Kaggle website on bitcoin historical data See how that matches with your research question and from there make a translation to KNIME (and yes that is easy. but not always straight forward and therefore time consuming)

In the end I think you should consider seriously to spent your time on analysis your more familiar with. But he, that’s only my opinion. Maybe there are real Harry Potters around this forum to help you in a much better way than I did.

Good luck,
gr. Hans

6 Likes

#3

@HansS

I like and have to admit sadly that I agree with your Formula 1/Playstation reference (although I am a PC gamer, myself).

I have so far only really referenced this project from Kaggle (and I do really like Kaggle):
Forecasting of Bitcoin Prices (time series, arima, R notebook)

I will certainly go through the link you provided. Thank you for that!

A Harry Potter situation would be amazing, but as you said, “magic,” which is an unlikely concept to make itself logically apparent in the modern, real-life setting.

The most I feel I can really hope for from this query is suggestions, ideas, advice - anything that’ll help me think beyond what I’ve already been thinking so I can get some stuff moving again in a productive direction.

Much gratitude for taking the time to respond to me.

Cheers :blush:
-Angela S

1 Like

#4

These things come to mind in addition to what @HansS said:

  • for a master thesis or smth I assume you need a well-defined question that you might have to define yourself. Ideally, your professor would provide you with one but if that is not the case you might have to be creative
  • one way could be to use the list of Kaggle Examples from the list and choose one that has well-structured data in it for Bitcoin and Ethereum (using R or Python since you will not find a generic KNIME example )
  • since you are not supposed to just copy an example I think it makes sense to add some sort of question
  • also because I am not sure if a pure time series prediction could give you future Bitcoin prices (from the look of the links people have tried that)
  • you might want to include a piece of additional information like
    ** do tweets/google searches about cryptocurrency influence the price of Bitcoin or Ethereum more?
    ** or the stock prices of Fortune 500 companies?
  • you might think about a question like: “is it easier to predict Bitcoin or Ethereum when using these 3 algorithms” (ARIMA, H2O.ai/GBM when converting the time variables into categories, Linear Regression)
  • your original contribution could be to set up such an example in KNIME (KNIME can actually run R and Python code)

Of course, I do not know what the acceptance criteria are for your thesis but typically you will have to write some sort of theoretical explanation of what you are researching and why you choose a certain method.

Anyway, I would suggest you discuss this with staff from your educational institution and also see if it is possible to get an extension of your deadline.

6 Likes

#5

@mlauber71,

Thank you for contributing this great list of things for me to consider moving forward.

I guess though I will reiterate I have just about 0 experience with any coding languages, so I don’t know much or how well I will be able to teach myself anything within the next month and a haf, but I will take your suggestions. I have been poking around plenty of articles related to my topic that make use of R especially.

As for sepaking with the staff of my educational institution, maybe because it is a private university in the EU (Spain), all the professors including the head of my department have all been unreachable as of about May due to Summer vacation releasing them from any contractual obligations with students until September 16th (approximately a week and a half before my thesis is due). As such I have been speaking a little here and there with my professors back in the US, but they all are pretty busy doing things themselves and haven’t given me much time or consideration. I have tried to email many potential resources, but to no avail .

Regards,
Angela S.

PS. If you or anyone wants to read it for your own speculation, here is the draft for my abstract which was approved by my direct thesis advisor prior to summer beginning.
x1_Abstract.docx (14.7 KB)

0 Likes

#6

Not really sure what to recommend next, I collected some papers that already tried some ideas mentioned. So you might have some literature to cite. And also someone on Kaggle has already collected twitter posts about Bitcoin.

I have not read the papers only flipped thru some. Generally speaking a problem with time series predictions is that you have the trend and often some seasonality (more calls on Monday than on Sunday), so if there is a pattern one can try and detect it. Problem is with external influences. They are hard to predict like some economic trouble or something. Some future events you can put within your time series model like:

  • you know when easter will be next year so you can model with what happened last easter (or summer vacations)
  • you know when you will launch you next big advertising campaign so you can take that into account
  • if you eg find a correlation between tweets and the weather lets assume. That ist great but maybe not that useful in the future because you do not know the exact weather. You could try and use something like the typical weather in May or June

Another idea could be to redo one of the attempts from the papers or Kaggle with KNIME so you move more in the direction of software and method comparison.

I would then mention three additional software/ideas that move away from strictly dealing with KNIME (sorry KNIME, will do it anyway):

If you want the ‘latest technologies’ with regards to trend analysis you might try your hand at a prediction comparison for Crypto between Prophet and H2O Driverless AI; but in any case, the time is quite short and I would assume you need to get a lot of things organized.


Predicting Bitcoin price fluctuation with Twitter sentiment analysis

Bitcoin Response to Twitter Sentiments
http://ceur-ws.org/Vol-2104/paper_199.pdf

Cryptocurrency Price Prediction Using Tweet Volumes and Sentiment Analysis
https://scholar.smu.edu/cgi/viewcontent.cgi?article=1039&context=datasciencereview

Forecasting Price of Cryptocurrencies Using Tweets Sentiment Analysis
https://www.researchgate.net/publication/328906736_Forecasting_Price_of_Cryptocurrencies_Using_Tweets_Sentiment_Analysis

Price Movement Prediction of Cryptocurrencies Using Sentiment Analysis and Machine Learning

https://www.mdpi.com/1099-4300/21/6/589/pdf

Bitcoin tweets from Kaggle

4 Likes

#7

@mlauber71

Holy heck you really pulled out a big bag of helpful and thought-provoking ideas together for me with this one. I got the email reply notification and immediately started perusing the links. So now I’ll leave a gigantic, bold-faced THANK YOU right here while I get lost in some of this stuff.

I had noticed some readings that noted the correlation of social media or news involvement, or also commodities/other economic factors (ie exchaneg rates, etc. global markets) and their influence on price movements but it all seemed really confusing for me at the time, so I only skimmed it. Besides your links, I’ll try to backtrack to those as well. Also previously, my advisor was pretty adamant that I use [specifically] hourly or by-the-minute historic price data (I get mine form the Cryptocompare free API-key) as my datasource. I’ll take a whack at it again and look beyond what I already have.

Thank you again (I am a “thank you” machine right now). If you think of anything else, no matter how insignificant you think it may be, feel free to toss your ideas my way. “The more the merrier” especially in situations of knowledge seeking, in my opinion.

Kindest regards,
-Angela S.

2 Likes

#8

I’m not sure if these count as tutorials, but I can recommend two master’s theses on time series that describe their work using KNIME, one of which included predictions of Bitcoin prices:

Good luck with your work!

2 Likes

#9

@MrSampson

Awesome! I’ll check these out. From what you’ve said, I have a feeling these will be quite informative. Thanks a bundle for the input! Have a

Best
-Angela S.

0 Likes