/biz/ - Business & Finance

Make discord fren

What does it trade?
How long have you been studying ML?

>>20131739
nope, fren
I am not instagram influencer

>>20131803
>How long have you been studying ML?
2 month at good pace with lots of book

>What does it trade ?
BTCUSD scalp at even tp/sl distance from entry price

Also, I have not tested extensively the algorithm, it's just cross validation scores from the training test, not the actual test. But its classifies quite well I'm surprised.
When you have a lot of features (indicators), Ridge Regressors seems to outperform anything.
I'm testing decision tree RN.

Pic related is my homemade trading bot, I have not run it in a while.
Actually someone paid me to do it, but back in the days I had not extensive time to work on algorithms, I was just sprinting on the framework and ui dev.

What does the architecture look like? RNN? LSTM?

What does the test data look like? Has to be more than just labels and data right? Doesn't the bot have to choose when to sell/buy in real time? What does this "precision" metric actually mean?

where do I start to learn this?
t. electrician

>>20132113
If you don't know maths it will sometimes be difficult, but I read:
Hands On Machine Learning by someone at Google.
Use it with either google collaboratory or the new Jupyter Lab, and read scikit-learn documentation and user manual as much as you can.

>>20132106
>What does the architecture look like? RNN? LSTM?
God, it's machine learning, not deep learning.
Deep Learning is very costly in processing power, and it would often overfit. None of this memetic shit is required for such serious problems. Here I'm just using Ridge Polynomial Regressor (Linear Regressor witch minimises the sum of features total weight sum, and that extends features with polynomial features module). Also data is scaled.
I have put 77 features in. But with poly augmentation (2nd degree), it makes many thousands.

ML is based on retrofitting so if you use retro data it'll always perform above average. Come back when can sustain 1 year of real gains.

>>20132247
also come back when we're not in a ridiculous bull market where any buy & holder is making much more than a trader or a trading algo (who do you think made more money, those who held TSLA or those who've been trading it up and down?)

As I expected, decision tree are based as well.

>>20131679
CS fag here,
where do I get good on this? Professor is just teaching us useless javascript shit that I will never use. I get intimidated by the thought of ML and how to test the trade bot (since I have very limited capital rn).
What are your algos and how did you approach the problem? I would try and feed it data based off indicators and choose the best position

>>20132247
>ML is based on retrofitting so if you use retro data it'll always perform above average. Come back when can sustain 1 year of real gains.
You're making a real point there. Maybe it will fail in real time.

>>20132106
>What does this "precision" metric actually mean?
https://en.wikipedia.org/wiki/Precision_and_recall

>>20132288
Fuck I interpreted it wrong, it's not based at all it's fucking imprecise. I hope I will have better results with random forest.

>>20132305
If you want I can share the notebook from where I genrated the dataset on Github.
I first parsed the data, aggregated in bigger candles, removed the parts where there were holes, computed indicators, and generated labels for different tp/sl setups.

>>20132367
sure! Would be interesting to see. I haven't taken an ML course yet so would be cool to dip my toes in

>>20132347
>What does this "precision" metric actually mean?
Let me elaborate my question.
Does a "success" mean you bought and then sold at a higher price? Or is it trying to predict a price X seconds/minutes into the future? What is the bot actually doing that can be considered a success or failure?

>>20132410
There is 3 parameters: tp_ratio, sl_ratio, and max_distance.
If from candle X, we hit take profit target before hitting stop loss target or reach X+max_distance, it's a success.
Here's the code to generate the labels.
I did it for a lots of parameters, but having even tp and sl in % seems to be nice since it's easy to understand the results without the bias of having 90% of success and 10% of failures.

>>20132305
You could always trade "on paper" before committing a bot to an exchange. Some exchanges let you trade with play money before using real money.

>>20132474
Does X+max_distance just mean max_distance amout of time passes without hitting either stop or take_profit?

Why would that be a success?

>>20132604
If Im understanding this correctly, I would be careful that your 80+% success rate isn't majorly inflated by the price simply not moving enough to make any difference in your profits.

>>20132634
The test I have made revealed it was not due to the price not moving.
And the poor performances close to 50% of the others Linear Regressors or SVM seems to indicate it's not the case.
In my tests, algo has 2 weeks to do 2%, it's far from enought.
Also I checked with graph (pic related) all the trading labels to see if they were ok so I would have seen (the pic related is not the same label parameters as the OP results).

>>20132687
What is the actual output of the bot?
Something like:
Buy Qty (min 0)
Stop Gap Price
Take Profits Price
?

You mentioned 72 features, did you manually code these in, or were they discovered during the learning process? I'm not 100% keen on when ML ends and DeepL starts in this regard

>>20132784
Also, what is the input of the system? Every candle (a candle is what, 4 data points?) over a certain period of time? What's that period?

>>20132784
>did you manually code these in, or were they discovered during the learning process?
Check in the link for details about indicators. You have the entire source codes.

>>20132397
>sure! Would be interesting to see. I haven't taken an ML course yet so would be cool to dip my toes in
Here it is, for the ML details, just read the notebooks from this repo:
https://github.com/ageron/handson-ml2

https://github.com/QuentinFAIDIDE/Crypto-Trading-Classification-ML-Datasets

>>20132837
see here
>>20132842

>>20132837
https://github.com/QuentinFAIDIDE/Crypto-Trading-Classification-ML-Datasets/blob/master/GenerateLabels.ipynb
Nice, you can even real the entire notebook from github with source code and graphs !

>>20132837
>what is the input of the system
All the indicators are generated and studied here:
https://github.com/QuentinFAIDIDE/Crypto-Trading-Classification-ML-Datasets/blob/master/GenerateIndicators.ipynb

>>20131927
>>20131679

This is an obvious bullshit. Trading cannot be automated in principle (since market is evolved faster than your fucking model which always reflect the past state). Otherwise wallstreet offices will be empty. However, that is not the case.

Ignore this bullshit.

/thread

>>20131679
Stop thinking about it. Literally all you have to do.

Buy some shit. Do *NOTHING* for one year. If you become euphoric, sell 50% and then wait 3-6 months until you find a new thing. Repeat.

Start with EPIC under a $500k cap.

>>20132932
>I am butthurt, the post.
Lots of people are doing automated trading bots, anon. I'm not saying I have the perfect one, I'm merely discussing classification tasks and datasets with other anons (though OP was bit on a bait tone you jumped right in).
If it can calm you down, I personally know from the guys who paid me people who are succesfully running automated trading bots.
You can also find lots of book of people who opened their algo trading firms.

>>20132948
>Stop thinking about it. Literally all you have to do.
True if you have proper capital and do not care loosing a few k$
Still requires analysis though.

>>20131679
how much money does that thing make you?

>>20132965

OK, lets talk pseudo-smart (because you are really pseudo) - applying ML classificators to a partially observable stochastic environment is no differ from fucking astrology and the results are "no worse than random".

Learn something other than python lmao

>>20131679
>only right 80% of the time
>not right literally every day of every year, 100% of the time for an easy quintillion times your money
Though in all seriousness, if you think you have something which is right 80% of the time after learning ML for 2 months you have another thing coming.

Put your money and trade it live, I dare you. pussa

>>20133010
Literally zero since I have not run it in real time yet as I said above.

>>20133016
>Learn something other than python lmao
What do you know about the languages I use, brainlet ?
Your post is equivalent to saying: "I think that I'm smart and I need to remind everyone I do low level languages".
Though you forgot the purpose of having different levels of abstractions for languages: some are really fast to implement and secure, while other are less secure, slower to implements, but runs faster.

>to a partially observable stochastic environment
Where did you get that idea from ? It's bullshit. You don't even know what you're talking about and throw random buzzwords

Also I'm not saying I have perfect ml algo, I'm just opening a thread to discuss about it, and posts like >>20132247 >>20132270 are good while yours are a bit cancerous t.b.h

.

>>20133045
>Put your money and trade it live, I dare you. pussa
I will first spend time optimizing and trying ensemble learning things.
Also I need to do backtest on the timeframes I removed from my dataset.
Only after that I will trade, but not with my money, I will use the little testing sum of money of my friend who paid me to do latter bot put me up on one of his account.

>>20132872
lol @ these indicators. this is babby's first ML code and in no way would you make money doing this. but good on you for getting a good discussion going in this shithole forum.

>>20133082
>You don't even know what you're talking about and throw random buzzwords

lol, so you didn't get the argument. ok. I will try to spoonfeed

classification algorithms are applicable only to a discrete, deterministic environments, which are stable and finite. language is such an environment, pictures, obviously, because cats have stable traits which does not change at all, etc. I hope you get the idea.

markets have no such stable traits, so each and any classificator will be plain wrong. it will capture noise and will produce a noise, no matter how much data you feed to it. no stable traits - no ML classification possible.

it is that simple lmao. to have pattern-recognition you have to have patterns. real, not abstract and imaginary.

you code will work, but your results will be noisy and inconsistent.

>>20133137
>this is babby's first ML code
You've not seen any of the actual ML code, bruh. I do lots of pre-processing after that, I do not feed that directly into the model.
I have already my bot ready to try, so hopefully in a few weeks I will prove you wrong.
The results from cross validation are there though, classification works pretty well.

Also, I have fed as much indicators purposedly for sorting them out as "usefull" or "bullshit". I've fed everything I could to be honest.
What are good indicators for ml to you ?

>>20133178
I know what stochastic means thank you, I'm from a math background, but your theory extrapolates a lot to me.

>no stable traits
Maybe if you were a bit digging further, you could argue that the results are get are from neighbours data which share the same features.
That would be a good point to make.
But it would also be wrong, because results on the testing dataset part indicates there is indeed stable traits.

>>20133102
Yeah, I know you know. I was goofing
I just wanted to show my funny picture.

>I'm just opening a thread to discuss about it
I've been algo trading this year.
None of my machine learning based algos make money. So I don't have any money in them.

What worked for me are algos based on gambling theory and game theory (I'm intentionally being vague) trying to capture very basic market behaviors like momentum and mean reversion.
These algos don't need training like machine learning, but similarly you can pick parameters to fit them to data.
The biggest help of my ML knowledge was to use bagging to hedge specification error. Also cluster learning.

Thank you, you guys made me realize I cannot take random bits of potentially neighbours related data for the testing dataset.
I will take random batches at different point to prevent fitting on neigbours I separated into different purposed sets.

>>20133189
I mean RSI is fine and stuff. ADX and CCI are a couple of others. It's just that now, people are using CNNs with prices and indicators laid out as images. Regression models lead to losses (typically) unless you're on some kind of up streak already.

Again, I would find an exchange where you can use play money to test your model in the real world on unseen data before trying with your own personal money.

>>20133330
>Regression models lead to losses
Even on ridge to cancel noise, and with polynomial feature expanding ? I read otherwise.

>I would find an exchange where you can use play money to test
Someone left me money to test out algos when I made the bot I mentioned earlier for him, he's rich af because of early crypto trading, he will prolly allow me to test it on his money he left me on some account one year ago. I will ask him.
That would kill me to have to port my bot to some other platform, I though I was done with framework dev but I guess we're never done with that kek.

Nice introductory level stuff

>>20133392
kek thank you
I'm really is at introductory level I admit it

>>20131927
>it's just cross validation scores from the training test, not the actual test
Are you aware of the concept of 'overfitting'? Usually ML algorithms can be made to work well on the training set but when they're used on new data which wasn't in the training set, their performance drops.

>>20133460
>Are you aware of the concept of 'over fitting'?
You never heard of cross validation, right ?
I can understand there is the possibility of over fitting due to related neighbors data lines being spread out in different batches (train / test / cross validation batches), but saying that cross validation score is invalid is outrageous.

>>20133381
Typically, yes. Try your model on unseen data for the next week and see what happens. As >>20133460 points out, you are probably overfitting.

Keep in mind that asset prices tend to be a random walk or geometric brownian motion, so it is difficult to build a model with robust predictive power.

>>20133515
Oh nevermind I'm retarded and wasn't thinking. In cross validation you only use a part of the 'training set' to do the actual training right?

>>20133522
>you are probably overfitting
Yeah I suspect spreading neighbors created a potential overfitt. Those results are juste too good.
But though, SVM and LinearRegressor would get terrible scores so I don't know if we can say that the model is bad and we're necessarily over-fitting yet.

>>20133535
>In cross validation you only use a part of the 'training set' to do the actual training right?
No you're not retarding you're just on a short attention span Taiwanese board, fren.
Yeah, it uses like N different batches and train on N-1, and validates on the remaining one, things like that.
https://scikit-learn.org/stable/modules/cross_validation.html#multimetric-cross-validation

But as I said I probably introduced overfitting with the spreading neighbors things.

>>20133515
Also, be aware of implicit overfitting.

>>20131679
>live psychological rollercoaster
nope, i trade with what i can afford to lose and practice mediation in the process.
I am zen.

>>20134165
This is the correct answer.

Why bother when there’s cryptohopper and 3commas? Seems like reinventingthe wheel to me *sip*

BTC doesn't have enough variance to make real gains with. You want to find coins that make at least 1% changes between peaks and troughs every few hours. I'm working on something similar but I don't think I'll be using ML as I don't see the benefit of doing so right now. I'm essentially looking for small gains that I can compound. 1% gains 4 times a day is still 32%/week and I think I can achieve more entries than that if I can identify more eligible coins to trade between.

>>20134165
>i trade with what i can afford to lose and practice mediation in the process.
Extremely based

>>20134314
Using 1% for your positions will eat up to 30% of your profit with fees (For Bitfinex with a taker fee on entrance and maker fee on exit).I personally don't like it and prefer waiting until 2 or 3% to have more like 10% and less of fees.
Btw this is outrageously high compared to forex or stock market.
Also, when opening a position on crypto market, I always mirror it on 4-6 shitcoins that follow btc trend but with higher volatility. So I can sometimes fill 5-6% during the day.
Just like at the screenshot how much position I closed in the last five days in the topbar of the bot:
>>20131927
Every tick a closed position.
Btw this is hand monitored by me for trade autorisations, it not at all machine learning in play in this screenshot.

>>20134314
>BTC doesn't have enough variance to make real gains with.
Have you heard of leverage, you dumbilly dumb ?

Or just get gunbot

post portfolio larper

>>20131927
are you using gnu linux sirs

>>20132474
>When you hear you should comment your code
Holy fookin shit lol, babbys first program? Even your code has reddit spacing.

>>20135599
>>20135731
>When you hear you should comment your code
>Holy fookin shit lol, babbys first program? Even your code has reddit spacing.
Fuck, some people are so dumb on this board, it's beyond hope.

>>20135827
You are commenting literally exactly what the line of code reads as most of the time, it's very rookie. Take it as constructive criticism. Don't believe me? Look at any source code from a Python project from legit companies, TF2 or Pytorch for example. You'll learn anon, don't worry.
t. FAANG engineer

>>20135945
maybe he designed that algorithm, wrote pseudocode comments to get the structure right, then filled in the actual code later? but i guess you don't design stuff, kid? you're more of a script kiddie than a developer right?

>>20135945
>rookie
It's been 10 years I've been coding either python, or C, or JS. I was writing code for FOSS software that was 1st on Mac Apple Store back in 2014, you are extrapolating a lot, anon, even though I am still young (23yo).
I just like to do the pseudocode in the comments before actually starting writing code on some important algorithms, doesn't mean anything, I do whatever I want.