/sci/ - Science & Math

[Anonymous No.12141566]

I want to learn mathematics behind machine learning and deep learning. I have have knowledge of matrices but nothing solid beyond that.
Where should I start?

[Anonymous No.12141971]

>>12141566

[Anonymous No.12142065]

>>12141566
Probability theory, multivariable calculus, numerical methods, programming, data structures and algorithms

[Anonymous No.12142079]

>>12142065
Oh and linear algebra of course

[Anonymous No.12142122]

>>12141566
To understand the absolute basics:
calculus I -> calculus II -> real analysis + linear algebra -> functional analysis -> relevant applied functional analysis (i.e. machine learning theory)

To understand the learning algos:
calculus I -> calculus II -> calculus III -> diffeq + numerical diffeq-> linear programming + dynamic programming -> advanced numerical optimization

Then, To understand some more advanced models:
Measure theory -> probability theory -> PhD level Mathematical statistics -> probabilistic neural networks
calc III -> mathematics of signals & systems -> signal processing -> machine learning in signal processing
PhD level algos -> adversarial networks
etc

[Anonymous No.12142127]

>>12142122
in math terms, tell me what is a machine learning

[Anonymous No.12142138]

>>12141566
literally basic calculus, linear algebra, and basic probability

[Anonymous No.12142146]

>>12142127
Depends on the origin of model and the method of parameter optimization. For example linear regression is sometimes considered a machine learning technique: it is applied functional analysis (projection in abstract vector spaces) but 'learning' happens only theoretically while proving the exact closed form solution. A multilayer neural network is an application of a certain Hahn Banach theorem (you can find it on Rudin) that is used in appproximation theory (more functional analysis), and learning happens by numerical techniques that maximize (or minimize) an objective function

[Anonymous No.12142166]

>>12142122
dont listen to this fag

OP you just gotta learn calc and then read papers. write code and implement these models yourself. learn as you go

[Anonymous No.12142177]

>>12142166
This guy is why we have marketing grads claiming they know machine learning because they can call a fit function on python, and consequently that's why the field lost a lot of credibility in the past 2 years.

[Anonymous No.12142185]

>>12141566
Calculus (lots) and optimization will get you most of the way.
What you really need is a fuckton of statistics and probability theory.

[Anonymous No.12142758]

>>12142146
what a long definition
ML: Minimizing a loss function

[Anonymous No.12142952]

>>12141566
Calculus multivariate.
Linear algebra.
Probability theory, math stats (undergrad) - with the inclusion of bayesian stats.
Learn to analyze, implement algorithms (pseudocode etc).
Some results require Calculus of variations and functional analysis to justify (this can wait) . There is a lot of optimization involved as well as some information theory and graph theory, BUT: given that you have solid understanding of the above, decent ml textbooks will introduce the essential concepts and their usage can be then justified using your prior knowledge.
If you do not know something, try to learn as you go by or accept is as a given if you understand the motivation and perhaps some technique used is not known to you. You can fill in the details later- be mindfull of where you are not fully in the know though, and you will be just fine.

[Anonymous No.12142997]

>>12142177
>in the past 2 years.
cmon now

[Anonymous No.12143337]

>>12141566
Statistician here. All ML models (including NNs of course) I've encountered can be complex but break down to extremely easy math. You need to know what +, -, * and / mean. And some calculus and linear algebra. I learned all of that in school already.

[Anonymous No.12143850]

>>12141566
You should start by finding some other area of interest that's actually productive since so-called 'machine learning' is dead-end bullshit.

[Anonymous No.12144177]

>>12141566
https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf

[Anonymous No.12145897]

>>12142127
basically this
>>12142758
To expand slightly, ML is broad enough that a single definition (mathematically) doesn't quite exist, but we can define almost all ML as:
1) Defining an objective function (e.g., y = b_0 + x*b_1, where x and y are provided data and b_0/b_1 are unknown)
2) Defining a loss/cost function (e.g.,mean squared error or log-loss)
3)implementing an algorithm that minimizes the loss/cost (e.g., gradient descent)
And boom! You have ML.
>>12143337
Also big stats dude here (not my degree, but I publish almost exclusively ML papers) and I 100% agree. You need baby-math knowledge to understand most ML algorithms.
NN are actually the easiest to learn, imo, because they're so simple. You literally need high school math at best. Even when you get into the really crazy architectures, most of it is intuitive/easy on the single-function level.

[Anonymous No.12146020]

>>12142065
>programming / data structures
>math

[Anonymous No.12146026]

>>12141971
Did anyone read all this?
Why isn't there exams list with questions and excercies to check if you know all that?

[Anonymous No.12146384]

>>12146026
No. These charts are made by guys that like to download thousands of books and just skim through them

[Anonymous No.12146661]

>>12142166
>calc then read papers
>write code and implement these models yourself
this is stuff you do to implement popular models in the industry, not to do ML research, and much less to actually understand the math behind ML.