/sci/ - Science & Math

what's convoluted about stats? it's pretty straight forward. if you don't take the time to understand the math that's your problem.

>>9801666
>it's pretty straight forward
Bullshit, if you think that you've probably made even less progress than I have at really understanding from first principles how it works.
I'm talking about actual understanding of where the rules come from, not memorizing formulae.

>>9801666
>>9801685
For example try explaining why the probability density function of x is the reciprocal of the standard deviation multiplied by the square root of pi multiplied by Euler's number raised to the power of the inverse of the difference of x and the mean squared divided by 2 multiplied by the standard deviation squared.
And knowing how to get back to that is just what's required to begin to have an understanding of a normal distribution.

>>9801655
i don't know why it's so hairy but you're wrong about people having no understanding of its mountains of derivations

>>9801728
I said most people, not all people.
Talk to any random person who makes use of probability distributions in some way as an incidental part of their work and the smart money would be on them not really understanding where the formulae they're using really came from or why exactly they work.

>>9801655
Because statistics is the theory of how to forcefully wrangle data that doesn't fit with a nice theory into a form that's possible to glean useful information from. It's ugly for the same reasons that dog food tastes like shit, it's what you make of the bits that are left over once the nice bits have been taken out.

>>9801759
That makes a lot of sense, thanks.

>>9801759
ooooh yeah right

>>9801774
Are you expressing a realization that his answer explains a lot, or sarcasm because you don't agree with it?

>>9801655
It is because statistics has to be taught to all the plebs who don't know a lick about Fourier transform, measure theory, real analysis or functional analysis.

If you find statistics confusing, you might be in some kind of statistics for engineers bullshit course where they just cram formulas into your head. If you don't know that the normal distribution gives you an stable eigenfunction for the Fourier transform, you should reconsider your life choices.

>>9801715
have you heard of the central limit theorem?

>>9801793
>the normal distribution gives you an stable eigenfunction for the Fourier transform
That's not anywhere close to a derivation from first principles.

>>9801800
If you don't know that there are at least 3 ways to approach the central limit theorem from first principles, you are probably part of what I am talking about. One approach, by combinatorics, for instance, leads to free probability. Though that is another story.

>>9801810
You're deflecting.
My own ignorance is irrelevant to the point that no one here has even begun explaining from first principles where something as basic to statistics as the probability function for the normal distribution is specifically derived.
As in where each term comes from and what it actually means, not a general reference to a possible way someone else could do it.
You're really just proving the point in that if it's assumed for the sake of argument you personally have some understanding of how the concept emerges, even you as someone who understands it struggle to explain it from the ground up with specifics.

>>9801820
Actually, I told you about the subjects you had to learn. You just didn't listen. If you study the Fourier transform, you will know it is the most important bijective, isometric operation on the Hilbert space L^2, which is the most important space among the L^p space in real analysis. The central limit theorem -- the most important theorem in classical statistics that establishes the right form for the normal distribution -- requires the distribution to have finite variance, which is to say, it has to be in L^2. Fourier transform, as you will learn from Tao, is rotation in L^2, and much like the mean ergodic theorem, the L^2-averaged sum will converge towards the center axis of rotation, which is the normal distribution. And that is but one way to understand the normal distribution.

>>9801715
The [math]\frac{1}{\sigma\sqrt{2\pi}}[/math] is because you want the integral of F(x) to be 1, it's just a factor for normalization reasons, though in certain derivations it comes up naturally. The [math]x-\mu[/math] is just needed so that if you take the mean of F(x) it is equal to [math]\mu[/math], same with the [math]\frac{1}{\sigma^2}[/math]. Each of the factors are really just expressing properties of the pdf

>>9801655
The equations are the way they are because they don’t hold back in showing how every parameter has an effect on distribution. The methods themselves aren’t the worst (the mgf is literally just a Taylor series expansion and some calculus to find a general form), but the expressions don’t hold anything back.

>>9801842
No, like I said, still not dealing with the specifics of the actual function.
You didn't even mention a single term of the actual function let alone explain what exactly it's doing. You're outlining the general idea and passing the buck to yet another abstract framework, not the same thing.

>>9801875
Actually, I did. Now I understand what Buddha said about arguing with fools. You don't have to believe me now because there is no way for you to understand me now, but save what I say and look at it again after you have learned real analysis and Fourier transform. In fact, probably the first function you will learn in Fourier transform then is the Gaussian function, which is the cornerstone of Fourier inversion. Then you will know what I mean, instead of whatever you think I mean now. It's like speaking Greek to someone who hasn't learned Greek.
Also, it is not abstract, just that you haven't learned it yet. For instance, your not knowing what a Lorentz manifold is would not be Einstein's fault. It is just that you have to keep learning. The more you learn, the more you understand stuff. You can't expect to understand things without learning.

>>9801907
I never claimed you don't understand it.
In fact the case where you do understand this just makes my point that much stronger since even someone who understands it repeatedly fails to explain the specific terms even when asked point blank to do so.
As an example of what you didn't begin to do, there's this post:
>>9801843
Where the anon at least mentions the actual terms of the function in question.

>>9801782
the former

>>9801655
Because unintuative solutions are innately unaesthetic.

>>9801875
>You didn't even mention a single term of the actual function let alone explain what exactly it's doing. You're outlining the general idea and passing the buck to yet another abstract framework, not the same thing.
Learn what real math is, buddy. And what "first principles" means.

>>9801956
I'm getting less convinced you understand any of what you're talking about with each subsequent post you make.
For one thing you're now acting surprised I'm pointing out you still haven't even mentioned the actual terms of the function when I already told you repeatedly that was what you were failing to do at least two posts prior to that one.
You probably went to school and received a working knowledge of what to do but because you're intellectually shallow you never even tried to figure out why it all works, with ********specifics******** rather than high level glossing over of messy details.

>>9801842
You kind of left out the fact that the measure on L^2 is translation invariant, which is kind of why you can just say the non-zero mean and variance not equal to 1 cases are essentially the same as the standard gaussian case, after which everything else follows

>>9801715
https://en.wikipedia.org/wiki/Gaussian_function
https://en.wikipedia.org/wiki/Gaussian_integral

>>9801992
lol this makes no sense and explains nothing the OP wanted to know

>>9802021
spotted the data ``````````scientist''''''''''

>>9801715
>>9802021
As >>9801799 posted, it's maybe best understood by studying the proof of the classical central limit theorem.

For any distribution and a function g(X) with a series expansion (sum of powers of X), expectation values E[g(X)] can be computed if you know E[X^n]. The characteristic function f(t)=E[exp(itX)] captures those data, and you can get X^n by computing -i(∂/∂t)^n f(t). The fourier transform of a Gaussian is a Gaussian. What remains to show is that the characteristic function of the proces given by the sum of indepenent variables is a Gaussian.
From that standpoint, you could say the Gaussian pops up because it's a nice object w.r.t. the Fourier transform - the niceness that comes out from the independence and linearity assumption of the process under considerations - and the characteristic function has that exp(itX) that ties to the Fourier transform.

To believe in the central limit theorem, what I find helpful is computing the n-ford convolution of a simple distribution, like e.g. a rectangle, where you see that if you keep on smearing one function against itself a bunch of time, you approach a Gaussian. E.g. if you compute the convolution of two squares you get this triangle (pic related) which is broader and pointier. Then if you do the convolution of the triangle with another rectangle, the new rectangle reaches the tip even sooner and the whole form becomes bumpier. Then if you compute the convolution of that bump with a fourth rectangle, then you get a smoother bump and so on and so on. This is the distribution of 4 independent random jumps with the max distance of half the width of the rectangle. If you got thin rectangles and compute 1000 jumps, you'd have a nice looking Gaussian

pic related if you do the triangle with the square, i.e. if you do 3 squares.

>>9801655

>haven't visited /sci/ in months because of shit
>try to post on /his/ and ignore stormfags and commies
>leave
>visit /sci/
>first thread

>one of my favorite derivations ever
>makes use of elementary calculus and great use of the Jacobian for undergraduates
>in one of the most well known books on Probability ever (Hoel Port Stone) specifically written for Mathematicians
>Gauss...GAUSS
>obscure

I bet OP is the kind of guy that has a Chegg account and crams before midterms

>>9801983
I'm not the one you are responding to. But now I see it doesn't matter because you're just determined to be a belligerent idiot. I explained everything and you said I said nothing, because whatever word you don't know is the same as "nothing" and you want people to explain in terms that you know only. You are not interested in learning any topics I mentioned. I could meet your criteria by unpacking the definitions of every term I used down to the level where a school kid can understand. But I would be working for free to educate someone who is both ignorant and belligerent, and someone who casts aspersions on my academic credentials despite not knowing who I am or why I'm able to give you such in-depth explanations. My goodwill can only go so far. Stay proud and stupid, anon.

>>9802237
No, "terms" as in use the terms of the actual function in your response, not "terms" as in "use terms I understand."
If your response doesn't even mention the standard deviation or 2pi or e or the additive inverse of x minus the mean squared then you're not explaining the actual terms of the function and where they come from specifically and what they're doing specifically.
I don't see how "at least mention the actual terms of the function" is some ridiculous expectation on my part here.

>>9801715
This is maybe the most embarrassing unironic post in the history of /sci/

>>9801759
this is beautiful

>>9801842
thanks bro

found The dumb op who cant stats

>>9801923
LMFAO come on OP.
Do you think an explanation of this concept needs to contain specific references to the terms in the form you posted?

This person is trying to give you an intuition for the subject by telling you what to read about in order to understand it better. You should at least attempt this for a little while before coming back and bitching that their explanation doesn't reference "specific terms" in the formula you posted.

BTW, I agree with you - stats is a difficult subject without background in analysis and combinatorics. It's just that your argument for it being hard ("someone who says they have an understanding of it can't explain it to me using these terms") is idiotic.

>>9801655

>>9801655
you need to take a course in probability theory if this doesn't make sense to you