/sci/ - Science & Math

So is this a (maybe oversimplified, but still) way of understanding the updating of the weight representing the slope in gradient descent for linear regression?

The update is given by
[math] w_1 \leftarrow w_1 + \alpha(y-h_w(x))*x [/math]
where (x,y) is a data pair, and h_w(x) is a line [math]w_1x + w_0[/math] being fitted

And what I am wondering about is whether this is a sensible way of understanding how positive an negative values of the "error vector" ends up working as hoped:
there are four cases, barring a perfect fit: [math]h_w(x)[/math] might be too high (higher than y), so that the parenthesized term is negative,
or it might be too low, so that it is positive.
x might also be negative, or it might be positive.

case 1: [math]h_w(x)[/math] too large, x positive. This is (a) in the image.
In this case, the second term in the update is negative, which results in a decrease in slope, which again adjusts (a) downwards.

case 2: [math]h_w(x)[/math] too large, x negative. This is (d) in the image.
In this case, the second term in the update is positive, which results in an increase in slope, which again adjusts (d) downwards.

case 3: positive delta*positive x, (c), increase slope (c upwards)

case 4: positive delta*negative x, (b), decrease slope (b upwards)

I am just having issues imagining the effect positive or negative values of X might have on the direction the weight is adjusted in, without mapping out each case like this, if it is even the right way of thinking of it.