/sci/ - Science & Math

I can't seem to figure out what I'm doing wrong with this calculation. I'm trying to find out how the derivative in a system passes back through a normalization layer, and I'm getting a result saying it isn't at all. This seems impossible considering that Transformer neural nets rely on gradients passing back thorough it to function. I'm pretty sure I've got the formulation of the original Layer Normalization function correct (from here: https://arxiv.org/abs/1607.06450)) so there must be something wrong with my math. Please tell me why I'm a moron and what I'm doing wrong, I've been stuck on this all day. Pardon my poor LaTeX skills.