https://HaitongLan.github.io/categories/ideas/Recent content in Ideas on Haitong LanHugoenHaitong's blogSat, 14 Nov 2026 00:00:00 +0800https://HaitongLan.github.io/p/mu-as-preconditioning/Sat, 14 Nov 2026 00:00:00 +0800https://HaitongLan.github.io/p/mu-as-preconditioning/I started thinking about multiplicative updates after reading about ScaledGD. Both algorithms scale their gradients using the current factors. Rewriting MU in this form reveals how its diagonal scaling responds to coordinate units and component correlation. Seeing the gradient inside the multiplicative update Scaled gradient descent (ScaledGD) makes a familiar promise in low-rank recovery. By adapting the gradient to the current factors, it can achieve convergence rates independent of the condition number of the target matrix under appropriate observation and initialization assumptions Tong, Ma, and Chi (2021).