"Learning rates, corrupted linear systems, and randomized Kaczmarz" by Nicholas Marshall from Oregon State University in STAG 213.

Link to Probability and Data Science Seminar.

Abstract: In this talk, we consider how the learning rate affects the performance of a relaxed randomized Kaczmarz algorithm for solving Ax≈b+ε, where Ax=b is a consistent linear system and ε has independent mean zero random entries. We derive a learning rate schedule that optimizes a bound on the expected error that is sharp in certain cases; in contrast to the exponential convergence of the standard randomized Kaczmarz algorithm, our optimized bound involves the reciprocal of the Lambert-W function of an exponential (This talk is based on joint work with Oscar Mickelin).