Background

Linear Algebra

Resource	Description
Stanford CS229 Linear Algebra Review	A great general purpose linear algebra primer.
Matrix Cookbook	Good resource for looking various linear algebra things up, no proofs.
3Blue1Brown Essence of Linear Algebra	Videos detailing what linear algebra actually "looks" like.
Convex Optimization (Appendix A)	Appendix A is math background for optimization.

Linear Systems

Resource	Description
Stanford EE263	Introduction to linear dynamical systems and linear algebra.
Tom Kailath's Linear Systems	Classic book on linear systems, no PDF though.

Optimization

Resource	Description
Numerical Optimization, Nocedal and Wright	The best book for general purpose (nonconvex) optimization. Implementation/algorithmic details on many common solution methods for convex and nonconvex problems. Interior point methods, active set methods, SQP, quasi-Newton methods.
Convex Optimization	Stephen Boyd's book, the Bible for Convex optimization.
CMU Convex Lectures	Ryan Tibshirani's convex lecture slides.
Stanford Convex Lectures	Stephen Boyd's lecture slides.
PDIP QP Solver	CVXGEN paper with all of the great implementation details for a classic primal-dual interior point (PDIP) method for solving quadratic programs (QPs).
CVXOPT Documentation	All the implementation details for convex conic solvers.
Algorithms for Optimization	Common gradient-based and derivative-free optimization algorithms in Julia.
Calculus of Variations and Optimal Control	Contains classic optimal control ideas.