STAT C100 Lecture 9: Lecture09 Modeling and Estimation II
Document Summary
Summary of model estimation (so far : define the model: simplified representation of the world. We did this both graphically and analytically. Today we will also discuss how to compute the loss numerically. (cid:395)ua(cid:396)ed loss is p(cid:396)o(cid:374)e to (cid:862)o(cid:448)e(cid:396)(cid:396)ea(cid:272)t(cid:863) to outlie(cid:396)s. Example above for data = [5, 7, 8, 9, 100]. New data point changes (cid:3118) from 7. 25 to 25. 8. New data point means (cid:3117) is now definitely 8, instead of 8 being one of many possible solutions. No simple analytic solution for average huber loss. Large unique optimum like squared loss. Numerical optimization: minimizing the loss function using brute force. Range of guesses may miss the minimum. Guesses may be too coarsely spaced: minimizing the loss function using the derivative. Negative to the left of the solution. Positive to the right of the solution. Derivative tells us which way to go: minimizing the loss function using gradient descent.
