Deep Learning: Notes
Deep Learning is to approximate the data distribution by learning the best-fit probability distribution from training data using a neural network. Supervised Learning What we have is a training set $\mathcal D_{\text{train}} = \{(x_i, y_i)\}_{i=1}^N$ sampled from $\mathcal D$. The empirical risk is defined as $$R_{\text{emp}}(h) = \frac{1}{N} \sum_{i=1}^n L(h(x_i), y_i),$$ where $L$ is the loss function, $h$ is the hypothesis function in the hypothesis space $\mathcal H$. And the expected risk is defined as, can not be computed directly since we do not know the data distribution $\mathcal D$, ...