Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The "deltas" are calculated by the error in how well the current state of the network predicts the output, backpropagated. Sequential runs are not commutative because the state changes.

Consider the trivial example of training a network to distinguish between sample A and sample B. Give it a hundred As in a row and it just learns "everything is A". Give it a hundred Bs in a row and it relearns "no, everything is B". To train it to distinguish, you must alternate As and Bs (and not too regularly, either!)



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: