วันศุกร์ที่ 27 กันยายน พ.ศ. 2567

Neural Net's Learning rate adjustment

 One cycle policy (OCP)  is increasing and decreasing learning rate (i.e. adaptive learning rate) between upper & lower bounds to avoid slow training as much as possible.

https://medium.com/@varunsivamani/one-cycle-policy-a-deep-understanding-6d4d352ec7b1

https://medium.com/dsnet/the-1-cycle-policy-an-experiment-that-vanished-the-struggle-in-training-neural-nets-184417de23b9