commaai
/
tinygrad
mirror of https://github.com/commaai/tinygrad.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

adjsut div factor to avoid underflow

This commit is contained in:
Yixiang Gao 2024-01-02 13:36:38 -08:00
parent 91ddda244f
commit 2e4d9ad936
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
examples/hlb_cifar10.py
View File

@ -94,7 +94,7 @@ def train_cifar():
'bias_decay': 1.08 * 6.45e-4 * BS/bias_scaler,
'non_bias_decay': 1.08 * 6.45e-4 * BS,
'final_lr_ratio': 0.025,
'initial_div_factor': 1e16,
'initial_div_factor': 1e6,
'label_smoothing': 0.20,
'momentum': 0.85,
'percent_start': 0.23,