Add Learning Rate Scheduler to neurallamConfig#107

Open

matschreiner wants to merge 24 commits intomllam:mainfrom matschreiner:scheduler

+237-46

Commits on Jan 24, 2025

add configuration
Jacob Mathias Schreiner
committed

Commits on Jan 27, 2025

post init
Jacob Mathias Schreiner
committed
check valid scheduler
Jacob Mathias Schreiner
committed
test valid and invalid config
Jacob Mathias Schreiner
committed
remove lr arg
Jacob Mathias Schreiner
committed
ar_model
Jacob Mathias Schreiner
committed
remove optimization_config
Jacob Mathias Schreiner
committed
cosine annealing test
Jacob Mathias Schreiner
committed
add model args fixture
Jacob Mathias Schreiner
committed
get from optimization config
Jacob Mathias Schreiner
committed
add test to check if lr_scheduler is working as expected
Jacob Mathias Schreiner
committed

Commits on Jan 28, 2025

use args fixture
Jacob Mathias Schreiner
committed
remove unused import
Jacob Mathias Schreiner
committed
add config instead
Jacob Mathias Schreiner
committed
move scheduler step to batch_end and add step frequency to the scheduler
Jacob Mathias Schreiner
committed
add bal_steps to log to model args, is needed for somet tests
Jacob Mathias Schreiner
committed
add learning rate to logs
Jacob Mathias Schreiner
committed
make lr a feature that is only accessed if the model has a trainer attached. Otherwise return the lr in the configuration
Jacob Mathias Schreiner
committed

Commits on Jan 29, 2025

Merge branch 'main' into scheduler
Jacob Mathias Schreiner
committed

Commits on Feb 3, 2025

trigger action
Jacob Mathias Schreiner
committed

Commits on Feb 4, 2025

change return type and enable resetting learning rate scheduler
Jacob Mathias Schreiner
committed
remove commented out code
Jacob Mathias Schreiner
committed
Merge branch 'main' into scheduler
Jacob Mathias Schreiner
committed
add metrics watch to modelconfig
Jacob Mathias Schreiner
committed