Skip to content

[Feature] Support tuning parameter-level optim hyperparameters. #463

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Sep 29, 2022
Merged

[Feature] Support tuning parameter-level optim hyperparameters. #463

merged 5 commits into from
Sep 29, 2022

Conversation

RangiLyu
Copy link
Owner

@RangiLyu RangiLyu commented Sep 2, 2022

New Feature

Add nanodet.optim.build_optimizer
Support use optimizer building function to customize parameter-level hyperparameters with config.

How to use

optimizer:
   name: AdamW
   lr: 0.001
   weight_decay: 0.05
   no_norm_decay: True  # do not add weight decay to norm layer (from bag of tricks)
   param_level_cfg:  # parameter-level config
     backbone:
       lr_mult: 0.1  # example: use smaller lr for the backbone

@codecov
Copy link

codecov bot commented Sep 2, 2022 •
edited
Loading

Codecov Report

Merging #463 (f56af5f) into main (a3b3452) will increase coverage by 0.19%.
The diff coverage is 88.60%.

@@            Coverage Diff             @@
##             main     #463      +/-   ##
==========================================
+ Coverage   74.51%   74.71%   +0.19%     
==========================================
  Files          67       70       +3     
  Lines        4509     4572      +63     
  Branches      760      777      +17     
==========================================
+ Hits         3360     3416      +56     
- Misses        968      971       +3     
- Partials      181      185       +4
Flag Coverage Δ
unittests 74.71% <88.60%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
nanodet/util/env_utils.py 86.95% <86.95%> (ø)
nanodet/optim/builder.py 88.88% <88.88%> (ø)
nanodet/trainer/task.py 55.23% <88.88%> (-0.08%) ⬇️
nanodet/optim/__init__.py 100.00% <100.00%> (ø)
nanodet/data/transform/warp.py 84.48% <0.00%> (-1.15%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@RangiLyu RangiLyu merged commit ad410c2 into main Sep 29, 2022
@RangiLyu RangiLyu deleted the optim_builder branch January 20, 2023 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@RangiLyu