在KaimingNormal/Uniform中增加mode参数 #73347
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
modified: python/paddle/nn/initializer/kaiming.py modified: test/legacy_test/test_initializer.py
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #73347 +/- ##
===========================================
Coverage ? 100.00%
===========================================
Files ? 1
Lines ? 9
Branches ? 0
===========================================
Hits ? 9
Misses ? 0
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
docs: PaddlePaddle/docs#7324 |
zhwesky2010
reviewed
Jun 16, 2025
| ) | ||
| if self._mode == 'fan_out' and self._fan_in is not None: | ||
| raise ValueError( | ||
| "The mode of MSRAInitializer is 'fan_out', " |
There was a problem hiding this comment.
KaimingNormal/KaimingUniform
modified: python/paddle/nn/initializer/kaiming.py
modified: python/paddle/nn/initializer/kaiming.py modified: test/legacy_test/test_initializer.py
|
@Qin-sx CI可能随机挂了，注意rerun下 |
嗯，收到，已重跑 |
zhwesky2010
reviewed
Jun 19, 2025
| @@ -62,6 +62,7 @@ class MSRAInitializer(Initializer): | |||
| seed (int32, optional): random seed. Default is 0. | |||
| negative_slope (float, optional): negative_slope (only used with leaky_relu). Default is 0.0. | |||
| nonlinearity(str, optional): the non-linear function. Default is relu. | |||
| mode(str, optional): the mode of initialization, can be 'fan_in' or 'fan_out'. When set to 'fan_in', the fan_in parameter is used for initialization. When set to 'fan_out', the out_features of trainable Tensor will be used. Default is 'fan_in'. | |||
There was a problem hiding this comment.
这个逻辑是不是得这么改一下，如果设置了 fan_in，则fan_in生效，mode不生效。如果没设置 fan_in，则mode生效。
fan_in与fan_out不可同时设置，给出明确的报错提示。
There was a problem hiding this comment.
嗯，我想的逻辑是以mode参数为判断标准，mode='fan_in'时，fan_in参数生效。mode='fan_out'时，fan_in参数不生效。感觉以fan_in参数为判断标准，逻辑不太好解释。
if self._mode == 'fan_in':
fan_in = f_in if self._fan_in is None else self._fan_in
if self._mode == 'fan_out':
fan_in = f_out以mode参数为判断标准的报错逻辑在这里
if self._mode not in ['fan_in', 'fan_out']:
raise ValueError(
"The mode of KaimingNormal/KaimingUniform should be 'fan_in' or 'fan_out', "
f"but received {self._mode}."
)
if self._mode == 'fan_out' and self._fan_in is not None:
raise ValueError(
"The mode of KaimingNormal/KaimingUniform is 'fan_out', "
"but fan_in is set. Please set fan_in to None."
)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Category
User Experience
PR Types
Improvements
Description
当前问题
paddle.nn.initializer.KaimingNormal目前只fan_in模式，不支持fan_out模式，fan_out需要自己计算out_feature值后填入fan_in参数
解决方案
paddle.nn.initializer.KaimingNormal中增加一个mode参数：