【Hackathon 6th No.23】为 paddle.nn.functional.embedding/paddle.nn.Embedding 增加参数 max_norm/norm_type/scale_grad_by_freq - part #63130
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
❌ The PR is not created using PR's template. You can refer to this Demo. |
|
@AndPuQing 你好，需要看一下大量CI没通过的原因 |
|
Sorry to inform you that 8b803d9's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
|
Sorry to inform you that c9d0326's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually. |
|
@AndPuQing 冲突麻烦修改一下 |
python/paddle/nn/functional/input.py
Outdated
| r""" | ||
| This operator is used to update the embedding weight by renorm. | ||
| """ | ||
| if in_dynamic_mode(): |
There was a problem hiding this comment.
- 这里能否 不新加kernel，直接复用已有kernel，因为embedding renorm kernel里其实也复用的是pnorm（
paddle.linalg.norm） - 如果不能复用norm，是不是因为inplace的问题，那能否直接在原有
paddle.linalg.norm的基础上改造使得其支持，尽可能复用已有的计算逻辑 paddle.nn.functional.embedding_renorm_需要公开出来不，torch有这个没
There was a problem hiding this comment.
1.2. 现在修改为了利用现有API组装这个算子，inplace 使用 paddle.scatter_
- 用户是可以直接调用
torch.embedding_renorm_的，但是没有doc
python/paddle/nn/functional/input.py
Outdated
| @@ -127,10 +129,33 @@ def one_hot( | |||
| return one_hot_out | |||
|
|
|||
|
|
|||
| @inplace_apis_in_dygraph_only | |||
There was a problem hiding this comment.
这个还需要吗，因为组合的API都是动静统一的，是因为组合里面的有纯动态图下的API吗？@AndPuQing
There was a problem hiding this comment.
scatter_ 是 inplace_apis_in_dygraph_only
Paddle/python/paddle/tensor/manipulation.py
Line 4078 in 900d27c
| @@ -17,6 +17,7 @@ | |||
| import numpy as np | |||
|
|
|||
| import paddle | |||
| from paddle.nn.functional.input import embedding_renorm_ | |||
|
@AndPuQing 这个还少一个参数scale_grad_by_freq，该任务完成了2/3 |
|
@AndPuQing API英文文档没有修改，请提交一个PR。该PR合入后，请在docs里修改API中文文档、API映射文档。 以上所有文档（英文文档、中文文档、映射文档）合入后，该任务算完成。（计做2/3） |
…ding 增加参数 max_norm/norm_type/scale_grad_by_freq - part (PaddlePaddle#63130) * Add embedding_renorm_ function to functional module * Fix embedding_renorm_ function signature * Add max_norm and norm_type parameters to Embedding class * Remove scale_grad_by_freq parameter from embedding function * remove scale_grad_by_freq parameter * add test case * refactor: use fabs and powf functions * refactor: use api to build embedding_renorm * remove embedding_renorm_ function from __all__ * fix * add static test
PR Category
Operator Mechanism
PR Types
New features
Description
为embedding添加了 max_norm, norm_type 参数，并添加了单测
实现方法：分别实现了CPU、GPU上的EmbeddingRenormKernel，并注册了inplace算子