Move fused_moe_permute and fused_moe_unpermute #73264
Conversation
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
/re-run all-failed |
3 similar comments
|
/re-run all-failed |
|
/re-run all-failed |
|
/re-run all-failed |
|
/re-run all-failed |
|
/re-run all-failed |
|
/re-run all-failed |
1 similar comment
|
/re-run all-failed |
|
/re-run all-failed |
1 similar comment
|
/re-run all-failed |
| def moe_permute( | ||
| X: Tensor, | ||
| XScale: Tensor | None, | ||
| expert_routemap_topk: Tensor, | ||
| expert_prob_topk: Tensor, | ||
| topk: int, | ||
| num_experts: int, | ||
| tokens_per_expert: list, | ||
| padding_multiplex: int, | ||
| name: str | None = None, | ||
| ): |
| from paddle import Tensor | ||
|
|
||
|
|
||
| def moe_unpermute( |
|
/re-run all-failed |
1 similar comment
|
/re-run all-failed |
|
/re-run approval |
| tokens_per_expert: list, | ||
| padding_multiplex: int, | ||
| name: str | None = None, | ||
| ): |
| X: Tensor, | ||
| XScale: Tensor | None, |
There was a problem hiding this comment.
为什么 Python 端还会有大写 X、XScale 这种变量名呢？用 x、x_scale
| num_experts: int, | ||
| MP: bool = True, | ||
| name: str | None = None, | ||
| ): |
| unzipped_token_probs: Tensor, | ||
| total_zipped_tokens: int, | ||
| num_experts: int, | ||
| MP: bool = True, |
There was a problem hiding this comment.
与AMP对应，是MixPrecision的缩写，这个后续我会单独和文档一起推进规范化
| return max_abs_err, max_rel_err | ||
|
|
||
|
|
||
| class TestFusedMoePermuteUnpermute(unittest.TestCase): |
There was a problem hiding this comment.
建议补充小算子组合实现的版本，作为参考答案，验证API的正确性
| print( | ||
| f"permute-unpermute tokens relative error: {max_rel_err}" | ||
| ) | ||
| self.assertLess( |
There was a problem hiding this comment.
这里误差的原因是什么？是否可以做到二进制完全对齐？误差范围为什么设置为1e-2
There was a problem hiding this comment.
存在一个token广播给多个专家的情况，在收集时会涉及到token的加权&reduce操作，会带来一定的浮点精度损失，这个损失来源于两部分：1.浮点乘法 2.累加误差，设置为1e-2是参考了bfloat16 reduce的阈值
| print( | ||
| f"ermute-unpermute probs max absolute error: {max_abs_err}, relative error: {max_rel_err}" | ||
| ) | ||
| self.assertLess( |
There was a problem hiding this comment.
这里有误差的原因是什么？是否可以做到二进制完全对齐？误差范围为什么设置为1e-5
|
/re-run all-failed |
| for dt in self.DTYPES: | ||
| for expert_num in self.EXPERT_NUMS: | ||
| for topk in self.TOPKS: |
There was a problem hiding this comment.
| for dt in self.DTYPES: | |
| for expert_num in self.EXPERT_NUMS: | |
| for topk in self.TOPKS: | |
| for (dt, expert_num, topk) in itertools.product(self.DTYPES, self.EXPERT_NUMS, self.TOPKS): |
使用 itertools.product 折叠多个 for 循环，减少锁进层级
| ): | ||
| print( | ||
| f"Testing with {expert_num} experts, topk {topk}, dtype {dt}" | ||
| ) |
There was a problem hiding this comment.
刚刚没注意，这几个无条件的 print 也要删掉，如果要打印误差，利用 self.assertEqual 等 API 的 err_msg 字段，不要无条件打印日志，会影响 CI 日志
PR Category
User Experience
PR Types
Improvements
Description
Add
paddle.nn.functional.fused_moe_permute&paddle.nn.functional.fused_moe_unpermutekernels/fusion/gpupcard-91067