[Typing] type checking with apis #64991

megemini · 2024-06-07T08:14:23Z

PR Category

User Experience

PR Types

Improvements

Description

修改 tools/type_checking.py ，使其可以单独检查指定的 api，具体用法为：

checking from input apis

> python type_checking.py paddle.abs paddle.abs_ paddle.sin

checking from spec, with increment api

> python type_checking.py

checking from spec, with full apis

> python type_checking.py --full-test

--full-test and apis should not be set at the same time.

@SigureMo

paddle-bot · 2024-06-07T08:14:29Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

SigureMo · 2024-06-07T18:33:55Z

tools/sampcd_processor_utils.py

@@ -598,41 +598,48 @@ def get_test_capacity(run_on_device="cpu"):
 def get_docstring(
    full_test: bool = False,
    filter_api: typing.Callable[[str], bool] | None = None,
+    apis: tuple[tuple[str, str], ...] | None = None,


这里是不是用 list 更合适？

SigureMo · 2024-06-07T18:35:15Z

tools/type_checking.py

-# We type-check the `Example` codes from docstring.
+# We type-check the `Example` codes from docstring, like:
+# 1. checking from input `apis`
+# > python type_checking.py paddle.abs paddle.abs_ paddle.sin


这里的 1 需要预先手动生成 spec 么？

不用～单独检查指定的 api （如 python type_checking.py paddle.abs paddle.abs_ paddle.sin）与通过 spec 检查（如 python type_checking.py）是互斥的～

也就是说，开发者可以用这个脚本在本地检查 api 的 typing ～

本来是想把这个功能单独写一个工具的，就像之前 xdoctest 那个，但是，出于以下考虑，整合在了这里：

type checking 需要动态获取 api 的签名

之前的 xdoctest 可以看作是静态检查，而 type checking 需要开发者改完之后重新打包安装，或者手动覆盖掉修改的文件，所以要麻烦很多～

mypy 的配置文件应该会不断更新

如果像之前那样单列工具，那么两边同步的工作会很多，不利用使用～

由此，这里把 type_checking.py 简单扩展了一下，方便开发者本地调试～当然，调试前还是需要自己手动重新打包安装～

看看还没有什么更好的方式？

执行方式：

> python type_checking.py paddle.abs paddle.acos ----------------Codeblock Type Checking Start-------------------- >>> Get docstring from api ... API_PR is diff from API_DEV: dict_keys(['paddle.abs', 'paddle.acos']) Total api: 2 >>> Running type checker ... >>> Print summary ... ----------------Check results-------------------- ----------------Check results-------------------- >>> Type checking is successful! >>> Type checking is successful! ----------------End of the Check-------------------- ----------------End of the Check--------------------

嗯嗯，这里没什么更多问题了，只是在 CI 中是否可以像 codestyle 流水线一样，失败后提示用户如何复现问题？

Paddle/tools/codestyle/pre_commit.sh

Lines 49 to 62 in ed8168d

echo "Your PR code style check failed."

echo "Please install pre-commit locally and set up git hook scripts:"

echo ""

echo " pip install pre-commit==2.17.0"

echo " pre-commit install"

echo ""

if [[ $num_diff_files -le 100 ]];then

echo "Then, run pre-commit to check codestyle issues in your PR:"

echo ""

echo " pre-commit run --files" $(echo ${diff_files} | tr "\n" " ")

echo ""

fi

echo "For more information, please refer to our codestyle check guide:"

echo "https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/codestyle_check_guide_cn.html"

仅提示失败的那几个就好～比如 API diff 含 paddle.abs、paddle.acos，前者成功，后者失败，那么提示使用 python type_checking.py paddle.acos 来复现

SigureMo · 2024-06-11T07:07:53Z

话说为什么会有一些分布式 API 也被检查了呢？

SigureMo · 2024-06-11T07:08:43Z

tools/type_checking.py

+            logger.error(
+                ">>> Please recheck the type annotations. Run `tools/type_checking.py` to check the typing issues:"
+            )
+            logger.error("> python type_checking.py " + " ".join(failed_apis))


Suggested change

logger.error("> python type_checking.py " + " ".join(failed_apis))

logger.error("> python tools/type_checking.py " + " ".join(failed_apis))

megemini · 2024-06-11T09:32:18Z

话说为什么会有一些分布式 API 也被检查了呢？

print_signatures.py 里是这么获取的 api ，我们只负责检查～

需要过滤一下？

SigureMo · 2024-06-11T09:35:28Z

print_signatures.py 里是这么获取的 api ，我们只负责检查～

喔喔，没注意用了 typing all，我以为用的是 typing

SigureMo

LGTMeow

[Change] type checking with apis

673b9de

paddle-bot bot added the contributor External developers label Jun 7, 2024

luotao1 added the PaddlePaddle Hackathon label Jun 7, 2024

luotao1 assigned luotao1 and SigureMo Jun 7, 2024

[Add] paddle.log typing

38d7b0c

megemini changed the title ~~[Change] type checking with apis~~ [Typing] type checking with apis Jun 7, 2024

[Fix] import annotations

df49e04

megemini changed the title ~~[Typing] type checking with apis~~ [Typing all] type checking with apis Jun 7, 2024

megemini changed the title ~~[Typing all] type checking with apis~~ [] type checking with apis Jun 7, 2024

megemini changed the title ~~[] type checking with apis~~ [Typing] type checking with apis Jun 7, 2024

SigureMo reviewed Jun 7, 2024

View reviewed changes

megemini added 2 commits June 8, 2024 12:24

[Change] typing

efb8764

[Update] type_checking logger

a7baab9

megemini changed the title ~~[Typing] type checking with apis~~ [Typing all] type checking with apis Jun 10, 2024

[Update] type_checking logger

e19788f

SigureMo reviewed Jun 11, 2024

View reviewed changes

megemini added 2 commits June 11, 2024 18:10

[Fix] ctrl+c interrupt

dfe0d1f

[Fix] ctrl+c interrupt on mac os

111f25a

megemini mentioned this pull request Jun 12, 2024

[Type Hints] 为公开 API 标注类型提示信息 #65008

Closed

SigureMo changed the title ~~[Typing all] type checking with apis~~ [Typing] type checking with apis Jun 12, 2024

SigureMo approved these changes Jun 12, 2024

View reviewed changes

SigureMo merged commit 26547f1 into PaddlePaddle:develop Jun 12, 2024

SigureMo mentioned this pull request Jun 15, 2024

[Typing][A-16,A-19] Add type annotations for base Layer and containers #65190

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Typing] type checking with apis #64991

[Typing] type checking with apis #64991

Uh oh!

megemini commented Jun 7, 2024

Uh oh!

paddle-bot bot commented Jun 7, 2024

Uh oh!

SigureMo Jun 7, 2024

Uh oh!

SigureMo Jun 7, 2024 •

edited

Loading

Uh oh!

megemini Jun 8, 2024 •

edited

Loading

Uh oh!

SigureMo Jun 9, 2024

Uh oh!

SigureMo commented Jun 11, 2024

Uh oh!

SigureMo Jun 11, 2024

Uh oh!

megemini commented Jun 11, 2024

Uh oh!

SigureMo commented Jun 11, 2024

Uh oh!

SigureMo left a comment

Uh oh!

Uh oh!

	echo "Your PR code style check failed."
	echo "Please install pre-commit locally and set up git hook scripts:"
	echo ""
	echo " pip install pre-commit==2.17.0"
	echo " pre-commit install"
	echo ""
	if [[ $num_diff_files -le 100 ]];then
	echo "Then, run pre-commit to check codestyle issues in your PR:"
	echo ""
	echo " pre-commit run --files" $(echo ${diff_files} \| tr "\n" " ")
	echo ""
	fi
	echo "For more information, please refer to our codestyle check guide:"
	echo "https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/dev_guides/git_guides/codestyle_check_guide_cn.html"

	logger.error("> python type_checking.py " + " ".join(failed_apis))
	logger.error("> python tools/type_checking.py " + " ".join(failed_apis))

[Typing] type checking with apis #64991

[Typing] type checking with apis #64991

Uh oh!

Conversation

megemini commented Jun 7, 2024

PR Category

PR Types

Description

Uh oh!

paddle-bot bot commented Jun 7, 2024

Uh oh!

SigureMo Jun 7, 2024

Choose a reason for hiding this comment

Uh oh!

SigureMo Jun 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

megemini Jun 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SigureMo Jun 9, 2024

Choose a reason for hiding this comment

Uh oh!

SigureMo commented Jun 11, 2024

Uh oh!

SigureMo Jun 11, 2024

Choose a reason for hiding this comment

Uh oh!

megemini commented Jun 11, 2024

Uh oh!

SigureMo commented Jun 11, 2024

Uh oh!

SigureMo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

SigureMo Jun 7, 2024 •

edited

Loading

megemini Jun 8, 2024 •

edited

Loading