Skip to content

动转静单测机制推全任务列表 #58356

@gouzil

Description

@gouzil

Note

SOT 开发小组对组合方式进行了修改,请查看: #58630#58633

Motivation

单测在项目开发中是很重要的一部分,我们动转静相关的单测主要放置在 test/dygraph_to_static,也一直保障着动转静功能的正确性。

而随着近期 SOT 动转静新模式和 PIR 底层 IR 的开发,两者都使用了动转静单测作为功能正确性的验证方式,为了能够同时跑多种模式,SOT 和 PIR 开发小组在 test/dygraph_to_static/dygraph_to_static_util.py 分别添加了相关的装饰器,用于选择具体跑在什么模式下。

但 SOT 和 PIR 的单测扩增机制非常不同,这将会影响之后单测的扩展(比如之后即将开展的 PIR 动转静(理想态)单测推全验证),为了能够方便、统一地对单测进行管理,#57824 中在 test/dygraph_to_static/dygraph_to_static_utils_new.py 添加了一套统一的动转静单测机制,方便灵活地启用各种单测 case。但 #57824 的主要目标只是将 SOT 合入 Paddle,为了确保动转静单测机制的一致性,我们需要尽快将新机制推全~

Design

新的机制利用了 Metaclass,将会在生成新的 class 的时候统一生成新的单测 case,比如对于如下单测代码:

class MyTest(Dy2StTestBase):
    @set_to_static_mode(
        ToStaticMode.LEGACY_AST | ToStaticMode.SOT
    )
    @set_ir_mode(IrMode.LEGACY_PROGRAM | IrMode.PIR)
    def test_case1(self):
        ...

在装饰器的执行阶段,会在函数 MyTest.test_case1 上 patch 两个属性,分别是 test_case1.to_static_mode = ToStaticMode.LEGACY_AST | ToStaticMode.SOTtest_case1.ir_mode = IrMode.LEGACY_PROGRAM | IrMode.PIR。而从 Metaclass Dy2StTestMeta 生成 MyTest 的时候,我们会根据 patch 的属性来生成新的单测 case,这里对于 test_case1,根据其 to_static_modeir_mode,我们共有 2x2 = 4 种组合,因此我们会生成如下四个单测 case(伪码描述,实际实现会略有不同):

class MyTest(unittest.TestCase):
    def test_case1__legacy_ast_legacy_program(self):
        fn = MyTest.test_case1
        fn = to_legacy_program_test(fn)
        fn = to_legacy_ast_test(fn)
        fn(self)

    def test_case1__legacy_ast_pir(self):
        fn = MyTest.test_case1
        fn = to_legacy_pir(fn)
        fn = to_legacy_ast_test(fn)
        fn(self)

    def test_case1__sot_legacy_program(self):
        fn = MyTest.test_case1
        fn = to_legacy_program_test(fn)
        fn = to_sot_test(fn)
        fn(self)

    def test_case1__sot_pir(self):
        fn = MyTest.test_case1
        fn = to_legacy_pir(fn)
        fn = to_sot_test(fn)
        fn(self)

Important

  • 新的机制一律使用 Dy2StTestBase 作为基类,如果不设置基类,根据新机制的原理,仅仅会在函数上 patch 一个属性,而不会跑两次,也就是没有任何效果,注意不仅仅是原来使用 dy2static_unittest 装饰的单测 case,只要使用了 dygraph_to_static_util 下装饰器的 case 都需要修改
  • 新的机制只能装饰 test 开头的 test case,非 test case 使用相关装饰器是没有效果的(相关装饰器一律以 test_ 作为前缀)
  • 我们保留了 compare_legacy_with_pir 这一个特殊的装饰器,因为它大多数情况是需要装饰在非 test case 上面的,也就是 test case 所调用的内部函数,根据我们新机制的原理,我们没有办法利用新机制实现这一点,因此保留了该装饰器,该装饰器仍然是旧的一次跑两种 case 的机制

Tasks

  • 推全动转静单测新机制,使新机制覆盖全部动转静单测
  • 确保新机制的正确性,测试生成的 case 数量符合预期,迭代完善单测新机制
  • 移除单测旧机制代码

Details

序号 文件 pr
1 test/dygraph_to_static/test_assert.py #58316
2 test/dygraph_to_static/test_ast_util.py #58316
3 test/dygraph_to_static/test_backward_without_params.py #58316
4 test/dygraph_to_static/test_basic_api_transformation.py #58389
5 test/dygraph_to_static/test_bert.py #58389
6 test/dygraph_to_static/test_bmn.py #58389
7 test/dygraph_to_static/test_break_continue.py #58389
8 test/dygraph_to_static/test_build_strategy.py #58389
9 test/dygraph_to_static/test_cache_program.py
10 test/dygraph_to_static/test_cast.py #58316
11 test/dygraph_to_static/test_cinn_prim_gelu.py #58389
12 test/dygraph_to_static/test_cinn_prim_layer_norm.py #58389
13 test/dygraph_to_static/test_cinn_prim_mean.py #58389
14 test/dygraph_to_static/test_cinn_prim.py #58389
15 test/dygraph_to_static/test_cinn.py #58389
16 test/dygraph_to_static/test_closure_analysis.py #58316
17 test/dygraph_to_static/test_container.py #58509
18 test/dygraph_to_static/test_convert_call_generator.py #58389
19 test/dygraph_to_static/test_convert_call.py #58389
20 test/dygraph_to_static/test_convert_operators.py #58389
21 test/dygraph_to_static/test_cpu_cuda_to_tensor.py #58389
22 test/dygraph_to_static/test_cycle_gan.py #58389
23 test/dygraph_to_static/test_declarative.py #58316
24 test/dygraph_to_static/test_decorator_transform.py #58316
25 test/dygraph_to_static/test_deepcopy.py #58316
26 test/dygraph_to_static/test_dict.py #58389
27 test/dygraph_to_static/test_drop_path.py #58389
28 test/dygraph_to_static/test_duplicate_output.py #58389
29 test/dygraph_to_static/test_error.py
30 test/dygraph_to_static/test_eval_frame.py
31 test/dygraph_to_static/test_fallback.py #58458
32 test/dygraph_to_static/test_fetch_feed.py #58458
33 test/dygraph_to_static/test_for_enumerate.py #58458
34 test/dygraph_to_static/test_full_name_usage.py #58458
35 test/dygraph_to_static/test_function_spec.py
36 test/dygraph_to_static/test_grad.py #58458
37 test/dygraph_to_static/test_gradient_aggregation.py #58458
38 test/dygraph_to_static/test_gradname_parse.py #58458
39 test/dygraph_to_static/test_grid_generator.py #58389
40 test/dygraph_to_static/test_ifelse.py #58458
41 test/dygraph_to_static/test_ignore_module.py
42 test/dygraph_to_static/test_inplace_assign.py #58464
43 test/dygraph_to_static/test_isinstance.py #58458
44 test/dygraph_to_static/test_jit_property_save.py #58458
45 test/dygraph_to_static/test_jit_setitem.py #58458
46 test/dygraph_to_static/test_lac.py #58509
47 test/dygraph_to_static/test_lambda.py #58458
48 test/dygraph_to_static/test_layer_hook.py #58458
49 test/dygraph_to_static/test_len.py #58458
50 test/dygraph_to_static/test_list.py #58458
51 test/dygraph_to_static/test_load_transformer.py #58316
52 test/dygraph_to_static/test_logging_utils.py
53 test/dygraph_to_static/test_logical.py #58464
54 test/dygraph_to_static/test_loop.py #58464
55 test/dygraph_to_static/test_lstm.py #58464
56 test/dygraph_to_static/test_mnist_amp.py(test_mnist.py) #58464
57 test/dygraph_to_static/test_mnist_pure_fp16.py(test_mnist.py) #58464
58 test/dygraph_to_static/test_mnist.py #58464
59 test/dygraph_to_static/test_mobile_net.py #58464
60 test/dygraph_to_static/test_multi_forward.py #58464
61 test/dygraph_to_static/test_new_ir_selectedrows.py #58464
62 test/dygraph_to_static/test_op_attr.py #58464
63 test/dygraph_to_static/test_origin_info.py #58464
64 test/dygraph_to_static/test_param_guard.py #58464
65 test/dygraph_to_static/test_params_no_grad.py #58464
66 test/dygraph_to_static/test_partial_program_hook.py #58465
67 test/dygraph_to_static/test_partial_program.py #58316
68 test/dygraph_to_static/test_place.py #58465
69 test/dygraph_to_static/test_print.py #58465
70 test/dygraph_to_static/test_program_translator.py #58465
71 test/dygraph_to_static/test_ptb_lm_v2.py #58465
72 test/dygraph_to_static/test_ptb_lm.py #58465
73 test/dygraph_to_static/test_pylayer.py #58465
74 test/dygraph_to_static/test_reinforcement_learning.py #58465
75 test/dygraph_to_static/test_resnet_amp.py #58465
76 test/dygraph_to_static/test_resnet_pure_fp16.py #58465
77 test/dygraph_to_static/test_resnet_v2.py #58465
78 test/dygraph_to_static/test_resnet.py #58465
79 test/dygraph_to_static/test_return.py #58465
80 test/dygraph_to_static/test_rollback.py #58465
81 test/dygraph_to_static/test_save_inference_model.py #58465
82 test/dygraph_to_static/test_save_load.py #58490
83 test/dygraph_to_static/test_se_resnet.py #58490
84 test/dygraph_to_static/test_sentiment.py #58509
85 test/dygraph_to_static/test_seq2seq.py #58490
86 test/dygraph_to_static/test_set_dynamic_shape.py #58316
87 test/dygraph_to_static/test_setter_helper.py
88 test/dygraph_to_static/test_simnet_v2.py #58490
89 test/dygraph_to_static/test_simnet.py #58490
90 test/dygraph_to_static/test_slice.py #58490
91 test/dygraph_to_static/test_spec_names.py #58316
92 test/dygraph_to_static/test_static_analysis.py
93 test/dygraph_to_static/test_tensor_hook.py #58490
94 test/dygraph_to_static/test_tensor_memcpy_on_cpu.py #58490
95 test/dygraph_to_static/test_tensor_memcpy_on_gpu.py #58490
96 test/dygraph_to_static/test_tensor_methods.py #58490
97 test/dygraph_to_static/test_tensor_shape.py #58316
98 test/dygraph_to_static/test_to_tensor.py #58490
99 test/dygraph_to_static/test_train_step.py #58490
100 test/dygraph_to_static/test_transformer.py #58490
101 test/dygraph_to_static/test_tsm.py #58490
102 test/dygraph_to_static/test_typehint.py #58490
103 test/dygraph_to_static/test_typing.py #58499
104 test/dygraph_to_static/test_unuseful_inputs.py #58499
105 test/dygraph_to_static/test_utils.py #58499
106 test/dygraph_to_static/test_variable_trans_func.py #58499
107 test/dygraph_to_static/test_warning.py #58499
108 test/dygraph_to_static/test_word2vec.py #58499
109 test/dygraph_to_static/test_write_python_container.py #58509
110 test/dygraph_to_static/test_yolov3.py #58499
111 test/legacy_test/test_cond.py #58509
112 test/legacy_test/test_while_loop_op.py #58509
113 test/legacy_test/test_while_op.py #58509
114 test/dygraph_to_static/test_no_gradient.py #58509

Metadata

Metadata

Labels

PFCCPaddle Framework Contributor Club,https://github.com/PaddlePaddle/community/tree/master/pfccstatus/close已关闭

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions