-
Notifications
You must be signed in to change notification settings - Fork 5.8k
Description
Motivation
单测在项目开发中是很重要的一部分,我们动转静相关的单测主要放置在 test/dygraph_to_static
,也一直保障着动转静功能的正确性。
而随着近期 SOT 动转静新模式和 PIR 底层 IR 的开发,两者都使用了动转静单测作为功能正确性的验证方式,为了能够同时跑多种模式,SOT 和 PIR 开发小组在 test/dygraph_to_static/dygraph_to_static_util.py
分别添加了相关的装饰器,用于选择具体跑在什么模式下。
但 SOT 和 PIR 的单测扩增机制非常不同,这将会影响之后单测的扩展(比如之后即将开展的 PIR 动转静(理想态)单测推全验证),为了能够方便、统一地对单测进行管理,#57824 中在 test/dygraph_to_static/dygraph_to_static_utils_new.py
添加了一套统一的动转静单测机制,方便灵活地启用各种单测 case。但 #57824 的主要目标只是将 SOT 合入 Paddle,为了确保动转静单测机制的一致性,我们需要尽快将新机制推全~
Design
新的机制利用了 Metaclass,将会在生成新的 class 的时候统一生成新的单测 case,比如对于如下单测代码:
class MyTest(Dy2StTestBase):
@set_to_static_mode(
ToStaticMode.LEGACY_AST | ToStaticMode.SOT
)
@set_ir_mode(IrMode.LEGACY_PROGRAM | IrMode.PIR)
def test_case1(self):
...
在装饰器的执行阶段,会在函数 MyTest.test_case1
上 patch 两个属性,分别是 test_case1.to_static_mode = ToStaticMode.LEGACY_AST | ToStaticMode.SOT
和 test_case1.ir_mode = IrMode.LEGACY_PROGRAM | IrMode.PIR
。而从 Metaclass Dy2StTestMeta
生成 MyTest
的时候,我们会根据 patch 的属性来生成新的单测 case,这里对于 test_case1
,根据其 to_static_mode
和 ir_mode
,我们共有 2x2 = 4 种组合,因此我们会生成如下四个单测 case(伪码描述,实际实现会略有不同):
class MyTest(unittest.TestCase):
def test_case1__legacy_ast_legacy_program(self):
fn = MyTest.test_case1
fn = to_legacy_program_test(fn)
fn = to_legacy_ast_test(fn)
fn(self)
def test_case1__legacy_ast_pir(self):
fn = MyTest.test_case1
fn = to_legacy_pir(fn)
fn = to_legacy_ast_test(fn)
fn(self)
def test_case1__sot_legacy_program(self):
fn = MyTest.test_case1
fn = to_legacy_program_test(fn)
fn = to_sot_test(fn)
fn(self)
def test_case1__sot_pir(self):
fn = MyTest.test_case1
fn = to_legacy_pir(fn)
fn = to_sot_test(fn)
fn(self)
Important
- 新的机制一律使用
Dy2StTestBase
作为基类,如果不设置基类,根据新机制的原理,仅仅会在函数上 patch 一个属性,而不会跑两次,也就是没有任何效果,注意不仅仅是原来使用dy2static_unittest
装饰的单测 case,只要使用了dygraph_to_static_util
下装饰器的 case 都需要修改 - 新的机制只能装饰
test
开头的 test case,非 test case 使用相关装饰器是没有效果的(相关装饰器一律以test_
作为前缀) - 我们保留了
compare_legacy_with_pir
这一个特殊的装饰器,因为它大多数情况是需要装饰在非 test case 上面的,也就是 test case 所调用的内部函数,根据我们新机制的原理,我们没有办法利用新机制实现这一点,因此保留了该装饰器,该装饰器仍然是旧的一次跑两种 case 的机制
Tasks
- 推全动转静单测新机制,使新机制覆盖全部动转静单测
- 确保新机制的正确性,测试生成的 case 数量符合预期,迭代完善单测新机制
- 移除单测旧机制代码
Details
序号 | 文件 | pr |
---|---|---|
1 | test/dygraph_to_static/test_assert.py | #58316 |
2 | test/dygraph_to_static/test_ast_util.py | #58316 |
3 | test/dygraph_to_static/test_backward_without_params.py | #58316 |
4 | test/dygraph_to_static/test_basic_api_transformation.py | #58389 |
5 | test/dygraph_to_static/test_bert.py | #58389 |
6 | test/dygraph_to_static/test_bmn.py | #58389 |
7 | test/dygraph_to_static/test_break_continue.py | #58389 |
8 | test/dygraph_to_static/test_build_strategy.py | #58389 |
10 | test/dygraph_to_static/test_cast.py | #58316 |
11 | test/dygraph_to_static/test_cinn_prim_gelu.py | #58389 |
12 | test/dygraph_to_static/test_cinn_prim_layer_norm.py | #58389 |
13 | test/dygraph_to_static/test_cinn_prim_mean.py | #58389 |
14 | test/dygraph_to_static/test_cinn_prim.py | #58389 |
15 | test/dygraph_to_static/test_cinn.py | #58389 |
16 | test/dygraph_to_static/test_closure_analysis.py | #58316 |
17 | test/dygraph_to_static/test_container.py | #58509 |
18 | test/dygraph_to_static/test_convert_call_generator.py | #58389 |
19 | test/dygraph_to_static/test_convert_call.py | #58389 |
20 | test/dygraph_to_static/test_convert_operators.py | #58389 |
21 | test/dygraph_to_static/test_cpu_cuda_to_tensor.py | #58389 |
22 | test/dygraph_to_static/test_cycle_gan.py | #58389 |
23 | test/dygraph_to_static/test_declarative.py | #58316 |
24 | test/dygraph_to_static/test_decorator_transform.py | #58316 |
25 | test/dygraph_to_static/test_deepcopy.py | #58316 |
26 | test/dygraph_to_static/test_dict.py | #58389 |
27 | test/dygraph_to_static/test_drop_path.py | #58389 |
28 | test/dygraph_to_static/test_duplicate_output.py | #58389 |
31 | test/dygraph_to_static/test_fallback.py | #58458 |
32 | test/dygraph_to_static/test_fetch_feed.py | #58458 |
33 | test/dygraph_to_static/test_for_enumerate.py | #58458 |
34 | test/dygraph_to_static/test_full_name_usage.py | #58458 |
36 | test/dygraph_to_static/test_grad.py | #58458 |
37 | test/dygraph_to_static/test_gradient_aggregation.py | #58458 |
38 | test/dygraph_to_static/test_gradname_parse.py | #58458 |
39 | test/dygraph_to_static/test_grid_generator.py | #58389 |
40 | test/dygraph_to_static/test_ifelse.py | #58458 |
42 | test/dygraph_to_static/test_inplace_assign.py | #58464 |
43 | test/dygraph_to_static/test_isinstance.py | #58458 |
44 | test/dygraph_to_static/test_jit_property_save.py | #58458 |
45 | test/dygraph_to_static/test_jit_setitem.py | #58458 |
46 | test/dygraph_to_static/test_lac.py | #58509 |
47 | test/dygraph_to_static/test_lambda.py | #58458 |
48 | test/dygraph_to_static/test_layer_hook.py | #58458 |
49 | test/dygraph_to_static/test_len.py | #58458 |
50 | test/dygraph_to_static/test_list.py | #58458 |
51 | test/dygraph_to_static/test_load_transformer.py | #58316 |
53 | test/dygraph_to_static/test_logical.py | #58464 |
54 | test/dygraph_to_static/test_loop.py | #58464 |
55 | test/dygraph_to_static/test_lstm.py | #58464 |
56 | test/dygraph_to_static/test_mnist_amp.py(test_mnist.py) | #58464 |
57 | test/dygraph_to_static/test_mnist_pure_fp16.py(test_mnist.py) | #58464 |
58 | test/dygraph_to_static/test_mnist.py | #58464 |
59 | test/dygraph_to_static/test_mobile_net.py | #58464 |
60 | test/dygraph_to_static/test_multi_forward.py | #58464 |
61 | test/dygraph_to_static/test_new_ir_selectedrows.py | #58464 |
62 | test/dygraph_to_static/test_op_attr.py | #58464 |
63 | test/dygraph_to_static/test_origin_info.py | #58464 |
64 | test/dygraph_to_static/test_param_guard.py | #58464 |
65 | test/dygraph_to_static/test_params_no_grad.py | #58464 |
66 | test/dygraph_to_static/test_partial_program_hook.py | #58465 |
67 | test/dygraph_to_static/test_partial_program.py | #58316 |
68 | test/dygraph_to_static/test_place.py | #58465 |
69 | test/dygraph_to_static/test_print.py | #58465 |
70 | test/dygraph_to_static/test_program_translator.py | #58465 |
71 | test/dygraph_to_static/test_ptb_lm_v2.py | #58465 |
72 | test/dygraph_to_static/test_ptb_lm.py | #58465 |
73 | test/dygraph_to_static/test_pylayer.py | #58465 |
74 | test/dygraph_to_static/test_reinforcement_learning.py | #58465 |
75 | test/dygraph_to_static/test_resnet_amp.py | #58465 |
76 | test/dygraph_to_static/test_resnet_pure_fp16.py | #58465 |
77 | test/dygraph_to_static/test_resnet_v2.py | #58465 |
78 | test/dygraph_to_static/test_resnet.py | #58465 |
79 | test/dygraph_to_static/test_return.py | #58465 |
80 | test/dygraph_to_static/test_rollback.py | #58465 |
81 | test/dygraph_to_static/test_save_inference_model.py | #58465 |
82 | test/dygraph_to_static/test_save_load.py | #58490 |
83 | test/dygraph_to_static/test_se_resnet.py | #58490 |
84 | test/dygraph_to_static/test_sentiment.py | #58509 |
85 | test/dygraph_to_static/test_seq2seq.py | #58490 |
86 | test/dygraph_to_static/test_set_dynamic_shape.py | #58316 |
88 | test/dygraph_to_static/test_simnet_v2.py | #58490 |
89 | test/dygraph_to_static/test_simnet.py | #58490 |
90 | test/dygraph_to_static/test_slice.py | #58490 |
91 | test/dygraph_to_static/test_spec_names.py | #58316 |
93 | test/dygraph_to_static/test_tensor_hook.py | #58490 |
94 | test/dygraph_to_static/test_tensor_memcpy_on_cpu.py | #58490 |
95 | test/dygraph_to_static/test_tensor_memcpy_on_gpu.py | #58490 |
96 | test/dygraph_to_static/test_tensor_methods.py | #58490 |
97 | test/dygraph_to_static/test_tensor_shape.py | #58316 |
98 | test/dygraph_to_static/test_to_tensor.py | #58490 |
99 | test/dygraph_to_static/test_train_step.py | #58490 |
100 | test/dygraph_to_static/test_transformer.py | #58490 |
101 | test/dygraph_to_static/test_tsm.py | #58490 |
102 | test/dygraph_to_static/test_typehint.py | #58490 |
103 | test/dygraph_to_static/test_typing.py | #58499 |
104 | test/dygraph_to_static/test_unuseful_inputs.py | #58499 |
105 | test/dygraph_to_static/test_utils.py | #58499 |
106 | test/dygraph_to_static/test_variable_trans_func.py | #58499 |
107 | test/dygraph_to_static/test_warning.py | #58499 |
108 | test/dygraph_to_static/test_word2vec.py | #58499 |
109 | test/dygraph_to_static/test_write_python_container.py | #58509 |
110 | test/dygraph_to_static/test_yolov3.py | #58499 |
111 | test/legacy_test/test_cond.py | #58509 |
112 | test/legacy_test/test_while_loop_op.py | #58509 |
113 | test/legacy_test/test_while_op.py | #58509 |
114 | test/dygraph_to_static/test_no_gradient.py | #58509 |