Skip to content

BatchNorm1D eval模式报错CUDNN_STATUS_NOT_SUPPORTED #49718

@justld

Description

@justld

bug描述 Describe the Bug

BatchNorm1D在eval模式,报错CUDNN_STATUS_NOT_SUPPORTED.

环境:
cuda 10.2
Tesla v100 32gb
paddlepaddle-gpu 2.4.1

复现代码:

import paddle
x = paddle.randn((69798, 32, 64))     # 怀疑是输入太大了,当输入维度是[49931, 32, 64]时没问题
norm = paddle.nn.BatchNorm1D(64, epsilon=1e-3, momentum=0.99)   
norm.eval()    
out = norm(x.transpose([0, 2, 1]))       

报错内容:
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 948, in call
return self.forward(*inputs, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/nn/layer/norm.py", line 725, in forward
use_global_stats=self._use_global_stats,
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/nn/functional/norm.py", line 233, in batch_norm
False,
OSError: (External) CUDNN error(9), CUDNN_STATUS_NOT_SUPPORTED.
[Hint: Please search for the error code(9) on website (https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cudnnStatus_t) to get Nvidia's official solution and advice about CUDNN Error.] (at /paddle/paddle/phi/kernels/gpu/batch_norm_kernel.cu:849)

其他补充信息 Additional Supplementary Information

No response

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions