-
Notifications
You must be signed in to change notification settings - Fork 5.8k
Description
bug描述 Describe the Bug
BatchNorm1D在eval模式,报错CUDNN_STATUS_NOT_SUPPORTED.
环境:
cuda 10.2
Tesla v100 32gb
paddlepaddle-gpu 2.4.1
复现代码:
import paddle
x = paddle.randn((69798, 32, 64)) # 怀疑是输入太大了,当输入维度是[49931, 32, 64]时没问题
norm = paddle.nn.BatchNorm1D(64, epsilon=1e-3, momentum=0.99)
norm.eval()
out = norm(x.transpose([0, 2, 1]))
报错内容:
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/fluid/dygraph/layers.py", line 948, in call
return self.forward(*inputs, **kwargs)
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/nn/layer/norm.py", line 725, in forward
use_global_stats=self._use_global_stats,
File "/usr/local/python3.7.0/lib/python3.7/site-packages/paddle/nn/functional/norm.py", line 233, in batch_norm
False,
OSError: (External) CUDNN error(9), CUDNN_STATUS_NOT_SUPPORTED.
[Hint: Please search for the error code(9) on website (https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cudnnStatus_t) to get Nvidia's official solution and advice about CUDNN Error.] (at /paddle/paddle/phi/kernels/gpu/batch_norm_kernel.cu:849)
其他补充信息 Additional Supplementary Information
No response