Describe the bug
Always the same speaker output in fastspeech2 aishell3 voice conversion

To Reproduce

1. PaddleSpeech语音克隆 always output the same speaker.
2. When I change the synthesizer to Tacotron2, everything works fine, the model can generate different speaker speech.
3. Here are some outputs I packed:
   output_sound_fastspeech2.zip
   output_sound_tacotron2.zip