PPO reward normalization works only for default gamma #203
Description
Problem Description
Current implementation of continuous action PPO uses gym.wrappers.NormalizeReward with default gamma value, for all other gamma's except default 0.99 this normalization will be not correct.
cleanrl/cleanrl/ppo_continuous_action.py
Line 92 in 94a685d
Possible Solution
Very easy, just add gamma=args.gamma as an argument to the normalization wrapper.