After further testing and usage, we found that applying a reward penalty for effective gripper actions (command close when open and vice versa) speeds up the training of tasks that involve learning to grasp. Specifically, it prevents the policy from excessively opening and closing the gripper. The code has been updated to provide a -0.1 reward penalty by default -- a value that we found to work well for most tasks.

Also, we edited the environment so that gripper actions are executed in a blocking fashion. The environment waits until the gripper fully opens or closes before returning the observation. This also reduces the training time by making the transitions more Markovian.