need to refactor off-policy critic first to avoid code duplication

I will do it.