Not learning reward?

[IanWangg]

Hi,

In the original code base, the algo does not learn the reward function, instead, the agent computes cumulative reward using the true reward function (directly get from env). Meaning we are actually using some true information from the environment, is that allowed in the offline setting? Also, after changing the 'learn_reward' option to True in the config file, the performance is much worse.

[IanWangg]

cannot reproduce the results presented by the paper on D4RL benchmark when using the learned reward model.

[Valerio-Colombo]

Hi,
I'm having the same problem with the hopper example provided. Has anyone solved the problem?
The rollout reward is stuck at very low values

[Valerio-Colombo]

I may have a solution. In the readme, it suggests to change the configuration file putting 'learn_reward' = True and 'reward_file' = None. If you put 'reward_file' = None the execution stops due to an error. What I find is to keep the 'reward_file' field, but comment out the reward function contained in there. If you simply comment the 'reward_file' flag, you will also delete the termination function used for the rollouts.
At least in the hopper example, these changes fixed the problem.