Meet errors when use trl grpo --reward_funcs my_lib.my_reward

Traceback (most recent call last):
  File "/root/miniforge3/envs/hf-trl/lib/python3.12/site-packages/trl/scripts/grpo.py", line 136, in <module>
    main(script_args, training_args, model_args)
  File "/root/miniforge3/envs/hf-trl/lib/python3.12/site-packages/trl/scripts/grpo.py", line 91, in main
    sys.path.insert(os.getcwd())
TypeError: insert expected 2 arguments, got 1
Traceback (most recent call last):
  File "/root/miniforge3/envs/hf-trl/bin/trl", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/root/miniforge3/envs/hf-trl/lib/python3.12/site-packages/trl/cli.py", line 104, in main
    launch_command(args)  # launch training
    ^^^^^^^^^^^^^^^^^^^^
  File "/root/miniforge3/envs/hf-trl/lib/python3.12/site-packages/accelerate/commands/launch.py", line 1213, in launch_command
    simple_launcher(args)
  File "/root/miniforge3/envs/hf-trl/lib/python3.12/site-packages/accelerate/commands/launch.py", line 795, in simple_launcher
    raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)
subprocess.CalledProcessError: Command '['/root/miniforge3/envs/hf-trl/bin/python3.12', '/root/miniforge3/envs/hf-trl/lib/python3.12/site-packages/trl/scripts/grpo.py', '--model_name_or_path', '/root/.cache/modelscope/hub/models/Qwen/Qwen3-4B', '--dataset_name', 'argilla/Capybara-Preferences', '--output_dir', 'res_v1', '--reward_funcs', 'my_lib.my_reward']' returned non-zero exit status 1.

That is because there is no current path in sys.path, it works if add sys.path.insert(0, os.getcwd())

Rewards functions for Command Line Interfaces GRPO trainer #3448

Description

Feature request

Motivation

Your contribution

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions