Describe the bug

I commit issue yesterday: #5085 , but @sheremet-va closed it and mark it to duplicate to #4497 .
I have tried ways of #4497 metioned, but still not working, and I don't think they are same problem, please try my demo in Reproduction, it still can not debugger in browser mode, THANKS~

Reproduction

https://github.com/gezilinll/vitest-browser-debugger

System Info

System:
    OS: macOS 13.5.2
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 288.28 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 18.15.0 - ~/Library/Caches/fnm_multishells/56721_1706690459670/bin/node
    Yarn: 1.22.19 - ~/Library/Caches/fnm_multishells/56721_1706690459670/bin/yarn
    npm: 9.5.0 - ~/Library/Caches/fnm_multishells/56721_1706690459670/bin/npm
    pnpm: 7.30.0 - ~/Library/Caches/fnm_multishells/56721_1706690459670/bin/pnpm
  Browsers:
    Chrome: 121.0.6167.139
    Safari: 16.6

Used Package Manager

pnpm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.