Skip to content

Conversation

fmeum
Copy link
Member

@fmeum fmeum commented Feb 13, 2025

What type of PR is this?

Feature

What does this PR do? Why is it needed?

This makes it easier to download all relevant patches with --remote_download_regex without creating a bunch of empty patch files.

Which issues(s) does this PR fix?

Other notes for review

@fmeum fmeum force-pushed the non-empty-patches branch 3 times, most recently from 0761444 to ab9d525 Compare February 13, 2025 15:56
@fmeum
Copy link
Member Author

fmeum commented Feb 13, 2025

@sluongng Could you test this?

Copy link
Contributor

@sluongng sluongng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok it was hard to find the right regex, so let's document it somewhere. But this is working for me

--remote_download_regex='.*_nogo_patch.*'

oA semi related problem I found was that in some cases, the same nogo.patch would be generated twice:

?master ~/work/buildbuddy/buildbuddy> find -L bazel-out -name 'nogo.patch' | cat
bazel-out/platform_linux_x86_64-fastbuild/bin/enterprise/tools/rbesxs/rbesxs_nogo_patch/nogo.patch
bazel-out/platform_linux_x86_64-fastbuild/bin/enterprise/tools/rbesxs/rbesxs_lib_nogo_patch/nogo.patch
bazel-out/platform_linux_x86_64-fastbuild/bin/enterprise/server/remote_execution/snaploader/snaploader_nogo_patch/nogo.patch
bazel-out/platform_linux_x86_64-fastbuild/bin/enterprise/server/vmexec/vmexec_nogo_patch/nogo.patch
bazel-out/platform_linux_x86_64-fastbuild/bin/cli/remotebazel/remotebazel_nogo_patch/nogo.patch
bazel-out/platform_linux_x86_64-fastbuild/bin/cli/remotebazel/remotebazel_test.internal_nogo_patch/nogo.patch
bazel-out/platform_linux_x86_64-fastbuild-ST-d45aeea7532a/bin/enterprise/server/vmexec/vmexec_nogo_patch/nogo.patch
bazel-out/platform_linux_x86_64-fastbuild-ST-9e98d01b8f6c/bin/enterprise/server/remote_execution/snaploader/snaploader_nogo_patch/nogo.patch
?master ~/work/buildbuddy/buildbuddy> cat bazel-out/platform_linux_x86_64-fastbuild/bin/enterprise/tools/rbesxs/rbesxs_nogo_patch/nogo.patch

--- a/enterprise/tools/rbesxs/rbesxs.go
+++ b/enterprise/tools/rbesxs/rbesxs.go
@@ -126,7 +126,7 @@
    if *iterations < 0 {
        log.Fatal("--iterations cannot be < 0")
    }
-   if !(*environment == "dev" || *environment == "prod") {
+   if (*environment != "dev" && *environment != "prod") {
        log.Fatal("--environment must be 'dev' or 'prod'")
    }
    if *target == "" {
?master ~/work/buildbuddy/buildbuddy> cat bazel-out/platform_linux_x86_64-fastbuild/bin/enterprise/tools/rbesxs/rbesxs_lib_nogo_patch/nogo.patch

--- a/enterprise/tools/rbesxs/rbesxs.go
+++ b/enterprise/tools/rbesxs/rbesxs.go
@@ -126,7 +126,7 @@
    if *iterations < 0 {
        log.Fatal("--iterations cannot be < 0")
    }
-   if !(*environment == "dev" || *environment == "prod") {
+   if (*environment != "dev" && *environment != "prod") {
        log.Fatal("--environment must be 'dev' or 'prod'")
    }
    if *target == "" {

I think this happens if the go_library is also directly used for go_binary or go_test. In such case, one nogo action will be generated for each of the target

@sluongng
Copy link
Contributor

sluongng commented Feb 14, 2025

It seems like

find -L bazel-out -name 'nogo.patch' -print0 |\
  xargs -0 -I{} sh -c 'patch -p1 -N --reject-file /dev/null < {}'

works nicely, though can be a bit annoying to remember.

@fmeum
Copy link
Member Author

fmeum commented Feb 14, 2025

`--remote_download_regex='._nogo_patch.'

Does --remote_download_regex=.*/nogo.patch$ not work? That's the one I wanted to make work as it is much easier to come up with.

I think this happens if the go_library is also directly used for go_binary or go_test. In such case, one nogo action will be generated for each of the target

We have two different compilation actions, the one for the library and the one for the internal test that embeds the library (i.e., recompiles its sources). It's probably not common for that to result in different nogo violations, but it's also not obvious that it can't.

@fmeum fmeum force-pushed the non-empty-patches branch from ab9d525 to 56d5732 Compare April 29, 2025 15:50
@fmeum fmeum requested a review from sluongng April 29, 2025 17:57
@fmeum fmeum marked this pull request as ready for review April 29, 2025 17:57
@fmeum fmeum requested review from tyler-french and linzhp and removed request for sluongng April 30, 2025 17:15
@fmeum
Copy link
Member Author

fmeum commented May 2, 2025

bazelbuild/bazel#25976 will make this approach work better by avoiding the eager creation of directories.

@fmeum fmeum force-pushed the non-empty-patches branch 5 times, most recently from 0115909 to 7139ae9 Compare May 2, 2025 10:34
@fmeum fmeum requested review from sluongng and linzhp May 2, 2025 11:54
@fmeum fmeum force-pushed the non-empty-patches branch from 3682b2c to 57b939a Compare May 2, 2025 12:12
@fmeum fmeum requested a review from linzhp May 2, 2025 14:18
fmeum added 5 commits May 13, 2025 15:49
This makes it easier to download all relevant patches with `--remote_download_regex` without creating a bunch of empty patch files.

# Conflicts:
#	go/private/actions/archive.bzl
#	go/private/actions/compilepkg.bzl
@fmeum fmeum force-pushed the non-empty-patches branch from 57b939a to cf4d4cf Compare May 13, 2025 13:49
@fmeum fmeum enabled auto-merge (squash) May 13, 2025 13:50
@fmeum fmeum merged commit fb90c46 into master May 13, 2025
4 checks passed
@fmeum fmeum deleted the non-empty-patches branch May 13, 2025 14:24
fmeum added a commit that referenced this pull request May 13, 2025
**What type of PR is this?**

Refactor

**What does this PR do? Why is it needed?**

Follow-up to #4269

**Which issues(s) does this PR fix?**

**Other notes for review**
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants