This issue follows from #42328 which is now closed, and where @mpx suggested I create a new one.

Overview

I have a //go:embed of ~700 files, ~70 directories and where some files have a leading underscore. These are not dot files, nor go source files, and it took me some time to work out that the reason I couldn't open some of them without error was that they were excluded from the build.

The draft proposal doesn't mention this restriction. It was written in July and the exclusion of these files was made in December by a change resulting from #42328.

It was counterintuitive to me that files would be arbitrarily excluded because they're considered "hidden" on some platforms. Why counterintuitive? I don't know of any other tools that automatically exclude dot or leading-underscore files, other than ls for dot files, which provides the well-known -a escape hatch. Git, find, tar, etc., include such files by default. Even the macOS Finder allows you to toggle display dot files with ⌘-shift-..

Related to this, I sometimes use "filesystem templates" that include empty directories that are later populated by code: these are either initially extracted to the filesystem, or used with a virtual in-memory filesystem. To embed such a template using //go:embed, I'd have to add placeholder files like a readme.txt with contents "This file intentionally left blank.". (When forced to do this, it is preferable to use .empty dot files, but they're also unsupported by //go:embed!)

I've also experienced a similar chicken or the egg problem with an empty embed dir; see [1] below.

Sometimes we need to include everything in an embed of a directory tree. Please give us a way out! :)

Suggestions and feedback

A simple opt-in flag like //go:embed -all dir to include everything in dir without filtering, and without declaring an error if the directory is empty [1]. This flag would instruct //go:embed to include all files, including those having a leading underscore, and any dot files. For the edge-case of a file named "-all", we can use a simple flag terminator: //go:embed -all -- -all. The flag package makes this implementation straightforward.

Otherwise having some explicit filtering option in the directive (with an "otherwise include-all") would be welcomed, as was suggested in #42325.

The proposal in #42328 for //go:embed dir vs. //go:embed dir/** is very subtle and reminds me of the arcane build constraint syntax that was thankfully recently revisited. An explicit inclusion-list, written by hand or code generated, is a big turn-off.

[1] In one project, I have a package that embeds a directory tree that is generated by another program. The generator needs to use the same package, which leads to a chicken or the egg problem. I currently resolve this by having a Makefile add a readme.txt file into the embed directory before each build (which unfortunately remains in the embed). I am not using the Makefile for any other purpose here.

Workarounds

1. Write more code and wrap my file tree in a store-only ZIP file and //go:embed that.
2. Use a code-generator tool: either to explicitly generate //go:embed filenames for the entire tree, or a 3rd party embed tool (and eat the slower build times).
3. Using some encoding to escape filenames that //go:embed otherwise denies me from using naturally.
4. Add placeholder (non-dot) files in empty directories that are managed by Makefiles.

The first two options are not preferred because they burden the project with an external generator. The third option would force design changes to a larger surface area of an otherwise already complex project just to simplify the build.

I'd love to avoid the use of Makefiles in the last workaround, especially when they serve no other purpose.