• Consolidate all workflows to GitHub Actions
  • Want to have draft/ready PR status as a filter, and Azure DevOps doesn't straightforwardly support it

Triggers

• Every push
  • Runs in the repo, so runs in the fork
• PR events
  • Default is open, reopen, synchronize
    • Synchronize is a catch-all for repo branch/history updates
  • Likely want a distinction between draft/ready
    • In draft state, run e.g. --testall --nonloc and report coverage; and maybe a couple other things?
      • Quick feedback for critical, functional problems
    • In ready state, run all the things, to get the PR actually ready for merge
    • In ready state and in release- branch merging to stable, run all of all the things?
• May want to have an action that makes all PRs draft-status if they're not already?
  • Triggers on pull_request.opened; no tests run
  • Contributor then triggers tests by either committing to the PR or by switching back to 'ready'
  • Turns out this doesn't really work - the draft conversion happens, but is overridden by something else. Not worth figuring out, and may be a toothache for contributors
  • At PR open, if the PR is opened in ready state, post bot comment about the option to switch to draft for streamlined CI.
• Also include a 'push (if no open associated PR)' trigger on the 'draft PR' workflows?
  • Test everything!
  • UPDATE: Avoiding duplicate workflows (both push and pull_request) is challenging, because there's no way(?) to query for a PR associated with the current commit/branch without using the REST API, and you can only use the REST API(?) inside a running workflow.
    • So, you could have the workflow get created, then have a step that queries the API to look for a PR, and sets an env var with whether to continue with the push; and then skip all subsequent steps if an open associated PR is found... but, meh.
• Obey the [skip ci] signal on the push/draft main PR workflows, but ignore it on ready-main-PR and stable/release PR workflows
  • Naaah... the whole point of the draft/ready dichotomy is so that in draft state the CI is quick enough that it's not a friction for it to run on every push.
• For PRs merging to a branch other than main or stable, only do the draft-PR/push light tests
• Add a job for PRs merging to stable that fail if not named as a release- branch.
  • Will need to handle PEP 440 version names
  • Logistically, this just needs to reject PRs incoming to stable that don't start with release-

Specific tasks/checks currently in place:

• flake8 linting
  • All main/stable PRs, ready state
• interrogate linting
  • Possibly replace by using flake8-docstrings-complete as part of flake8, instead?
    • Future decision
  • All main/stable PRs, ready state
• Single test run, current primary Python on windows and linux
  • PLUS codecov on linux for main/stable PRs, ready state
  • All PRs, any state
• Main tests
  • All main/stable PRs, ready state
  • Full matrix of Pythons and platforms (like azure-coretest.yml)
    • Perhaps just one macOS?
• Ensuring sdist builds, and that the sdist payload can itself run tests
  • Release PRs only, ready state
    • Figure out how to store a variable that's durable within the script scope
      • Use one to store the unpacked sdist dir
      • ChatGPT: echo "MY_VAR=some_value" >> $GITHUB_ENV
      • You can access the variable in subsequent steps using ${{ env.MY_VAR }} or $MY_VAR (depending on the shell).
    • Figure out if there's a way to durably change pwd mid-workflow-job
      • Doesn't look like it; a stored env variable appears to be the cleanest way
• Docs build with warnings as errors
  • Release PRs only, ready-state
• Doctests in code
  • All main/stable PRs, ready-state
• Doctests in README
  • Release PRs, ready-state
  • Only the Python. Removed the manual shell testing, too brittle/flaky.
  • Doesn't really look like there are other good options out there at the moment for shell doctests. :-/
• Coverage on tests, ensuring all of them ran
  • Release PRs, ready-state
• Never-fail flake8-noqa
  • Release PRs, ready-state
  • This is currently set up wrong in tox.ini
  • Also, the CI script doesn't need to install flake8-noqa, tox will do that