Documentation preview for this PR (built with commit 9c023fb; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

actually what is livedoc release? I mostly just use the Documentation preview link for development which still works… I think.

and what rule determines which user can deploy to which netlify URL?

actually what is livedoc release? I mostly just use the Documentation preview link for development which still works… I think.

See the updated PR description. The doc-release is published on every release (so about weekly).

and what rule determines which user can deploy to which netlify URL?

No user deploys. The netlify URL is determined by the sage repo secrets NETLIFY_SITE_ID and NETLIFY_AUTH_TOKEN which currently points to my netlify site.

You can add "Fixes #40463" to the PR description.

Thanks. A test running at my repo is not yet finished; hence "Draft" state yet.

Is there a published version of the built PDF documentation accessible somewhere? I can't find it.

The test is building pdfs. I will show you when it finished.

I assume meson builds things faster than make (but doc-build is not 100% meson either, it still uses bootstrap at some parts), it's unfortunate that meson live doc doesn't work. If I understood correctly the logic is moved to doc-build-pdf just because doc-build-pdf still uses make (while doc-build uses meson).

Still, working slow code is better than broken code.

(also, any idea which part of the code is responsible for deploying the documentation preview on each pull request, and why does that one actually work?)

I assume meson builds things faster than make (but doc-build is not 100% meson either, it still uses bootstrap at some parts), it's unfortunate that meson live doc doesn't work.

It does work. But doc-release (=live doc + pdf docs) needs pdf docs, for which meson does not work yet (right?).

If I understood correctly the logic is moved to doc-build-pdf just because doc-build-pdf still uses make (while doc-build uses meson).

Yes.

Still, working slow code is better than broken code.

For once a weak release, "slow" does not matter much.

(also, any idea which part of the code is responsible for deploying the documentation preview on each pull request, and why does that one actually work?)

See doc-publish.yml

But doc-release (=live doc + pdf docs) needs pdf docs, for which meson does not work yet (right?).

not really. sage --docbuild en/tutorial pdf works just fine in meson, provided you install the sage_docbuild package into the Python environment first. So is sage --docbuild reference/stats pdf.

sage --docbuild en/reference pdf, however, is currently broken… (with or without the en)

It does work.

I assume the original # The following fails randomly comments (that you deleted) means that it does not work. I haven't checked what's the error.

Thanks for fixing this!

A few initial questions/comments:

• What's the reason for moving the livedocs to the pdf run? They are just a different 'flavor' of the html build, right? Perhaps it's easier to just move the pdf build to the html (as a separate job) and then rename the workflow simply to "doc build".
• I would prefer if it uses meson directly, if that's now working.
• The released html seems to have broken styles, eg https://doc-release--sagemath.netlify.app/html/en/reference/rings_standard/ doesn't render nicely for me.
• Could you please also add this "doc-release" url somewhere in the documentation and readme, so that it's easier to find.

• What's the reason for moving the livedocs to the pdf run? They are just a different 'flavor' of the html build, right? Perhaps it's easier to just move the pdf build to the html (as a separate job) and then rename the workflow simply to "doc build".
• I would prefer if it uses meson directly, if that's now working.

I think this is already discussed above: none of us knows why livedoc on meson fails (yet), so a temporary workaround is to use it in the pdf, which is still based on make.

• The released html seems to have broken styles, eg https://doc-release--sagemath.netlify.app/html/en/reference/rings_standard/ doesn't render nicely for me.

probably just a mistake caused while doing the manual upload, if you change the sagemath to sagemath-test (automated upload) then it works well. In any case I don't see it as an evidence of the code being incorrect.

• Could you please also add this "doc-release" url somewhere in the documentation and readme, so that it's easier to find.

As mentioned in the first message, it's linked from the wiki. I agree it's hard to find though, having in the README would be good.

The README, however, links to https://doc.sagemath.org/html/en/index.html—which makes sense, since this is user-facing, and unreleased versions are not visible to most users. Maybe https://doc.sagemath.org/html/en/developer/index.html#section-updating-documentation would be suitable.

It does work.

I assume the original # The following fails randomly comments (that you deleted) means that it does not work. I haven't checked what's the error.

It works with "-j1", as you suggested as a fix.

Okay, so currently we have meson builds HTML doc, and make builds PDF doc.

Why do we need to build the HTML again in the "Build live doc" step, if meson can already build HTML with -j1 workaround?

• I would prefer if it uses meson directly, if that's now working.

I think this is already discussed above: none of us knows why livedoc on meson fails (yet), so a temporary workaround is to use it in the pdf, which is still based on make.

Right. Later if everything works well with meson, all could be combined into single "doc-build.yml" and "doc-publish.yml". However, since building pdf docs takes more time, having separate workflows might be still preferred.

• The released html seems to have broken styles, eg https://doc-release--sagemath.netlify.app/html/en/reference/rings_standard/ doesn't render nicely for me.

probably just a mistake caused while doing the manual upload, if you change the sagemath to sagemath-test (automated upload) then it works well. In any case I don't see it as an evidence of the code being incorrect.

Correct.

• Could you please also add this "doc-release" url somewhere in the documentation and readme, so that it's easier to find.

As mentioned in the first message, it's linked from the wiki. I agree it's hard to find though, having in the README would be good.

The README, however, links to https://doc.sagemath.org/html/en/index.html—which makes sense, since this is user-facing, and unreleased versions are not visible to most users.

Right. "doc-release" is for developers. That provides a sample to be eventually released for users.

On the other hand, https://doc.sagemath.org provides the official doc for stable releases for ordinary users. I hope that that also provides live doc...

Maybe https://doc.sagemath.org/html/en/developer/index.html#section-updating-documentation would be suitable.

I looked around. But I don't see a suitable place; the section is focused on local development.

Perhaps Wiki itself should be advertised, but isn't it wildly visible?

For your information, "doc-release" is also essential for "other version" selector (see below the Sage logo on the left panel) to work. It is currently broken.

"doc-release" is the source of "other versions" info. See https://doc-release--sagemath.netlify.app/html/en/versions.txt

I updated the Wiki to emphasize the doc-release.

Why do we need to build the HTML again in the "Build live doc" step, if meson can already build HTML with -j1 workaround?

"again" because it is "live doc". And in doc-build-pdf because doc-release should contain pdf docs.

An alternative approach is to build live doc with '-j1' in "doc-build" and publish doc-release with pdf docs built in "doc-build-pdf" (after wasting time waiting for pdf docs) But this approach would slow down doc previews for PRs.

"again" because it is "live doc"

Just to clarify, the point is the Sage Live tab on the documentation page right?

I've actually never heard of the feature before. It needs more advertisement, I suppose.

Perhaps Wiki itself should be advertised, but isn't it wildly visible?

I've never heard of the wiki before either.

I think one useful place to introduce it is in the message by github-actions itself. Currently it says

            [Documentation preview for this PR](${{ steps.deploy-netlify.outputs.NETLIFY_URL }}/html/en) (built with commit ${{ steps.source-run-info.outputs.sourceHeadSha }}; [changes](${{ steps.deploy-netlify.outputs.NETLIFY_URL }}/CHANGES.html)) is ready! :tada:
            This preview will update shortly after each push to this PR.

we could add a

This message is automatically generated by <something>.
Refer to https://doc.sagemath.org/html/en/developer/[...] for an explanation how it works.

An alternative approach is to build live doc with '-j1' in "doc-build" and publish doc-release with pdf docs built in "doc-build-pdf" (after wasting time waiting for pdf docs) But this approach would slow down doc previews for PRs.

I like this approach better. The live docs are only build on releases and take quite some time, right? So in this case, it would be okay to wait for the live docs + pdf build before publishing them together.

Btw, what is the difference between the doc-release and doc-develop netlify sites?

I would be fine changing the first sentence on https://doc-develop--sagemath.netlify.app/html/en/index.html to "Documentations in other languages and the most recent beta release are available too." and then link the doc-release. And a similar sentence can be added in the readme.

For your info, in #40597, I've now fixed the pdf-meson build and migrated the CI to it.

I've actually never heard of the feature before. It needs more advertisement, I suppose.

It was first advertised here: https://github.com/sagemath/sage/wiki/Sage-10.2-Release-Tour#documentation

... we could add a

This message is automatically generated by <something>.
Refer to https://doc.sagemath.org/html/en/developer/[...] for an explanation how it works.

I guess we are talking about doc-release and live doc; this thing is different from doc previews for PRs.

I think we may add a subsection about the automatic generation of various docs for developers available in github interface (perhaps also about the Wiki :-) in the section about the github repo: https://doc-release--sagemath.netlify.app/html/en/developer/github# However, then this issue is somewhat beyond the present PR. I may do this work in a new PR.

An alternative approach is to build live doc with '-j1' in "doc-build" and publish doc-release with pdf docs built in "doc-build-pdf" (after wasting time waiting for pdf docs) But this approach would slow down doc previews for PRs.

I like this approach better. The live docs are only build on releases and take quite some time, right? So in this case, it would be okay to wait for the live docs + pdf build before publishing them together.

I gave up that approach and took the present approach, because that approach would slow down generation of doc previews for PRs since the doc-publish action needs to wait for the completion of building pdf docs, which takes long.

Btw, what is the difference between the doc-release and doc-develop netlify sites?

doc-develop is used when doc preview diffs (changes) for a PR is generated, by comparing it with doc-pr-xxxxx.

I would be fine changing the first sentence on https://doc-develop--sagemath.netlify.app/html/en/index.html to "Documentations in other languages and the most recent beta release are available too." and then link the doc-release. And a similar sentence can be added in the readme.

I don't agree. That sentence is about the specific doc version. It is inappropriate to talk about other versions in the same sentence.

Anyway, let's deal with the advertisement issue in a new PR mentioned in the previous comment.

For your info, in #40597, I've now fixed the pdf-meson build and migrated the CI to it.

I don't know meson and how things are going about meson transition. Hopefully in a sequel to this PR, you may convert pdf doc build stuffs to meson and perhaps then merge doc-build.yml and doc-build-pdf.yml (or not because of the delayed doc previews issue). Let's get this in for now.

Okay, now there is a merge conflict with #40597. Easiest way out is to create a separate workflow that is copied from the pre-meson migration, I guess.

Okay, now there is a merge conflict with #40597.

There is no merge conflict, fortunately.

Easiest way out is to create a separate workflow that is copied from the pre-meson migration, I guess.

No. Just replacing make with meson seems enough.

I am running tests. Let's see.

Easiest way out is to create a separate workflow that is copied from the pre-meson migration, I guess.

No. Just replacing make with meson seems enough.

Hmm. The situation is more complicated than I supposed.

I realized that reusing pdf docs built in doc-build-pdf for live doc is problematic since the workflow is supposed to build pdf docs with TESTS blocks included. Doc previews for PRs are also supposed to contain TESTS blocks. See https://doc-pr-40681--sagemath.netlify.app/html/en/developer/github#hunk1

Hence I ended up separating out doc-build-livedoc from doc-build-pdf, and accordingly renamed doc-publish-pdf to doc-publish-livedoc workflows, responsible to publish doc-release. Note that the live doc in doc-release is supposed to present a user-friendly doc and hence it does not contain TESTS blocks.

By the way, TESTS blocks currently don't appear both in doc previews for PRs and in PDF docs! This seems to be broken for months, like the spurious pdf icons problem.

setuptools(_scm) update is in 10.8.beta1. So reverting it might be hard to insist upon. What exactly does it do here?

setuptools(_scm) update is in 10.8.beta1. So reverting it might be hard to insist upon.

I separated out the "sagemath" kernel issue from this PR. See #40586 (comment). Previously, reverting it was needed to test this PR. Hopefully now I don't need it.

What exactly does it do here?

Nothing now.

Since the "sagemath" kernel issue is not strictly in the scope of this PR, I removed the "reverting to old setuptools" commit here.

OK, sorry, I missed "revert the revert".

I hope that the "sagemath" kernel issue is dealt with in other PR before the next release.

it's hopefully dealt with in #40700

I hope that the "sagemath" kernel issue is dealt with in other PR before the next release.

it's hopefully dealt with in #40700

Right. The new beta is still haunted with the sagemath kernel issue, even in meson. I am testing this PR with your #40700. The hope is shared with me.

... The new beta is still haunted with the sagemath kernel issue, even in meson. I am testing this PR with your #40700. The hope is shared with me.

I was wrong. The problem was with my branch. With -j1 (single thread) added to the meson command, the workflows complete successfully, and doc-release is published. #40700 is not necessary to test this PR.

On the other hand, the live doc is broken with the last release. Precisely speaking, "Sage", Python", "Sage Live" tabs in Examples do not appear. See https://doc-release--sagemath-test.netlify.app/html/en/reference/

Anyway, this PR itself is in good shape. Ready for review again. Thanks for your patience.

On the other hand, the live doc is broken with the last release. Precisely speaking, "Sage", Python", "Sage Live" tabs in Examples do not appear. See https://doc-release--sagemath-test.netlify.app/html/en/reference/

I suspect that this is just a symptom of the missing sagemath kernel in the meson build...

#40700 notes that using --user for the kernel installation option is not always what's needed

Positive?