Skip to content

Conversation

aabadie
Copy link
Contributor

@aabadie aabadie commented Mar 2, 2018

Contribution description

This PR is an attempt to move the static-test from travis to circleci. The main ideas behind are:

  • use the riotbuild docker image to run the static-test and be consistent with workers of Murdock
  • have a per PR and online access to the generated documentation (this was my initial goal in fact) using the build artifacts feature provided by circleci.

I tested the solution on my own fork and it works. One can see the results here: https://circleci.com/gh/aabadie/RIOT/8 and the generated documentation here: https://8-44016379-gh.circle-artifacts.com/0/tmp/doc/html/index.html

The main problem I see is the time it takes to upload the artifacts to the final report. The total build time is 9 minutes and the upload takes a bit more than 7 minutes.

Issues/PRs references

None

@aabadie aabadie added Area: doc Area: Documentation Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR Area: CI Area: Continuous Integration of RIOT components labels Mar 2, 2018
@aabadie aabadie changed the title ci: setup static tests on circleci ci: setup static test on circleci Mar 2, 2018
@kaspar030
Copy link
Contributor

have a per PR and online access to the generated documentation (this was my initial goal in fact) using the build artifacts feature provided by circleci.

we could also save murdock's output.

@aabadie
Copy link
Contributor Author

aabadie commented Mar 6, 2018

we could also save murdock's output

I thought of it but the effort was not the same, at least for me.

@miri64
Copy link
Member

miri64 commented Feb 26, 2019

Is there still interest in this PR?

@stale
Copy link

stale bot commented Aug 30, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

@stale stale bot added the State: stale State: The issue / PR has no activity for >185 days label Aug 30, 2019
@stale stale bot removed the State: stale State: The issue / PR has no activity for >185 days label Sep 9, 2019
@aabadie
Copy link
Contributor Author

aabadie commented Sep 10, 2019

I've updated the initial goal of this PR to only build the documentation and use Circle CI Artifacts to get a preview of the generated documentation.
The last run result (on my fork) is available here: https://1526-44016379-gh.circle-artifacts.com/0/tmp/doc/html/index.html

The bad news is that now it take more than 25 minutes to upload the artifacts to S3 (the storage backend on CircleCI). This is because the doxygen documentation generates a lot of single file and they are copied one by one.
The advantage is that it requires no setup to get the preview (except the circleci config file, which is rather simple).

@aabadie aabadie changed the title ci: setup static test on circleci ci: use circleci to get a preview of the generated documentation Sep 10, 2019
@miri64
Copy link
Member

miri64 commented Sep 10, 2019

The bad news is that now it take more than 25 minutes to upload the artifacts to S3 (the storage backend on CircleCI).

Are there other backend options?

@aabadie
Copy link
Contributor Author

aabadie commented Sep 10, 2019

Are there other backend options?

I don't think so: https://circleci.com/docs/2.0/artifacts/

@stale
Copy link

stale bot commented Mar 13, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

@stale stale bot added the State: stale State: The issue / PR has no activity for >185 days label Mar 13, 2020
@aabadie
Copy link
Contributor Author

aabadie commented Mar 13, 2020

still useful, although slow.

@stale stale bot removed the State: stale State: The issue / PR has no activity for >185 days label Mar 13, 2020
@fjmolinas
Copy link
Contributor

Can we revive this one? As long as circle-ci does not block a PR getting merged I find this quite useful (even if slow), I have had many times issues or question about whether my locally generated doc would match what would be displayed on riot.org.

@aabadie aabadie force-pushed the pr/circleci branch 2 times, most recently from 2dcafe1 to 6371c03 Compare June 24, 2020 14:48
@miri64
Copy link
Member

miri64 commented Jun 24, 2020

What needs to be configured to add the status of circle-ci to the PR?

@aabadie
Copy link
Contributor Author

aabadie commented Jun 24, 2020

What needs to be configured to add the status of circle-ci to the PR?

I think integration of external applications, similar to Travis I guess.

If 25 minutes is acceptable before the documentation preview is available, using circleci is useful. Unfortunately, because of the huge amount of single files to upload (+19k), 99% of the time is spent in uploading the artifacts.
To speed-up things there are alternatives:

  • retrieve the documentation in an archive, still using circleci : this is way faster (a couple of minutes) but no online preview.
  • use github actions: there's one for uploading artifacts. I just tested in ci: setup github actions #14308 and it works, just takes 15 minutes to complete but has the same limitation: the documentation is available in an archive, so no online preview.

@miri64
Copy link
Member

miri64 commented Jun 24, 2020

We could (and probably should), like with Travis make the results non-binding. If for any reason the artefacts are of any interest, the wait time is not much longer as with current Murdock ;-).

@miri64
Copy link
Member

miri64 commented Jun 24, 2020

I think integration of external applications, similar to Travis I guess.

You have some doc on that?

@aabadie
Copy link
Contributor Author

aabadie commented Jun 24, 2020

You have some doc on that?

Could this one help: https://circleci.com/docs/2.0/gh-bb-integration/#section=projects ?

@miri64
Copy link
Member

miri64 commented Jun 24, 2020

Can you do a force push to see if it worked?

@miri64
Copy link
Member

miri64 commented Jun 24, 2020

@aabadie
Copy link
Contributor Author

aabadie commented Jun 24, 2020

Integration seems to work, I see a new circleci item.

@miri64
Copy link
Member

miri64 commented Jun 24, 2020

Integration seems to work, I see a new circleci item.

You seem to need to log in (via GitHub) to be able to see the progress.

@aabadie
Copy link
Contributor Author

aabadie commented Jun 24, 2020

So after 27 minutes of copy... the documentation artifacts are available here. And the generated documentation is available at https://2803-44016379-gh.circle-artifacts.com/0/doc/index.html

@miri64 miri64 added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: skip compile test If set, CI server will run only non-compile jobs, but no compile jobs or their dependent jobs labels Jun 24, 2020
Copy link
Member

@miri64 miri64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then I say: Let's try this out :-).

@miri64 miri64 added Reviewed: 1-fundamentals The fundamentals of the PR were reviewed according to the maintainer guidelines Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines Reviewed: 3-testing The PR was tested according to the maintainer guidelines Reviewed: 4-code-style The adherence to coding conventions by the PR were reviewed according to the maintainer guidelines labels Jun 24, 2020
@miri64 miri64 added the Reviewed: 5-documentation The documentation details of the PR were reviewed according to the maintainer guidelines label Jun 24, 2020
@miri64 miri64 added this to the Release 2020.07 milestone Jun 24, 2020
@miri64 miri64 merged commit 6b3e68a into RIOT-OS:master Jun 24, 2020
@aabadie aabadie deleted the pr/circleci branch June 24, 2020 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: CI Area: Continuous Integration of RIOT components Area: doc Area: Documentation CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: skip compile test If set, CI server will run only non-compile jobs, but no compile jobs or their dependent jobs Discussion: RFC The issue/PR is used as a discussion starting point about the item of the issue/PR Reviewed: 1-fundamentals The fundamentals of the PR were reviewed according to the maintainer guidelines Reviewed: 2-code-design The code design of the PR was reviewed according to the maintainer guidelines Reviewed: 3-testing The PR was tested according to the maintainer guidelines Reviewed: 4-code-style The adherence to coding conventions by the PR were reviewed according to the maintainer guidelines Reviewed: 5-documentation The documentation details of the PR were reviewed according to the maintainer guidelines
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants