Skip to content

Support external blob #1672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Apr 2, 2025
Merged

Support external blob #1672

merged 2 commits into from
Apr 2, 2025

Conversation

BraveY
Copy link

@BraveY BraveY commented Feb 28, 2025

Relevant Issue (if applicable)

If there are Issues related to this PullRequest, please list it.

Details

Please describe the details of PullRequest.

Types of changes

What types of changes does your PullRequest introduce? Put an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • [x ] New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist

Go over all the following points, and put an x in all the boxes that apply.

  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

@BraveY BraveY requested a review from a team as a code owner February 28, 2025 04:48
@BraveY BraveY requested review from bergwolf, imeoer and Desiki-high and removed request for a team February 28, 2025 04:48
@codecov Codecov
Copy link

codecov bot commented Feb 28, 2025 •
edited
Loading

Codecov Report

Attention: Patch coverage is 59.72540% with 704 lines in your changes missing coverage. Please review.

Project coverage is 55.26%. Comparing base (871e1c6) to head (cfd0be1).
Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
contrib/nydusify/pkg/converter/converter.go 72.77% 70 Missing and 28 partials ⚠️
builder/src/directory.rs 0.00% 88 Missing ⚠️
builder/src/core/node.rs 23.76% 72 Missing and 5 partials ⚠️
contrib/nydusify/pkg/external/modctl/remote.go 58.77% 36 Missing and 18 partials ⚠️
contrib/nydusify/pkg/external/modctl/modctl.go 80.08% 36 Missing and 9 partials ⚠️
...trib/nydusify/pkg/snapshotter/external/external.go 53.60% 30 Missing and 15 partials ⚠️
builder/src/core/context.rs 48.75% 32 Missing and 9 partials ⚠️
src/bin/nydus-image/main.rs 0.00% 37 Missing ⚠️
...ydusify/pkg/snapshotter/external/backend/walker.go 64.83% 23 Missing and 9 partials ⚠️
builder/src/attributes.rs 71.15% 24 Missing and 6 partials ⚠️
... and 18 more

❗ There is a different number of reports uploaded between BASE (871e1c6) and HEAD (cfd0be1). Click for more details.

HEAD has 8 uploads less than BASE
Flag BASE (871e1c6) HEAD (cfd0be1)
9 1
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1672      +/-   ##
==========================================
- Coverage   60.28%   55.26%   -5.02%     
==========================================
  Files         146      196      +50     
  Lines       49086    55347    +6261     
  Branches    46398    46817     +419     
==========================================
+ Hits        29591    30588     +997     
- Misses      18026    23169    +5143     
- Partials     1469     1590     +121
Files with missing lines Coverage Δ
api/src/config.rs 88.78% <100.00%> (+0.03%) ⬆️
builder/src/core/bootstrap.rs 77.63% <100.00%> (+0.29%) ⬆️
builder/src/core/chunk_dict.rs 70.18% <ø> (ø)
builder/src/core/tree.rs 80.86% <100.00%> (-0.02%) ⬇️
builder/src/merge.rs 72.03% <100.00%> (ø)
builder/src/stargz.rs 75.00% <100.00%> (+0.14%) ⬆️
builder/src/tarball.rs 63.26% <100.00%> (+0.57%) ⬆️
contrib/nydusify/pkg/cache/cache.go 39.90% <100.00%> (ø)
...trib/nydusify/pkg/chunkdict/generator/generator.go 0.00% <ø> (ø)
contrib/nydusify/pkg/committer/commiter.go 0.00% <ø> (ø)
... and 34 more

... and 34 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@BraveY BraveY force-pushed the external-blob branch 5 times, most recently from 9ca650d to 24a62f1 Compare March 1, 2025 13:02
@imeoer imeoer changed the title [WIP]External blob Support external blob Mar 21, 2025
@BraveY BraveY closed this Mar 22, 2025
@BraveY BraveY reopened this Mar 22, 2025
@BraveY BraveY closed this Mar 22, 2025
@BraveY BraveY reopened this Mar 22, 2025
@BraveY BraveY requested a review from a team as a code owner March 22, 2025 08:40
@BraveY BraveY requested review from changweige and removed request for a team March 22, 2025 08:40
@BraveY
Copy link
Author

BraveY commented Mar 22, 2025

@codecov retest

@BraveY BraveY force-pushed the external-blob branch 4 times, most recently from e30592b to 26b3955 Compare April 1, 2025 08:39
@BraveY
builder: suport build external model image from modctl
6192a2b 
builder: add support for build external model image from modctl in local
context or remote registery.

feat(nydusify): add support for mount external large model images

chore: introduce GoReleaser for RPM package generation

nydusify(feat): add support for model image in check command

nydusify(test): add support for binary-based testing in external model's smoke tests

Signed-off-by: Yan Song <yansong.ys@antgroup.com>

Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
imeoer
imeoer reviewed Apr 2, 2025
View reviewed changes
imeoer
imeoer reviewed Apr 2, 2025
View reviewed changes
@imeoer
Copy link
Collaborator

imeoer commented Apr 2, 2025

Let's remove the unready nydusd (runtime) implemention. :)

@BraveY
nydusify: remove nydusd code which is working in progress
cfd0be1 
- remove the unready nydusd (runtime) implemention.
- remove the debug code.
Signed-off-by: Yang Kaiyong <yangkaiyong.yky@antgroup.com>
@BraveY
Copy link
Author

BraveY commented Apr 2, 2025

Let's remove the unready nydusd (runtime) implemention. :)

done

@BraveY BraveY closed this Apr 2, 2025
@BraveY BraveY reopened this Apr 2, 2025
imeoer
imeoer approved these changes Apr 2, 2025
View reviewed changes
Copy link
Collaborator

@imeoer imeoer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, LGTM!

@imeoer imeoer merged commit 8593498 into dragonflyoss:master Apr 2, 2025
45 of 46 checks passed
@BraveY BraveY deleted the external-blob branch April 14, 2025 07:08
@BraveY BraveY mentioned this pull request Aug 25, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@imeoer imeoer imeoer approved these changes

@bergwolf bergwolf Awaiting requested review from bergwolf bergwolf is a code owner automatically assigned from dragonflyoss/nydus-reviewers

@Desiki-high Desiki-high Awaiting requested review from Desiki-high Desiki-high is a code owner automatically assigned from dragonflyoss/nydus-reviewers

@changweige changweige Awaiting requested review from changweige changweige is a code owner automatically assigned from dragonflyoss/nydus-maintainers

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@BraveY @imeoer