Skip to content

feat: support uploading images file by file #753

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 15 commits into from
Jun 18, 2025
Merged

feat: support uploading images file by file #753

merged 15 commits into from
Jun 18, 2025

Conversation

ptpt
Copy link
Member

@ptpt ptpt commented Jun 10, 2025 •
edited
Loading

Upload images file by file concurrently.

Benefits:

Upload speed comparison

Before: single thread uploading a zipfile of 2GB images

2.00G/2.00G [02:21<00:00, 15.2MB/s]

Now: parallel uploading with 128 workers

2.00G/2.00G [00:52<00:00, 41.0MB/s]

Now: parallel uploading with 64 workers

2.00G/2.00G [00:50<00:00, 42.8MB/s]

@ptpt ptpt merged commit 812f3fc into main Jun 18, 2025
19 checks passed
@ptpt ptpt deleted the feature-upload-images-file-by-file branch June 18, 2025 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
cla signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ptpt @facebook-github-bot