Feature/backup mode #1024
Merged
Feature/backup mode #1024
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* feat: add backup api * refactor: delete exp count in callback * refactor: create folder not only local * refactor: log handler and callback params * refactor: again! * chore: revert log
There was a problem hiding this comment.
Pull Request Overview
This PR adds a new “backup” logging mode that retains all cloud-mode features locally without uploading, introduces a shared backup model layer, and integrates backup callbacks across the SDK and runtime.
- Extend ProxyType and write logic to skip proxying when proxy_type is
"none". - Introduce
BackupHandler, backup models (Pydantic), and writer utilities for serializing/deserializing and writing backup data. - Update environment, SDK initialization, run operator, and all callbackers (local/cloud/backup) to respect the new backup mode.
Reviewed Changes
Copilot reviewed 29 out of 29 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| swanlab/log/type.py | Add 'none' to ProxyType to disable terminal proxy. |
| swanlab/log/log.py | Guard log uploads when no messages are present. |
| swanlab/log/backup/models.py | New backup data models with JSON (de)serialization. |
| swanlab/log/backup/backup_handler.py | BackupHandler for async writing of backup entries. |
| swanlab/log/backup/writer.py | Helper to write runtime info and media buffers. |
| swanlab/env.py | Extend allowed modes with "backup". |
| swanlab/data/utils.py | Import BackupCallback and login info type. |
| swanlab/data/sdk.py | Support "backup" in MODES and docs. |
| swanlab/data/run/main.py | Integrate should_save logic for backup mode. |
| swanlab/data/run/helper.py | Adjusted on_stop signature to include epoch. |
| swanlab/data/run/callback.py | Centralize backup start/stop and proxy registration. |
| swanlab/data/callbacker/local.py | Hook in backup callbacks alongside local behavior. |
| swanlab/data/callbacker/cloud.py | Hook in backup callbacks alongside cloud behavior. |
| swanlab/data/callbacker/backup.py | Define no-op backup-specific callbacks. |
| swanlab/api/upload/model.py | Include LogModel and adjust metric path prefix. |
| swanlab/api/upload/init.py | Export LogModel. |
| swanlab/api/http.py | Expose history_exp_count and adjust mount_project. |
SAKURA-CAT
added a commit
that referenced
this pull request
Jun 5, 2025
SAKURA-CAT
added a commit
that referenced
this pull request
Jun 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🏠 主要功能
完成了日志备份功能的 80%：
backup模式，此模式下支持云端模式的所有功能但不上传文件，并保存相关备份文件到run_dir文件夹内backup、cloud、local三大模式都支持备份日志文件，并且在设计上同时支持云端的所有功能local模式的数据格式，现在local模式只在swanboard回调层面区分了部分不兼容的图表类型（比如 3D 点云）和功能（比如硬件监控），相关文件依旧在run_dir文件夹内，它是backup模式保存数据的全集，这意味着用户可以将local模式下的日志输出保存到云端回调模型和上传接口模型之间的中间层——备份模型，允许在回调模型和上传接口模型之间互相切换，这是日志备份功能的基础。💡 设计思想
backup模式新增BackupCallback回调类，方便触发备份器事件run_dir文件夹内，包括a. backup.swanlab：对一次实验的整体性描述，包含日志等信息
b. files：继承原本实验元信息保存的设计思想，保存conda环境、python环境、系统信息、用户config配置对象等文件
c. media：保存实验产出的媒体文件
⌛️ 待完成