Skip to content

Conversation

yuya-takeyama
Copy link
Owner

@yuya-takeyama yuya-takeyama commented Aug 10, 2025

Summary

BREAKING CHANGE: Complete restructure of JSON output format, separating planning from execution results.

New Features

1. Separate Plan and Result Outputs

  • --plan-json-file: Outputs the execution plan before any operations
  • --result-json-file: Outputs actual execution results (not generated in dry-run mode)

2. Improved JSON Structure

Plan JSON (--plan-json-file)

{
  "files": [
    {
      "action": "create",     // "skip", "create", "update", "delete"
      "source": "/local/path/file.txt",
      "target": "s3://bucket/prefix/file.txt",
      "reason": "new file"    // Explanation for the action
    }
  ],
  "summary": {
    "skip": 5,      // Files unchanged (matching checksums)
    "create": 3,    // New files to upload
    "update": 2,    // Files to update
    "delete": 1     // Files to delete
  }
}

Result JSON (--result-json-file)

{
  "files": [
    {
      "action": "created",    // Past tense: "skipped", "created", "updated", "deleted"
      "source": "/local/path/file.txt",
      "target": "s3://bucket/prefix/file.txt"
    }
  ],
  "errors": [              // Always an array, even when empty
    {
      "action": "create",
      "source": "/local/path/failed.txt",
      "target": "s3://bucket/prefix/failed.txt",
      "error": "Permission denied"
    }
  ],
  "summary": {
    "skipped": 5,    // Successfully skipped (unchanged)
    "created": 3,    // Successfully created
    "updated": 2,    // Successfully updated
    "deleted": 1,    // Successfully deleted
    "failed": 1      // Failed operations (count of errors array)
  }
}

Key Changes

  1. Plan vs Result Separation

    • Plan shows what will be done (present tense)
    • Result shows what was done (past tense)
  2. Error Handling

    • Failed operations go to errors array with error messages
    • Failed operations do NOT count toward success metrics
    • Example: If update fails, it counts as failed, not updated
    • errors array is always present (empty array [] when no errors)
  3. Simplified Structure

    • Removed total_files field (can be calculated from summary)
    • Consistent file representation across plan and result
    • Clear separation between successful and failed operations
  4. Dry-run Behavior

    • --plan-json-file: Always generated when specified
    • --result-json-file: NOT generated in dry-run mode
  5. Skip Tracking

    • Files with unchanged checksums are included as "skip" in plan
    • Successfully skipped files appear as "skipped" in result
    • Excluded files (via --exclude patterns) are NOT included in JSON output

Migration Guide

Before (v0.2.x)

{
  "changes": [...],
  "summary": {
    "total_files": 10,
    "created": 5,
    "updated": 3,
    "deleted": 2,
    "failed": 0
  }
}

After (v0.3.0)

  • For planning: Use --plan-json-file
  • For results: Use --result-json-file
  • Structure completely changed (see examples above)

Testing

Tested with real S3 bucket (yuyat-strict-s3-sync-test):

  • ✅ Plan JSON output with all action types
  • ✅ Result JSON output with successful operations
  • ✅ Empty errors array when no failures
  • ✅ Skip tracking for unchanged files
  • ✅ Create/Update/Delete operations
  • ✅ Dry-run mode (no result JSON output)

Test Plan

  • Unit tests pass
  • Build successful
  • Manual testing with real S3 bucket
  • Verify plan output format
  • Verify result output format
  • Test error array initialization

🤖 Generated with Claude Code

yuya-takeyama and others added 2 commits August 11, 2025 04:28
- Add SkippedFile type to track excluded files with reason
- Collect skipped files during local and S3 file scanning
- Include skipped files and count in JSON result output
- Update Summary struct with Skipped field

This enhancement provides visibility into which files were excluded
from sync operations when using --exclude patterns.

Co-Authored-By: Claude <noreply@anthropic.com>
BREAKING CHANGE: JSON output structure changed
- Replaced "changes" with "files" array containing all file operations
- Removed separate "skipped" array for excluded files
- Files with unchanged checksums are now included with action "skip"
- Excluded files (via --exclude patterns) are NOT included in output
- Each file entry now includes "reason" field explaining the action

This provides a cleaner, more consistent JSON structure where all
processed files are in a single array with their action type and reason.

Co-Authored-By: Claude <noreply@anthropic.com>
@yuya-takeyama yuya-takeyama changed the title feat: add skipped files tracking to JSON output refactor: restructure JSON output to use unified Files array Aug 10, 2025
BREAKING CHANGE: Restructured JSON output format
- Added --plan-json-file for execution plan output
- Modified --result-json-file for execution results only
- result-json-file is not output in dry-run mode

Plan JSON structure:
- files: array of planned operations
- summary: counts by action type (skip, create, update, delete)

Result JSON structure:
- files: successfully executed operations
- errors: failed operations with error messages
- summary: execution counts (skipped, created, updated, deleted, failed)
- Removed total_files field

Key changes:
- Plan shows what will be done (present tense actions)
- Result shows what was done (past tense actions)
- Failed operations appear in errors array, not in summary counts

Co-Authored-By: Claude <noreply@anthropic.com>
@yuya-takeyama yuya-takeyama changed the title refactor: restructure JSON output to use unified Files array feat: separate plan and result JSON outputs with improved structure Aug 10, 2025
yuya-takeyama and others added 2 commits August 11, 2025 04:51
Ensure JSON output consistency by initializing the errors array
as an empty array [] instead of null when there are no errors.

Co-Authored-By: Claude <noreply@anthropic.com>
- Document --plan-json-file and --result-json-file options
- Add examples for both plan and result JSON structures
- Clarify dry-run behavior with result JSON
- Show difference between present tense (plan) and past tense (result)

Co-Authored-By: Claude <noreply@anthropic.com>
@yuya-takeyama yuya-takeyama marked this pull request as ready for review August 10, 2025 19:55
@yuya-takeyama yuya-takeyama merged commit ef87f33 into main Aug 10, 2025
1 check passed
@yuya-takeyama yuya-takeyama deleted the yuya-takeyama/feat/add-skipped-files-to-json-output branch August 10, 2025 19:55
@yuya-takeyama-tagpr yuya-takeyama-tagpr bot mentioned this pull request Aug 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant