Skip to content

dvc import: breaks progress bar when importing folders #10677

@petrchmelar

Description

@petrchmelar

Bug Report

dvc import: breaks progress bar when importing folders

Description

It seems that progress bar is not working correctly when importing folder.

Image

Reproduce

# prepare repo
mkdir reproducer && cd reproducer
git init && dvc init 

# generate dummy data
mkdir data
dd if=/dev/zero of=./data/file0 bs=4k iflag=fullblock,count_bytes count=1G
dd if=/dev/zero of=./data/file1 bs=4k iflag=fullblock,count_bytes count=1G

# commit data
dvc add data
git add .
git commit -m "Initial commit" 

# try to import
dvc import . -o imported-data data

Expected

Progress bar working.

Environment information

Output of dvc doctor:

╰─❯ dvc doctor
DVC version: 3.59.0 (pip)
-------------------------
Platform: Python 3.10.12 on Linux-5.15.0-117-generic-x86_64-with-glibc2.35
Subprojects:
        dvc_data = 3.16.8
        dvc_objects = 5.1.0
        dvc_render = 1.0.2
        dvc_task = 0.40.2
        scmrepo = 3.3.9
Supports:
        http (aiohttp = 3.11.11, aiohttp-retry = 2.9.1),
        https (aiohttp = 3.11.11, aiohttp-retry = 2.9.1),
        s3 (s3fs = 2024.9.0, boto3 = 1.36.3)
Config:
        Global: /home/pchmelar/.config/dvc
        System: /etc/xdg/dvc
Cache types: hardlink, symlink
Cache directory: nfs4 on 10.11.72.32:/mlops_data/workplace
Caches: local
Remotes: None
Workspace directory: nfs4 on 10.11.72.32:/mlops_data/workplace
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/f179e68f4c2a6cf27b5dc4aa96041e39

Additional Information (if any):

It can be probably fixed by adding child progress bars on line https://github.com/iterative/dvc/blob/main/dvc/fs/dvc.py#L581
I will prepare bugfix pull request

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions