Skip to content

Local file is lost even if the download fails #479

New issue
New issue
Closed
#582
Closed
Local file is lost even if the download fails#479
#582
Assignees
ahmethakanbesel
Milestone
v2.2.0
@kucukaslan

Description

@kucukaslan
kucukaslan
opened on Aug 1, 2022

Issue & steps to reproduce

$ echo "Example text." > tmp
$ cat tmp
Example text.
$ s5cmd cp s3://nonexistentbucket/key tmp
ERROR "cp s3://nonexistentbucket/key tmp": InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records. status code: 403, request id: PAYWDH7HHVPAHC8G, host id: cnI/cBhYZKKJ159f8mMs4KHOn1+jYtvfpYnXgMuKMRd9pZ10FBdi/cGuVxyr+iwrKbk2kP7Opx4=
$ cat tmp
cat: tmp: No such file or directory

We would expect the local file to remain untouched since the download operation did not even started.

Proposed Solution

Instead of creating the file before the download request a custom WriterAt, which will create the file once the first write request came, can be used. Following snippets are going to change:

s5cmd/command/cp.go

Lines 511 to 517 in 123b1c7

file, err := dstClient.Create(dsturl.Absolute())
if err != nil {
return err
}
defer file.Close()
size, err := srcClient.Get(ctx, srcurl, file, c.concurrency, c.partSize)

s5cmd/storage/fs.go

Lines 208 to 214 in 0431f50

func (f *Filesystem) Create(path string) (*os.File, error) {
if f.dryRun {
return &os.File{}, nil
}
return os.Create(path)
}

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

s5cmd

Status

Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions