Import problem when command line import stopped and restarted #1238
Description
Detailed description of the problem [REQUIRED]
Importing from command line with symlinks gives too vague error message, and situation is not handled well (in my opinion), when there is a problem, for example left overs from previous import process that has been killed mid process because of reboot or other various reasons.
Steps to reproduce the issue
Steps to reproduce the behavior:
- Start import of pic folder (
nice -n 19 php artisan lychee:sync /storage/NAS/photo/2021/ -vvvv --import_via_symlink=1 --skip_duplicates=1) - stop import process (crtl+C for example)
- Start import again to continue the process.
problem: Could not import file, that seems to happen to the one that was being processed when the previous import was terminated.
Command line tool output:
/storage/NAS/photo/2021/: 67% /storage/NAS/photo/2021/IMG_20210912_203350.jpg: Skipped duplicate /storage/NAS/photo/2021/IMG_20210912_203408.jpg: Skipped duplicate /storage/NAS/photo/2021/: 67% /storage/NAS/photo/2021/IMG_20210914_085702.jpg: Could not import file /storage/NAS/photo/2021/: 68% /storage/NAS/photo/2021/: 68% /storage/NAS/photo/2021/: 68%
Log:
2022-03-03 11:06:03 -- notice -- App\Actions\Photo\Strategies\StrategyDuplicate::storeFile -- 31 -- Nothing to store, image is a duplicate 2022-03-03 11:06:04 -- notice -- App\Actions\Photo\Strategies\StrategyDuplicate::hydrate -- 79 -- Skipped upload of existing photo because skipDuplicates is activated 2022-03-03 11:06:04 -- error -- App\Actions\Import\Exec::do -- 256 -- Could not import file (/storage/NAS/photo/2021/IMG_20210914_085702.jpg) 2022-03-03 11:06:18 -- notice -- App\Actions\Photo\Extensions\ImageEditing::createThumb -- 165 -- Photo URL is e2a91dad86a3825c5b3af74186dbf97b.jpg 2022-03-03 11:06:28 -- notice -- App\Image\ImagickHandler::crop -- 161 -- Saving thumb to /storage/www/Lychee/public/uploads/thumb/e2a91dad86a3825c5b3af74186dbf97b.jpeg 2022-03-03 11:07:02 -- notice -- App\Image\ImagickHandler::crop -- 161 -- Saving thumb to /storage/www/Lychee/public/uploads/thumb/e2a91dad86a3825c5b3af74186dbf97b@2x.jpeg
This seems to because a conflicting link (and maybe an entry to database, how to check?) was already in place, but database-entry is not complete(?) so it is not recognised as a duplicate(?). Photo is not shown in the gallery. Link to photo is in place in Lychee/public/uploads/big/
What I expected would happen: Lychee either just deletes the offending link/file and restarts that ones import, or at least gives more verbose error message about what is wrong so it can be corrected manually (how?)
If latter, there should be some --force parameter to delete offending/blocking stuff out of the way.
Output of the diagnostics [REQUIRED]
(Settings => Diagnostics or https://example.com/Diagnostics or php artisan lychee:diagnostics)
Diagnostics
-----------
Warning: Dropbox import not working. dropbox_key is empty.
Warning: user_agent for PHP is not set. You may experience problems when importing images via URL.
Warning: Using 32 bit PHP, recommended upgrade to 64 bit
Error: PHP zip extension not activated
System Information
------------------
Lychee Version (release): 4.4.0
DB Version: 4.4.0
composer install: --no-dev
APP_ENV: production
APP_DEBUG: true
System: Linux
PHP Version: 8.1
PHP User agent:
Max uploaded file size: 500M
Max post size: 500M
Max execution time: 0
MySQL Version: 10.5.12-MariaDB-0+deb11u1
Imagick: 1
Imagick Active: 1
Imagick Version: 1691
GD Version: 2.3.0
Config Information
------------------
version: 040400
check_for_updates: 0
sorting_Photos_col: taken_at
sorting_Photos_order: ASC
sorting_Albums_col: max_taken_at
sorting_Albums_order: ASC
imagick: 1
skip_duplicates: 1
small_max_width: 0
small_max_height: 360
medium_max_width: 1920
medium_max_height: 1080
lang: en
layout: 1
image_overlay_type: exif
default_license: none
compression_quality: 90
full_photo: 1
delete_imported: 0
Mod_Frame: 1
Mod_Frame_refresh: 30
thumb_2x: 1
small_2x: 1
medium_2x: 1
landing_page_enable: 0
landing_owner: JT
landing_title: JT
landing_subtitle:
landing_facebook:
landing_flickr:
landing_twitter:
landing_instagram:
landing_youtube:
landing_background:
site_title: Lychee - moerkoe
site_copyright_enable: 0
site_copyright_begin: 2019
site_copyright_end: 2019
additional_footer_text:
display_social_in_gallery: 0
public_search: 0
SL_enable: 0
SL_for_admin: 0
public_recent: 0
recent_age: 1
public_starred: 0
downloadable: 0
photos_wraparound: 1
map_display: 1
zip64: 1
map_display_public: 0
map_provider: Wikimedia
force_32bit_ids: 0
map_include_subalbums: 1
update_check_every_days: 3
has_exiftool: 1
share_button_visible: 1
import_via_symlink: 1
has_ffmpeg: 1
location_decoding: 0
location_decoding_timeout: 30
location_show: 1
location_show_public: 0
rss_enable: 0
rss_recent_days: 7
rss_max_items: 100
prefer_available_xmp_metadata: 0
editor_enabled: 1
lossless_optimization: 0
swipe_tolerance_x: 150
swipe_tolerance_y: 250
local_takestamp_video_formats: .avi|.mov
log_max_num_line: 1000
unlock_password_photos_with_url_param: 0
nsfw_visible: 1
nsfw_blur: 0
nsfw_warning: 0
nsfw_warning_admin: 0
map_display_direction: 1
album_subtitle_type: oldstyle
upload_processing_limit: 4
public_photos_hidden: 1
new_photos_notification: 0
Browser and system
Debian stable, current stable version of Lychee (both freshly installed 2 days ago). Browser: current chrome on mac os.