Add folder hash for quick scan detection #4220
Conversation
There was a problem hiding this comment.
Summary of Changes
Hello @deluan, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces a significant optimization to the library scanning process. By adding a hash to each folder entry in the database, the scanner can now quickly determine if a folder's contents have changed without needing to perform a full directory walk and comparison every time. This should drastically speed up subsequent scans after the initial full scan.
Highlights
- Database Schema Change: Adds a new
hashcolumn to thefoldertable via a database migration to store a hash representing the folder's content state. - Folder Hashing Logic: Implements a method (
folderEntry.hash()) to calculate an MD5 hash for a folder based on its modification time, sorted lists of audio and image file names, number of playlists, number of subfolders, and image update timestamp. This hash provides a quick way to check if relevant folder contents have changed. - Optimized Scan Detection: Modifies the scanning logic (
folderEntry.isOutdated()) to use the stored folder hash. A folder is now considered 'outdated' (requiring a full rescan) if it was last updated before a full scan started OR if its current calculated hash differs from the previously stored hash. - Data Model and Repository Updates: Introduces a
FolderUpdateInfostruct to hold both theUpdatedAttime and the newHash. TheFolderRepository.GetLastUpdatesmethod is updated to return this new struct, allowing the scanner to retrieve the previous hash along with the last update time. - Unit Tests: Adds new unit tests for the folder hashing logic to ensure it produces stable hashes regardless of file order, and tests the updated
isOutdated()logic to verify it correctly identifies changes based on the hash.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configureGemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
The pull request introduces a folder hashing mechanism to speed up the detection of outdated folders during library scans. This involves adding a Hash column to the folder table, updating data models and repositories, and integrating the hash comparison into the scanning logic. Unit tests for the hashing and outdated detection have also been added.
The review identified a key issue with the database migration's down function, which needs to be implemented to ensure schema changes can be properly rolled back. Additionally, a potential performance optimization regarding redundant hash computations in the scanning logic was noted.
There was a problem hiding this comment.
Pull Request Overview
This PR enhances the quick scan detection by introducing a folder hash mechanism to determine if a folder is outdated based on its current contents rather than just its modification time. Key changes include:
- Adding a new "Hash" column to the folder table and corresponding migration.
- Updating the scan logic to compute a hash from folder attributes.
- Introducing unit tests to verify the new folder hash functionality.
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| scanner/walk_dir_tree.go | Removal of redundant folderEntry definitions now replaced by folder_entry.go |
| scanner/phase_1_folders.go | Updated lastUpdates map to use FolderUpdateInfo instead of time.Time |
| scanner/folder_entry.go | New implementation for folderEntry with folder hash calculation |
| persistence/folder_repository.go | Updated GetLastUpdates to return FolderUpdateInfo including the hash |
| model/folder.go | Added new "Hash" field and introduced FolderUpdateInfo type |
| db/migrations/20250701010101_add_folder_hash.go | New migration for adding the folder hash column to the database |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Summary
Smarter quick scan change detection, using a hash of the information from the folder, not just the ModTime of the folder.
Hashcolumn to folder table and migrationTesting
make testhttps://chatgpt.com/codex/tasks/task_b_684aef16bbb8832e9ac41836e89087d0