Skip to content

Conversation

JadeCara
Copy link
Contributor

@JadeCara JadeCara commented Jul 14, 2025

Closes #ENG-899

Description Of Changes

Discovered an error where skipping/completing some manual tasks was causing the request to hang. This PR addresses that issue.

Code Changes

  • Cleaned up some unnecessary changes
  • Added some temporary checks which will be updated when we start adding manual task dependencies.

Steps to Confirm

  • In fidesplus pointing at this branch - create a manual task integration with both access and erasure fields.
  • Create a an access privacy request.
Screenshot 2025-07-10 at 1 03 23 PM
  • Verify there is only one manual task instance for that request. (if you have multiple integrations you can expect that number of instances)
fides=# SELECT * FROM manual_task_instance WHERE entity_id = 'pri_efe0712a-5208-4d89-b9e6-da100be95294';
-[ RECORD 1 ]---+-----------------------------------------
id              | man_3c598813-e2ae-4be4-8894-c992897086a3
created_at      | 2025-07-10 18:46:09.385489+00
updated_at      | 2025-07-10 18:46:09.385489+00
task_id         | man_ed6e68a9-7f49-43d0-bf42-cd09775e44d1
config_id       | man_9ed7fa90-e5b5-4a50-b1df-65f64243ee03
entity_id       | pri_efe0712a-5208-4d89-b9e6-da100be95294
entity_type     | privacy_request
status          | pending
due_date        | 
completed_at    | 
completed_by_id | 
  • Create an erasure privacy request.
Screenshot 2025-07-10 at 1 05 17 PM
  • Verify there is only one manual task instance for that request. (if you have multiple integrations you can expect that number of instances)
fides=# SELECT * FROM manual_task_instance WHERE entity_id = 'pri_58243c66-ef58-4704-848f-1f5e665e634d';
-[ RECORD 1 ]---+-----------------------------------------
id              | man_293cdec7-0536-432f-bf74-6623a43bbe8d
created_at      | 2025-07-10 18:54:09.981685+00
updated_at      | 2025-07-10 18:54:09.981685+00
task_id         | man_40fb48d0-b3b1-44ef-9077-69a0fe6aa409
config_id       | man_05147435-c5f1-4d14-874c-d5ab4226b61c
entity_id       | pri_58243c66-ef58-4704-848f-1f5e665e634d
entity_type     | privacy_request
status          | pending
due_date        | 
completed_at    | 
completed_by_id | 
  • Super safety double check the config ids fore these requests to ensure one of each config is creating and that those configs are for the right type of task (Access vs erasure)
    config_id | man_9ed7fa90-e5b5-4a50-b1df-65f64243ee03 != config_id | man_05147435-c5f1-4d14-874c-d5ab4226b61c

The access config:

fides=# SELECT * FROM manual_task_config where id = 'man_9ed7fa90-e5b5-4a50-b1df-65f64243ee03';
-[ RECORD 1 ]----+-----------------------------------------
id               | man_9ed7fa90-e5b5-4a50-b1df-65f64243ee03
created_at       | 2025-07-07 22:50:00.123959+00
updated_at       | 2025-07-07 22:50:00.123959+00
task_id          | man_ed6e68a9-7f49-43d0-bf42-cd09775e44d1
config_type      | access_privacy_request
version          | 3
is_current       | t
execution_timing | pre_execution

The erasure config

fides=# SELECT * FROM manual_task_config where id = 'man_05147435-c5f1-4d14-874c-d5ab4226b61c';
-[ RECORD 1 ]----+-----------------------------------------
id               | man_05147435-c5f1-4d14-874c-d5ab4226b61c
created_at       | 2025-07-10 18:35:20.017722+00
updated_at       | 2025-07-10 18:35:20.017722+00
task_id          | man_40fb48d0-b3b1-44ef-9077-69a0fe6aa409
config_type      | erasure_privacy_request
version          | 1
is_current       | t
execution_timing | pre_execution
  • Verify that completing access and Erasure manual tasks or skipping access and erasure manual tasks allows the privacy request to complete as expected

Pre-Merge Checklist

  • Issue requirements met
  • All CI pipelines succeeded
  • CHANGELOG.md updated
    • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
    • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
    • Updates unreleased work already in Changelog, no new entry necessary
  • Followup issues:
    • Followup issues created
    • No followup issues
  • Database migrations:
    • Ensure that your downrev is up to date with the latest revision on main
    • Ensure that your downgrade() migration is correct and works
      • If a downgrade migration is not possible for this change, please call this out in the PR description!
    • No migrations
  • Documentation:
    • Documentation complete, PR opened in fidesdocs
    • Documentation issue created in fidesdocs
    • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
    • No documentation updates required

Copy link

vercel bot commented Jul 14, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
fides-plus-nightly ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 15, 2025 4:28pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
fides-privacy-center ⬜️ Ignored (Inspect) Jul 15, 2025 4:28pm

@JadeCara JadeCara changed the title simplified and corrected some manual task graph creation issues [ENG-899] corrected some manual task graph creation issues Jul 14, 2025
Copy link

codecov bot commented Jul 14, 2025

Codecov Report

Attention: Patch coverage is 45.45455% with 6 lines in your changes missing coverage. Please review.

Project coverage is 86.78%. Comparing base (b9653cb) to head (efc62c1).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...rc/fides/api/task/manual/manual_task_graph_task.py 14.28% 4 Missing and 2 partials ⚠️

❌ Your patch status has failed because the patch coverage (45.45%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6335      +/-   ##
==========================================
+ Coverage   86.76%   86.78%   +0.01%     
==========================================
  Files         448      448              
  Lines       28612    28603       -9     
  Branches     3180     3177       -3     
==========================================
- Hits        24826    24822       -4     
+ Misses       3050     3048       -2     
+ Partials      736      733       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@thabofletcher
Copy link
Contributor

🚀 🚀 🚀

@JadeCara JadeCara merged commit 356ed96 into main Jul 15, 2025
40 of 41 checks passed
@JadeCara JadeCara deleted the ENG-899-research-and-correct-task-instance-creation-2 branch July 15, 2025 17:55
Copy link

cypress bot commented Jul 15, 2025

fides    Run #13120

Run Properties:  status check passed Passed #13120  •  git commit 356ed96f23: [ENG-899] corrected some manual task graph creation issues (#6335)
Project fides
Branch Review main
Run status status check passed Passed #13120
Run duration 00m 50s
Commit git commit 356ed96f23: [ENG-899] corrected some manual task graph creation issues (#6335)
Committer JadeWibbels
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 5
View all changes introduced in this branch ↗︎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants