Skip to content

Add concept of block events and some design notes #1059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 31, 2025
Merged

Conversation

ml-evs
Copy link
Member

@ml-evs ml-evs commented Mar 4, 2025

This PR adds the concept of "events" that can be registered on block types; essentially, server-side functions that can be triggered separately from parsing/ingestion for things like setting parameters. It also introduces the handling of particular events raised by Bokeh plots in the base data block in Vue.

This is all described in a new design document for this current version of datablocks, also committed in this PR.

I think eventually we can add several lifecycle hooks to the base block for handling certain things (remake block from scratch etc) that can be reused by derived block types, though I haven't put any effort into doing this now. I'm already using this functionality in a custom block that sets some parameters based on user feedback that requires reanalysing all block data (essentially a slider setting some offset time parameter).

For review, I recommend reading the design doc in the rtd preview at https://the-datalab--1059.org.readthedocs.build/en/1059/design/blocks/.

There's still a few open questions about how errors in events should be handled, and what methods should be triggered afterwards (mostly due to the fact we always call the to_web() block method in the update-block route.

Copy link

codecov bot commented Mar 4, 2025

Codecov Report

❌ Patch coverage is 88.88889% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.95%. Comparing base (7fb9869) to head (9717f48).
⚠️ Report is 5 commits behind head on main.

Files with missing lines Patch % Lines
pydatalab/src/pydatalab/routes/v0_1/blocks.py 0.00% 6 Missing ⚠️
pydatalab/src/pydatalab/apps/xrd/blocks.py 87.87% 4 Missing ⚠️
pydatalab/src/pydatalab/blocks/base.py 96.49% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1059      +/-   ##
==========================================
+ Coverage   74.56%   74.95%   +0.38%     
==========================================
  Files          67       67              
  Lines        4561     4659      +98     
==========================================
+ Hits         3401     3492      +91     
- Misses       1160     1167       +7     
Files with missing lines Coverage Δ
pydatalab/src/pydatalab/bokeh_plots.py 83.39% <100.00%> (+0.80%) ⬆️
pydatalab/src/pydatalab/blocks/base.py 82.19% <96.49%> (+7.72%) ⬆️
pydatalab/src/pydatalab/apps/xrd/blocks.py 75.56% <87.87%> (+5.29%) ⬆️
pydatalab/src/pydatalab/routes/v0_1/blocks.py 45.36% <0.00%> (-3.00%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

cypress bot commented Mar 4, 2025

datalab    Run #3667

Run Properties:  status check passed Passed #3667  •  git commit 82b22c0dd2 ℹ️: Merge 9717f487d57d6639e05bbffe3be1b6ce8ebb72e4 into 7fb986982b94f4028c638b03875e...
Project datalab
Branch Review ml-evs/blocks
Run status status check passed Passed #3667
Run duration 10m 25s
Commit git commit 82b22c0dd2 ℹ️: Merge 9717f487d57d6639e05bbffe3be1b6ce8ebb72e4 into 7fb986982b94f4028c638b03875e...
Committer Matthew Evans
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 336
View all changes introduced in this branch ↗︎

@ml-evs ml-evs marked this pull request as ready for review March 10, 2025 22:06
@ml-evs ml-evs force-pushed the ml-evs/blocks branch 2 times, most recently from 2f892ca to f765a97 Compare March 19, 2025 18:56
@ml-evs
Copy link
Member Author

ml-evs commented Mar 19, 2025

Just pinging @BenjaminCharmes and @jdbocarsly for review on this one!

Copy link
Contributor

@BenjaminCharmes BenjaminCharmes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀 very promising!
XRD Block works perfectly fine.

@ml-evs
Copy link
Member Author

ml-evs commented Mar 21, 2025

Made some fixes post-discussion with @jdbocarsly. Known limitations still:

  • The event handler currently triggers per block on the page, with the wrong data. We need to find a way to scope the handler, and then dispatch it correctly to the right block. Ideally this would be encapsulated in a function so that we don't have too much fiddly boilerplate in the Bokeh callbacks.
  • We should probably rename bokehStateUpdate
  • The whole block currently updates; it would be nice to be able to also do partial updates

@ml-evs ml-evs force-pushed the ml-evs/blocks branch 2 times, most recently from f7fc272 to 34c71dd Compare April 4, 2025 23:16
@ml-evs ml-evs force-pushed the ml-evs/blocks branch 2 times, most recently from 827928a to 3f6c7e2 Compare April 21, 2025 14:46
@ml-evs ml-evs added this to the v0.6.x milestone May 9, 2025
@ml-evs ml-evs modified the milestones: v0.7.x, v0.6.x May 20, 2025
@ml-evs ml-evs self-assigned this Jun 7, 2025
@ml-evs ml-evs force-pushed the ml-evs/blocks branch 2 times, most recently from 45e5591 to 9ae754c Compare June 26, 2025 11:43
@ml-evs ml-evs force-pushed the ml-evs/blocks branch 2 times, most recently from f642195 to 2147107 Compare July 29, 2025 23:19
@ml-evs ml-evs requested a review from BenjaminCharmes July 29, 2025 23:20
@ml-evs ml-evs added enhancement New feature or request datablock An issue pertaining to a specific datablock labels Jul 29, 2025
Copy link
Contributor

@BenjaminCharmes BenjaminCharmes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, everything works great!

@ml-evs ml-evs merged commit f1150eb into main Jul 31, 2025
15 checks passed
@ml-evs ml-evs deleted the ml-evs/blocks branch July 31, 2025 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
datablock An issue pertaining to a specific datablock enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants