Skip to content

Conversation

cbentejac
Copy link
Contributor

Description

This PR adds the support for plugins in Meshroom, building on top of #2703.

A Plugin is a collection of at least one NodePlugin, which contains the description of a node, and can be associated to templates. Plugins can be loaded and unloaded with the NodePluginManager, which is in charge of the registration and unregistration of all NodePlugins (a node cannot be instantiated until its NodePlugin has been registered).

All collections of nodes are now considered as plugins, may they be provided through the MESHROOM_NODES_PATH or through the MESHROOM_PLUGINS_PATH environment variables. In the case of MESHROOM_NODES_PATH, a Python module containing node descriptions is expected. For MESHROOM_PLUGINS_PATH, it is expected that the nodes are contained within a Python module that is itself located in a meshroom folder.

Features list

  • Add a "plugins" core module containing new classes: Plugin, NodePlugin, NodePluginManager;
  • Fully manage the loading and registration of nodes through the NodePluginManager;
  • Add unit tests for the newly added classes.

Implementations remarks

Currently, the whole process of going through folders in search of Python packages remains in the __init__.py of the core module. This is because the method that is used to find files containing nodes is the same as the one used to discover submitters. Submitters are, for now, not included in the "plugins" module and are handled as they were previously.

@cbentejac cbentejac added this to the Meshroom 2025.1.0 milestone May 27, 2025
@cbentejac cbentejac self-assigned this May 27, 2025
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements support for plugins in Meshroom by introducing a new “plugins” core module with Plugin, NodePlugin, and NodePluginManager classes, refactoring node registration and discovery to use the new pluginManager, and updating tests accordingly.

  • Refactor of node registration from nodesDesc to pluginManager.
  • Updates of unit tests and integration of plugin-based node discovery.
  • Addition of new plugin and node files for PluginA and PluginB.

Reviewed Changes

Copilot reviewed 30 out of 30 changed files in this pull request and generated 1 comment.

File Description
tests/test_compatibility.py Refactored node registration and unregistration to use pluginManager.
tests/test_attributeChoiceParam.py Updated node registration and tear down to use NodePlugin and pluginManager.
tests/plugins/meshroom/pluginB/PluginBNodeB.py Added new plugin node with potential type mismatch in IntParam default value.
Remaining files Various code refactoring and formatting changes to switch from nodesDesc to pluginManager integration.

Copy link

codecov bot commented May 27, 2025

Codecov Report

Attention: Patch coverage is 91.00775% with 58 lines in your changes missing coverage. Please review.

Project coverage is 78.46%. Comparing base (99a6dc0) to head (d3738fb).
Report is 45 commits behind head on develop.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
meshroom/core/plugins.py 87.64% 22 Missing ⚠️
meshroom/core/__init__.py 73.33% 16 Missing ⚠️
meshroom/core/graph.py 31.25% 11 Missing ⚠️
meshroom/core/attribute.py 87.27% 7 Missing ⚠️
meshroom/common/qt.py 50.00% 1 Missing ⚠️
meshroom/core/nodeFactory.py 80.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2733      +/-   ##
===========================================
+ Coverage    77.15%   78.46%   +1.30%     
===========================================
  Files           41       47       +6     
  Lines         6167     6542     +375     
===========================================
+ Hits          4758     5133     +375     
  Misses        1409     1409              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cbentejac cbentejac marked this pull request as ready for review June 2, 2025 15:34
@cbentejac cbentejac force-pushed the dev/pluginsSupport branch from 265f51d to 4e0c15d Compare June 5, 2025 12:35
cbentejac added 17 commits June 5, 2025 14:38
Add a new module named `plugins` with a `ProcessEnv` class which
contains the paths describing the environment needed for the plugin's
(node's) process.
A `Plugin` object contains a collection of nodes, represented as
`NodePlugin` objects. This commit adds the implementation of the `Plugin`
class with the methods to add and remove `NodePlugins`, as well as an
empty class for the `NodePlugin` objects themselves.
The validation of the node descriptions will be handled directly within
the `NodePlugin` objects and there is thus no need for this method to
exist outside of plugins.py.
The `NodePluginManager` class manages is used to load, manage,
and register plugins and nodes.
Templates that are available for a plugin are detected and gathered
upon the plugin's creation.
Nodes are now registered and unregistered through the `NodePluginManager`.
Also comply more with PEP8 linting rules.
The plugin manager is now effectively used for all the operations that
involve registering or unregistering nodes.
…...`

This prevents ambiguities between `NodePlugin` objects that have been
registered (and are thus instantiable) and those that belong to `Plugin`
objects but have not been registered.
A `NodePlugin` may be part of a `Plugin` even if it has not been
registered. `containsNode` allows to check whether a `NodePlugin` is
contained within a `Plugin`, and `belongsToPlugin`
cbentejac added 3 commits June 5, 2025 16:12
This will replace all the nodes in the current graph with a new instance
of the same type. If any change has occured in the description of these
nodes since their last instantiation, the new nodes will reflect them.
"Reload All Nodes" will reload all the descriptions for `NodePlugins`
that are part of loaded plugins, whether they are registered or not.

Once the reload of the descriptions has been performed, the node instances
from the current graph will be reloaded as well to reflect any change
in the description.
@cbentejac cbentejac force-pushed the dev/pluginsSupport branch 6 times, most recently from 2b64879 to 074a459 Compare June 5, 2025 14:46
cbentejac added 2 commits June 5, 2025 16:54
On some systems such as GitHub's Windows CI, the `write` operation
is too fast and does not cause a change in the timestamp of the
file we're reloading, hence causing the test to fail for external
reasons.

Adding a sleep does not change anything to the test functionally, but
on the contrary ensures that we are actually testing the feature.

https://stackoverflow.com/questions/19059877/python-os-path-getmtime-time-not-changing
@cbentejac cbentejac force-pushed the dev/pluginsSupport branch from 074a459 to a6d80b3 Compare June 5, 2025 14:56
cbentejac and others added 6 commits June 6, 2025 16:06
…atus

If the plugin has successfully been reloaded, return `True`. If it has
not been reloaded for any reason (either an error or because no
modification has been made to it since it has been loaded last), then
return `False`.
When requesting a `NodePlugin` to reload, the status of the reloading
is now saved. If it has been successful, all the nodes of this type in
the graph will effectively be removed. If it has not (either because of
an error, or because no change has been done to the description), then
nothing will happen.

Replacing only the nodes whose description has been updated prevents
from having to update heavy graphs for very few nodes to change (if any).
Warning only on a module/file without any element in it and provide
full-path to ease debugging.
For package/folder, it does not generate any message.
@fabiencastan fabiencastan changed the title Add support for plugins New Plugin Architecture for Node Registration Jun 7, 2025
@fabiencastan fabiencastan merged commit 64f806b into develop Jun 7, 2025
5 checks passed
@fabiencastan fabiencastan deleted the dev/pluginsSupport branch June 7, 2025 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants