Making material pickability controllable #704

hmaarrfk · 2024-04-03T01:42:00Z

My first observation are:

My pointer_move event is no longer getting triggered for the one I attached to my scene.
It still doesn't make the lines on the example code not block the pip. Maybe i missed something?

xref: #702

General todo list

Get pointer picking to work as expected on examples/feature_demo/image_click_events.py
Go through each of the demos to assess if they are broken or not.
Fix the broken demo files
Decide on appropriate default values for the pickability of different material.

hmaarrfk · 2024-04-03T02:16:32Z

In building #700 i learned that a "pick" is generated everytime you move your mouse, so for the very least, the background of the scene should be made pickable to get the mouse movement events.

pygfx/materials/_base.py

almarklein · 2024-04-03T11:32:41Z

I think I know what's missing. In blender.py, the get_color_descriptors() will produce a descriptor for the picking, but this should happen conditionally now. I think we can add a material_write_pick arg to get_color_descriptors().

hmaarrfk · 2024-04-05T03:33:45Z

I think I tried something like your suggestion in 65cb368

but it seems that have crashed quite badly on me.

I'll have to sleep on this. maybe read more about GPUs

almarklein · 2024-04-05T07:37:43Z

I think I tried something like your suggestion in 65cb368

Not exactly what I meant. Sorry for being not sufficiently clear. I meant that in the implementation of the get_color_descriptors() methods that produce a color descriptor for picking, this should be optional. In particular, these cases:

pygfx/pygfx/renderers/wgpu/engine/blender.py

Lines 80 to 96 in 0189f81

    
           def get_color_descriptors(self, blender): 
        
               bf, bo = wgpu.BlendFactor, wgpu.BlendOperation 
        
               return [ 
        
                   { 
        
                       "format": blender.color_format, 
        
                       "blend": { 
        
                           "alpha": (bf.one, bf.zero, bo.add), 
        
                           "color": (bf.one, bf.zero, bo.add), 
        
                       }, 
        
                       "write_mask": wgpu.ColorWrite.ALL, 
        
                   }, 
        
                   { 
        
                       "format": blender.pick_format, 
        
                       "blend": None, 
        
                       "write_mask": wgpu.ColorWrite.ALL, 
        
                   }, 
        
               ]

pygfx/pygfx/renderers/wgpu/engine/blender.py

Lines 184 to 200 in 0189f81

    
           def get_color_descriptors(self, blender): 
        
               bf, bo = wgpu.BlendFactor, wgpu.BlendOperation 
        
               return [ 
        
                   { 
        
                       "format": blender.color_format, 
        
                       "blend": { 
        
                           "alpha": (bf.one, bf.one_minus_src_alpha, bo.add), 
        
                           "color": (bf.one, bf.one_minus_src_alpha, bo.add), 
        
                       }, 
        
                       "write_mask": wgpu.ColorWrite.ALL, 
        
                   }, 
        
                   { 
        
                       "format": blender.pick_format, 
        
                       "blend": None, 
        
                       "write_mask": wgpu.ColorWrite.ALL, 
        
                   }, 
        
               ]

pygfx/pygfx/renderers/wgpu/engine/blender.py

Lines 408 to 424 in 0189f81

    
           def get_color_descriptors(self, blender): 
        
               bf, bo = wgpu.BlendFactor, wgpu.BlendOperation 
        
               return [ 
        
                   { 
        
                       "format": blender.frontcolor_format, 
        
                       "blend": { 
        
                           "alpha": (bf.one, bf.zero, bo.add), 
        
                           "color": (bf.one, bf.zero, bo.add), 
        
                       }, 
        
                       "write_mask": wgpu.ColorWrite.ALL, 
        
                   }, 
        
                   { 
        
                       "format": blender.pick_format, 
        
                       "blend": None, 
        
                       "write_mask": wgpu.ColorWrite.ALL, 
        
                   }, 
        
               ]

And these methods of the passes are all called via this method of the blender object:

pygfx/pygfx/renderers/wgpu/engine/blender.py

Lines 572 to 573 in 0189f81

    
           def get_color_descriptors(self, pass_index): 
        
               return self.passes[pass_index].get_color_descriptors(self)

almarklein · 2024-04-05T07:40:58Z

I'll have to sleep on this. maybe read more about GPUs

Not so much related to GPU's. The pipeline.py is probably the trickiest code in pygfx to follow, because it's where a lot of things come together.

hmaarrfk · 2024-04-06T03:41:41Z

my notes

validate_helpers2.py
- PASS: Used the mouse to move the axes.
- PASS: This continue to work even after write_pick is false
validate_line_2d.py
- PASS: can still control rotating the camera
validate_line_dash.py
- PASS: can still control camera
validate_line_thickness_space.py
- PASS: can still control camera
validate_meshslice.py
- FAIL: hover over mesh no longer produced text on the screen.
- FIX: Needed to make the mesh pickable
validate_skybox.py
- FAIL: Skybox disappeared????? But the reflection in the ball is still good???
- FIX: Neded to make the skybox pickable???? what??? why....
- REASON: It seems that some feature of the skybox shader is using write_pick as a detection of the opaque pass.
collection_line.py
- PASS: can zoom in with mouse
cube_wx.py
- UNTESTED
light_view.py
- UNRELATED FAIL: AttributeError: module 'pylinalg' has no attribute 'Quaternion'
- PASS: Camera control
reactive_rendering.py:
- UNREALTED: ModuleNotFoundError: No module named 'observ'
- MUST CHECK: event_handlers added
examples/other/sponza_scene.py
- UNRELATED FAIL: RuntimeError: Could not find 'glTF-Sample-Models' directory.
clipping_planes.py
- PASS: can rotate things
cube.py
- FAIL: background disappeared
- FIX: Neded to make the background pickable???? what??? why....
- REASON: It seems that some feature of the background shader is using write_pick as a detection of the opaque pass.
light_basic.py
- seems like i can move around. seems like the light is working
object_bounding_box.py
- seems like the bounding box shows up
orbit_camera.py
- FAIL: background doesn't show up
- FIX: Same as other background stuffs
points_basic.py
- FAIL: background not rendered
- FIX: Background(pick_write=True)

Feature

aomap.py
colormap_channels.py
colormap_image.py
colormap_mesh.py
custom_object1.py
custom_object2.py
custom_object3.py
cylinder.py
dynamic_env_map.py -- Background pick_write bug
env_maps.py -- Background pick_write bug
flat_shaded_torus.py
fly_controller.py
geometry_cubes.py
geometry_image.py
geometry_klein_bottle.py
geometry_mobius.py
geometry_plane.py
geometry_polyhedron.py
geometry_polyhedron_subdivisions.py
geometry_torus_knot.py
helpers_gizmo.py -- fixed
helpers_stats.py
image_click_events.py -- You can now click the points!!! and it clicks through them!!!
image_overlay.py
image_plus_points.py
instancing_mesh.py
light_directional_shadow.py
lightmap.py
light_shadow.py
light_spotlight_shadows.py
light_spotlights.py
line_basic.py
line_performance.py
line_segments.py
line_thick.py
look_at.py
manual_matrix_update.py
map_click_events_to_world.py -- looks black... add pick_write=True... -- background hack
mesh_depth_material.py
mesh_dynamic.py
mesh_slice.py
multiprocessing_zmq
multi_select.py -- Had to add picking to cubes
multi_slice1.py -- background hack
multi_slice2.py -- background hack
paint_to_texture.py -- background hack, image must be write_pick=True
panzoom_camera.py -- background hack
pbr2.py -- background skybox hack
pbr.py -- background skybox hack
physical_color.py
picking_color.py -- background hack -- cube write pick
picking_mesh.py -- background hack -- write pick on objects
picking_points.py -- write_pick enabled
quad_mesh.py -- background hack -- write pick on objects
README.rst
scene_in_a_scene.py -- background hack
scene_overlay.py
scene_subplots1.py
scene_subplots2.py -- background hack
scene_subplots_video.py
show_image.py
show_scene.py
show_stl.py
skybox.py -- background hack
skybox_rotate.py -- background hack
spheres.py
synced_video.py
text_contrast.py -- background hack
text_snake.py -- and object pick write
text_waterfall.py -- background hack
transparency1.py
transparency2.py
volume_render1.py -- object pick write
volume_render2.py
volume_slice1.py
volume_slice2.py
volume_slice3.py -- pick_write object
wireframe1.py
wireframe2.py
wireframe_material.py
world_bounding_box.py

This was found in pygfx#704 It seems that write_pick was previously used a sentinel to detect the type of pass that was getting used. It is currently defined as true when: * BasePass * OpaquePass * FullOpaquePass * SimpleSinglePass * FrontmostTransparencyPass

pygfx/renderers/wgpu/engine/pipeline.py

pygfx/renderers/wgpu/engine/blender.py

#711) This was found in #704 It seems that write_pick was previously used a sentinel to detect the type of pass that was getting used. It is currently defined as true when: * BasePass * OpaquePass * FullOpaquePass * SimpleSinglePass * FrontmostTransparencyPass

pygfx/renderers/wgpu/engine/pipeline.py

hmaarrfk · 2024-04-09T13:50:08Z

Ok i added a note in the docs about the change in the default value.

Hopefully this is good to go.

docs/advanced_shaders.rst

Korijn · 2024-04-10T11:02:45Z

Have you confirmed toggling pick_write at runtime works as expected?

Looks good to go other than some small typos, see my inline comments.

hmaarrfk · 2024-04-10T11:20:47Z

Have you confirmed toggling pick_write at runtime works as expected?

No.

I need to think of a "demo" test for this.

Co-authored-by: Korijn van Golen <korijn@gmail.com>

Korijn · 2024-04-10T13:57:36Z

Have you confirmed toggling pick_write at runtime works as expected?

No.

I need to think of a "demo" test for this.

We'll merge, would be great if you could still follow up on that check.

Thanks a ton!

almarklein · 2024-04-10T14:12:03Z

Have you confirmed toggling pick_write at runtime works as expected?

I just checked, and it does not 😛 Created #720 to track. I can have a look.

hmaarrfk · 2024-04-10T14:24:45Z

😮‍💨

hmaarrfk requested a review from Korijn as a code owner April 3, 2024 01:42

hmaarrfk marked this pull request as draft April 3, 2024 01:42

almarklein reviewed Apr 3, 2024

View reviewed changes

pygfx/materials/_base.py Outdated Show resolved Hide resolved

hmaarrfk force-pushed the make_write_pick_optional branch 2 times, most recently from 08d0377 to f9014ff Compare April 6, 2024 03:40

hmaarrfk mentioned this pull request Apr 6, 2024

Use the render_mask to control when the background shader is displayed #711

Merged

hmaarrfk commented Apr 6, 2024

View reviewed changes

pygfx/renderers/wgpu/engine/pipeline.py Outdated Show resolved Hide resolved

hmaarrfk force-pushed the make_write_pick_optional branch 2 times, most recently from 611d3b7 to 90f27e3 Compare April 6, 2024 13:26

almarklein reviewed Apr 8, 2024

View reviewed changes

pygfx/renderers/wgpu/engine/blender.py Show resolved Hide resolved

Korijn reviewed Apr 8, 2024

View reviewed changes

pygfx/renderers/wgpu/engine/pipeline.py Outdated Show resolved Hide resolved

hmaarrfk force-pushed the make_write_pick_optional branch from 90f27e3 to 9825550 Compare April 9, 2024 09:24

hmaarrfk marked this pull request as ready for review April 9, 2024 13:49

hmaarrfk added 3 commits April 9, 2024 09:50

Enable the controlability of the material picking

8bd0da7

Adjust examples to ensure pickable objects are picked

44bbda0

Mention that write pick is False in the docs by default

799da22

hmaarrfk force-pushed the make_write_pick_optional branch from a78bd05 to 799da22 Compare April 9, 2024 13:50

Korijn reviewed Apr 10, 2024

View reviewed changes

docs/advanced_shaders.rst Outdated Show resolved Hide resolved

Korijn reviewed Apr 10, 2024

View reviewed changes

docs/advanced_shaders.rst Outdated Show resolved Hide resolved

Korijn reviewed Apr 10, 2024

View reviewed changes

docs/advanced_shaders.rst Outdated Show resolved Hide resolved

Improve documentaton wording

8887544

Co-authored-by: Korijn van Golen <korijn@gmail.com>

almarklein approved these changes Apr 10, 2024

View reviewed changes

Korijn approved these changes Apr 10, 2024

View reviewed changes

Merge branch 'main' into make_write_pick_optional

dfaec10

Korijn enabled auto-merge (squash) April 10, 2024 13:57

Korijn merged commit 3aaa371 into pygfx:main Apr 10, 2024

almarklein mentioned this pull request Apr 10, 2024

Make it possible to dynamically change material.pick_write #720

Closed

kushalkolar mentioned this pull request Apr 11, 2024

pygfx optional picking fastplotlib/fastplotlib#481

Closed

almarklein mentioned this pull request Apr 15, 2024

Making objects "unpickable" when render mode is ordered1 #702

Closed

Uh oh!

Making material pickability controllable #704

Making material pickability controllable #704

Uh oh!

Conversation

hmaarrfk commented Apr 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hmaarrfk commented Apr 3, 2024

Uh oh!

Uh oh!

almarklein commented Apr 3, 2024

Uh oh!

hmaarrfk commented Apr 5, 2024

Uh oh!

almarklein commented Apr 5, 2024

Uh oh!

almarklein commented Apr 5, 2024

Uh oh!

hmaarrfk commented Apr 6, 2024

Feature

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmaarrfk commented Apr 9, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Korijn commented Apr 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hmaarrfk commented Apr 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Korijn commented Apr 10, 2024

Uh oh!

almarklein commented Apr 10, 2024

Uh oh!

hmaarrfk commented Apr 10, 2024

Uh oh!

Uh oh!

hmaarrfk commented Apr 3, 2024 •

edited

Loading

Korijn commented Apr 10, 2024 •

edited

Loading

hmaarrfk commented Apr 10, 2024 •

edited

Loading