Support Multipass shaders (buffer A ...) #30
Conversation
|
Little update:
new breaking examples found, that might be unrelated to this PR, but I will note them down for later reference:
|
|
I think I finally fixed the compatibility issue. There is some small visual issues which look like precision problems to me (not sure yet). And the performance is horrible it seems... Please let me know if you find any shaders that are broken (not due to missing features, wgpu bugs) Will work on tests, examples and documentation to hopefully get this ready for next week. E: found this one seemingly broken: |
|
I think this is finally ready for review - and I welcome some feedback.
|
|
Cool stuff! |
wgpu_shadertoy/shadertoy.py
Outdated
| device=self._device, format=wgpu.TextureFormat.bgra8unorm | ||
| device=self._device, format=self._format |
There was a problem hiding this comment.
Can also set to None to have it select the preferred format. Less code, unless you need self._format. Note that the format is also accessible in texture.format on the texture obtained via get_current_texture().
There was a problem hiding this comment.
hm, I had a look and the only other solution I can think of is to make it a property of the ImageRenderPass, as it's needed to create the render pipeline. The awful part is that at init time for the Image class, the canvas(_present_context) might not be accessible via the parent instance of Shadertoy.
It could be returned by this method instead of passed via an attribute. It could be useful to have available when translating the snapshot back into RGB.
|
My thesis is nearly submitted, so I can spent time again getting this finished. For the performance issue I want to try the following: ping pong the render target to reduce the number of copies (which are likely the slowdown). Also it might be the case that buffers don't followed the A-B-C-D order but instead the order in which they were added. Which I believe might be recorded in the json we get from the API. I will try to find some examples that proof the behavior. E: maybe this can be merged as a MVP and performance improvements can be made in follow up PRs instead |
|
CI failures are addressed in #36 which isn't included in this branch |
|
found some more time to figure out the performance issues. I read somewhere that the queue is smart and might figure out of some copies and be avoided. Doesn't seem to make much difference. To avoid the guesswork I added some rudimentary profiling which still needs some work. But looking at the data already shows that something is up. the slower rendertimes on the right are full screen (instead of small window). But the constant spikes are a problem - will find the time to run more experiments in the following days and try different systems (to rule out weirdness of Intel GPUs). Seems like something(I suspect the copies) is causing the GPU to hang for ~60m. |
|
while making the vertex and fragment code templates simpler - I still need to do a yflip for the image pass. I couldn't find any setup for the gui canvas to accepted a flipped image. This works mostly fine, however I have been informed of Perhaps there is a solution to this down the line in rendercanvas? I would love to migrate to that once this PR gets merged. |
No, the definition of the viewport is simply part of the wgpu spec (https://wgpu-py.readthedocs.io/en/stable/guide.html#coordinate-system). IIRC this is reversed from OpenGL.
Aha, I gues that one will be multiplied with |
Rewrite rules are one option, but can be really complicated, so there might be better ideas. Doing a whole renderpass or compute pass just to flip the out image also seems wasteful. Plus there might be other functionality that is currently broken which I don't know about yet. I will track this as a standalone issue as it already exists before this PR too. |
|
Since this is a massive PR and by now the git history is sorta messed up. Would it help to split this up into a few smaller pieces? I am thinking: bug fixes, wgpu v0.18.1 support, input refactor/API stuff, buffer/Multipass(will have the improved shader code snippets), fixes to resizing, and optionally the profiling stuff (which I think isn't even reliable right now). My goal is to get this reviewed and merged to have a release v0.2 done before 5th February for a paper(my thesis) deadline which links to this branch. |
|
Makes sense! It's a lot easier to review and test in smaller batches |
|
closed as #43 has been merged instead. The branch will stay because some other projects still point to it. |
update:
this branch will be split into smaller PRs:
part of #4
approximately 17.5% of public Shadertoys are multipass. Multipass allows up to 4 buffers (A through D) to be rendered as a texture. These can also be used to store data and enable quite some more experiences.
Some of the challenges include timing as well as cross inputs.
Buffer passes can seemingly take the exact same inputs as the main "Image" renderpass, including other buffers (and themselves?)
This PR starts to bloat a little and contains some refactor for the whole channel input concept... still in flux
Instead, will try to implement BufferTexture as a
ShadertoyChannelsubclass so it can hold for example the sampler settings.Additionally, there will likely be a RenderPass base class and subclasses for Image, Buffer(a-d) and later cube and sound.
So the main Shadertoy class contains several render passes, and all of these get their inputs(channels) attached.
I even started to try and sketch it out - but will have to sleep through this for a few more days... my concepts change every day but I need to just try and work on the ideas for a bit.
The render order should be Buffer A through D and then Image. So you can keep temporal data, by using itself has an input.
TODOs:
additional tests cases for inferred input types, empty channels(caching conflict with pytests)test coverage for examples in readme!(different PR)(maybe) some debug mode where you can render the buffers to canvas?(you can use RenderDoc with "capture child processes")