Add microphone/audio device input #45
Conversation
|
Hey, Awesome stuff! Thanks for the contribution. |
| parser.add_argument('--from_id', type=str, default="llSGDh", | ||
| help='Shadertoy ID (default: llSGDh) https://www.shadertoy.com/view/llSGDh by iq CC-BY-NC-SA-3.0') |
There was a problem hiding this comment.
having the code directly makes the example available offline for for those without an API key set.
| parser.add_argument('--list-audio-devices', action='store_true', | ||
| help='List all available audio input devices and exit') | ||
| parser.add_argument('--device-index', type=int, default=None, | ||
| help='Audio device index to use (default: system default)') |
There was a problem hiding this comment.
the example shouldn't really need interactivity for testing. This cl arg makes sense for cli.py
| if args.list_audio_devices: | ||
| print("Available audio input devices:") | ||
| print(sd.query_devices()) | ||
| sys.exit(0) |
There was a problem hiding this comment.
it should be fine to just print all for the example. Although on my system there were 51 devices - so maybe just print those that work as microphones
| return np.zeros(num_samples, dtype=np.float32) | ||
|
|
||
| class NoiseAudioDevice(AudioDevice): | ||
| """An AudioDevice that always returns silence.""" |
There was a problem hiding this comment.
| """An AudioDevice that always returns silence.""" | |
| """An AudioDevice that always returns white noise.""" |
| sample_rate: int = 44100, | ||
| buffer_duration_seconds: float = 5.0, | ||
| chunk_size: int = 1024, | ||
| device_index: int | None = None, |
There was a problem hiding this comment.
technically we still support 3.9 which doesn't allow this syntax
|
Finally had a look at first half of this and tried it out too. Will have to take a closer look at the update logic after this is rebased with the multipass changes especially around updating. I can think of cases where the input might be used in different passes for example (will look for or write a shadertoy example to test this). Also I am not 100% sure if this matches the website experience since the website source doesn't give any details :/ Maybe a good time to add the |
|
hey @richinsley are you still up for working on this? Otherwise I can work on the rebase.
I have been linked the following: https://gist.github.com/soulthreads/2efe50da4be1fb5f7ab60ff14ca434b8 which has some details on the data transformation - my aim is to match the website as closely as possible. |
|
@Vipitis absolutely. I got caught up in crunch time at work and haven't had a chance to swing back to this. However I have some time now, so I'll handle the rebase. I'll also go over the link you provided and make those adjustments. I had to make assumptions the first time (particularly with how the FFT was handled) |
|
Closing in favor of a new PR with an updated branch |
This adds an abstract audio device class with a sub-class that creates an audio device using the sounddevice package.
ShadertoyChannelMusic is implemented to pull the audio data from it's given audio device and generates the waveform and fft data.
infer_subclass detects ctype of 'mic' and creates the ShadertoyChannelMusic instance.
An example (shadertoy/examples/shadertoy_mic_input.py) was added that demonstrates how to select and add the audio device to the ShaderToy instance.