Hey all! I’ve been thinking about how to best unit-test my shader code, since TDD has been very helpful for me. But I’ve been able to find surprisingly little information. If possible, I’d love to hear from your experiences!
You must log in or register to comment.
Where did that t come from, and does it survive a driver update? The snapshot from one GPU becomes the reference for hardware you don’t have, so the threshold has to absorb whatever the vendor’s rounding does before it can catch anything you changed.
The code snippet, or what exactly? Everything comes from my brain.
Yeah, that’s what I tried to express with the snippet. But even if a driver update every now and then invalidates your snapshots - so what? You can either manually verify them, or just regenerate them from the last known working commit and keep going.
I’ve now written ~60 tests for ~10 shader functions, and it’s been an enormous help. Can’t recommend it enough!
My current plan is to write unit tests using Sub-Viewports that each have a small wrapper shader to draw the result of the tested functions to e.g. 256x256px textures, which I can then compare against snapshots. Depending on the effects of different hardware/driver environments, it might be necessary to check that for some threshold
t:vec3 res = abs(result - snapshot); // residuals max(res.r, res.g, res.b) < tThis could technically also allow for assertion functions (one condition per pixel), but getting any kind of usable error messages would mean a ton of work, so I don’t think that it’d be worth it - though I am tempted :)
Edit: turns out Godot has native support for comparing two images, even better! https://docs.godotengine.org/en/stable/classes/class_image.html#class-image-method-compute-image-metrics