Want top quality direct illumination? Are you constrained by a very tight ray budget? Are you in keto diet and carbs are calling? Don't worry about the first two, in this short blog post I describe a simple yet powerful idea that can help you achieve really good results reducing both the needed ray budget and the reservoir storage in half. This is based on the paper entitled Rearchitecting Spatiotemporal Resampling for Production [1] which I will refer to as The Paper from now on. About the carbs, can't help, they are delicious.
Quick recap, ReSTIR allows us to have high quality samples of lights, one for each pixel, first we sample lights using unshadowed path contributions, then we combine the new samples with the previous frame reservoir and surrounding pixel reservoirs. After all that sampling and combining, we will have a light which we check for visibility, basically we trace a shadow ray, and if it is visible, we use it to shade our pixel. If none of that made sense or you need more info, you can check my other post describing the whole process in more detail here [2].
The Paper makes an interesting observation about that shadow ray that we need to trace, specifically when we are combining reservoirs from neighboring pixels. If they are close to each other, we could forgo tracing the ray .... wait, what? why? Think about it, visibility has been determined by that other pixel, it did the tracing and since it is pretty close we can trust it and just shade the pixel. Of course, closeness is a must, otherwise light leaking would be noticeable. Yes, this method will introduce bias :(
So how can we use that observation to our advantage? We can forgo tracing half of the shadow rays if we use checkerboarding. Half of the pixels would go through the whole ReSTIR process, they would be responsible of introducing new light samples and tracing shadow rays, you know, breadwinners. The other half would be use a simplified version of ReSTIR, they would take a neighboring pixel's reservoir as its own temporal reservoir and combine it with other neighbors' reservoirs, basically freeloaders . These freeloaders would not trace a shadow ray and of course would not sample new lights, their reservoirs are discarded as they can't be trusted, we didn't check for visibility. And this is where storage is cut in half.
As you might have guessed, those freeloaders must combine reservoirs from really close pixels to avoid light leaking, or at least to minimize it. I've called the resulting algorithm ReSTIR El-Cheapo.
There is one key advantage of this method over upscaling. Despite the fact that I'm calling half of the pixels freeloaders in reality they are resampling those reservoirs based on their individual characteristics, meaning they are using their particular geometry and material to choose a light and once chosen they are shaded individually, yielding high quality images.
Could something similar be done in a quarter resolution fashion? Like have one quarter of breadwinners and 3 quarters of freeloaders? Let's call it ReSTIR El-Very-Cheapo, the pixel diagram would look something like this:
1/4 of the shadow rays, 1/4 of the needed storage. Interesting, but of course the resulting image is noisier and the checkerboarding is more noticeable, also there are less new samples introduced to our reservoirs.
Maybe the whole thing could be fine tuned to minimize all that noise. To be honest this whole El-Very-Cheapo idea occurred to me while I was drafting this blog post, so let's say it's very new, fresh out of the oven ... mmm, carbs.
The following image compares 3 ReSTIR implementations, I've added vanilla ReSTIR with all it's shadow rays and whatnot. You'll notice that there's some loss of detail and that shadows look blockier at the edge the cheaper we go.
Wait a second, all the screenshots are from Q2RTX, why not Jedi Outcast? What's going on? ... 🙊
Footnotes
[1]Rearchitecting Spatiotemporal Resampling for Production
[2]How to add thousands of lights to your renderer and not die in the process