Pigeon Devlog #1 — Building a Vulkan Renderer From Scratch
Practical Part:
I started a new renderer from zero and I named it Pigeon. Behavioral scientists use pigeons in experiments. They were used not because they are exceptionally smart, but because they are consistent, trainable, and respond predictably to feedback. That is the spirit of this project: small, controlled experiments that learn from results—try, measure, adjust, repeat.
 |
| Bistro Scene rendered in Pigeon |
Right now, Pigeon loads and renders glb models without
problems. I parse meshes, materials, and transforms, then upload vertex and
index buffers to GPU memory. I set up image samplers for textures and a basic
PBR-friendly path for albedo and normals, keeping the pipeline simple to debug.
It is still a minimal engine, but it already shows the shape of a real tool.
Vulkan is different from older APIs because it gives
explicit control. I create instances, devices, and queues so I know exactly
where work is going. I record command buffers that describe every draw. I
manage synchronization with fences and semaphores to avoid hidden stalls. I
bind resources through descriptor sets, and I build pipelines that lock shaders
and fixed-function state for predictable performance. This is more work than
OpenGL, but it removes mystery. When something is slow, I can see why.
To reach this point, I am reading the Modern Vulkan Cookbook
and studying its code to understand real-world patterns. The book helps me
with best practices for memory allocation, staging uploads, and swapchain
recreation. Reading others’ code shows me practical tricks: how they structure
frame graphs, when they prefer push constants, how they group descriptor sets
by frequency of change. I learn faster by mixing theory and working examples.
My current focus is bindless rendering. In normal Vulkan,
shaders access a limited number of textures or buffers through fixed descriptor
bindings. With a bindless approach, shaders can index large arrays of resources
using integer IDs. This reduces CPU overhead from rebinding, and it matches
modern content where scenes can have hundreds of materials. The challenge is
descriptor indexing rules, GPU limits, and how to organize heaps so that
lookups are fast but still portable.
For the next two weeks, I will implement Bruneton's atmospheric
scattering model. This is important for my thesis direction, where I plan to replace the second
ray march to the light with a learned predictor. To prepare for that, I am
designing the renderer with clean interfaces around light queries so I can plug
in a neural approximation later. I will use my renderer—with Bruneton’s atmospheric scattering and real-time volumetric rendering—as the high-quality reference and compare it against the neural network implementation
Theory Part:
1. Deep
Scattering [Kallweit et al., 2017] The main idea of this paper is to use a
deep radiance-predicting neural network (DRPNN) to predict the difficult part of cloud lighting,
which is the multi-scattered light. It does this by taking density samples from
the cloud in a special hierarchical way to feed the network. The result is very
fast and looks good, but the limitation is that it only works for one type of
cloud material at a time; you must retrain it for different looks. Note: this is a path-tracing project, not real-time.
2. Faster
RPNN [Panin & Nikolenko, 2019] This paper makes the first method (RPNN)
faster. It precomputes some information into "light probes" using a
"baking network." Then, at render time, a smaller network uses these
probes to get the final image. This gives a 2-3x speedup, but it does not help
speed up lighting from the environment, which can still be slow.
3. Deep
Real-time Volumetric Rendering Using Multi-feature Fusion [Hu et al., 2023]
This paper presents a very fast, real-time method. Instead of only using cloud
density, it gives the network more information like transmittance and material
properties (phase function, albedo). This makes the network much smaller and
faster, and allows changing the material look without retraining. It is two
orders of magnitude faster than RPNN but can have some small errors in complex
media because it uses mipmaps.
4. A
real-time rendering method for high albedo... [Fang, Feng, & Cui, 2024]
Similar to the previous paper, this work also uses a neural network with
multiple features (density, transmittance, phase) to render in real time. It
uses a 3D-CNN and an attention module to process the features. The paper says
it has good results and is fast, but it does not give specific numbers for
performance or quality comparison, which makes it hard to evaluate. (NOT COMPLETE PAPER, IDK)
5. Optimizing
Light Samples for Real-time volumetric clouds [Blomqvist, 2025] This paper
focuses on a different problem: how to render with very few light samples
without getting artifacts. The method is to randomly move (jitter) the samples
using different noise types and then use filters to clean the image over time.
The key result is that a special noise called FAST gives the best numerical
quality, but the paper notes it did not test if it also looks best to a human
user. (NOT A NN PAPER)
6. Real-Time Volumetric Shadows using 1D Min-Max Mipmaps [Chen et al., 2010] This paper presents a fast way to calculate volumetric shadows in real time. The main technique is to use "epipolar rectification" to simplify the shadow calculation, and then use a simple data structure (1D min-max mipmap) to find the lit parts of a ray very quickly on the GPU. It is fast (55 FPS), but it only works for single-scattering light and uniform media. (NOT A NN PAPER)
7. Real-time Neural Radiance Caching for Path Tracing (Müller et al., 2021) — The paper proposes a neural radiance cache (NRC) that learns indirect lighting online while you render, instead of pretraining a network for each scene. It trains a small MLP on-the-fly using self-training: extend a sparse set of short path-traced samples by a few extra bounces, use the cache’s own prediction at the tail as a low-noise target, and keep the cache adapting every frame. This avoids heavy precomputation and works with fully dynamic lights, geometry, and materials.
References
Kallweit, S., Müller, T., McWilliams, B., Gross, M., & Novák, J. (2017).
Deep scattering: Rendering atmospheric clouds with radiance-predicting neural
networks. ACM Transactions on Graphics, 36(6), Article 231. https://doi.org/10.1145/3130800.3130880
Panin, M., & Nikolenko, S. (2019). Faster RPNN: Rendering clouds with
latent space light probes. In SA ’19: SIGGRAPH Asia 2019 Technical Briefs
(pp. 21–24). ACM. https://doi.org/10.1145/3355088.3365150
Hu, J., Yu, C., Liu, H., Yan, L., Wu, Y., & Jin, X. (2023). Deep
real-time volumetric rendering using multi-feature fusion. In SIGGRAPH ’23:
ACM SIGGRAPH Conference Proceedings. ACM. https://doi.org/10.1145/3588432.3591493
Fang, S., Feng, H., & Cui, K. (2024). A real-time rendering method
for high albedo anisotropic materials with multiple scattering. arXiv. https://arxiv.org/abs/2401.14051
Blomqvist, A. (2025). Optimizing light samples for real-time volumetric
clouds: Through stochastic rendering and noise reduction (Master’s thesis,
KTH Royal Institute of Technology). DiVA portal. https://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-367820
Chen, J., Baran, I., Durand, F., & Jarosz, W. (2011). Real-time
volumetric shadows using 1D min-max mipmaps. In Proceedings of the 2011 ACM
SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D ’11) (pp.
39–46). ACM. https://doi.org/10.1145/1944745.1944752
Müller, T., Rousselle, F., Novák, J., & Keller, A. (2021). Real-time neural radiance caching for path tracing. ACM Transactions on Graphics, 40(4), Article 36. https://doi.org/10.1145/3450626.3459812
Comments
Post a Comment