RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915308915130368 |
|---|---|
| author | Zeng, Chong Dong, Yue Peers, Pieter Wu, Hongzhi Tong, Xin |
| author_facet | Zeng, Chong Dong, Yue Peers, Pieter Wu, Hongzhi Tong, Xin |
| contents | We present RenderFormer, a neural rendering pipeline that directly renders an image from a triangle-based representation of a scene with full global illumination effects and that does not require per-scene training or fine-tuning. Instead of taking a physics-centric approach to rendering, we formulate rendering as a sequence-to-sequence transformation where a sequence of tokens representing triangles with reflectance properties is converted to a sequence of output tokens representing small patches of pixels. RenderFormer follows a two stage pipeline: a view-independent stage that models triangle-to-triangle light transport, and a view-dependent stage that transforms a token representing a bundle of rays to the corresponding pixel values guided by the triangle-sequence from the view-independent stage. Both stages are based on the transformer architecture and are learned with minimal prior constraints. We demonstrate and evaluate RenderFormer on scenes with varying complexity in shape and light transport. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_21925 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination Zeng, Chong Dong, Yue Peers, Pieter Wu, Hongzhi Tong, Xin Graphics Computer Vision and Pattern Recognition Machine Learning We present RenderFormer, a neural rendering pipeline that directly renders an image from a triangle-based representation of a scene with full global illumination effects and that does not require per-scene training or fine-tuning. Instead of taking a physics-centric approach to rendering, we formulate rendering as a sequence-to-sequence transformation where a sequence of tokens representing triangles with reflectance properties is converted to a sequence of output tokens representing small patches of pixels. RenderFormer follows a two stage pipeline: a view-independent stage that models triangle-to-triangle light transport, and a view-dependent stage that transforms a token representing a bundle of rays to the corresponding pixel values guided by the triangle-sequence from the view-independent stage. Both stages are based on the transformer architecture and are learned with minimal prior constraints. We demonstrate and evaluate RenderFormer on scenes with varying complexity in shape and light transport. |
| title | RenderFormer: Transformer-based Neural Rendering of Triangle Meshes with Global Illumination |
| topic | Graphics Computer Vision and Pattern Recognition Machine Learning |
| url | https://arxiv.org/abs/2505.21925 |