Rendering in games can be a complex topic, different approaches with their own tradeoffs are available, new techniques arise, others fall out of fashion… Studios using in-house engines need to make choices that will fit the type of game they’re aiming for.
It can be very interesting to know which decisions were actually made to ship the final product, what worked and what didn’t.

There are several resources available online detailing game post-mortems, whether it’s GDC / Siggraph talks or even content hosted by game studios themselves.
But not everything is available: public documentation requires time and effort, with little direct return-on-investment for the studio besides good PR among developers. And that’s when the company doesn’t simply prohibit employees from disclosing any information publicly.

Below is a collection of frame breakdowns I gathered online (including a few of mines), all written by volunteers and enthusiasts who decided to reverse-engineer some games and document their findings.

Game Release ▼	Game Title ▼		Link	Author ▼	Published ▼
2002/12/13		Zelda - Wind Waker (GC)	Analysis	by Nathan Gordon	2016/11/06
2002/12/13		Zelda - Wind Waker (GC)	Analysis	by Soenke Seidel	2012/08/23
2005/10/18		Shadow of the Colossus (PS2)	Analysis	by Léna Piquet	2012/10/07
2007/02/16		Supreme Commander	Analysis	by Adrian Courrèges	2015/06/23
2009/11/17		Assassin’s Creed II	Analysis	by Léna Piquet	2015/12/05
2011/08/23		Deus Ex: Human Revolution	Analysis	by Adrian Courrèges	2015/03/10
2011/10/18		Batman: Arkham City	Analysis	by Léna Piquet	2012/09/21
2012/02/21		Syndicate (2012)	Analysis	by Léna Piquet	2024/01/31
2013/09/17		GTA V	Analysis	by Adrian Courrèges	2015/11/02
2014/02/25		Castlevania: Lords of Shadow 2	Analysis	by Emilio López	2015/11/28
2014/09/30		Middle-earth: Shadow of Mordor	Analysis	by Emilio López	2017/12/27
2014/10/06		Alien Isolation	Analysis	by Gen Afanasev	2018/01/16
2015/05/18		The Witcher 3: Wild Hunt	Analysis	by Mateusz / Astralis	2017/09/07
2015/06/23		Batman: Arkham Knight	Analysis	by Balázs Török	2020/04/03
2015/09/01		Metal Gear Solid V: The Phantom Pain	Analysis	by Adrian Courrèges	2017/12/15
2015/11/10		Rise of the Tomb Raider	Analysis	by Emilio López	2018/12/31
2015/12/16		Waves 2: Notorious	Analysis	by David Maas	2022/09/21
2016/01/26		The Witness	Analysis	by Thomas Poulet	2017/08/21
2016/05/13		DOOM 2016	Analysis	by Adrian Courrèges	2016/09/09
2016/05/24		Overwatch	Analysis	by Alain Galvan	2020/04/13
2016/12/06		Shadow Tactics: Blades of the Shogun	Analysis	by Kosmonaut	2017/01/09
2017/08/01		Slime Rancher	Analysis	by David Maas	2022/12/07
2018/05/25		Detroit: Become Human	Analysis	by M. A. Moniem	2024/11/27
2018/06/12		Jurassic World Evolution	Analysis	by Emilio López	2021/03/12
2018/08/01		Yakuza 0	Analysis	by Silent	2019/02/24
2018/09/14		Shadow of the Tomb Raider	Analysis	by Balázs Török	2019/04/19
2018/10/26		Red Dead Redemption 2	Analysis	by Hüseyin	2020/06/19
2018/11/07		Ni No Kuni 2	Analysis	by Thomas Poulet	2017/08/21
2019/01/25		Resident Evil 2 (Remake)	Analysis	by Anton Schreiner	2019/08/01
2019/01/25		Resident Evil 2 (Remake)	Analysis	by M. A. Moniem	2022/07/20
2019/02/15		Metro Exodus	Analysis	by Balázs Török	2019/03/27
2019/02/15		Metro Exodus	Analysis	by Anton Schreiner	2019/08/11
2019/04/23		Mortal Kombat 11	Analysis	by Alain Galvan	2020/04/28
2019/05/14		A Plague Tale: Innocence	Analysis	by Balázs Török	2019/06/16
2019/08/27		Control	Analysis	by Alain Galvan	2021/11/06
2020/03/20		DOOM Eternal	Analysis	by Simon Coenen	2020/08/30
2020/04/16		Minecraft RTX	Analysis	by Alain Galvan	2020/05/19
2020/07/14		Death Stranding	Analysis	by M. A. Moniem	2022/11/30
2020/09/25		Mafia: Definitive Edition	Analysis	by Emilio López	2021/08/23
2020/12/10		Cyberpunk 2077	Analysis	by Hang Zhang	2020/12/12
2020/12/10		Cyberpunk 2077	Analysis	by Angelo Pesce	2020/12/17
2021/05/21		Knockout City	Analysis	by Ashley Taylor	2023/12/15
2022/01/14		God of War (PC)	Analysis	by M. A. Moniem	2022/01/18
2022/02/22		Elden Ring	Analysis	by M. A. Moniem	2022/05/25
2022/04/21		Teardown	Analysis	by Steven Wittens	2023/01/24
2022/04/21		Teardown	Analysis	by Jake Ryan	2023/02/20
2023/06/05		Diablo IV	Analysis	by M. A. Moniem	2023/06/28

I’ll try to maintain the list updated so if I’m missing something don’t hesitate to reach out.

Keep in mind most of this is the fruit of reverse-engineering and some amount of guess-work can be involved.
For those looking for the ground-truth and more explanation, you can hear it straight from the horse’s mouth:
studios research division (Activision, EA, Ready At Dawn…), Siggraph’s Advances in Real-Time Rendering or GDC Vault talks.

I also recommend Simon Trümpler’s website which presents technical breakdowns of some graphical effects implemented in commercial games.

At my day job I get to optimize several games for the Nintendo Switch or the NVIDIA Shield, some of them using Unreal Engine 4.
While UE4 is very powerful and offers a large selection of knobs to balance visual quality and performance some of the post-effects can end up being significantly heavy on a Tegra X1 GPU even at the lowest quality settings.

Below is a small collection of customizations/hacks I wrote in order to optimize the runtime cost of certain effects while remaining as close as possible to the original visuals of a vanilla UE4. The idea is to provide a drop-in replacement you can easily integrate into your own game to achieve better performance on a X1. Here I will be mainly writing about:

• BokehDOF
• SSAO
• Dynamic Resolution

Bokeh Depth-of-Field

Depth-of-field techniques have seen a lot of changes recently in the latest UE4 versions, some getting deprecated in favor of the new DiaphragmDOF implementation. Historically UE4 supported 3 different approaches:

No DOF

GaussianDOF

Physically incorrect, not very pleasant visually with light fall-off looking wrong. But extremely lightweight and the fastest option available.

BokehDOF

Better physical accuracy, correctly uses a flat kernel unlike Gaussian. Fully artist-driven with custom bokeh sprite, but gets slow at large radius.

CircleDOF

Aims for the best physical accuracy relying on lens properties, only disk shape. Can get very expensive and noisy as the bokeh radius increases.

Here I will be writing about a drop-in replacement for BokehDOF called GatherDOF.

BokehDOF produces very pleasant visual results but the main issue is its bandwidth cost.
To give some idea of how BokehDOF operates, it basically spawns one bokeh sprite per original pixel of the scene, each sprite size being proportional to the circle-of-confusion value of the pixel it originates from.
(See also the MGS V graphics study for more in-depth insights, it’s using roughly the same method.)

BokehDOF Sprite

For a 1080p scene that means drawing around 2 million quads, blended on the top of each other. As pixels get furthermore out-of-focus the sprite size increases and performance takes a nose-dive with quad overdraw saturating the bandwidth.

GatherDOF was implemented with the idea of producing a visual result close to BokehDOF but with a different approach: “gather” neighbor texel values instead of “scattering” bokeh sprites. The two algorithms are completely different — their cost as well — however the end result is quite similar visually:

No DOF

BokehDOF
15ms
( TX1@768MHz )

GatherDOF
2ms
( TX1@768MHz )

The Metal Gear series achieved world-wide recognition when Metal Gear Solid became a best-seller on the original PlayStation almost two decades ago. The title introduced many players to the genre of “tactical espionage action”, an expression coined by Hideo Kojima the creator of the franchise.

Though in my case the first time I played as Snake wasn’t with this game but with the Ghost Babel spin-off on GBC, a lesser-known but nevertheless excellent title with an impressive depth.

The final chapter Metal Gear Solid V: The Phantom Pain was released in 2015 and brings the series to a whole new level of graphics quality thanks to the Fox Engine developed by Kojima Productions. The analysis below is based on the PC version of the game with all the quality knobs set to maximum. Some of the information I present here has already been made public in the GDC 2013 session “Photorealism Through the Eyes of a FOX”.

Dissecting a Frame

Here is a frame taken from the very beginning of the game, during the prologue when Snake tries to make his way out of the hospital. Snake is lying on the floor trying to blend in among the other corpses, he’s at the bottom of the screen with his naked shoulder. Not the most glamorous scene but it illustrates well the different effects the engine can achieve.

Right in front of Snake two soldiers are standing up, they’re looking at some burning silhouette at the end of the hallway. I’ll simply refer to that mysterious individual as the “Man on Fire” not to spoil anything about the story.

So let’s see how this frame is rendered!

Depth Pre-Pass

This pass renders only the geometry of the terrain underneath the hospital as viewed from the point of view of the player and outputs its depth information to a depth buffer. The terrain mesh is generated from the heightmap you can see below: it’s a 16-bit floating point texture containing the terrain elevation value (view from the top). The engine divides the heightmap into different tiles, for each tile a draw call is dispatched with a flat grid of 16x16 vertices. The vertex shader reads the heightmap and modifies on-the-fly the vertex position to match the elevation value. The terrain is rasterized in about 150 draw calls.

Terrain Elevation

Depth Map: 5% Depth Map: 10% Depth Map: 40% Depth Map: 100%

G-Buffer Generation

MGS V uses a deferred renderer like many games of its generation, if you already read the GTA V study you will notice several similar elements. So instead of calculating directly the final lighting value of each pixel as the scene is rendered, the engine first stores the properties of each pixels (like albedo colors, normals…) in several render targets called G-Buffer and will later combine all this information together.

All the following buffers are generated at the same time:

G-Buffer Generation: 25%

 

Albedo	Normal
Specular	Depth

G-Buffer Generation: 50%

 

Albedo	Normal
Specular	Depth

G-Buffer Generation: 75%

 

Albedo	Normal
Specular	Depth

G-Buffer Generation: 100%

 

Albedo	Normal
Specular	Depth

If you use sprites with transparency in your game — and you most likely do for the UI at least — you’ll probably want to pay attention to the fully transparent pixels of your textures (or should I say “texels”).

Even with an alpha of 0, a pixel still has some RGB color value associated with it.
That color shouldn’t matter, right? After all if the pixel is completely transparent, who cares about its color…

Well this color does actually matter, getting it wrong leads to artifacts which are present in many games out there. Most of the time the corruption is very subtle and you won’t notice it but in some cases it’s really standing out.

Corruption Example

Time for a real-world example! Here is the XMB of my PS3 (home menu), browsing some game demos installed.
At first Limbo is selected, I just press “Up” to move the focus to The Unfinished Swan (both great games by the way).

Start

Press up.
Limbo moves down.

White background
fades in.

Artifacts

Do you see what’s happening in the Limbo logo area?
The Unfinished Swan white background fades-in and we end up with the Limbo logo — pure white — drawn on the top of a background which is also pure white. Everything should be completely white in that area, so why do we have these weird gray pixels?

The corruption most likely comes from the Limbo texture using wrong RGB colors for its fully transparent pixels.

Texture Filtering

The artifacts are actually due to the way the GPU filters a texture when it renders a sprite on the screen. Let’s see how this all works with a simple example.

Here is a tiny 12x12 pixel texture of a red cross:

On the left is a zoomed view, the checkerboard pattern is just here to emphasize the completely transparent area with an alpha of 0.
You could use this sprite as an icon to display health points in your UI or as a texture for some in-game med-kit model… (No wait! You don’t want to do this actually!)

Let’s make three versions of this sprite, by changing only the color value of the pixels with an alpha of 0.

Transparent Area: Green Actual Image File:

Transparent Area: Blue Actual Image File:

Transparent Area: Red Actual Image File:

(You can download the image files and do some color picking to confirm the RGB value of the transparent pixels)

These 3 sprites look exactly the same on your screen, right? It makes sense: we only modified the color value of the transparent pixels, which end up being invisible anyway.
But let’s see what happens when these sprites are in motion. Here is a zoomed view to better see the screen pixels:

Well that’s quite some corruption we have here! Some brown tint for the first sprite, purple tint for the second one.
The 3rd one… is correct, it’s how it’s supposed to look.

Let’s focus on the blue version:

As you can see the issue happens when the texture is not perfectly pixel-aligned with the screen pixels. This can be explained by the bilinear filtering the GPU performs when rendering a sprite on the screen: when sampling a texture, the GPU averages the color values of the closest neighbors of the coordinates requested, both in the vertical and horizontal direction.

So if we consider the case where the sprite is misaligned by exactly half a pixel:

Each pixel of the screen will sample the sprite texture right in the middle between 2 texels. This is what happens with the pixel you see on the left: it fetches the sprite texture in the middle between a solid red texel and a transparent blue texel. The averaged color is:

$matht$ 0.5 * [ [\color{#ff2c2c}{1}], [\color{#00c300}{0}], [\color{#2f9fff}{0}], [1] ] + 0.5 * [ [\color{#ff2c2c}{0}], [\color{#00c300}{0}], [\color{#2f9fff}{1}], [0] ] = [ [\color{#ff2c2c}{0.5}], [\color{#00c300}{0}], [\color{#2f9fff}{0.5}], [0.5] ] $matht$

Which is some translucent purple looking like this: █

DOOM pioneered fundamental changes in game design and mechanics back in 1993, it was a world-wide phenomenon which propelled to fame iconic figures like John Carmack and John Romero…

23 years later, id Software now belongs to Zenimax, all the original founders are gone but it didn’t prevent the team at id from showing all its talent by delivering a great game.

The new DOOM is a perfect addition to the franchise, using the new id Tech 6 engine where ex-Crytek Tiago Sousa now assumes the role of lead renderer programmer after John Carmack’s departure.
Historically id Software is known for open-sourcing their engines after a few years, which often leads to nice remakes and breakdowns. Whether this will stand true with id Tech 6 remains to be seen but we don’t necessarily need the source code to appreciate the nice graphics techniques implemented in the engine.

How a Frame is Rendered

We’ll examine the scene below where the player attacks a Gore Nest defended by some Possessed enemies, right after obtaining the Praetor Suit at the beginning of the game.

Unlike most Windows games released these days, DOOM doesn’t use Direct3D but offers an OpenGL and Vulkan backend.
Vulkan being the new hot thing and Baldur Karlsson having recently added support for it in RenderDoc, it was hard resisting picking into DOOM internals. The following observations are based on the game running with Vulkan on a GTX 980 with all the settings on Ultra, some are guesses others are taken from the Siggraph presentation by Tiago Sousa and Jean Geffroy.

Mega-Texture Update

First step is the Mega-Texture update, a technique already present in id Tech 5 used in RAGE and now also used in DOOM.
To give a very basic explanation, the idea is that a few huge textures (16k x 8k in DOOM) are allocated on the GPU memory, each of these being a collection of 128x128 tiles.

16k x 8k storage with 128 x 128 pages

All these tiles are supposed to represent the ideal set of actual textures at the good mipmap level which will be needed by the pixel shaders later to render the particular scene you’re looking at.
When the pixel shader reads from a “virtual texture” it simply ends up reading from some of these 128x128 physical tiles.
Of course depending on where the player is looking at, this set is going to change: new models will appear on screen, referencing other virtual textures, new tiles must be streamed in, old ones streamed out…
So at the beginning of a frame, DOOM updates a few tiles through vkCmdCopyBufferToImage to bring some actual texture data into the GPU memory.

More information about Mega-Textures here and here.

Shadow Map Atlas

For each light casting a shadow, a unique depth map is generated and saved into one tile of a giant 8k x 8k texture atlas. However not every single depth map is calculated at every frame: DOOM heavily re-uses the result of the previous frame and regenerates only the depth maps which need to be updated.

8k x 8k Depth Buffer
(Previous Frame)

8k x 8k Depth Buffer
(Current Frame)

When a light is static and casts shadows only on static objects it makes sense to simply keep its depth map as-is instead of doing unnecessary re-calculation. If some enemy is moving under the light though, the depth map must be generated again.
Depth map sizes can vary depending on the light distance from the camera, also re-generated depth maps don’t necessarily stay inside the same tile within the atlas.
DOOM has specific optimizations like caching the static portion of a depth map, computing then only the dynamic meshes projection and compositing the results.