Leveraging scene view extensions and modules to add custom global shader passes the “right” way.

This post demonstrates adding a custom postprocessing pass using global shaders to an Unreal project without requiring any modification to engine source code. The contents of this post were implemented in Unreal 4.27 and Unreal 5.0.3, and assume that you have access to the Unreal github repository and a basic understanding of graphics programming.

Preamble

Unreal doesn’t make it easy to use custom shaders. They would really, really appreciate it if you would just use materials for every single shader in your game, and honestly how dare you for asking otherwise. The only documentation that Unreal provides for adding custom shader code is not only quite outdated (the author of that documentation has since written a now-also-outdated update on his personal blog), but it explicitly tells you to modify engine source to get it to work. It’s not a particularly well kept secret that essentially every video game made with Unreal uses a modified engine. Indeed, the main saving grace of the sparsity of Unreal’s documentation is how easy it is to get one’s hands on the source code. Nevertheless, it’s a somewhat strange admission for the first party documentation of commercial software to invite the user to fork it.

In the time since the aforementioned documentation was written, a new feature was added to Unreal’s rendering ecosystem: Scene View Extensions. As of writing, there is no documentation for this feature anywhere except as a (surprisingly detailed) comment in the header file. View Extensions implement an interface with functions that are called at various times in Unreal’s capital-R Render function. Simply inherit from FSceneViewExtensionBase and register it, and you’ve got custom code running in the middle of the render pipeline!

Of course, it isn’t quite that easy, but that’s what this blog post is for!

Step 1: Setting Up Your Unreal Project

Before we can actually get underway, the first step is to download and build the Unreal Engine source code. There are a lot of good reasons to do this for any Unreal project, but in our particular case we will be referencing private engine classes, so having the source code is a requirement. Here is the documentation page for downloading the source.

Once we’ve got Unreal built from source, the next step is to actually run Unreal and create our project. For this demonstration I used the first person shooter template, but there’s no reason that this shouldn’t work for any project type. I won’t go into details on how you set up your project because it doesn’t matter (for the examples below the project is called “MyGame”), but an extremely critical step for your own sanity is to install the RenderDoc plugin. If you haven’t used it before, it is an absolute godsend for understanding what the hell is going on while working on shaders.

If you’re planning to follow along closely with my examples, the last step of our setup is to configure an actor in the scene with a custom stencil value, which I’ll be using later in my shader code. First go to Edit > Project Settings > Engine > Rendering > Postprocessing, and set Custom Depth-Stencil Pass to Enabled with Stencil. Then add a cube to the scene, and in the details pane for the StaticMeshComponent under Rendering, enable Render CustomDepth Pass and set CustomDepth Stencil Value to 1. Save your scene, and close Unreal.

Step 2: Defining A Custom Module

Now that the project is set up, it’s time to set up our code structure. In order to get our custom shaders to work, we will need to create a custom module. This is necessary because shaders are compiled at a specific point in the editor startup, and the point where your main game module is run is too late. The modules documentation page should cover most of what I’m doing in this section, but there are a few extra items to hit. My folder structure for the entire project looks like this:

Source
├── MyGame
│   ├── MyGame.Build.cs
│   ├── MyGameCharacter.cpp
│   └── ...
├── MyCustomModule
│   ├── Private
│   │   ├── MyCustomModule.cpp
│   │   └── ...
│   ├── Public
│   │   ├── MyCustomModule.h
│   │   └── ...
│   ├── Shaders
│   │   └── ...
│   └── MyCustomModule.Build.cs
├── MyGame.Target.cs
└── MyGameEditor.Target.cs

.h files go in Public, .cpp files go in Private, .usf/.ush files go in Shaders. Easy enough. Now it’s time to hook things together, one file at a time.

MyGame.Build.cs

This is a simple change: add "MyCustomModule" to the array passed to PublicDependencyModuleNames, so we’ll actually be able to use our module in our game.

MyCustomModule.Build.cs

// MyCustomModule/MyCustomModule.Build.cs

using UnrealBuildTool;
using System.Collections.Generic;
using System.IO;
public class MyCustomModule : ModuleRules {
    public MyCustomModule(ReadOnlyTargetRules Target) : base(Target) {
        PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;

        PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "Renderer", "RenderCore", "RHI"});

        string EnginePath = Path.GetFullPath(Target.RelativeEnginePath);
        PublicIncludePaths.Add(EnginePath + "Source/Runtime/Renderer/Private");
    }
}

We add "Renderer", "RenderCore", "RHI" to PublicDependencyModuleNames, since we’ll be working with all of those modules, but even more importantly, we will also add Source/Runtime/Renderer/Private to our public include paths, so we can reference some private engine classes in our implementation later.

*.Target.cs

In both of the Target.cs files, we’ll add our custom module right under the game module: ExtraModuleNames.Add("MyCustomModule");.

MyGame.uproject

In the .uproject file, we will add our custom module under our game module, but we’ll use PostConfigInit for the loading phase rather than Default. This loads the module earlier so our shaders get compiled.

// MyGame.uproject
...
"Modules": [
{
  "Name": "MyGame",
  "Type": "Runtime",
  "LoadingPhase": "Default"
},
{
  "Name": "MyCustomModule",
  "Type": "Runtime",
  "LoadingPhase": "PostConfigInit"
}
],
...

MyCustomModule.cpp and MyCustomModule.h

The last yak to shave is the module class itself. There are two important aspects to this setup: passing the Shaders directory to AddShaderSourceDirectoryMapping, and calling the IMPLEMENT_MODULE macro with our module class.

// MyCustomModule/Public/MyCustomModule.h

#pragma once

#include "CoreMinimal.h"
#include "Modules/ModuleManager.h"

class FMyCustomModule: public IModuleInterface {
public:
	virtual void StartupModule() override;
};

// MyCustomModule/Private/MyCustomModule.cpp

#include "MyCustomModule.h"

void FMyCustomModule::StartupModule() {
	FString BaseDir = FPaths::Combine(FPaths::GameSourceDir(), TEXT("MyCustomModule"));
	FString ModuleShaderDir = FPaths::Combine(BaseDir, TEXT("Shaders"));
	AddShaderSourceDirectoryMapping(TEXT("/MyCustomModule"), ModuleShaderDir);
}

IMPLEMENT_MODULE(FMyCustomModule, MyCustomModule)

With all this done, we’ll want to regenerate our VS project files by running GenerateProjectFiles.bat or hitting File > Refresh Visual Studio Project in Unreal. You can confirm that the module is loading by adding a UE_LOG call to StartupModule().

Step 3: Shaders

With all of the boring stuff out of the way, it’s time to actually work on our implementation. The gameplan for our custom render pass is to create a mask using the stencil buffer (which we configured an actor with earlier), and then use that mask to modify the scene color. I’ll be doing this in a slightly roundabout way, since this demo is intended to be scaffolding for more interesting shader applications. There are three files necessary for our shaders, one per directory in our module.

Shader Classes

// MyCustomModule/Private/MyShaders.cpp

#include "MyShaders.h"

IMPLEMENT_SHADER_TYPE(, FCombineShaderPS, TEXT("/MyCustomModule/MyShaders.usf"), TEXT("CombineMainPS"), SF_Pixel);
IMPLEMENT_SHADER_TYPE(, FUVMaskShaderPS, TEXT("/MyCustomModule/MyShaders.usf"), TEXT("UVMaskMainPS"), SF_Pixel);

This file uses the magic IMPLEMENT_SHADER_TYPE macro to hook our shader classes to our shader code. You’ll note that the path for the .usf file does not match our exact directory structure. When we called AddShaderSourceDirectoryMapping in MyCustomModule.cpp, we created a virtual MyCustomModule directory that the contents of the Shaders directory are loaded into. As long as the setup in that file and this one matches, your shaders should be loaded correctly.

// MyCustomModule/Public/MyShaders.h

#include "GlobalShader.h"
#include "Runtime/Renderer/Private/ScreenPass.h"

BEGIN_SHADER_PARAMETER_STRUCT(FUVMaskShaderParameters, )
	SHADER_PARAMETER_RDG_TEXTURE(Texture2D, InputTexture)
	SHADER_PARAMETER_RDG_TEXTURE(Texture2D, SceneColor)
	SHADER_PARAMETER_RDG_TEXTURE_SRV(Texture2D<uint2>, InputStencilTexture)
	SHADER_PARAMETER_SAMPLER(SamplerState, InputSampler)
	SHADER_PARAMETER_STRUCT(FScreenPassTextureViewportParameters, ViewParams)
	RENDER_TARGET_BINDING_SLOTS()
END_SHADER_PARAMETER_STRUCT()

class FUVMaskShaderPS : public FGlobalShader {
public:
	DECLARE_EXPORTED_SHADER_TYPE(FUVMaskShaderPS, Global, );
	using FParameters = FUVMaskShaderParameters;
	SHADER_USE_PARAMETER_STRUCT(FUVMaskShaderPS, FGlobalShader);
};

BEGIN_SHADER_PARAMETER_STRUCT(FCombineShaderParameters, )
	SHADER_PARAMETER(FLinearColor, Color)
	SHADER_PARAMETER_RDG_TEXTURE(Texture2D, SceneColor)
	SHADER_PARAMETER_RDG_TEXTURE(Texture2D, InputTexture)
	SHADER_PARAMETER_SAMPLER(SamplerState, InputSampler)
	SHADER_PARAMETER_STRUCT(FScreenPassTextureViewportParameters, ViewParams)
	RENDER_TARGET_BINDING_SLOTS()
END_SHADER_PARAMETER_STRUCT()

class FCombineShaderPS : public FGlobalShader {
public:
	DECLARE_EXPORTED_SHADER_TYPE(FCombineShaderPS, Global, );
	using FParameters = FCombineShaderParameters;
	SHADER_USE_PARAMETER_STRUCT(FCombineShaderPS, FGlobalShader);
};

This file is pretty hefty with arcane Unreal macros, but it’s actually a pretty simple setup. We defined two different classes, one for each shader we declared in our .cpp file, with respective structs for passing uniforms along. The structs are fairly similar, both taking in a texture of the scene color, a sampler, a struct containing data about our view, and binding slots for our outputs. FUVMaskShaderParameters additionally takes a texture of ints for our custom stencil data, and FCombineShaderParameters takes an extra input texture (the result of the previous pass) and a color.

Inside our shader classes, we’re doing three things. First, we’re calling DECLARE_EXPORTED_SHADER_TYPE, which attaches some functions to our class that are required to get the shader to compile. Second, we’re pulling our parameter structs into our classes and calling them FParameters in the local scope. This is important for our third step, when we call SHADER_USE_PARAMETER_STRUCT. If you compare this implementation to the one in the official docs, you’ll note that when they define their shader class, they override the constructor and call .Bind() on the shader parameter they defined. SHADER_USE_PARAMETER_STRUCT saves us the trouble of doing that by binding all of the properties of FParameters (which we’ve just changed to be our custom struct!), which will give us a very nice workflow of directly handing our shader parameter structs to our shaders when the time is right.

Shader Code

// MyCustomModule/Shaders/MyShaders.usf

#include "/Engine/Public/Platform.ush"
#include "/Engine/Private/Common.ush"
#include "/Engine/Private/ScreenPass.ush"

SCREEN_PASS_TEXTURE_VIEWPORT(ViewParams)

SamplerState InputSampler;
Texture2D SceneColor;
Texture2D InputTexture;
Texture2D<uint2> InputStencilTexture;
float4 Color;

float2 PosToUV(float2 Pos) {
	float2 ViewportUV = ((Pos - ViewParams_ViewportMin.xy) * ViewParams_ViewportSizeInverse.xy);
	return ViewportUV * ViewParams_UVViewportSize + ViewParams_UVViewportMin;
}

void UVMaskMainPS(
	float4 SvPosition : SV_POSITION, 
	out float4 UVMask : SV_Target0, 
	out float4 CopyColor : SV_Target1
) {
	uint2 stencil = InputStencilTexture.Load(uint3(SvPosition.xy, 0));
	float2 UV = PosToUV(SvPosition.xy);
	if (stencil.y == 1) {
		UVMask = float4(UV.x, UV.y, 0, 1);
	} else {
		UVMask = float4(0, 0, 0, 0);
	}

	CopyColor = SceneColor.SampleLevel(InputSampler, UV, 0);
}

float4 CombineMainPS(float4 SvPosition : SV_POSITION) : SV_Target0 {
	float2 UV = PosToUV(SvPosition.xy);
	float4 samp = Texture2DSample(InputTexture, InputSampler, UV);

	if (length(samp.xyz) > 0) {
		return Color;
	}
	return Texture2DSample(SceneColor, InputSampler, UV);
}

These pixel shaders should hopefully be pretty straightforward, if slightly wonky. Touching on notable aspects of the code in order:

1. SCREEN_PASS_TEXTURE_VIEWPORT declares the members of ViewParams as uniforms along with our others. These are for…
2. PosToUV(), a helper function to change the space of our SV_POSITION value from (0..ViewportSize) to (0..1). The values like ViewParams_ViewportMin are the ones that were declared by SCREEN_PASS_TEXTURE_VIEWPORT.
3. InputStencilTexture.Load() is used and directly passed our SV_POSITION rather than a sampler with a UV, because of its unique status as a stencil texture. Load reads texel data without filtering or sampling - it doesn’t make sense to filter a stencil and potentially get an interpolated value.
4. UVMaskMainPS has two outputs: our mask and a direct copy of our scene color. More on that later.
5. Fans of the show might have an idea of why I’ve added the extra step of overlaying the UVs on the mask…
6. Texture2DSample (defined in Common.ush) is just an alias of TEXTURE.SampleLevel() that automatically selects the appropriate MIP level.

Step 4: Scene View Extension

We’re nearly there. We’ve written our shaders and got them compiling, now we need to put them to use.

Scene View Extension

// MyCustomModule/Public/MyViewExtension.h

#pragma once

#include "SceneViewExtension.h"

class MYCUSTOMMODULE_API FMyViewExtension : public FSceneViewExtensionBase {
	FLinearColor HighlightColor;
public:
	FMyViewExtension(const FAutoRegister& AutoRegister, FLinearColor CustomColor);

	//~ Begin FSceneViewExtensionBase Interface
	virtual void SetupViewFamily(FSceneViewFamily& InViewFamily) override {};
	virtual void SetupView(FSceneViewFamily& InViewFamily, FSceneView& InView) override {};
	virtual void BeginRenderViewFamily(FSceneViewFamily& InViewFamily) override {};
	virtual void PreRenderViewFamily_RenderThread(FRHICommandListImmediate& RHICmdList, FSceneViewFamily& InViewFamily) override {};
	virtual void PreRenderView_RenderThread(FRHICommandListImmediate& RHICmdList, FSceneView& InView) override {};
	virtual void PostRenderBasePass_RenderThread(FRHICommandListImmediate& RHICmdList, FSceneView& InView) override {};
	virtual void PrePostProcessPass_RenderThread(FRDGBuilder& GraphBuilder, const FSceneView& View, const FPostProcessingInputs& Inputs) override;
	//~ End FSceneViewExtensionBase Interface
};

Here we are inheriting from FSceneViewExtensionBase and overriding the abstract functions on it. We’ve also defined a custom constructor so we can pass some data from the game to our View Extension. Most importantly, however, is the MYCUSTOMMODULE_API DLL export macro. This is generated when your module is built (global search ModuleApiDefine in *.cs files if you’re curious) and ensures that the View Extension is exported and usable by your game module - you will get linker errors if you don’t include it.

In Unreal 5, the ISceneViewExtension class has been enhanced with overloaded functions which take an FRDGBuilder reference rather than FRHICommandListImmediate, e.g.:

// From Engine\Source\Runtime\Engine\Public\SceneViewExtension.h
class ISceneViewExtension {
	...
	virtual void PreRenderView_RenderThread(FRHICommandListImmediate& RHICmdList, FSceneView& InView) {};
	virtual ENGINE_API void PreRenderView_RenderThread(FRDGBuilder& GraphBuilder, FSceneView& InView);
	...
}

// From Engine\Source\Runtime\Engine\Private\SceneViewExtension.cpp
void ISceneViewExtension::PreRenderView_RenderThread(FRDGBuilder& GraphBuilder, FSceneView& InView) {
	AddPass(GraphBuilder, RDG_EVENT_NAME("PreRenderView_RenderThread"), [this, &InView](FRHICommandListImmediate& RHICmdList) {
		PreRenderView_RenderThread(RHICmdList, InView);
	});
}

The RDG function is the one that is actually called in the render thread, and by default is simply a wrapper for the RHI function, so it should be safe to override either one. For the scope of this blog post I only interacted with PrePostProcessPass_RenderThread (which receives an FRDGBuilder reference in both UE4 and UE5) since it also receives the FPostProcessingInputs struct containing references to the screen textures.

// MyCustomModule/Private/MyViewExtension.cpp

#include "MyViewExtension.h"
#include "MyShaders.h"

#include "PixelShaderUtils.h"
#include "PostProcess/PostProcessing.h"

// Yoinked from Engine\Plugins\Experimental\ColorCorrectRegions\Source\ColorCorrectRegions\Private\ColorCorrectRegionsSceneViewExtension.cpp
// This is how it appears in Unreal 5.0.3 - in UE4 it uses FVector2D instead of FVector2f but is otherwise identical
FScreenPassTextureViewportParameters GetTextureViewportParameters(const FScreenPassTextureViewport& InViewport) {
	const FVector2f Extent(InViewport.Extent);
	const FVector2f ViewportMin(InViewport.Rect.Min.X, InViewport.Rect.Min.Y);
	const FVector2f ViewportMax(InViewport.Rect.Max.X, InViewport.Rect.Max.Y);
	const FVector2f ViewportSize = ViewportMax - ViewportMin;

	FScreenPassTextureViewportParameters Parameters;

	if (!InViewport.IsEmpty()) {
		Parameters.Extent = FVector2f(Extent);
		Parameters.ExtentInverse = FVector2f(1.0f / Extent.X, 1.0f / Extent.Y);

		Parameters.ScreenPosToViewportScale = FVector2f(0.5f, -0.5f) * ViewportSize;	
		Parameters.ScreenPosToViewportBias = (0.5f * ViewportSize) + ViewportMin;	

		Parameters.ViewportMin = InViewport.Rect.Min;
		Parameters.ViewportMax = InViewport.Rect.Max;

		Parameters.ViewportSize = ViewportSize;
		Parameters.ViewportSizeInverse = FVector2f(1.0f / Parameters.ViewportSize.X, 1.0f / Parameters.ViewportSize.Y);

		Parameters.UVViewportMin = ViewportMin * Parameters.ExtentInverse;
		Parameters.UVViewportMax = ViewportMax * Parameters.ExtentInverse;

		Parameters.UVViewportSize = Parameters.UVViewportMax - Parameters.UVViewportMin;
		Parameters.UVViewportSizeInverse = FVector2f(1.0f / Parameters.UVViewportSize.X, 1.0f / Parameters.UVViewportSize.Y);

		Parameters.UVViewportBilinearMin = Parameters.UVViewportMin + 0.5f * Parameters.ExtentInverse;
		Parameters.UVViewportBilinearMax = Parameters.UVViewportMax - 0.5f * Parameters.ExtentInverse;
	}

	return Parameters;
}


FMyViewExtension::FMyViewExtension(const FAutoRegister& AutoRegister, FLinearColor CustomColor) : FSceneViewExtensionBase(AutoRegister) {
	HighlightColor = CustomColor;
}

void FMyViewExtension::PrePostProcessPass_RenderThread(FRDGBuilder& GraphBuilder, const FSceneView& View, const FPostProcessingInputs& Inputs) {
	if ((*Inputs.SceneTextures)->CustomDepthTexture->Desc.Format != PF_DepthStencil) return; // This check was only necessary in UE5 - see below

	checkSlow(View.bIsViewInfo); // can't do dynamic_cast because FViewInfo doesn't have any virtual functions.
	const FIntRect Viewport = static_cast<const FViewInfo&>(View).ViewRect;
	FScreenPassTexture SceneColor((*Inputs.SceneTextures)->SceneColorTexture, Viewport);
	FGlobalShaderMap* GlobalShaderMap = GetGlobalShaderMap(GMaxRHIFeatureLevel);

	RDG_EVENT_SCOPE(GraphBuilder, "My Render Pass");

	// Viewport parameters
	const FScreenPassTextureViewport SceneColorTextureViewport(SceneColor);
	const FScreenPassTextureViewportParameters SceneTextureViewportParams = GetTextureViewportParameters(SceneColorTextureViewport);

	// Render targets
	FScreenPassRenderTarget SceneColorCopyRenderTarget;
	SceneColorCopyRenderTarget.Texture = GraphBuilder.CreateTexture((*Inputs.SceneTextures)->SceneColorTexture->Desc, TEXT("Scene Color Copy"));
	FScreenPassRenderTarget UVMaskRenderTarget;
	UVMaskRenderTarget.Texture = GraphBuilder.CreateTexture((*Inputs.SceneTextures)->SceneColorTexture->Desc, TEXT("UV Mask"));

	// Shader setup
	TShaderMapRef<FUVMaskShaderPS> UVMaskPixelShader(GlobalShaderMap);
	FUVMaskShaderPS::FParameters* UVMaskParameters = GraphBuilder.AllocParameters<FUVMaskShaderPS::FParameters>();
	UVMaskParameters->SceneColor = (*Inputs.SceneTextures)->SceneColorTexture;
	UVMaskParameters->InputStencilTexture = GraphBuilder.CreateSRV(FRDGTextureSRVDesc::CreateWithPixelFormat((*Inputs.SceneTextures)->CustomDepthTexture, PF_X24_G8));
	UVMaskParameters->InputSampler = TStaticSamplerState<SF_Point, AM_Clamp, AM_Clamp, AM_Clamp>::GetRHI();
	UVMaskParameters->ViewParams = SceneTextureViewportParams;
	UVMaskParameters->RenderTargets[0] = UVMaskRenderTarget.GetRenderTargetBinding();
	UVMaskParameters->RenderTargets[1] = SceneColorCopyRenderTarget.GetRenderTargetBinding();

	FPixelShaderUtils::AddFullscreenPass(
		GraphBuilder,
		GlobalShaderMap,
		FRDGEventName(TEXT("UV Mask")),
		UVMaskPixelShader,
		UVMaskParameters,
		Viewport);

	// Shader setup
	TShaderMapRef<FCombineShaderPS> CombinePixelShader(GlobalShaderMap);
	FCombineShaderPS::FParameters* CombineParameters = GraphBuilder.AllocParameters<FCombineShaderPS::FParameters>();
	CombineParameters->SceneColor = SceneColorCopyRenderTarget.Texture;
	CombineParameters->InputTexture = UVMaskRenderTarget.Texture;
	CombineParameters->InputSampler = TStaticSamplerState<SF_Point, AM_Clamp, AM_Clamp, AM_Clamp>::GetRHI();
	CombineParameters->Color = HighlightColor;
	CombineParameters->ViewParams = SceneTextureViewportParams;
	CombineParameters->RenderTargets[0] = FRenderTargetBinding(SceneColor.Texture, ERenderTargetLoadAction::ELoad);

	FPixelShaderUtils::AddFullscreenPass(
		GraphBuilder,
		GlobalShaderMap,
		FRDGEventName(TEXT("Combine")),
		CombinePixelShader,
		CombineParameters,
		Viewport);
}

And here it is, the sinewy tendons of the program. Some notes:

1. GetTextureViewportParameters populates the ViewParams struct with appropriate values. We only use a couple of the values in this struct in our actual shaders, but I opted to leave the copypasta as-is.
2. RDG_EVENT_SCOPE groups our two passes together when we view them in RenderDoc.
3. We create two render targets for the two outputs of our first pass. We need to copy the scene color to a new texture because we can’t use the scene color target as both an input and an output simultaneously.
4. We pass the CustomDepthTexture into the UV mask pass as PF_X24_G8 format, which is the format for reading PF_DepthStencil textures.
5. FPixelShaderUtils::AddFullscreenPass is an extremely useful helper function, which provides a vertex shader and a fullscreen quad for our postprocessing pass with no fuss.

The CustomDepthTexture->Desc.Format != PF_DepthStencil check at the beginning of the RenderThread function only seems to be necessary in Unreal 5. CustomDepthTexture is set to a dummy texture with PF_B8G8R8A8 format in the very first frame that is rendered, and a (seemingly) new validation check when we create our InputStencilTexture SRV will fail, asserting that “PF_X24_G8 is only to read stencil from a PF_DepthStencil texture”.

I would like to take this opportunity to complain that the FPostProcessingInputs struct argument to PrePostProcessPass_RenderThread, which contains the actual information from the scene textures, is defined in a private file, and is the primary reason that we had to import private engine files into our module to get this to work. It seems like an oversight to me that PrePostProcessPass_RenderThread, a function on a public interface, would hide the actually useful parts of the render pipeline in a struct that you cannot use without a workaround.

A Note On ViewParams

Something that threw me for a bit of a loop is the distinction between ViewportSize and Extent, with respect to our viewport rectangle. If you look at any of the Unreal render passes using RenderDoc, you will see that all of the scene textures have a two pixel wide black border on the bottom and right sides of the image.

I assume that this is some kind of allowance for texture sampling and window resizing, but if you’re not careful when defining your UVs in your shaders, you could end up sampling these black borders and getting some nasty artifacts in your resulting image. So when trying to get the size of a texture, in most cases you will want to use ViewportSize, whose dimensions go up to the black borders, rather than Extent, whose dimensions include the black borders.

Step 5: Actually Using The Module

The final step in our journey is to actually use this custom module in our game. We already set up the import and building process for it, so the only thing left to do is actually reference it in our game code. This is extremely simple, and the hardest part of it in the context of an actual project would be deciding on where to do it. For this demo, I opted to place it in AMyGameCharacter::BeginPlay(). This has the advantage of avoiding potential runtime errors with my render passes immediately when opening the editor, since the View Extension will only be executed once I hit Play and the character is created. If this is not an advantage for your particular project, you might try registering your View Extension using a subsystem or in an AGameMode class. As long as the lifetime of the object or actor aligns with the desired lifetime of your custom shader, I don’t think you can go wrong.

// MyGame/MyGameCharacter.h

...
class AMyGameCharacter : public ACharacter {
	...
	TSharedPtr<class FMyViewExtension, ESPMode::ThreadSafe> MyViewExtension;
	...
};

// MyGame/MyGameCharacter.cpp

...
#include "MyViewExtension.h"
...

void AMyGameCharacter::BeginPlay() {
	Super::BeginPlay();
	...
	MyViewExtension = FSceneViewExtensions::NewExtension<FMyViewExtension>(FLinearColor::Green);
}

And that’s all there is to it! By calling FSceneViewExtensions::NewExtension, our custom View Extension will be registered with the rendering pipeline, and our custom functions will be called at key points in the render process.

Here is the result of the shader, the cube that was configured with a custom stencil value of 1 has been overlayed with the color passed to the View Extension.

And here are the passes as seen in RenderDoc’s event list, both grouped under the RDG_EVENT_SCOPE “My Render Pass” at the top of the PostProcessing pass.

Conclusion

So there you have it, custom postprocessing without materials or engine modification. Obviously, the particular effect in this demo could be accomplished fairly trivially with a postprocessing material, but hopefully this gives you a framework for doing more interesting things in your own project. This solution is admittedly limited when compared to what can be accomplished if you get your hands dirty with the engine source, and I haven’t investigated the applications of the other View Extension callbacks, but if you exist in the very particular cross section of “just wants to write HLSL for a postprocessing pass” and “doesn’t want to modify the engine”, I hope this helped.

References

My solution for this blog post leaned heavily on the in-engine plugin ColorCorrectRegionsSceneViewExtension. If you’re looking to extend this demo for your own project, that class is considerably more robust.

If you want to go deeper on custom rendering in Unreal, there is no better resource online than Léna Piquet (aka Froyok)’s blog. She is a true master of Unreal rendering, and singlehandedly a better source of documentation for the engine than you will find on most of the internet combined.

Thanks for reading! Follow me on social media:

• Twitter
• Mastodon
• Cohost