Binding Data To Shaders

Intro

Are you a graphics programmer? If so, I think odds are good that we have a shared lived experience: the pain of plumbing data from the CPU to the GPU. Do you want to be a graphics programmer? I think odds are good that you will soon experience the same pain.

Given the amount of pain associated with shader data binding, I thought it’d be a good idea to document my current approach to the problem. This post is going to be fairly DX12-centric, as this is the primary backend my renderer currently supports, but I will likely update this post in the future when I flesh out the Vulkan backend or port the engine to console.

Big Picture

By far the best resource I constantly refer to for the details is the resource binding spec. This post isn’t meant to be a tutorial on how to use the API itself, but we’ll still cover the data binding problem at a 10,000 foot view first to get our bearings.

Root Signatures

In simplest terms, shaders are subprograms that run on the GPU, and aside from simple shaders that can hardcode all the data they need within the shader itself, typical shaders need to be fed data to operate on. I say “subprogram” because aside from compute shaders, most shaders operate in conjunction with other shaders to constitute a full end-to-end program (e.g. VS/PS linked together).

The way this abstraction works is that shaders describe a bunch of “slots” (aka bindings) through which data is fed. These slots can describe specific resources like buffers and textures, or they can describe data fed directly to the shader as “constant data,” without the indirection through a resource. Each binding slot is assigned a numerical index (technically, a pair of indices) which we use when defining the root signature later.

Alone, the shader doesn’t dictate to the application how the data and resources are bound. Only what needs to be bound. At runtime, each pipeline used is ultimately associated with a root signature which describes precisely how data should be fed to each binding. Root signatures are collections of root parameters, and each parameter is mapped to a binding slot (or slot range) of the shaders within the pipeline.

To actually use a shader program at runtime, the following needs to happen:

• The root signature associated with the program needs to be set on the command list.
• A shader program (pipeline state object) needs to be set on the command list.
• For each relevant root parameter, a root argument needs to be supplied with a descriptor, a descriptor table, or data, depending on the parameter type.
• Finally, a draw or dispatch can be invoked on the command list depending on the program type.

The driver is responsible for ensuring that all the root arguments supplied in this way are fed to each draw or dispatch. In the case of back-to-back draws/dispatches that use the same root signature, we don’t have to set every root argument every time. When a draw/dispatch occurs, the value of each root argument is simply whatever value was last supplied to the command list.

This is one of the main reasons the root arguments are limited to 64 DWORDs. The limit implies that some drivers have limited storage space to buffer root argument data, and exceeding this buffer space on some implementations would cause root argument data to spill.

As such, DX12 (and other graphics backends) expect that most of the data access occur via some form of indirection. If dispatches A and B access resources { A1, A2, ..., AN } and { B1, B2, ..., BN } respectively, instead of changing all N resources in root arguments directly, we instead swap one root argument which effectively redirects the shader “somehow” to a different set of descriptors. Mechanically, the most vanilla way this is done is by creating separate descriptor tables for A and B containing the resource descriptors needed, and swapping the table as a single root argument.

Indirection isn’t free of course. While indirection may make the root signature smaller and improve driver performance, it imposes an indirection penalty to actually access the data within the shader itself. Because GPUs are typically excellent at hiding this sort of latency, I’ve found it’s not actually easy to measure this impact except in very specific low-occupancy workloads, so I wouldn’t say indirection needs to be top-of-mind most of the time. It’s a good thing to be aware of nonetheless.

Bindless

Aside from a shader advertising specific binding slots, the shader can also access resources directly from the descriptor heap via ResourceDescriptorHeap. The idea is that with just a 4-byte DWORD resource index, a shader can access any resource available within the currently bound descriptor heap. The exact mechanism for how that resource index is supplied is entirely at the developer’s discretion:

• It could be embedded in the root signature directly as a root constant.
• It could be stored in a different buffer.
• It could be fetched from a texture.
• It could be hard-coded (e.g. the “debug print” buffer is always bound at resource index 0).

Vendor recommendations

Here are some general guidelines I’ve pulled from various sources on what each vendor would like to see with respect to resource binding. While I’ve observed some minor changes here and there over the last decade, most of these guidelines have stuck around for a while. This is great news because we can try our best to honor these recommendations and expect our architecture will (probably) perform well across a number of hardware and driver generations both past and future.

AMD

This information is taken mostly from the RDNA Performance Guide but much of it applies to GCN also.

• AMD suggests keeping the root signature data below 13 DWORDs to avoid spilling data to memory. This is probably the most direct piece of documentation that implies the internal buffer size used to handle root signature versioning that exists.
• A related suggestion is to sort root parameters in decreasing order of change frequency. This recommendation matches Microsoft’s recommendations, and makes sense if you imagine how root signature versioning might be implemented under the hood.
• AMD would prefer that we minimize the following:
  • Root argument access (the set of shader stages for which a root argument is visible)
  • Root argument updates
  • Root signature changes

In summary, AMD wants us to have tighter root signatures so it can version the data efficiently without spilling as much as possible.

Nvidia

This information comes from the Advanced API Performance: Descriptors post from Nvidia’s technical blog (formerly published as part of a DirectX “Dos and Donts” guide).

• Nvidia suggests using the full 64 DWORD limit to bind data, in contrast to AMD’s recommendation, suggesting they don’t version root argument data with the same sort of memory hierachy.
• Nvidia also recommends the use of resource indirection, mainly as a way to make more room for other data (which presumably has hotter access patterns).
• There’s a special callout for Pascal, where constant buffer views (CBVs) are faster than shader resource views (SRVs) since they have a separate dedicated data path.

The suggested use of the root signature is quite different to AMD’s recommendations due to differences in the underlying architecture.

Intel

Intel publishes recommendations for Gen11 as well as Arc.

On the whole, their recommendations are similar to AMD’s recommendations so I won’t repeat it all here. In short, Intel prefers tight root signatures in addition minimal state changes. For the integrated architectures in particular, we can expect that with narrower silicon than a discrete GPU, the indirection penalty will be felt a bit more.

Vendor recommendation summary

While Nvidia suggests just YOLO packing tons of data in the root signature directly, I don’t think this is a viable strategy for a few reasons:

• We obviously want a strategy that is vendor-neutral (best effort), so we have to go with the lowest common denominator.
• In practice, the data to be accessed by different draws and dispatches varies quite a lot, so while we could model this by having tons of bespoke root signatures, this would mean lots of root signature state-resetting each time the RS changes on the command list.
• A design where state changes are associated with RS changes sort of adds another orthogonal dimension to our draw/dispatch batching algorithm, which already needs to minimize PSO changes, account for depth sorting, and so on.

The simplest approach, in my mind, is to have as few root signatures as possible (maybe a couple). Bindless is what opens up a ton of doors here, since a generic root signature can now describe any number of resources in an arbitrary hierarchy compared to before.

Shader binding design goals

When designing the renderer backend for the Rookery, here are the main design problems I wanted to resolve:

Shared C++ and HLSL Code

One of the main sources of bugs in rendering code is mismatching data layouts. When C++ and HLSL code is decoupled, it’s quite easy to introduce bugs where the data a shader expects doesn’t quite match the data provided by the CPU. As such, I wanted to build an abstraction that allowed headers to be included directly in both C++ and HLSL source files.

This alone doesn’t guarantee that data layouts will match. Unlike C++ where a struct declaration also decides its layout, in HLSL, the struct layout depends on context!

For example:

struct Foo
{
    float x;
    float y;
};

struct Bar
{
    Foo foo[2];
};

ConstantBuffer<Bar> cb : register(b0, space0);
StructuredBuffer<Bar> sb : register(t0, space0);

In the code above, the layout of Bar depends on whether it is accessed through the constant buffer or the structured buffer (in the case of the constant buffer, an additional 2 bytes of padding is present between the two array elements).

Mara provides an excellent tool for visualizing struct layout differences with and without constant buffer packing rules. Alternatively, you can inspect the DXIL record entries to see how your shader is accessing the data.

All that said, however our abstraction works, it should also be aware of this footgun.

Reasonable flexibility

Some designs impose a higher-level structure to slots by ascribing some semantic meaning to each slot. For example, you might have a scheme where there are fixed slots assigned to each shader to store scene data, material data, draw data, and so on.

While this type of scheme really does make a good amount of sense, I think it lives one level higher in the abstraction stack, and a typical game engine will have a lot of utility shaders that don’t neatly map to this type of scheme.

Understandable and debuggable

This honestly should go without saying, but at this point, I’ve used enough macro-infested shader bindings APIs for several lifetimes.

Macros? Out. Codegen? Out. I want to be able to bind data to a shader, step through code to see precisely if things work as expected in a debugger, and not have the IDE choke when navigating types.

Implementation

Code demo

Before looking at the exact implementation mechanics, let’s get a feel for things by seeing how the current binding interface in the engine works.

Suppose there’s a shader (VS/PS/CS, it doesn’t matter) that we want to associate with a buffer and some constant data. In a shared HLSL/C++ header (I use .hlsli.hpp for such files), the bindings are expressed like so:

// FooBindings.hlsli.hpp

// RK_HLSL is set when compiling with DXC. These headers define
// the binding interface for HLSL and C++ respectively.
#if RK_HLSL
#include <ShaderBindings.hlsli>
#else
#include <gpu/ShaderBindings.hpp>
#endif

// There is no padding in this struct in either C++ or HLSL.
struct Bar
{
    float3 a;
    float2 b;
    uint2 c;
};

// There is no padding in this struct either.
struct FooBindings
{
    StructuredBufferBinding<Bar> bar;
    Float2Binding f;
    UintBinding bar_offset;
};

Then, in the shader itself, we’d use the bindings like so:

#include <FooBindings.hlsli.hpp>

float4 ps_main(PSInput input) : SV_Target0
{
    FooBindings foo = root_data<FooBindings>();
    
    // Getting an instance of Bar from the buffer at an offset.
    // This also shows how the constant bar offset is retrieved.
    Bar bar = foo.bar.load(foo.bar_offset.get());
    
    // Accessing the float2 data is similar.
    float2 f = foo.f.get();

    // Rest of the shader ...
}

In C++, when issuing the draw, the pseudocode looks roughly like this:

// The FooBindings type is just a simple struct.
FooBindings foo;

// bar_buffer is just a reference to a GpuBuffer type.
foo.bar = bar_buffer;

// Binding constant data is pretty easy.
foo.f = float2{1.f, 2.f};
foo.bar_offset = 3u;

// Issuing the actual draw with these bindings looks like this:

// The DrawCommand encapsulates all the state needed to issue a draw.
// It can be stored and replayed as often as needed, or just generated
// on-the-fly.
DrawCommand draw;

// The bind method is templated and embeds all the relevant data
// into the command.
draw.bind(foo);

// Other data relevant to the draw.
draw.vertex_count = 6;
draw.instance_count = 1;
draw.pso = some_pso;

// Internally, the command list sets the root arguments, and also
// performs state transitions needed. If bar_buffer wasn't already
// in the SRV state, a barrier would be emitted to perform the
// access transition.
//
// For example, if the buffer was previously in a UAV state, a
// barrier would be emitted to flush writes before reading them in
// this draw. This automatic barrier generation can be bypassed
// if necessary, but the default behavior is conservative barrier
// placement.
command_list.draw(draw);

Some other aspects of the API that may not be completely clear from the interface demo above:

• The number of root argument DWORDs is actually restricted to 12 DWORDs.
• Compile-time validation asserts if the bindings struct (FooBindings in this case) exceeds the size limit.
• The FooBindings struct can be instantiated on the CPU anywhere, anytime, and it can ge stored, copied, or moved as needed.
• The interface can also bind textures, samplers, byte address buffers, scalar/vector/matrix types, and also resource UAVs of all the aforementioned resource types.
• Padding/alignment is never a concern with this interface. The DWORDs bound to the matrix are always tightly packed. The main restriction here is that I don’t yet provide any mechanism for binding 16-bit types at the top-level. This is actually quite possible, but I haven’t had a use case for it yet.
• Resources can also be accessed with ResourceDescriptorHeap and SamplerDescriptorHeap directly in the shader, although no automatic barrier placement happens when resources are accessed this way.

Overall, this interface has been quite pleasant to work with, and while plumbing data to shaders still isn’t fun by any means, this API has certainly taken the edge off. Hopefully, it’s clear from the demo that there is very little boilerplate here, and also, there is very little code duplication.

OK, so that’s how the end-product work, let’s look at how this works under the hood.

The Bindings.hlsli file

Earlier, in the FooBindings.hlsli.hpp file, we saw that if RK_HLSL was defined, the shader would include a file called ShaderBindings.hlsli. This header is responsible for powering this entire thing in HLSL-land.

// ShaderBindings.hlsli
#pragma once

struct RootConstants
{
    // 12 tightly-packed DWORDs that constant all the root
    // arguments supported. This is the only type in the
    // entire engine subject to the legacy 4x32-bit
    // constant buffer layout rules.
    uint4 data_[3];
};

// The only top-level register binding that will exist in
// ANY shader throughout the engine.
//
// If the root signature changes, this declaration needs to
// change also (and nothing else).
ConstantBuffer<RootConstants> root_data : register(b0, space0);

// This helper function just casts the root_data constants
// to a binding struct like FooBinding earlier.
template <typename T>
T root_data()
{
    return (T)root_data.data_;
}

// There are templated binding types for each supported
// resource, scalar, vector, or sampler that can be bound.
// This is one example, used to bind structured buffers
// as seen earlier.
template <typename T>
struct StructuredBufferBinding
{
    StructuredBuffer<T> get()
    {
        // Because resource indices are set in the root
        // constants, this access is never non-uniform.
        return ResourceDescriptorHeap[index_];
    }

    T load(int element_index)
    {
        return get()[element_index];
    }
    
    // When the root_data cast operation is performed
    // above, the DWORD constant associated with this
    // binding gets copied into index_
    uint index_;
};

// This is an example vector binding for binding constants
// like float2 or int3 directly.

template <int N>
struct FloatBinding
{
    // The constants are bound as uints, so we use asfloat
    // to do a bit-cast to floating point values as opposed
    // to a numeric cast.
    vector<float, N> get()
    {
        return asfloat(data_);
    }
    
    vector<uint, N> data_;
};

struct FloatBinding
{
    float get()
    {
        return asfloat(data_);
    }
    
    uint data_;
};

using Float2Binding = FloatBinding<2>;
using Float3Binding = FloatBinding<3>;
using Float4Binding = FloatBinding<4>;

// ... and so on for the other types. UAVs are fetched from
// the resource heap as above, and samplers are fetched from
// the sampler heap. Signed int types need to use `asint` to
// do a similar bit cast from the unsigned constants.

Notice that to access the resources here, we are just indexing the resource or sampler heap directly as opposed to using the “inline descriptor” feature. This simplifies the C++ binding code, since what gets ultimately bound are just DWORD constants.

There’s some API uniformity here also. All the get accessors are used to access the data in the binding directly, be it a resource or a constant. In the case of resource types, load (and optionally store) methods are available as convenience functions for accessing data within the underlying resource.

Canonical root signatures

There is one root signature in the engine for all graphics pipelines, and one resource for all compute pipelines. The only difference is that the graphics pipeline root signature also includes several static samplers that are commonly used. Aside from the static samplers, both canonical root signatures just advertise 12 DWORD constant root arguments.

At the start of command list recording, each canonical root signature is bound, along with the global descriptor heaps for resources and samplers. The global sampler heap is still useful, even with static sampler support since I want to support dynamic runtime sampler configuration changes.

Implementing C++ binding types

As with HLSL, our C++ code needs to understand all the binding types for resources, samplers, scalars, and vector quantities.

// ShaderBindings.hpp
#pragma once

enum class ShaderBindingType : u8
{
    Unknown,
    RawBuffer,
    RWRawBuffer,
    StructuredBuffer,
    // ... other resource/sampler types
    Uint,
    Uint2,
    // ... other scalar/resource types
};

// All binding types will have 1 or more shader binding slots
// as members. The slot can be thought of as a type-erased
// location to store root argument data.
struct ShaderBindingSlot
{
    constexpr static int root_constant_dword_count = 12;

    // This could be a sampler also, but I'm omitting the
    // union type and other details for brevity.
    void const* resource_   = nullptr;
    u32 value_              = 0u;
    ShaderBindingType type_ = ShaderBindingType::Unknown;
};

// This helper is used to control the number of slots
// occupied by a binding type...
template <ShaderBindingType Type>
constexpr inline int binding_slot_count_v = 1;

// ... for example:
template <>
constexpr inline int binding_slot_count_v<Uint2> = 2;

// In addition to the specialization above, all the other
// vector types need to occupy as many binding slots as
// there are vector components.

// The various binding types inherit this class.
template <ShaderBindingType Type>
class ShaderBinding
{
protected:
    ShaderBinding()
    {
        // Only the first slot actually needs the type to
        // be present since the other slots occupied by the
        // binding all implicitly share the same type.
        // We'll see how this is used later.
        slots_[0].type_ = Type;
    }

    // Resource bindings only every occupy one slot.
    template <typename T>
    void set_resource(T const* resource)
    {
        slots_[0].resource_ = resource;
    }
    
    void set_value(u32 value, int index)
    {
        slots_[index].value_ = value;
    }

    ShaderBindingSlot slots_[binding_slot_count_v<Type>] = {};
};

// Here's an example implementation for the structured
// buffer binding type.
template <typename T>
class StructuredBufferBinding
    : public ShaderBinding<ShaderBindingType::StructuredBuffer>
{
public:
    void operator=(GpuBuffer const& buffer)
    {
        set_resource(&buffer);
    }
};

// This variable template is used to associate a vector
// register type with a binding type...
template <typename T, int N>
constexpr inline ShaderBindingType register_binding_v
    = ShaderBindingType::Unknown;

// ... for example, specializing the float2 binding.
template <>
constexpr inline ShaderBindingType register_binding_v<float, 2>
    = ShaderBindingType::Float2Binding;

// Scalar and vector bindings are done in a generic way.
// This is where the register_binding_v template is used.
template <typename T, int N>
class RegisterBinding :
    public ShaderBinding<register_binding_v<T, N>>
{
public:
    // Scalar quantities with a component count of one
    // are copied into the singular slot.
    void operator=(T value)
        requires(N == 1)
    {
        std::memcpy(&slots_[0].value_, &value, 4);
    }

    // Vector quantities are copied one by one.
    void operator=(Register<T, N, 1> const& value)
        requires(N > 1)
    {
        for (int i = 0; i != N; ++i)
        {
            std::memcpy(&slots_[i].value_, &value[i], 4);
        }
    }
}

Summarizing the code snippet above, we have a base class which defines a binding slot. The binding slot can optionally store either a value or a resource pointer. The slot also stores a type used by the command list abstraction to write the appropriate data into the command list root arguments. While this class looks “ugly” in one sense, I find it to be far more tractable than macro-based abstractions, and at the end of the day, each binding slot really isn’t much state at all. The total code footprint is quite small compared to many equivalent APIs I’ve worked with.

As seen previously, the DrawCommand encapsulates all the information needed to record a draw on the command list, including binding data. Earlier, this looked like:

FooBindings foo;

foo.bar = bar_buffer;
foo.f = float2{1.f, 2.f};
foo.bar_offset = 3u;

DrawCommand draw;

// What happens here?
draw.bind(foo);

The bind function there needs to accept a template and is implemented as a member function of the DrawCommand type. Here is an only slightly abbreviated snippet to show what this looks like:

struct DrawCommand
{
    GraphicsPipeline const* pso_            = nullptr;
    GpuBuffer const* index_buffer_          = nullptr;
    ShaderBindingSlot const* binding_slots_ = nullptr;

    // The vertex count is interpreted as an index count
    // if the index buffer is present.
    int vertex_count_       = 0;
    int index_offset_       = 0;
    int instance_count_     = 0;
    int binding_slot_count_ = 0;

    template <typename T>
    void bind(T const& bindings)
    {
        binding_slots_      = (ShaderBindingSlot const*)&bindings;
        binding_slot_count_ = count;
    }
};

That’s it really. The shader binding slots are type erased, so the draw command just tracks the initial ShaderBindingSlot address and a count for how many total slots are present. The “real” code has more stuff here to handle things like lifetime tracking and such, but I’m just focusing on the core mechanics here.

In addition to a DrawCommand type, there is also a DispatchCommand type which encapsulates information for recording a compute shader dispatch. The binding interface for a DispatchCommand is identical.

Writing root constants

All that remains is to actually extract the root constant data and set it on the recording command list.

FooBindings foo;

foo.bar = bar_buffer;
foo.f = float2{1.f, 2.f};
foo.bar_offset = 3u;

DrawCommand draw;

draw.bind(foo);

// What happens here?
command_list.draw(draw);

Basically, when draw happens, we do the following:

• Visit each slot and examine the slot type.
  • If the slot refers to a resource, we cast the type-erased resource pointer to the expected resource type and do access verification.
  • If a texture layout transition or barrier is needed, we issue the barrier.
• While iterating each slot, we also write the slot’s DWORD value.
  • For resources, we query the resource SRV or UAV index and write that.
  • For constant data, we write the embedded u32 value.
  • Vector types occupy a known number of slots, so we write as many DWORD values as needed.

After the above, we have a simple array of up to 12 DWORD values we can send directly to SetGraphicsRoot32BitConstants or SetComputeRoot32BitConstants. All 12 values are available to all shader stages. The root arguments are slim enough that I haven’t observed this to be a problem, and restricting shader stage visibility hasn’t been worth it for my use cases.

It’s important to observe that at draw time, the original struct binding type (FooBindings in our example) is not known to the routine above. This way, binding types can be defined in any module, and they ultimately turn into a small collection of up to 12 type-erased shader binding slots.

Outro

As described, shader binding in the Rookery can be summarized like so:

• There are two canonical root signatures used for all shader programs, one for graphics and one for compute.
• These root signatures are set at the start of every command list, along with the global descriptor and sampler heaps.
• Binding data to every shader is done through a single HLSL register binding.
• A binding struct are collections of fields, each of which derives the ShaderBinding type.
• At draw time, the bindings are iterated to extract a set of up to 12 DWORD root arguments.
• Resources bound directly in as root arguments will be transitioned to the correct texture layout.
• Barriers needed to flush previous writes in the recording pipeline are performed implicitly as well.
• In HLSL, a single call to root_data<T> retrieves the binding struct, at which point data bound to each slot can be retrieved via accessor.

Overall, I’ve been happy with the design due to its simplicity and ease-of-use. This design isn’t without compromises of course:

• Resources that aren’t bound directly to a root argument slot don’t have automatic barriers/transitions. I have a separate mechanism for resources accessed indirectly in this way. In practice, this isn’t much of an issue because most resources accessed indirectly tend to be read-only textures and such.
• As mentioned, I’m currently exposing all 12 root arguments to all shaders in the pipeline. I haven’t measured any impact for this decision, but I may or may not change this in the future when I test on a broader spectrum of hardware.
• All resource access has at least one level of indirection, since I don’t ever bind descriptors directly into root arguments. This choice vastly simplifies how the binding types are implemented in HLSL, which has been a worthwhile tradeoff.
• On the performance front, I don’t currently support CBVs at all. While the interface described could do this, I’d rather never have to worry about legacy constant buffer layout rules ever. By eschewing CBV support and sharing struct definitions between C++ and HLSL with matching layout semantics, I eliminate an entire class of bugs. With the time saved, I’m sure I can make up the difference on Pascal (then one GPU architecture relevant to me that cares about this) in other ways, many times over.
• Right now, the API just writes all root constants with each draw and dispatch. It’s not too hard to add some user-space versioning on top of the system as described though.

One last callout is that this approach is compatible with Vulkan as well and I have an initial prototype for this but am holding out for broader VK_EXT_descriptor_heap support before fully ironing out the implementation. Bindless indexing with Vulkan is possible without VK_EXT_descriptor_heap, but is a bit less convenient since you have to work around Vulkan’s descriptor set abstraction (which I believe is a historical mistake, see Faith Ekstrand’s talk from XDC 2025 for a nice description of the problem and how Vulkan developers are seeking to improve things).

And that’s it! I still don’t like plumbing data between the GPU and CPU, but I figure every effort spent to reduce time spent debugging issues here to preserve my sanity is time well spent.

Discuss this post on Patreon.