Dynamically Constructing Generic Types in Swift

Swift's type system is powerful but sometimes challenging, especially when dealing with generic types at runtime.

In this post, I'll explore how to dynamically construct generic types in Swift - a technique that's particularly useful for framework developers and those working with advanced Swift patterns.

The Challenge

In statically typed languages like Swift, we typically work with types that are known at compile time.

But what if we need to construct a generic type dynamically at runtime? For example, imagine we need to create an instance of Optional<T> or Array<T> where T is only known at runtime.

This is a common challenge when:

• Building reflection systems
• Implementing serialization/deserialization frameworks
• Creating DSL UI frameworks (like SwiftUI/OpenSwiftUI)

Understanding Swift's Type Metadata

To dynamically construct generic types, we need to understand Swift's runtime type system.

Swift maintains type metadata for every type used in our program, which contains information about the type's structure, inheritance, and generic parameters.

Swift Metadata Layout for Structs

Before diving into the struct example, it's important to understand the layout of Swift metadata for structs.

On ABI-stable platforms (like iOS, macOS, etc.), struct metadata¹ follows a specific layout (each row represents 8 bytes):

While this diagram shows struct metadata specifically, other Swift types such as classes, enums, and others follow similar patterns with type-specific variations. For our example, focusing on the struct implementation is sufficient.

Also, we'll use VWT to represent the Value Witness Table and NTD to represent the Nominal Type Descriptor for brevity in the following sections.

Implement Metadata type in Swift

First, we define a Metadata² type to help us handle the runtime type information:

typedef struct OG_SWIFT_NAME(_Metadata) OGSwiftMetadata {
} OGSwiftMetadata;

typedef const OGSwiftMetadata *OGTypeID OG_SWIFT_STRUCT OG_SWIFT_NAME(Metadata);

Then we can add nominal descriptor support to our metadata type:

OG_EXPORT
OG_REFINED_FOR_SWIFT
void const* _Nullable OGTypeNominalDescriptor(OGTypeID typeID) OG_SWIFT_NAME(getter:Metadata.nominalDescriptor(self:));

void const* OGTypeNominalDescriptor(OGTypeID typeID) {
    auto metadata = reinterpret_cast<OG::swift::metadata const*>(typeID);
    return metadata->nominal_descriptor();
}

#include <swift/Runtime/Metadata.h>
#include <swift/Runtime/HeapObject.h>
using namespace swift;

namespace OG {
namespace swift {
class metadata: public Metadata {
public:
    OG_INLINE OG_CONSTEXPR
    TypeContextDescriptor const* descriptor() const OG_NOEXCEPT {
        switch (getKind()) {
            case MetadataKind::Class: {
                const auto cls = static_cast<const ClassMetadata *>(getType());
                // We may build this with a newer OS SDK but run on old OS.
                // So instead of using `isTypeMetadata` / `(Data & SWIFT_CLASS_IS_SWIFT_MASK)`,
                // we manully use 3 here to check isTypeMetadata
                if ((cls->Data & 3) == 0) return nullptr;
                return cls->getDescription();
            }
            case MetadataKind::Struct:
            case MetadataKind::Enum:
            case MetadataKind::Optional: {                
                return static_cast<const TargetValueMetadata<InProcess> *>(getType())->Description;
            }
            default:
                return nullptr;
        }
    }
    
    OG_INLINE OG_CONSTEXPR
    TypeContextDescriptor const* nominal_descriptor() const OG_NOEXCEPT {
        auto descriptor = this->descriptor();
        if (descriptor == nullptr) {
            return nullptr;
        }
        switch(descriptor->getKind()) {
            case ContextDescriptorKind::Struct:
            case ContextDescriptorKind::Enum:
                return descriptor;
            default:
                return nullptr;
        }
    }    
}; /* OG::swift::metadata */
} /* OG::swift */
} /* OG */

Also remember to add some Swifty interface for the Metadata type:

extension Metadata {
    @inlinable
    @inline(__always)
    public init(_ type: any Any.Type) {
        self.init(rawValue: unsafeBitCast(type, to: UnsafePointer<_Metadata>.self))
    }
    
    @inlinable
    @inline(__always)
    public var type: any Any.Type {
        unsafeBitCast(rawValue, to: Any.Type.self)
    }
}

Finally, we'll add a generic type information accessor to the Metadata type:

extension Metadata {
    package func genericType(at index: Int) -> any Any.Type {
        UnsafeRawPointer(rawValue)
            .advanced(by: index &* 8)    // Each generic argument takes 8 bytes
            .advanced(by: 16)            // Skip 16 bytes (kind + NTD) to reach generic arguments
            .assumingMemoryBound(to: Any.Type.self)
            .pointee
    }
}

Implement ProtocolDescriptor type

Before we dive into our concrete example, another important part is to implement the ProtocolDescriptor type.

// MARK: - ProtocolDescriptor

package protocol ProtocolDescriptor {
    static var descriptor: UnsafeRawPointer { get }
}

extension ProtocolDescriptor {
    package static func conformance(of type: any Any.Type) -> TypeConformance<Self>? {
        guard let conformance = swiftConformsToProtocol(type, descriptor) else {
            return nil
        }
        return TypeConformance(storage: (type, conformance))
    }
}

// MARK: - TypeConformance

package struct TypeConformance<P> where P: ProtocolDescriptor {
    package let storage: (type: any Any.Type, conformance: UnsafeRawPointer)

    package init(storage: (type: any Any.Type, conformance: UnsafeRawPointer)) {
        self.storage = storage
    }

    package var type: any Any.Type {
        storage.type
    }

    package var conformance: UnsafeRawPointer {
        storage.conformance
    }

    package var metadata: UnsafeRawPointer {
        unsafeBitCast(storage.type, to: UnsafeRawPointer.self)
    }
}

@_silgen_name("swift_conformsToProtocol")
private func swiftConformsToProtocol(
    _ type: Any.Type,
    _ protocolDescriptor: UnsafeRawPointer
) -> UnsafeRawPointer?

And then we can implement the ConditionalProtocolDescriptor type:

// MARK: - ConditionalProtocolDescriptor

package protocol ConditionalProtocolDescriptor: ProtocolDescriptor {
    static func fetchConditionalType(key: ObjectIdentifier) -> ConditionalTypeDescriptor<Self>?

    static func insertConditionalType(key: ObjectIdentifier, value: ConditionalTypeDescriptor<Self>)
}

// MARK: - ConditionalTypeDescriptor

package struct ConditionalTypeDescriptor<P> where P: ConditionalProtocolDescriptor {
    fileprivate enum Storage {
        case atom(TypeConformance<P>)
        indirect case optional(any Any.Type, ConditionalTypeDescriptor<P>)
        indirect case either(any Any.Type, f: ConditionalTypeDescriptor<P>, t: ConditionalTypeDescriptor<P>)
    }

    private var storage: Storage

    var count: Int

    fileprivate init(storage: Storage, count: Int) {
        self.storage = storage
        self.count = count
    }
}

Runtime Construction of Generic Types for Struct Example

Now that we understand the challenge and have explored all the infrastructure needed to deal with Swift's runtime type system, let's explore a practical approach using SwiftUI's _ConditionalContent as our example. This type is particularly interesting because it demonstrates the complexity of constructing generic types at runtime:

@frozen
public struct _ConditionalContent<TrueContent, FalseContent> {
    @frozen
    public enum Storage {
        case trueContent(TrueContent)
        case falseContent(FalseContent)
    }

    public let storage: Storage
}

What we want to achieve is add a ConditionalTypeDescriptor.init(_:) method to init a ConditionalTypeDescriptor type from any dynamic Any.Type type:

For Optional<T> type, it's straightforward

1. We first get a NTD for Optional<Void> type.
2. Since every Optional<T> type will have the same NTD of Optional enum type, we use it to check if the type is an Optional type.
3. If it is, we then call descriptor method to recursively get the ConditionalTypeDescriptor instance for the wrapped type T, and init an optional case for ConditionalTypeDescriptor with the wrapped type.

let optionalTypeDescriptor: UnsafeRawPointer = Metadata(Void?.self).nominalDescriptor!

extension ConditionalTypeDescriptor {
    fileprivate static func descriptor(type: any Any.Type) -> Self {
        if let descriptor = P.fetchConditionalType(key: ObjectIdentifier(type)) {
            return descriptor
        } else {
            let descriptor = ConditionalTypeDescriptor(type)
            P.insertConditionalType(key: ObjectIdentifier(type), value: descriptor)
            return descriptor
        }
    }

    init(_ type: any Any.Type) {
        let storage: Storage
        let count: Int

        let metadata = Metadata(type)
        let descriptor = metadata.nominalDescriptor

        switch descriptor {
            case optionalTypeDescriptor:
                let wrappedDescriptor = Self.descriptor(type: metadata.genericType(at: 0))
                storage = .optional(type, wrappedDescriptor)
                count = wrappedDescriptor.count + 1
            default: ...
        }
        self.init(storage: storage, count: count)
    }
}

For _ConditionalContent<Void, Void>.Storage type, it's a bit more complex, and we can try to use the same approach to construct it:

1. We first get a NTD for _ConditionalContent<Void, Void> type.
2. Since every _ConditionalContent<TrueContent, FalseContent> type will have the same NTD of _ConditionalContent struct type, we use it to check if the type is a _ConditionalContent type.
3. If it is, we then get the descriptor for the dynmanic TrueContent and FalseContent types, and init an either case for ConditionalTypeDescriptor with the TrueContent and FalseContent types.

The tricky and challenging part is how to get the descriptor for the dynamic _ConditionalContent<TrueContent, FalseContent>.Storage type.

private let conditionalTypeDescriptor: UnsafeRawPointer = Metadata(_ConditionalContent<Void, Void>.self).nominalDescriptor!

extension ConditionalTypeDescriptor {
    init(_ type: any Any.Type) {
        let storage: Storage
        let count: Int

        let metadata = Metadata(type)
        let descriptor = metadata.nominalDescriptor

        switch descriptor {
            case optionalTypeDescriptor: ...
            case conditionalTypeDescriptor:
                let falseDescriptor = Self.descriptor(type: metadata.genericType(at: 1))
                let trueDescriptor = Self.descriptor(type: metadata.genericType(at: 0))
                // How to get the dynamic metadata type for _ConditionalContent<TrueContent, FalseContent>.Storage here?
                storage = .either(?, f: falseDescriptor, t: trueDescriptor)
                count = falseDescriptor.count + trueDescriptor.count
            default: ...
        }
        self.init(storage: storage, count: count)
    }
}

Let's just skip it and focus on the last default part for now.

It's pretty straightforward, we just get the conformance for the type, and init an atom case for ConditionalTypeDescriptor with the conformance.

default: 
    storage = .atom(P.conformance(of: type)!)
    count = 1

Finally, let's deal with the most challenging aspect of this implementation: constructing the _ConditionalContent<?, ?>.Storage type at runtime.

While I haven't found an ideal solution yet, I've developed a working approach that leverages Swift's runtime type system.

The current implementation follows these steps:

1. Get the NTD for _ConditionalContent<Void, Void>.Storage
2. Advance the NTD pointer by 12 bytes to get the metadata accessor function pointer
3. Invoke the accessor function with corresponding arguments to get the final type we want

Here's how it looks in code³:

typealias Accessor =  @convention(c) (UInt, Metadata, Metadata) -> Metadata
let nominal = Metadata(_ConditionalContent<Void, Void>.Storage.self).nominalDescriptor!
// Accessing the metadata accessor function pointer:
// NTD memory layout (simplification):
// [0-4]:   Flags
// [4-8]:   Parent
// [8-12]:  Name
// [12-16]: Accessor function pointer relative offset
let accessorRelativePointer = nominal.advanced(by: 12)
let accessor = unsafeBitCast(
    accessorRelativePointer.advanced(by:Int(accessorRelativePointer.assumingMemoryBound(to: Int32.self).pointee)),
    to: Accessor.self
)
let type = accessor(0, Metadata(metadata.genericType(at: 0)), Metadata(metadata.genericType(at: 1)))

While this solution works, it's admittedly not as elegant as I'd prefer. It relies on implementation details of Swift's runtime that could change in future versions, making it somewhat fragile.

I'm actively exploring more robust alternatives and would welcome insights from the community. If you've tackled similar challenges or have suggestions for improving this approach, I'd love to hear from you!

This exploration into Swift's type system demonstrates both the power and complexity of working with generic types at runtime. While Swift's static type system provides safety and performance, these techniques allow us to push beyond those boundaries when necessary for advanced use cases.

1. Ref TypeMetadata#struct-metadata↩

2. The implementation detail can be found at OpenGraph's Runtime part↩

3. The implementation detail can be found at OpenSwiftUI↩