跳转到正文
Retep's
返回

[TIL] IREE 编译流水线导读

编辑文章

本文由英文原文翻译。代码、公式与技术名称保留原文。

[TIL] IREE 编译流水线

本文用一个小型向量程序,介绍 IREE 各流水线阶段的工作,以及生成的 IR。

核心设计是分离主机编排与设备计算。二者按不同速度降级:GPU kernel 已经变成 LLVM/NVVM IR 时,外围程序可能仍使用 Stream 操作;之后 kernel 成为不透明载荷,而主机程序变成调用 HAL 运行时的 VM 字节码。

程序与分析依据

输入是一个简单的四元素向量程序:

func.func @simple_mul(%arg0: tensor<4xf32>,
                      %arg1: tensor<4xf32>) -> tensor<4xf32> {
  %0 = "arith.mulf"(%arg0, %arg1)
      : (tensor<4xf32>, tensor<4xf32>) -> tensor<4xf32>
  return %0 : tensor<4xf32>
}

实际计算为 C[i] = A[i] * B[i]。输入 [1, 2, 3, 4] 与 [10, 20, 30, 40] 时,结果为 [10, 40, 90, 160]。

各阶段概览

使用 --compile-to 查看各阶段结果。一个阶段可能执行多个 pass,同一份产物通常同时包含多种方言,这就是渐进式降级。

阶段边界主要职责代表操作或属性
input将受支持输入规范化为公共表示util.func、张量运算
abi建立面向应用的调用约定hal.tensor.import、hal.tensor.export
preprocessing全局优化前的预处理ABI 包装中的张量计算
global-optimization优化结构化张量计算linalg.generic、计算屏障
dispatch-creation划分可调度工作区域dispatch 区域与计算边界
flow提取和组织可执行计算flow.executable、flow.dispatch
stream规划资源与执行依赖stream.resource.alloca、stream.cmd.execute
executable-sources具体化目标变体和接口hal.executable.variant、绑定布局
executable-configurations选择代码生成策略和参数translation_info、lowering_config
executable-targets将 kernel 降级为面向目标的 IRllvm.func、nvvm.read.ptx.sreg.tid.x
hal降级编排并序列化设备程序队列、fence、命令缓冲区、二进制
vm用 VM 指令和导入表示主机控制vm.func、vm.call、vm.rodata
最终输出编码字节码并封装运行时模块.vmfb

1. Input、ABI 与预处理:让程序可调用

Input

程序从 func.func 和张量运算开始,公共函数导入后使用 IREE utility 方言表示:

util.func public @simple_mul(%arg0: tensor<4xf32>,
                            %arg1: tensor<4xf32>) -> tensor<4xf32> {
  %0 = arith.mulf %arg0, %arg1 : tensor<4xf32>
  util.return %0 : tensor<4xf32>
}

输入转换将受支持的源表示转换为后续编译器可理解的形式。PyTorch 程序需要先经过受支持的导入或导出路径,再进入类似流水线。这里手写 MLIR 已经是简单张量计算,因此这一步很直接。

ABI 与预处理

下一步建立 ABI 边界,区分外部对象与内部表示,定义程序如何与外部交互。公共签名接收并返回 !hal.buffer_view:

%a = hal.tensor.import %arg0 "input0"
    : !hal.buffer_view -> tensor<4xf32>
%b = hal.tensor.import %arg1 "input1"
    : !hal.buffer_view -> tensor<4xf32>
%c = arith.mulf %a, %b : tensor<4xf32>
%view = hal.tensor.export %c "output0"
    : tensor<4xf32> -> !hal.buffer_view

buffer view 是面向运行时的对象,描述缓冲区及张量元数据。函数内部仍基于张量优化,reflection 保留原签名:

sync func @simple_mul(%input0: tensor<4xf32>,
                     %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)

这样分离了外部对象模型与内部数学表示。import、export 标记边界,出现它们不意味着必然发生物理数据复制。

预处理通常涉及规范化。这里程序太小,几乎没有可简化的内容,因此与 ABI 阶段结果相同。完整结果如下:

module {
  util.func public @simple_mul(%arg0: !hal.buffer_view, %arg1: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}} {
    %0 = hal.tensor.import %arg0 "input0" : !hal.buffer_view -> tensor<4xf32>
    %1 = hal.tensor.import %arg1 "input1" : !hal.buffer_view -> tensor<4xf32>
    %2 = arith.mulf %0, %1 : tensor<4xf32>
    %3 = hal.tensor.export %2 "output0" : tensor<4xf32> -> !hal.buffer_view
    util.return %3 : !hal.buffer_view
  }
}

2. 全局优化

全局优化通常包括图级改写,例如公共子表达式消除和常量折叠,与目标硬件无关。

示例中的逐元素乘法转换为结构化的 linalg.generic:

indexing_maps = [
  affine_map<(d0) -> (d0)>,
  affine_map<(d0) -> (d0)>,
  affine_map<(d0) -> (d0)>
]
iterator_types = ["parallel"]

三个映射描述循环索引如何访问输入 A、输入 B 和输出 C,均使用相同索引。循环体执行标量乘法并返回结果:

^bb0(%a: f32, %b: f32, %out: f32):
  %product = arith.mulf %a, %b : f32
  linalg.yield %product : f32

结构化迭代使计算能够参与融合、分块和分发变换。这里使用第一个输入张量作为 destination-style outs。标量体忽略 %out,因此不依赖旧输出内容。但这一阶段的张量语义并不代表原地修改调用者输入;后续别名与缓冲区化决策仍须保持程序语义。

外围 iree_tensor_ext.compute_barrier.start/end 限制编译器变换,例如阻止 reshape 跨边界传播。它们是编译器层面的边界,不一定需要生成实际运行时操作。

全局优化发生在张量结构仍可见时。对于大模型,可以在 dispatch 边界限制组合范围之前,跨较大计算图进行优化。

全局优化结果:

module {
  util.func public @simple_mul(%arg0: !hal.buffer_view, %arg1: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}} {
    %0 = hal.tensor.import %arg0 "input0" : !hal.buffer_view -> tensor<4xf32>
    %1 = iree_tensor_ext.compute_barrier.start %0 : tensor<4xf32> -> tensor<4xf32>
    %2 = hal.tensor.import %arg1 "input1" : !hal.buffer_view -> tensor<4xf32>
    %3 = iree_tensor_ext.compute_barrier.start %2 : tensor<4xf32> -> tensor<4xf32>
    %4 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>], iterator_types = ["parallel"]} ins(%1, %3 : tensor<4xf32>, tensor<4xf32>) outs(%1 : tensor<4xf32>) {
    ^bb0(%in: f32, %in_0: f32, %out: f32):
      %7 = arith.mulf %in, %in_0 : f32
      linalg.yield %7 : f32
    } -> tensor<4xf32>
    %5 = iree_tensor_ext.compute_barrier.end %4 : tensor<4xf32> -> tensor<4xf32>
    %6 = hal.tensor.export %5 "output0" : tensor<4xf32> -> !hal.buffer_view
    util.return %6 : !hal.buffer_view
  }
}

3. Flow

Flow 围绕 dispatch 组织可执行工作,决定哪些操作属于一个执行单元。此前只有计算图,尚未明确哪些计算应组合为设备上的 SPMD 工作。flow.executable 容纳相应可执行计算,dispatch 成为调度单位。例如逐元素操作自然适合 SPMD,而归约会改变执行边界、grid 和 block 形状,需要另外处理。

Flow 后,外层函数调用:

%c = flow.dispatch
    @simple_mul_dispatch_0::@simple_mul_dispatch_0_elementwise_4_f32(%a, %b)
    : (tensor<4xf32>, tensor<4xf32>) -> tensor<4xf32>

计算位于 flow.executable 中,包含导出入口和嵌套函数。后者接收两个只读 dispatch tensor 和一个只写输出 tensor,加载输入切片、执行 Linalg,再存储结果。

这建立了清楚的分工:外围程序指定调用哪段计算,executable 描述如何计算结果。dispatch 边界同时是调度与编译边界。

导出中仍包含:

%x, %y, %z = iree_tensor_ext.dispatch.workgroup_count_from_slice()

这是 workgroup 数量计算的占位。workgroup 是分布式执行单位,在 CUDA 中对应线程块。组数和每组线程数是不同概念,此时都不能直接由四元素张量形状推出。

Flow 阶段完整结果:

module {
  flow.executable private @simple_mul_dispatch_0 {
    flow.executable.export public @simple_mul_dispatch_0_elementwise_4_f32 workgroups() -> (index, index, index) {
      %x, %y, %z = iree_tensor_ext.dispatch.workgroup_count_from_slice()
      flow.return %x, %y, %z : index, index, index
    }
    builtin.module {
      func.func @simple_mul_dispatch_0_elementwise_4_f32(%arg0: !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>>, %arg1: !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>>, %arg2: !iree_tensor_ext.dispatch.tensor<writeonly:tensor<4xf32>>) {
        %0 = iree_tensor_ext.dispatch.tensor.load %arg0, offsets = [0], sizes = [4], strides = [1] : !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>> -> tensor<4xf32>
        %1 = iree_tensor_ext.dispatch.tensor.load %arg1, offsets = [0], sizes = [4], strides = [1] : !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>> -> tensor<4xf32>
        %2 = tensor.empty() : tensor<4xf32>
        %3 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>], iterator_types = ["parallel"]} ins(%0, %1 : tensor<4xf32>, tensor<4xf32>) outs(%2 : tensor<4xf32>) {
        ^bb0(%in: f32, %in_0: f32, %out: f32):
          %4 = arith.mulf %in, %in_0 : f32
          linalg.yield %4 : f32
        } -> tensor<4xf32>
        iree_tensor_ext.dispatch.tensor.store %3, %arg2, offsets = [0], sizes = [4], strides = [1] : tensor<4xf32> -> !iree_tensor_ext.dispatch.tensor<writeonly:tensor<4xf32>>
        return
      }
    }
  }
  util.func public @simple_mul(%arg0: !hal.buffer_view, %arg1: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}} {
    %0 = hal.tensor.import %arg0 "input0" : !hal.buffer_view -> tensor<4xf32>
    %1 = hal.tensor.import %arg1 "input1" : !hal.buffer_view -> tensor<4xf32>
    %2 = flow.dispatch @simple_mul_dispatch_0::@simple_mul_dispatch_0_elementwise_4_f32(%0, %1) : (tensor<4xf32>, tensor<4xf32>) -> tensor<4xf32>
    %3 = hal.tensor.export %2 "output0" : tensor<4xf32> -> !hal.buffer_view
    util.return %3 : !hal.buffer_view
  }
}

4. Stream

Stream 让资源大小和调度关系显式可见。调度工作时需要知道:

Stream 将外部资源访问改写为带显式缓冲区大小的 stream.tensor.import:

%a = stream.tensor.import %arg0
    : !hal.buffer_view -> tensor<4xf32>
      in !stream.resource<external>{%c16}

external 资源类别表示生命周期跨越程序边界,例如由主机分配后复制到设备,或来自其他 kernel 的资源。

输出被显式分配:

%result, %ready = stream.resource.alloca uninitialized
    : !stream.resource<external>{%c16} => !stream.timepoint

分配同时返回资源及其可用时间点。这里的 alloca 是 Stream 资源操作,并不承诺使用 CPU 栈内存。

执行等待资源可用,并声明访问:

stream.cmd.dispatch @simple_mul_dispatch_0::@... {
  ro %a[0 for 16] : ...,
  ro %b[0 for 16] : ...,
  wo %result[0 for 16] : ...
}

它标注两次 16 字节读取和一次 16 字节写入。外围 stream.cmd.execute 返回完成时间点,stream.timepoint.await 将后续结果使用与执行完成关联起来。

Stream 将数据依赖与具体队列、同步 API 分离。较早的异步形式可能包含 stream.async.transfer、stream.async.dispatch,通过资源值表达依赖。调度与分配再逐步降级为命令区域和时间点。stream.async.transfer 表示资源状态或 affinity 转换,可能需要跨内存空间复制,但优化也可以改写或移除它。

executable 内的 !stream.binding 参数和 stream.binding.subspan 描述如何从绑定资源获得张量视图。Linalg 计算仍然处于高层,使 IREE 能在确定 kernel 最终实现前先分析资源调度。

Stream 完整结果:

module {
  stream.executable private @simple_mul_dispatch_0 {
    stream.executable.export public @simple_mul_dispatch_0_elementwise_4_f32 workgroups() -> (index, index, index) {
      %x, %y, %z = iree_tensor_ext.dispatch.workgroup_count_from_slice()
      stream.return %x, %y, %z : index, index, index
    }
    builtin.module {
      func.func @simple_mul_dispatch_0_elementwise_4_f32(%arg0: !stream.binding {stream.alignment = 64 : index}, %arg1: !stream.binding {stream.alignment = 64 : index}, %arg2: !stream.binding {stream.alignment = 64 : index}) {
        %c0 = arith.constant 0 : index
        %0 = stream.binding.subspan %arg0[%c0] : !stream.binding -> !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>>
        %1 = stream.binding.subspan %arg1[%c0] : !stream.binding -> !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>>
        %2 = stream.binding.subspan %arg2[%c0] : !stream.binding -> !iree_tensor_ext.dispatch.tensor<writeonly:tensor<4xf32>>
        %3 = iree_tensor_ext.dispatch.tensor.load %0, offsets = [0], sizes = [4], strides = [1] : !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>> -> tensor<4xf32>
        %4 = iree_tensor_ext.dispatch.tensor.load %1, offsets = [0], sizes = [4], strides = [1] : !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>> -> tensor<4xf32>
        %5 = tensor.empty() : tensor<4xf32>
        %6 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>], iterator_types = ["parallel"]} ins(%3, %4 : tensor<4xf32>, tensor<4xf32>) outs(%5 : tensor<4xf32>) {
        ^bb0(%in: f32, %in_0: f32, %out: f32):
          %7 = arith.mulf %in, %in_0 : f32
          linalg.yield %7 : f32
        } -> tensor<4xf32>
        iree_tensor_ext.dispatch.tensor.store %6, %2, offsets = [0], sizes = [4], strides = [1] : tensor<4xf32> -> !iree_tensor_ext.dispatch.tensor<writeonly:tensor<4xf32>>
        return
      }
    }
  }
  util.func public @simple_mul(%arg0: !hal.buffer_view, %arg1: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}} {
    %c0 = arith.constant 0 : index
    %c16 = arith.constant 16 : index
    %c4 = arith.constant 4 : index
    %element_type_f32 = hal.element_type<f32> : i32
    %dense_row_major = hal.encoding_type<dense_row_major> : i32
    hal.buffer_view.assert<%arg0 : !hal.buffer_view> message("input0") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %0 = stream.tensor.import %arg0 : !hal.buffer_view -> tensor<4xf32> in !stream.resource<external>{%c16}
    hal.buffer_view.assert<%arg1 : !hal.buffer_view> message("input1") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %1 = stream.tensor.import %arg1 : !hal.buffer_view -> tensor<4xf32> in !stream.resource<external>{%c16}
    %result, %result_timepoint = stream.resource.alloca uninitialized : !stream.resource<external>{%c16} => !stream.timepoint
    %2 = stream.cmd.execute await(%result_timepoint) => with(%0 as %arg2: !stream.resource<external>{%c16}, %1 as %arg3: !stream.resource<external>{%c16}, %result as %arg4: !stream.resource<external>{%c16}) {
      stream.cmd.dispatch @simple_mul_dispatch_0::@simple_mul_dispatch_0_elementwise_4_f32 {
        ro %arg2[%c0 for %c16] : !stream.resource<external>{%c16},
        ro %arg3[%c0 for %c16] : !stream.resource<external>{%c16},
        wo %arg4[%c0 for %c16] : !stream.resource<external>{%c16}
      }
    } => !stream.timepoint
    %3 = stream.timepoint.await %2 => %result : !stream.resource<external>{%c16}
    %4 = stream.tensor.export %3 : tensor<4xf32> in !stream.resource<external>{%c16} -> !hal.buffer_view
    util.return %4 : !hal.buffer_view
  }
}

5. Executable Sources

这个阶段后,程序拥有默认设备 affinity,以及面向 CUDA 的设备声明。分配和执行等操作都关联到该设备。

executable 变为:

hal.executable @simple_mul_dispatch_0
  └─ hal.executable.variant @cuda_nvptx_fb
       ├─ hal.executable.export
       └─ builtin.module containing the kernel

variant 是面向某个执行目标的实现。即使本例只有一种,也允许存在多个变体。导出建立包含三个 storage-buffer binding 的 pipeline layout:

绑定含义访问
0A只读
1B只读
2Ckernel 写入

kernel 通过 hal.interface.binding.subspan binding(0) 获取数据,绑定 1、2 同理。设备元数据记录 subgroup 大小、最大 block 维度和可用 workgroup 内存,作为代码生成约束。

这个阶段不修改 kernel 内部计算。

module attributes {stream.affinity.default = #hal.device.affinity<@__device_0>} {
  util.global private @__device_0 = #hal.device.target<"cuda", [#hal.executable.target<"cuda", "cuda-nvptx-fb", {iree_codegen.target_info = #iree_gpu.target<arch = "sm_60", features = "+ptx76", wgp = <compute = fp64|fp32|fp16|int64|int32|int16|int8, storage = b64|b32|b16|b8, subgroup = shuffle|arithmetic, subgroup_size_choices = [32], max_workgroup_sizes = [1024, 1024, 1024], max_thread_count_per_workgroup = 1024, max_workgroup_memory_bytes = 49152, max_workgroup_counts = [2147483647, 65535, 65535]>>}>]> : !hal.device
  hal.executable private @simple_mul_dispatch_0 {
    hal.executable.variant public @cuda_nvptx_fb target(<"cuda", "cuda-nvptx-fb", {iree_codegen.target_info = #iree_gpu.target<arch = "sm_60", features = "+ptx76", wgp = <compute = fp64|fp32|fp16|int64|int32|int16|int8, storage = b64|b32|b16|b8, subgroup = shuffle|arithmetic, subgroup_size_choices = [32], max_workgroup_sizes = [1024, 1024, 1024], max_thread_count_per_workgroup = 1024, max_workgroup_memory_bytes = 49152, max_workgroup_counts = [2147483647, 65535, 65535]>>}>) {
      hal.executable.export public @simple_mul_dispatch_0_elementwise_4_f32 ordinal(0) layout(#hal.pipeline.layout<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) count(%arg0: !hal.device) -> (index, index, index) {
        %x, %y, %z = iree_tensor_ext.dispatch.workgroup_count_from_slice()
        hal.return %x, %y, %z : index, index, index
      }
      builtin.module {
        func.func @simple_mul_dispatch_0_elementwise_4_f32() {
          %c0 = arith.constant 0 : index
          %0 = hal.interface.binding.subspan layout(<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) binding(0) alignment(64) offset(%c0) flags("ReadOnly|Indirect") : !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>>
          %1 = hal.interface.binding.subspan layout(<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) binding(1) alignment(64) offset(%c0) flags("ReadOnly|Indirect") : !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>>
          %2 = hal.interface.binding.subspan layout(<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) binding(2) alignment(64) offset(%c0) flags(Indirect) : !iree_tensor_ext.dispatch.tensor<writeonly:tensor<4xf32>>
          %3 = iree_tensor_ext.dispatch.tensor.load %0, offsets = [0], sizes = [4], strides = [1] : !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>> -> tensor<4xf32>
          %4 = iree_tensor_ext.dispatch.tensor.load %1, offsets = [0], sizes = [4], strides = [1] : !iree_tensor_ext.dispatch.tensor<readonly:tensor<4xf32>> -> tensor<4xf32>
          %5 = tensor.empty() : tensor<4xf32>
          %6 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>], iterator_types = ["parallel"]} ins(%3, %4 : tensor<4xf32>, tensor<4xf32>) outs(%5 : tensor<4xf32>) {
          ^bb0(%in: f32, %in_0: f32, %out: f32):
            %7 = arith.mulf %in, %in_0 : f32
            linalg.yield %7 : f32
          } -> tensor<4xf32>
          iree_tensor_ext.dispatch.tensor.store %6, %2, offsets = [0], sizes = [4], strides = [1] : tensor<4xf32> -> !iree_tensor_ext.dispatch.tensor<writeonly:tensor<4xf32>>
          return
        }
      }
    }
  }
  util.func public @simple_mul(%arg0: !hal.buffer_view, %arg1: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}} {
    %c0 = arith.constant 0 : index
    %c16 = arith.constant 16 : index
    %c4 = arith.constant 4 : index
    %element_type_f32 = hal.element_type<f32> : i32
    %dense_row_major = hal.encoding_type<dense_row_major> : i32
    hal.buffer_view.assert<%arg0 : !hal.buffer_view> message("input0") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %0 = stream.tensor.import on(#hal.device.affinity<@__device_0>) %arg0 : !hal.buffer_view -> tensor<4xf32> in !stream.resource<external>{%c16}
    hal.buffer_view.assert<%arg1 : !hal.buffer_view> message("input1") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %1 = stream.tensor.import on(#hal.device.affinity<@__device_0>) %arg1 : !hal.buffer_view -> tensor<4xf32> in !stream.resource<external>{%c16}
    %result, %result_timepoint = stream.resource.alloca uninitialized on(#hal.device.affinity<@__device_0>) : !stream.resource<external>{%c16} => !stream.timepoint
    %2 = stream.cmd.execute on(#hal.device.affinity<@__device_0>) await(%result_timepoint) => with(%0 as %arg2: !stream.resource<external>{%c16}, %1 as %arg3: !stream.resource<external>{%c16}, %result as %arg4: !stream.resource<external>{%c16}) {
      stream.cmd.dispatch @simple_mul_dispatch_0::@cuda_nvptx_fb::@simple_mul_dispatch_0_elementwise_4_f32 {
        ro %arg2[%c0 for %c16] : !stream.resource<external>{%c16},
        ro %arg3[%c0 for %c16] : !stream.resource<external>{%c16},
        wo %arg4[%c0 for %c16] : !stream.resource<external>{%c16}
      }
    } => !stream.timepoint
    %3 = stream.timepoint.await %2 => %result : !stream.resource<external>{%c16}
    %4 = stream.tensor.export on(#hal.device.affinity<@__device_0>) %3 : tensor<4xf32> in !stream.resource<external>{%c16} -> !hal.buffer_view
    util.return %4 : !hal.buffer_view
  }
}

6. Executable Configurations:选择降级策略

该阶段加入了对预期 GPU 映射最精炼的说明:

translation_info = #iree_codegen.translation_info<
  pipeline = #iree_gpu.pipeline<TileAndFuse>
  workgroup_size = [32, 1, 1]
  subgroup_size = 32
>

Linalg 操作获得:

lowering_config = #iree_gpu.lowering_config<{
  thread = [1], workgroup = [32]
}>

这些属性分别回答不同问题:

四元素问题可以放进一个 block,仅四个线程需要进行有效乘法。这里由配置表达映射,下一份产物会展示生成的控制流。

TileAndFuse 选择代码生成流水线。名称并不意味着本例中发生了多操作融合,因为这里只有一个计算操作。策略选择与具体变换分离,让配置可检查,也为用户选择和调优提供入口。

缓冲区接口也被规范化:binding subspan 现在生成 memref<4xf32, ...>,iree_codegen.load_from_buffer、store_to_buffer 连接它们与剩余张量计算。边界出现 memref 并不意味着完整缓冲区化已完成。

module attributes {stream.affinity.default = #hal.device.affinity<@__device_0>} {
  util.global private @__device_0 = #hal.device.target<"cuda", [#hal.executable.target<"cuda", "cuda-nvptx-fb", {iree_codegen.target_info = #iree_gpu.target<arch = "sm_60", features = "+ptx76", wgp = <compute = fp64|fp32|fp16|int64|int32|int16|int8, storage = b64|b32|b16|b8, subgroup = shuffle|arithmetic, subgroup_size_choices = [32], max_workgroup_sizes = [1024, 1024, 1024], max_thread_count_per_workgroup = 1024, max_workgroup_memory_bytes = 49152, max_workgroup_counts = [2147483647, 65535, 65535]>>}>]> : !hal.device
  hal.executable private @simple_mul_dispatch_0 {
    hal.executable.variant public @cuda_nvptx_fb target(<"cuda", "cuda-nvptx-fb", {iree_codegen.target_info = #iree_gpu.target<arch = "sm_60", features = "+ptx76", wgp = <compute = fp64|fp32|fp16|int64|int32|int16|int8, storage = b64|b32|b16|b8, subgroup = shuffle|arithmetic, subgroup_size_choices = [32], max_workgroup_sizes = [1024, 1024, 1024], max_thread_count_per_workgroup = 1024, max_workgroup_memory_bytes = 49152, max_workgroup_counts = [2147483647, 65535, 65535]>>}>) {
      hal.executable.export public @simple_mul_dispatch_0_elementwise_4_f32 ordinal(0) layout(#hal.pipeline.layout<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) count(%arg0: !hal.device) -> (index, index, index) {
        %x, %y, %z = iree_tensor_ext.dispatch.workgroup_count_from_slice()
        hal.return %x, %y, %z : index, index, index
      }
      builtin.module {
        func.func @simple_mul_dispatch_0_elementwise_4_f32() attributes {translation_info = #iree_codegen.translation_info<pipeline = #iree_gpu.pipeline<TileAndFuse> workgroup_size = [32, 1, 1] subgroup_size = 32>} {
          %c0 = arith.constant 0 : index
          %0 = hal.interface.binding.subspan layout(<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) binding(0) alignment(64) offset(%c0) flags("ReadOnly|Indirect") : memref<4xf32, #hal.descriptor_type<storage_buffer>>
          %1 = hal.interface.binding.subspan layout(<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) binding(1) alignment(64) offset(%c0) flags("ReadOnly|Indirect") : memref<4xf32, #hal.descriptor_type<storage_buffer>>
          %2 = hal.interface.binding.subspan layout(<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) binding(2) alignment(64) offset(%c0) flags(Indirect) : memref<4xf32, #hal.descriptor_type<storage_buffer>>
          %3 = iree_codegen.load_from_buffer %0 : memref<4xf32, #hal.descriptor_type<storage_buffer>> -> tensor<4xf32>
          %4 = iree_codegen.load_from_buffer %1 : memref<4xf32, #hal.descriptor_type<storage_buffer>> -> tensor<4xf32>
          %5 = tensor.empty() : tensor<4xf32>
          %6 = linalg.generic {indexing_maps = [affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>, affine_map<(d0) -> (d0)>], iterator_types = ["parallel"]} ins(%3, %4 : tensor<4xf32>, tensor<4xf32>) outs(%5 : tensor<4xf32>) attrs =  {lowering_config = #iree_gpu.lowering_config<{thread = [1], workgroup = [32]}>} {
          ^bb0(%in: f32, %in_0: f32, %out: f32):
            %7 = arith.mulf %in, %in_0 : f32
            linalg.yield %7 : f32
          } -> tensor<4xf32>
          iree_codegen.store_to_buffer %6, %2 : tensor<4xf32> into memref<4xf32, #hal.descriptor_type<storage_buffer>>
          return
        }
        iree_codegen.dispatch_config @simple_mul_dispatch_0_elementwise_4_f32 {
        ^bb0(%arg0: !hal.device):
          %x, %y, %z = iree_tensor_ext.dispatch.workgroup_count_from_slice()
          iree_codegen.yield %x, %y, %z : index, index, index
        }
      }
    }
  }
  util.func public @simple_mul(%arg0: !hal.buffer_view, %arg1: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}} {
    %c0 = arith.constant 0 : index
    %c16 = arith.constant 16 : index
    %c4 = arith.constant 4 : index
    %element_type_f32 = hal.element_type<f32> : i32
    %dense_row_major = hal.encoding_type<dense_row_major> : i32
    hal.buffer_view.assert<%arg0 : !hal.buffer_view> message("input0") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %0 = stream.tensor.import on(#hal.device.affinity<@__device_0>) %arg0 : !hal.buffer_view -> tensor<4xf32> in !stream.resource<external>{%c16}
    hal.buffer_view.assert<%arg1 : !hal.buffer_view> message("input1") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %1 = stream.tensor.import on(#hal.device.affinity<@__device_0>) %arg1 : !hal.buffer_view -> tensor<4xf32> in !stream.resource<external>{%c16}
    %result, %result_timepoint = stream.resource.alloca uninitialized on(#hal.device.affinity<@__device_0>) : !stream.resource<external>{%c16} => !stream.timepoint
    %2 = stream.cmd.execute on(#hal.device.affinity<@__device_0>) await(%result_timepoint) => with(%0 as %arg2: !stream.resource<external>{%c16}, %1 as %arg3: !stream.resource<external>{%c16}, %result as %arg4: !stream.resource<external>{%c16}) {
      stream.cmd.dispatch @simple_mul_dispatch_0::@cuda_nvptx_fb::@simple_mul_dispatch_0_elementwise_4_f32 {
        ro %arg2[%c0 for %c16] : !stream.resource<external>{%c16},
        ro %arg3[%c0 for %c16] : !stream.resource<external>{%c16},
        wo %arg4[%c0 for %c16] : !stream.resource<external>{%c16}
      }
    } => !stream.timepoint
    %3 = stream.timepoint.await %2 => %result : !stream.resource<external>{%c16}
    %4 = stream.tensor.export on(#hal.device.affinity<@__device_0>) %3 : tensor<4xf32> in !stream.resource<external>{%c16} -> !hal.buffer_view
    util.return %4 : !hal.buffer_view
  }
}

7. Executable Targets:具体化 GPU 实现

张量操作、Linalg 和配置阶段函数体被 LLVM、NVVM 操作替换。入口接收三个 !llvm.ptr<1> 参数,对 NVVM 目标而言地址空间 1 是全局内存。导出的 workgroup 数量为 (1, 1, 1),每组线程数为 (32, 1, 1)。

kernel 读取线程 ID:

%tid = nvvm.read.ptx.sreg.tid.x range <i32, 0, 32> : i32

控制流等价于:

for (int i = threadIdx.x; i < 4; i += 32) {
  C[i] = A[i] * B[i];
}

线程 0–3 各处理一个元素,线程 4–31 在初始边界检查后退出,不会发生越界访问。

llvm.getelementptr 计算地址,llvm.load、llvm.fmul、llvm.store 实现运算。中间 vector<1xf32> 只是单元素向量表示,并非每线程计算四个元素。

readonly、noalias 和对齐信息等指针属性帮助后续优化。乘法彼此独立,因此不需要共享内存协作或线程屏障。

这里仍是面向目标的 MLIR,还不是 PTX 文本。与此同时,外层函数仍包含 Stream 操作,清楚展示了 kernel 降级和编排降级彼此独立。

module attributes {stream.affinity.default = #hal.device.affinity<@__device_0>} {
  util.global private @__device_0 = #hal.device.target<"cuda", [#hal.executable.target<"cuda", "cuda-nvptx-fb", {iree_codegen.target_info = #iree_gpu.target<arch = "sm_60", features = "+ptx76", wgp = <compute = fp64|fp32|fp16|int64|int32|int16|int8, storage = b64|b32|b16|b8, subgroup = shuffle|arithmetic, subgroup_size_choices = [32], max_workgroup_sizes = [1024, 1024, 1024], max_thread_count_per_workgroup = 1024, max_workgroup_memory_bytes = 49152, max_workgroup_counts = [2147483647, 65535, 65535]>>}>]> : !hal.device
  hal.executable private @simple_mul_dispatch_0 {
    hal.executable.variant public @cuda_nvptx_fb target(<"cuda", "cuda-nvptx-fb", {iree_codegen.target_info = #iree_gpu.target<arch = "sm_60", features = "+ptx76", wgp = <compute = fp64|fp32|fp16|int64|int32|int16|int8, storage = b64|b32|b16|b8, subgroup = shuffle|arithmetic, subgroup_size_choices = [32], max_workgroup_sizes = [1024, 1024, 1024], max_thread_count_per_workgroup = 1024, max_workgroup_memory_bytes = 49152, max_workgroup_counts = [2147483647, 65535, 65535]>>}>) {
      hal.executable.export public @simple_mul_dispatch_0_elementwise_4_f32 ordinal(0) layout(#hal.pipeline.layout<bindings = [#hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, "ReadOnly|Indirect">, #hal.pipeline.binding<storage_buffer, Indirect>], flags = Indirect>) count(%arg0: !hal.device) -> (index, index, index) {
        %c1 = arith.constant 1 : index
        hal.return %c1, %c1, %c1 : index, index, index
      } attributes {subgroup_size = 32 : index, workgroup_size = [32 : index, 1 : index, 1 : index]}
      builtin.module {
        llvm.func @simple_mul_dispatch_0_elementwise_4_f32(%arg0: !llvm.ptr<1> {llvm.align = 16 : i32, llvm.noalias, llvm.nonnull, llvm.noundef, llvm.readonly}, %arg1: !llvm.ptr<1> {llvm.align = 16 : i32, llvm.noalias, llvm.nonnull, llvm.noundef, llvm.readonly}, %arg2: !llvm.ptr<1> {llvm.align = 16 : i32, llvm.noalias, llvm.nonnull, llvm.noundef}) attributes {gpu.known_block_size = array<i32: 32, 1, 1>} {
          %0 = llvm.mlir.constant(64 : i64) : i64
          %1 = llvm.mlir.constant(true) : i1
          %2 = llvm.mlir.constant(32 : i32) : i32
          %3 = llvm.mlir.constant(4 : i32) : i32
          %4 = nvvm.read.ptx.sreg.tid.x range <i32, 0, 32> : i32
          %5 = llvm.sext %4 : i32 to i64
          llvm.intr.assume %1 ["align"(%arg0, %0 : !llvm.ptr<1>, i64)] : i1
          llvm.intr.assume %1 ["align"(%arg1, %0 : !llvm.ptr<1>, i64)] : i1
          llvm.intr.assume %1 ["align"(%arg2, %0 : !llvm.ptr<1>, i64)] : i1
          %6 = llvm.trunc %5 : i64 to i32
          llvm.br ^bb1(%6 : i32)
        ^bb1(%7: i32):  // 2 preds: ^bb0, ^bb2
          %8 = llvm.icmp "slt" %7, %3 : i32
          llvm.cond_br %8, ^bb2, ^bb3
        ^bb2:  // pred: ^bb1
          %9 = llvm.zext %7 : i32 to i64
          %10 = llvm.getelementptr %arg0[%9] : (!llvm.ptr<1>, i64) -> !llvm.ptr<1>, f32
          %11 = llvm.load %10 <alignment = 4> : !llvm.ptr<1> -> vector<1xf32>
          %12 = llvm.getelementptr %arg1[%9] : (!llvm.ptr<1>, i64) -> !llvm.ptr<1>, f32
          %13 = llvm.load %12 <alignment = 4> : !llvm.ptr<1> -> vector<1xf32>
          %14 = llvm.fmul %11, %13 : vector<1xf32>
          %15 = llvm.getelementptr %arg2[%9] : (!llvm.ptr<1>, i64) -> !llvm.ptr<1>, f32
          llvm.store %14, %15 <alignment = 4> : vector<1xf32>, !llvm.ptr<1>
          %16 = llvm.add %7, %2 : i32
          llvm.br ^bb1(%16 : i32)
        ^bb3:  // pred: ^bb1
          llvm.return
        }
      }
    }
  }
  util.func public @simple_mul(%arg0: !hal.buffer_view, %arg1: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}} {
    %c0 = arith.constant 0 : index
    %c16 = arith.constant 16 : index
    %c4 = arith.constant 4 : index
    %element_type_f32 = hal.element_type<f32> : i32
    %dense_row_major = hal.encoding_type<dense_row_major> : i32
    hal.buffer_view.assert<%arg0 : !hal.buffer_view> message("input0") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %0 = stream.tensor.import on(#hal.device.affinity<@__device_0>) %arg0 : !hal.buffer_view -> tensor<4xf32> in !stream.resource<external>{%c16}
    hal.buffer_view.assert<%arg1 : !hal.buffer_view> message("input1") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %1 = stream.tensor.import on(#hal.device.affinity<@__device_0>) %arg1 : !hal.buffer_view -> tensor<4xf32> in !stream.resource<external>{%c16}
    %result, %result_timepoint = stream.resource.alloca uninitialized on(#hal.device.affinity<@__device_0>) : !stream.resource<external>{%c16} => !stream.timepoint
    %2 = stream.cmd.execute on(#hal.device.affinity<@__device_0>) await(%result_timepoint) => with(%0 as %arg2: !stream.resource<external>{%c16}, %1 as %arg3: !stream.resource<external>{%c16}, %result as %arg4: !stream.resource<external>{%c16}) {
      stream.cmd.dispatch @simple_mul_dispatch_0::@cuda_nvptx_fb::@simple_mul_dispatch_0_elementwise_4_f32 {
        ro %arg2[%c0 for %c16] : !stream.resource<external>{%c16},
        ro %arg3[%c0 for %c16] : !stream.resource<external>{%c16},
        wo %arg4[%c0 for %c16] : !stream.resource<external>{%c16}
      }
    } => !stream.timepoint
    %3 = stream.timepoint.await %2 => %result : !stream.resource<external>{%c16}
    %4 = stream.tensor.export on(#hal.device.affinity<@__device_0>) %3 : tensor<4xf32> in !stream.resource<external>{%c16} -> !hal.buffer_view
    util.return %4 : !hal.buffer_view
  }
}

8. HAL:封装 Kernel 并构造运行时执行

HAL 完成下一次重要转换:序列化设备实现,将外围调度转换为显式 HAL 操作。

Kernel 成为可执行数据

variant 中的 LLVM/NVVM 模块被替换为:

hal.executable.binary public @cuda_nvptx_fb attributes {
  data = dense<"0x..."> : vector<1484xi8>,
  format = "cuda-nvptx-fb",
  mime_type = "application/x-flatbuffers"
}

这份 1,484 字节载荷包含 CUDA executable FlatBuffer,包括 PTX 和入口元数据。解码后可见:

.version 7.6
.target sm_60
.address_size 64

核心指令读取 %tid.x,跳过 ID 大于 3 的线程,加载 A、B,执行 mul.rn.f32,再存储 C。由于每个参与线程最多执行一次迭代,早先的循环简化为一次边界检查。

包内是 PTX,之后由 CUDA 加载路径和驱动继续处理为可执行设备代码。

初始化创建可复用状态

util.initializer 枚举运行时设备,检查 CUDA 和可执行格式支持,再创建 !hal.executable。另一个初始化器通过 @__simple_mul_memoize_apply 录制并缓存命令缓冲区。

录制使用绑定槽 0、1、2 和固定 workgroup 数量 (1, 1, 1),并记录执行屏障、结束命令缓冲区。该屏障约束命令执行顺序,不是 CUDA 线程间屏障。

初始化准备 executable 与命令对象,并不执行模型乘法。

每次调用提供资源并提交工作

公共函数检查输入视图和底层缓冲区,用 hal.device.queue.alloca 获取 16 字节输出,再通过 hal.device.queue.execute.indirect 提交缓存命令缓冲区。

间接提交为已录制槽位提供实际 A、B、C 缓冲区,将稳定命令结构与逐次调用数据分离。依赖可以概括为:

Allocate C ──signal allocation fence──► Execute cached command buffer
                                               │
                                       signal completion fence
                                               │
                                      Await completion, return C

hal.buffer_view.create 将结果包装为形状 [4]、类型 f32 的视图。返回视图本身不会复制到 CPU。函数期待兼容输入缓冲区,调用工具执行的上传在本函数 IR 之外。

HAL 完整结果:

module {
  util.global private @__device_0 : !hal.device
  util.initializer {
    %c18_i32 = arith.constant 18 : i32
    %false = arith.constant false
    %c0 = arith.constant 0 : index
    %c1 = arith.constant 1 : index
    %0 = util.null : !hal.device
    %device_count = hal.devices.count : index
    cf.br ^bb1(%c0, %c0, %0 : index, index, !hal.device)
  ^bb1(%1: index, %2: index, %3: !hal.device):  // 2 preds: ^bb0, ^bb4
    %4 = util.cmp.eq %3, %0 : !hal.device
    %5 = arith.cmpi slt, %1, %device_count : index
    %6 = arith.andi %4, %5 : i1
    cf.cond_br %6, ^bb2, ^bb5
  ^bb2:  // pred: ^bb1
    %device_n = hal.devices.get %1 : !hal.device
    %ok, %value = hal.device.query<%device_n : !hal.device> key("hal.device.id" :: "cuda") : i1, i1 = false
    cf.cond_br %value, ^bb3, ^bb4(%false : i1)
  ^bb3:  // pred: ^bb2
    %ok_0, %value_1 = hal.device.query<%device_n : !hal.device> key("hal.executable.format" :: "cuda-nvptx-fb") : i1, i1 = false
    cf.br ^bb4(%value_1 : i1)
  ^bb4(%7: i1):  // 2 preds: ^bb2, ^bb3
    %8 = arith.cmpi eq, %2, %c0 : index
    %9 = arith.select %7, %c1, %c0 : index
    %10 = arith.addi %2, %9 : index
    %11 = arith.andi %7, %8 : i1
    %12 = arith.select %11, %device_n, %0 : !hal.device
    %13 = arith.addi %1, %c1 : index
    cf.br ^bb1(%13, %10, %12 : index, index, !hal.device)
  ^bb5:  // pred: ^bb1
    cf.cond_br %4, ^bb6, ^bb7
  ^bb6:  // pred: ^bb5
    util.status.check_ok %c18_i32, "HAL device `__device_0` not found or unavailable: #hal.device.target<\22cuda\22, [#hal.executable.target<\22cuda\22, \22cuda-nvptx-fb\22, {iree_codegen.target_info = #iree_gpu.target<arch = \22sm_60\22, features = \22+ptx76\22, wgp = <compute = fp64|fp32|fp16|int64|int32|int16|int8, storage = b64|b32|b16|b8, subgroup = shuffle|arithmetic, subgroup_size_choices = [32], max_workgroup_sizes = [1024, 1024, 1024], max_thread_count_per_workgroup = 1024, max_workgroup_memory_bytes = 49152, max_workgroup_counts = [2147483647, 65535, 65535]>>}>]>"
    cf.br ^bb7
  ^bb7:  // 2 preds: ^bb5, ^bb6
    util.global.store %3, @__device_0 : !hal.device
    util.return
  }
  util.global private @__device_0_query_0_hal_executable_format_cuda_nvptx_fb : i1
  util.initializer {
    %__device_0 = util.global.load @__device_0 : !hal.device
    %ok, %value = hal.device.query<%__device_0 : !hal.device> key("hal.executable.format" :: "cuda-nvptx-fb") : i1, i1 = false
    util.global.store %value, @__device_0_query_0_hal_executable_format_cuda_nvptx_fb : i1
    util.return
  }
  util.global private @__device_0_executable_0_simple_mul_dispatch_0 : !hal.executable
  util.initializer {
    %c-1_i64 = arith.constant -1 : i64
    %c-1 = arith.constant -1 : index
    %c0 = arith.constant 0 : index
    %c14_i32 = arith.constant 14 : i32
    %0 = util.null : !hal.executable
    %__device_0_query_0_hal_executable_format_cuda_nvptx_fb = util.global.load @__device_0_query_0_hal_executable_format_cuda_nvptx_fb : i1
    %__device_0 = util.global.load @__device_0 : !hal.device
    %1 = arith.select %__device_0_query_0_hal_executable_format_cuda_nvptx_fb, %c0, %c-1 : index
    %2 = arith.cmpi eq, %1, %c0 : index
    cf.cond_br %2, ^bb1, ^bb2
  ^bb1:  // pred: ^bb0
    %executable = hal.executable.create device(%__device_0 : !hal.device) affinity(%c-1_i64) target(@simple_mul_dispatch_0::@cuda_nvptx_fb) : !hal.executable
    cf.br ^bb3(%executable : !hal.executable)
  ^bb2:  // pred: ^bb0
    util.status.check_ok %c14_i32, "HAL device `__device_0` does not support any variant of executable `simple_mul_dispatch_0`; available formats: [cuda-nvptx-fb]"
    cf.br ^bb3(%0 : !hal.executable)
  ^bb3(%3: !hal.executable):  // 2 preds: ^bb1, ^bb2
    util.global.store %3, @__device_0_executable_0_simple_mul_dispatch_0 : !hal.executable
    util.return
  }
  hal.executable private @simple_mul_dispatch_0 {
    hal.executable.binary public @cuda_nvptx_fb attributes {data = dense<"..."> : vector<1484xi8>, format = "cuda-nvptx-fb", mime_type = "application/x-flatbuffers"}
  }
  util.func private @__simple_mul_memoize_apply() -> !hal.command_buffer attributes {inlining_policy = #util.inline.never} {
    %c-1_i64 = arith.constant -1 : i64
    %c3 = arith.constant 3 : index
    %c0 = arith.constant 0 : index
    %c1 = arith.constant 1 : index
    %c2 = arith.constant 2 : index
    %c16 = arith.constant 16 : index
    %__device_0 = util.global.load immutable @__device_0 : !hal.device
    %__device_0_executable_0_simple_mul_dispatch_0 = util.global.load immutable @__device_0_executable_0_simple_mul_dispatch_0 : !hal.executable
    %cmd = hal.command_buffer.create device(%__device_0 : !hal.device) mode("None") categories("Transfer|Dispatch") affinity(%c-1_i64) bindings(%c3) : !hal.command_buffer
    %function_id = hal.executable.lookup.function target(%__device_0_executable_0_simple_mul_dispatch_0 : !hal.executable) function(@simple_mul_dispatch_0::@cuda_nvptx_fb::@simple_mul_dispatch_0_elementwise_4_f32) : i64
    hal.command_buffer.dispatch<%cmd : !hal.command_buffer> target(%__device_0_executable_0_simple_mul_dispatch_0 : !hal.executable)[%function_id] workgroups([%c1, %c1, %c1]) bindings([
      (%c0 : index)[%c0, %c16], 
      (%c1 : index)[%c0, %c16], 
      (%c2 : index)[%c0, %c16]
    ]) flags("None")
    hal.command_buffer.execution_barrier<%cmd : !hal.command_buffer> source("Dispatch|Transfer|CommandRetire") target("CommandIssue|Dispatch|Transfer") flags("None")
    hal.command_buffer.finalize<%cmd : !hal.command_buffer>
    util.return %cmd : !hal.command_buffer
  }
  util.global private @__simple_mul_memoize_result_0_device_0 : !hal.command_buffer
  util.initializer {
    %0 = util.call @__simple_mul_memoize_apply() : () -> !hal.command_buffer
    util.global.store %0, @__simple_mul_memoize_result_0_device_0 : !hal.command_buffer
    util.return
  }
  util.func public @simple_mul(%arg0: !hal.buffer_view, %arg1: !hal.buffer_view) -> !hal.buffer_view attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}} {
    %c-1_i32 = arith.constant -1 : i32
    %c0_i64 = arith.constant 0 : i64
    %0 = util.null : !hal.fence
    %c-1_i64 = arith.constant -1 : i64
    %c0 = arith.constant 0 : index
    %c16 = arith.constant 16 : index
    %c4 = arith.constant 4 : index
    %memory_type = hal.memory_type<"DeviceVisible|DeviceLocal"> : i32
    %buffer_usage = hal.buffer_usage<"TransferSource|TransferTarget|Transfer|DispatchStorageRead|DispatchStorageWrite|DispatchStorage"> : i32
    %__device_0 = util.global.load immutable @__device_0 : !hal.device
    %__simple_mul_memoize_result_0_device_0 = util.global.load immutable @__simple_mul_memoize_result_0_device_0 : !hal.command_buffer
    %element_type_f32 = hal.element_type<f32> : i32
    %dense_row_major = hal.encoding_type<dense_row_major> : i32
    hal.buffer_view.assert<%arg0 : !hal.buffer_view> message("input0") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %buffer = hal.buffer_view.buffer<%arg0 : !hal.buffer_view> : !hal.buffer
    %allocator = hal.device.allocator<%__device_0 : !hal.device> : !hal.allocator
    hal.buffer.assert<%buffer : !hal.buffer> message("tensor") allocator(%allocator : !hal.allocator) minimum_length(%c16) type(DeviceVisible) usage("TransferSource|TransferTarget|Transfer|DispatchStorageRead|DispatchStorageWrite|DispatchStorage")
    hal.buffer_view.assert<%arg1 : !hal.buffer_view> message("input1") shape([%c4]) type(%element_type_f32) encoding(%dense_row_major)
    %buffer_0 = hal.buffer_view.buffer<%arg1 : !hal.buffer_view> : !hal.buffer
    hal.buffer.assert<%buffer_0 : !hal.buffer> message("tensor") allocator(%allocator : !hal.allocator) minimum_length(%c16) type(DeviceVisible) usage("TransferSource|TransferTarget|Transfer|DispatchStorageRead|DispatchStorageWrite|DispatchStorage")
    %fence = hal.fence.create device(%__device_0 : !hal.device) flags("None") : !hal.fence
    %transient_buffer = hal.device.queue.alloca<%__device_0 : !hal.device> affinity(%c-1_i64) wait(%0) signal(%fence) pool(%c0_i64) type(%memory_type) usage(%buffer_usage) flags("None") : !hal.buffer{%c16}
    %fence_1 = hal.fence.create device(%__device_0 : !hal.device) flags("None") : !hal.fence
    hal.device.queue.execute.indirect<%__device_0 : !hal.device> affinity(%c-1_i64) wait(%fence) signal(%fence_1) commands(%__simple_mul_memoize_result_0_device_0) bindings([
      (%buffer : !hal.buffer)[%c0, %c16], 
      (%buffer_0 : !hal.buffer)[%c0, %c16], 
      (%transient_buffer : !hal.buffer)[%c0, %c16]
    ]) flags("None")
    %status = hal.fence.await until([%fence_1]) timeout_millis(%c-1_i32) flags("None") : i32
    util.status.check_ok %status, "failed to wait on timepoint"
    %view = hal.buffer_view.create buffer(%transient_buffer : !hal.buffer)[%c0, %c16] shape([%c4]) type(%element_type_f32) encoding(%dense_row_major) : !hal.buffer_view
    util.return %view : !hal.buffer_view
  }
}

9. VM:将编排编码为可移植控制代码

最后 VM 流水线生成 vm.module,其中包含函数、导入、导出、全局变量和只读数据。

HAL 对象变为 VM 引用:

!hal.buffer_view → !vm.ref<!hal.buffer_view>
!hal.device      → !vm.ref<!hal.device>

HAL 操作变为运行时导入调用:

vm.import private @hal.buffer_view.buffer(
    %view: !vm.ref<!hal.buffer_view>) -> !vm.ref<!hal.buffer>

%buffer = vm.call @hal.buffer_view.buffer(%view)
    : (!vm.ref<!hal.buffer_view>) -> !vm.ref<!hal.buffer>

这是运行时边界:VM 执行控制指令并调用管理设备资源的 HAL 模块。导入的 minimum_version 约束 HAL 接口版本,不是 LLVM 版本。字符串和 executable 字节变为 vm.rodata,CUDA 载荷仍为 1,484 字节。初始化调用 @hal.executable.create 时,vm.const.ref.rodata 提供其引用。

等待可以挂起 VM 执行

fence 等待转换为:

vm.call.variadic.yieldable @hal.fence.await(...)
    -> ^resume(i32)

导入立即完成时,控制流携带结果进入后继;若让出执行,则等待后在该处恢复。后继检查状态,成功时创建输出视图,失败时清理并执行 vm.fail。

同步公共 ABI 要求返回结果前完成工作。内部 VM 可以支持挂起和恢复,HAL 同步模式也可能阻塞原生线程。同步的应用契约并不要求每一层采用相同的等待实现。

module attributes {vm.toplevel} {
  vm.module public @module {
    vm.global.ref private mutable @__device_0 : !vm.ref<!hal.device>
    vm.global.ref private mutable @__device_0_executable_0_simple_mul_dispatch_0 : !vm.ref<!hal.executable>
    vm.global.ref private mutable @__simple_mul_memoize_result_0_device_0 : !vm.ref<!hal.command_buffer>
    vm.rodata private @_utf8_hal_device_id_C6650FF277232B5A {alignment = 1 : i64} "hal.device.id"
    vm.rodata private @_utf8_cuda_342A062BE26F4639 {alignment = 1 : i64} "cuda"
    vm.rodata private @_utf8_hal_executable_format_E03EECB63A2AAF52 {alignment = 1 : i64} "hal.executable.format"
    vm.rodata private @_utf8_cuda_nvptx_fb_91E5BBD19AE47511 {alignment = 1 : i64} "cuda-nvptx-fb"
    vm.rodata private @simple_mul_dispatch_0_cuda_nvptx_fb {alignment = 16 : i64, mime_type = "application/x-flatbuffers"} dense<"0x43444131000000008C0500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000434441319AFAFFFF08000000100100000100000004000000A6FAFFFF640000002000000001000000010000000C0000007C00000020000000030000000300000000000000030000000000000002000000000000002700000073696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F663332002700000073696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F6633320042FBFFFF40000000040000004EFBFFFF08000000010000002600000073616D706C65732F7374617469635F6C6962726172792F73696D706C655F6D756C2E6D6C697200002700000073696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F663332000100000004000000C0FBFFFF04000000330400002F2F0A2F2F2047656E657261746564206279204C4C564D204E56505458204261636B2D456E640A2F2F0A0A2E76657273696F6E20372E360A2E74617267657420736D5F36300A2E616464726573735F73697A652036340A0A092F2F202E676C6F626C0973696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F6633320A0A2E76697369626C65202E656E7472792073696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F663332280A092E706172616D202E753634202E707472202E676C6F62616C202E616C69676E2031362073696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F6633325F706172616D5F302C0A092E706172616D202E753634202E707472202E676C6F62616C202E616C69676E2031362073696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F6633325F706172616D5F312C0A092E706172616D202E753634202E707472202E676C6F62616C202E616C69676E2031362073696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F6633325F706172616D5F320A290A2E6D61786E7469642033322C20312C20310A7B0A092E726567202E70726564200925703C323E3B0A092E726567202E623332200925723C353E3B0A092E726567202E62363420092572643C383E3B0A0A096D6F762E75333220092572312C20257469642E783B0A09736574702E67742E75333220092570312C202572312C20333B0A0940257031206272612009244C5F5F4242305F323B0A096C642E706172616D2E6236342009257264342C205B73696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F6633325F706172616D5F305D3B0A096C642E706172616D2E6236342009257264352C205B73696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F6633325F706172616D5F315D3B0A096C642E706172616D2E6236342009257264362C205B73696D706C655F6D756C5F64697370617463685F305F656C656D656E74776973655F345F6633325F706172616D5F325D3B0A096D756C2E776964652E7533322009257264372C202572312C20343B0A096164642E7336342009257264312C20257264342C20257264373B0A096164642E7336342009257264322C20257264352C20257264373B0A096164642E7336342009257264332C20257264362C20257264373B0A096C642E676C6F62616C2E6E632E62333220092572322C205B257264315D3B0A096C642E676C6F62616C2E6E632E62333220092572332C205B257264325D3B0A096D756C2E726E2E66333220092572342C202572322C202572333B0A0973742E676C6F62616C2E62333220095B257264335D2C202572343B0A244C5F5F4242305F323A0A097265743B0A0A7D0A0006000800040008000C00040008001400200000000400080000000000140018001C000000"> : vector<1484xi8>
    vm.rodata private @_utf8_simple_mul_dispatch_0_elem_BAECB3686C800B2E {alignment = 1 : i64} "simple_mul_dispatch_0_elementwise_4_f32"
    vm.func private @__simple_mul_memoize_apply() -> !vm.ref<!hal.command_buffer> attributes {vm.unwind, inlining_policy = #util.inline.never} {
      %c13 = vm.const.i32 13
      %c28 = vm.const.i32 28
      %c2 = vm.const.i32 2
      %null = vm.const.ref.zero : !vm.ref<!hal.buffer>
      %c1 = vm.const.i32 1
      %c3 = vm.const.i32 3
      %zero = vm.const.i32.zero
      %c16 = vm.const.i64 16
      %zero_0 = vm.const.i64.zero
      %c-1 = vm.const.i64 -1
      %__device_0 = vm.global.load.ref @__device_0 : !vm.ref<!hal.device>
      %__device_0_executable_0_simple_mul_dispatch_0 = vm.global.load.ref @__device_0_executable_0_simple_mul_dispatch_0 : !vm.ref<!hal.executable>
      %ref = vm.call @hal.command_buffer.create(%__device_0, %zero, %c3, %c-1, %c3) : (!vm.ref<!hal.device>, i32, i32, i64, i32) -> !vm.ref<!hal.command_buffer>
      %_utf8_simple_mul_dispatch_0_elem_BAECB3686C800B2E = vm.const.ref.rodata @_utf8_simple_mul_dispatch_0_elem_BAECB3686C800B2E : !vm.buffer
      %0 = vm.call @hal.executable.lookup.function(%__device_0_executable_0_simple_mul_dispatch_0, %_utf8_simple_mul_dispatch_0_elem_BAECB3686C800B2E) {nosideeffects} : (!vm.ref<!hal.executable>, !vm.buffer) -> i64
      vm.call.variadic @hal.command_buffer.dispatch(%ref, %__device_0_executable_0_simple_mul_dispatch_0, %0, %c1, %c1, %c1, %zero_0, [], [(%zero, %zero, %null, %zero_0, %c16), (%zero, %c1, %null, %zero_0, %c16), (%zero, %c2, %null, %zero_0, %c16)]) : (!vm.ref<!hal.command_buffer>, !vm.ref<!hal.executable>, i64, i32, i32, i32, i64, i32 ..., tuple<i32, i32, !vm.ref<!hal.buffer>, i64, i64> ...)
      vm.call @hal.command_buffer.execution_barrier(%ref, %c28, %c13, %zero_0) : (!vm.ref<!hal.command_buffer>, i32, i32, i64) -> ()
      vm.call @hal.command_buffer.finalize(%ref) : (!vm.ref<!hal.command_buffer>) -> ()
      vm.return %ref : !vm.ref<!hal.command_buffer>
    }
    vm.import private @hal.buffer.assert(%buffer : !vm.ref<!hal.buffer>, %message : !vm.buffer, %allocator : !vm.ref<!hal.allocator>, %minimum_length : i64, %memory_types : i32, %buffer_usage : i32)
    vm.import private @hal.buffer_view.create(%buffer : !vm.ref<!hal.buffer>, %source_offset : i64, %source_length : i64, %element_type : i32, %encoding_type : i32, %shape : i64 ...) -> !vm.ref<!hal.buffer_view> attributes {nosideeffects}
    vm.import private @hal.buffer_view.assert(%buffer_view : !vm.ref<!hal.buffer_view>, %message : !vm.buffer, %element_type : i32, %encoding_type : i32, %shape : i64 ...)
    vm.import private @hal.buffer_view.buffer(%buffer_view : !vm.ref<!hal.buffer_view>) -> !vm.ref<!hal.buffer> attributes {nosideeffects}
    vm.import private @hal.command_buffer.create(%device : !vm.ref<!hal.device>, %modes : i32, %command_categories : i32, %queue_affinity : i64, %binding_capacity : i32) -> !vm.ref<!hal.command_buffer> attributes {minimum_version = 6 : i32}
    vm.import private @hal.command_buffer.finalize(%command_buffer : !vm.ref<!hal.command_buffer>)
    vm.import private @hal.command_buffer.execution_barrier(%command_buffer : !vm.ref<!hal.command_buffer>, %source_stage_mask : i32, %target_stage_mask : i32, %flags : i64)
    vm.import private @hal.command_buffer.dispatch(%command_buffer : !vm.ref<!hal.command_buffer>, %executable : !vm.ref<!hal.executable>, %function_id : i64, %workgroup_x : i32, %workgroup_y : i32, %workgroup_z : i32, %flags : i64, %constants : i32 ..., %bindings : tuple<i32, i32, !vm.ref<!hal.buffer>, i64, i64> ...) attributes {minimum_version = 7 : i32}
    vm.import private @hal.device.allocator(%device : !vm.ref<!hal.device>) -> !vm.ref<!hal.allocator> attributes {nosideeffects}
    vm.import private @hal.device.query.i64(%device : !vm.ref<!hal.device>, %category : !vm.buffer, %key : !vm.buffer) -> (i32, i64) attributes {nosideeffects}
    vm.import private @hal.device.queue.alloca(%device : !vm.ref<!hal.device>, %queue_affinity : i64, %wait_fence : !vm.ref<!hal.fence>, %signal_fence : !vm.ref<!hal.fence>, %pool : i64, %memory_types : i32, %buffer_usage : i32, %allocation_size : i64, %flags : i64) -> !vm.ref<!hal.buffer>
    vm.import private @hal.device.queue.execute.indirect(%device : !vm.ref<!hal.device>, %queue_affinity : i64, %wait_fence : !vm.ref<!hal.fence>, %signal_fence : !vm.ref<!hal.fence>, %command_buffer : !vm.ref<!hal.command_buffer>, %flags : i64, %binding_table : tuple<!vm.ref<!hal.buffer>, i64, i64> ...)
    vm.import private @hal.devices.count() -> i32 attributes {nosideeffects}
    vm.import private @hal.devices.get(%index : i32) -> !vm.ref<!hal.device> attributes {nosideeffects}
    vm.import private @hal.executable.create(%device : !vm.ref<!hal.device>, %queue_affinity : i64, %executable_format : !vm.buffer, %executable_data : !vm.buffer, %constants : !vm.buffer) -> !vm.ref<!hal.executable> attributes {nosideeffects}
    vm.import private @hal.executable.lookup.function(%executable : !vm.ref<!hal.executable>, %function_name : !vm.buffer) -> i64 attributes {nosideeffects, minimum_version = 7 : i32}
    vm.import private @hal.fence.create(%device : !vm.ref<!hal.device>, %flags : i64) -> !vm.ref<!hal.fence>
    vm.import private @hal.fence.await(%timeout_millis : i32, %flags : i64, %fences : !vm.ref<!hal.fence> ...) -> i32 attributes {vm.yield}
    vm.rodata private @_utf8_input0_DCE99660CEB3F6B {alignment = 1 : i64} "input0"
    vm.rodata private @_utf8_tensor_FC1814BC4A58F22A {alignment = 1 : i64} "tensor"
    vm.rodata private @_utf8_input1_B898B726583C85DA {alignment = 1 : i64} "input1"
    vm.func private @simple_mul(%arg0: !vm.ref<!hal.buffer_view>, %arg1: !vm.ref<!hal.buffer_view>) -> !vm.ref<!hal.buffer_view> attributes {iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}, vm.unwind, vm.yield} {
      %c16 = vm.const.i32 16
      %c1 = vm.const.i32 1
      %c553648160 = vm.const.i32 553648160
      %c3075 = vm.const.i32 3075
      %c48 = vm.const.i32 48
      %c4 = vm.const.i64 4
      %c16_0 = vm.const.i64 16
      %zero = vm.const.i64.zero
      %c-1 = vm.const.i64 -1
      %null = vm.const.ref.zero : !vm.ref<!hal.fence>
      %c-1_1 = vm.const.i32 -1
      %__device_0 = vm.global.load.ref @__device_0 : !vm.ref<!hal.device>
      %__simple_mul_memoize_result_0_device_0 = vm.global.load.ref @__simple_mul_memoize_result_0_device_0 : !vm.ref<!hal.command_buffer>
      %_utf8_input0_DCE99660CEB3F6B = vm.const.ref.rodata @_utf8_input0_DCE99660CEB3F6B : !vm.buffer
      vm.call.variadic @hal.buffer_view.assert(%arg0, %_utf8_input0_DCE99660CEB3F6B, %c553648160, %c1, [%c4]) : (!vm.ref<!hal.buffer_view>, !vm.buffer, i32, i32, i64 ...)
      %ref = vm.call @hal.buffer_view.buffer(%arg0) {nosideeffects} : (!vm.ref<!hal.buffer_view>) -> !vm.ref<!hal.buffer>
      %ref_2 = vm.call @hal.device.allocator(%__device_0) {nosideeffects} : (!vm.ref<!hal.device>) -> !vm.ref<!hal.allocator>
      %_utf8_tensor_FC1814BC4A58F22A = vm.const.ref.rodata @_utf8_tensor_FC1814BC4A58F22A : !vm.buffer
      vm.call @hal.buffer.assert(%ref, %_utf8_tensor_FC1814BC4A58F22A, %ref_2, %c16_0, %c16, %c3075) : (!vm.ref<!hal.buffer>, !vm.buffer, !vm.ref<!hal.allocator>, i64, i32, i32) -> ()
      %_utf8_input1_B898B726583C85DA = vm.const.ref.rodata @_utf8_input1_B898B726583C85DA : !vm.buffer
      vm.call.variadic @hal.buffer_view.assert(%arg1, %_utf8_input1_B898B726583C85DA, %c553648160, %c1, [%c4]) : (!vm.ref<!hal.buffer_view>, !vm.buffer, i32, i32, i64 ...)
      %ref_3 = vm.call @hal.buffer_view.buffer(%arg1) {nosideeffects} : (!vm.ref<!hal.buffer_view>) -> !vm.ref<!hal.buffer>
      vm.call @hal.buffer.assert(%ref_3, %_utf8_tensor_FC1814BC4A58F22A, %ref_2, %c16_0, %c16, %c3075) : (!vm.ref<!hal.buffer>, !vm.buffer, !vm.ref<!hal.allocator>, i64, i32, i32) -> ()
      %ref_4 = vm.call @hal.fence.create(%__device_0, %zero) : (!vm.ref<!hal.device>, i64) -> !vm.ref<!hal.fence>
      %ref_5 = vm.call @hal.device.queue.alloca(%__device_0, %c-1, %null, %ref_4, %zero, %c48, %c3075, %c16_0, %zero) : (!vm.ref<!hal.device>, i64, !vm.ref<!hal.fence>, !vm.ref<!hal.fence>, i64, i32, i32, i64, i64) -> !vm.ref<!hal.buffer>
      %ref_6 = vm.call @hal.fence.create(%__device_0, %zero) : (!vm.ref<!hal.device>, i64) -> !vm.ref<!hal.fence>
      vm.call.variadic @hal.device.queue.execute.indirect(%__device_0, %c-1, %ref_4, %ref_6, %__simple_mul_memoize_result_0_device_0, %zero, [(%ref, %zero, %c16_0), (%ref_3, %zero, %c16_0), (%ref_5, %zero, %c16_0)]) : (!vm.ref<!hal.device>, i64, !vm.ref<!hal.fence>, !vm.ref<!hal.fence>, !vm.ref<!hal.command_buffer>, i64, tuple<!vm.ref<!hal.buffer>, i64, i64> ...)
      vm.call.variadic.yieldable @hal.fence.await(%c-1_1, %zero, %ref_6) {segment_sizes = dense<[-1, -1, 1]> : vector<3xi16>, segment_types = [i32, i64, !vm.ref<!hal.fence>]} : (i32, i64, !vm.ref<!hal.fence>) -> ^bb1 (i32)
    ^bb1(%0: i32):  // pred: ^bb0
      vm.cond_br %0, ^bb3, ^bb2
    ^bb2:  // pred: ^bb1
      %ref_7 = vm.call.variadic @hal.buffer_view.create(%ref_5, %zero, %c16_0, %c553648160, %c1, [%c4]) {nosideeffects} : (!vm.ref<!hal.buffer>, i64, i64, i32, i32, i64 ...) -> !vm.ref<!hal.buffer_view>
      vm.return %ref_7 : !vm.ref<!hal.buffer_view>
    ^bb3:  // pred: ^bb1
      vm.discard.refs %ref_5 : !vm.ref<!hal.buffer>
      vm.fail %0, "failed to wait on timepoint"
    }
    vm.export @simple_mul attributes {iree.abi.stub, iree.reflection = {iree.abi.declaration = "sync func @simple_mul(%input0: tensor<4xf32>, %input1: tensor<4xf32>) -> (%output0: tensor<4xf32>)"}}
    vm.export @__init
    vm.func private @__init() attributes {vm.unwind} {
      %c1 = vm.const.i32 1
      %null = vm.const.ref.zero : !vm.buffer
      %c14 = vm.const.i32 14
      %c-1 = vm.const.i64 -1
      %c18 = vm.const.i32 18
      %zero = vm.const.i32.zero
      %zero_0 = vm.const.i64.zero
      %c1_1 = vm.const.i64 1
      %null_2 = vm.const.ref.zero : !vm.ref<!hal.device>
      %0 = vm.call @hal.devices.count() {nosideeffects} : () -> i32
      %1 = vm.ext.i32.i64.s %0 : i32 -> i64
      vm.br ^bb1(%zero_0, %zero_0, %null_2 : i64, i64, !vm.ref<!hal.device>)
    ^bb1(%2: i64, %3: i64, %4: !vm.ref<!hal.device>):  // 2 preds: ^bb0, ^bb4
      %rnz = vm.cmp.nz.ref %4 : !vm.ref<!hal.device>
      %5 = vm.xor.i32 %rnz, %c1 : i32
      %slt = vm.cmp.lt.i64.s %2, %1 : i64
      %6 = vm.and.i32 %5, %slt : i32
      vm.cond_br %6, ^bb2, ^bb5
    ^bb2:  // pred: ^bb1
      vm.discard.refs %4 : !vm.ref<!hal.device>
      %7 = vm.trunc.i64.i32 %2 : i64 -> i32
      %ref = vm.call @hal.devices.get(%7) {nosideeffects} : (i32) -> !vm.ref<!hal.device>
      %_utf8_hal_device_id_C6650FF277232B5A = vm.const.ref.rodata @_utf8_hal_device_id_C6650FF277232B5A : !vm.buffer
      %_utf8_cuda_342A062BE26F4639 = vm.const.ref.rodata @_utf8_cuda_342A062BE26F4639 : !vm.buffer
      %8:2 = vm.call @hal.device.query.i64(%ref, %_utf8_hal_device_id_C6650FF277232B5A, %_utf8_cuda_342A062BE26F4639) {nosideeffects} : (!vm.ref<!hal.device>, !vm.buffer, !vm.buffer) -> (i32, i64)
      %nz = vm.cmp.nz.i64 %8#1 : i64
      %9 = vm.select.i32 %8#0, %nz, %zero : i32
      vm.cond_br %9, ^bb3, ^bb4(%zero : i32)
    ^bb3:  // pred: ^bb2
      %_utf8_hal_executable_format_E03EECB63A2AAF52 = vm.const.ref.rodata @_utf8_hal_executable_format_E03EECB63A2AAF52 : !vm.buffer
      %_utf8_cuda_nvptx_fb_91E5BBD19AE47511 = vm.const.ref.rodata @_utf8_cuda_nvptx_fb_91E5BBD19AE47511 : !vm.buffer
      %10:2 = vm.call @hal.device.query.i64(%ref, %_utf8_hal_executable_format_E03EECB63A2AAF52, %_utf8_cuda_nvptx_fb_91E5BBD19AE47511) {nosideeffects} : (!vm.ref<!hal.device>, !vm.buffer, !vm.buffer) -> (i32, i64)
      %nz_3 = vm.cmp.nz.i64 %10#1 : i64
      %11 = vm.select.i32 %10#0, %nz_3, %zero : i32
      vm.br ^bb4(%11 : i32)
    ^bb4(%12: i32):  // 2 preds: ^bb2, ^bb3
      %eq = vm.cmp.eq.i64 %3, %zero_0 : i64
      %13 = vm.select.i64 %12, %c1_1, %zero_0 : i64
      %14 = vm.add.i64 %3, %13 : i64
      %15 = vm.and.i32 %12, %eq : i32
      %ref_4 = vm.select.ref %15, %ref, %null_2 : !vm.ref<!hal.device>
      %16 = vm.add.i64 %2, %c1_1 : i64
      vm.br ^bb1(%16, %14, %ref_4 : i64, i64, !vm.ref<!hal.device>)
    ^bb5:  // pred: ^bb1
      vm.discard.refs %null_2 : !vm.ref<!hal.device>
      vm.cond_br %5, ^bb6, ^bb7
    ^bb6:  // pred: ^bb5
      vm.discard.refs %null, %4 : !vm.buffer, !vm.ref<!hal.device>
      vm.fail %c18, "HAL device `__device_0` not found or unavailable: #hal.device.target<\22cuda\22, [#hal.executable.target<\22cuda\22, \22cuda-nvptx-fb\22, {iree_codegen.target_info = #iree_gpu.target<arch = \22sm_60\22, features = \22+ptx76\22, wgp = <compute = fp64|fp32|fp16|int64|int32|int16|int8, storage = b64|b32|b16|b8, subgroup = shuffle|arithmetic, subgroup_size_choices = [32], max_workgroup_sizes = [1024, 1024, 1024], max_thread_count_per_workgroup = 1024, max_workgroup_memory_bytes = 49152, max_workgroup_counts = [2147483647, 65535, 65535]>>}>]>"
    ^bb7:  // pred: ^bb5
      %_utf8_hal_executable_format_E03EECB63A2AAF52_5 = vm.const.ref.rodata @_utf8_hal_executable_format_E03EECB63A2AAF52 : !vm.buffer
      %_utf8_cuda_nvptx_fb_91E5BBD19AE47511_6 = vm.const.ref.rodata @_utf8_cuda_nvptx_fb_91E5BBD19AE47511 : !vm.buffer
      %17:2 = vm.call @hal.device.query.i64(%4, %_utf8_hal_executable_format_E03EECB63A2AAF52_5, %_utf8_cuda_nvptx_fb_91E5BBD19AE47511_6) {nosideeffects} : (!vm.ref<!hal.device>, !vm.buffer, !vm.buffer) -> (i32, i64)
      %nz_7 = vm.cmp.nz.i64 %17#1 : i64
      %18 = vm.select.i32 %17#0, %nz_7, %zero : i32
      %19 = vm.select.i64 %18, %zero_0, %c-1 : i64
      %eq_8 = vm.cmp.eq.i64 %19, %zero_0 : i64
      vm.global.store.ref %4, @__device_0 : !vm.ref<!hal.device>
      vm.cond_br %eq_8, ^bb8, ^bb9
    ^bb8:  // pred: ^bb7
      %simple_mul_dispatch_0_cuda_nvptx_fb = vm.const.ref.rodata @simple_mul_dispatch_0_cuda_nvptx_fb : !vm.buffer
      %ref_9 = vm.call @hal.executable.create(%4, %c-1, %_utf8_cuda_nvptx_fb_91E5BBD19AE47511_6, %simple_mul_dispatch_0_cuda_nvptx_fb, %null) {nosideeffects} : (!vm.ref<!hal.device>, i64, !vm.buffer, !vm.buffer, !vm.buffer) -> !vm.ref<!hal.executable>
      vm.global.store.ref %ref_9, @__device_0_executable_0_simple_mul_dispatch_0 : !vm.ref<!hal.executable>
      %ref_10 = vm.call @__simple_mul_memoize_apply() : () -> !vm.ref<!hal.command_buffer>
      vm.global.store.ref %ref_10, @__simple_mul_memoize_result_0_device_0 : !vm.ref<!hal.command_buffer>
      vm.return
    ^bb9:  // pred: ^bb7
      vm.discard.refs %null, %4, %_utf8_cuda_nvptx_fb_91E5BBD19AE47511_6 : !vm.buffer, !vm.ref<!hal.device>, !vm.buffer
      vm.fail %c14, "HAL device `__device_0` does not support any variant of executable `simple_mul_dispatch_0`; available formats: [cuda-nvptx-fb]"
    }
  }
}

编辑文章
分享文章:

上一篇
[TIL] Triton 线性布局
下一篇
如何正确地写 Layout:深入理解 TileLang 布局推导