本文跟踪一个最小向量加法 kernel 在 Triton 编译器中的转换过程:
Python DSL → TTIR → TTGIR → LLVM IR → PTX
核心认识是:Triton 张量是逻辑张量,并不直接等于硬件向量。TTGIR 将元素分配给 GPU 线程,LLVM IR 则描述每个线程执行的标量工作。
Kernel
@triton.jit
def add_kernel(x, y, out, n: tl.constexpr, BLOCK: tl.constexpr):
offsets = tl.program_id(0) * BLOCK + tl.arange(0, BLOCK)
mask = offsets < n
values = tl.load(x + offsets, mask=mask) + tl.load(y + offsets, mask=mask)
tl.store(out + offsets, values, mask=mask)
使用 n = BLOCK = 128、4 个 warp 和 SM80 目标编译。两个编译期参数都从函数签名中消失,成为常量。
TTIR:逻辑张量计算
关键 TTIR 如下:
%n = arith.constant dense<128> : tensor<128xi32>
%block = arith.constant 128 : i32
%pid = tt.get_program_id x : i32
%base = arith.muli %pid, %block : i32
%range = tt.make_range {start = 0, end = 128} : tensor<128xi32>
%base_vector = tt.splat %base : i32 -> tensor<128xi32>
%offsets = arith.addi %base_vector, %range : tensor<128xi32>
%mask = arith.cmpi slt, %offsets, %n : tensor<128xi32>
%x_vector = tt.splat %x : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>>
%x_ptrs = tt.addptr %x_vector, %offsets
%x_values = tt.load %x_ptrs, %mask
%y_vector = tt.splat %y : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>>
%y_ptrs = tt.addptr %y_vector, %offsets
%y_values = tt.load %y_ptrs, %mask
%values = arith.addf %x_values, %y_values : tensor<128xf32>
%out_vector = tt.splat %out : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>>
%out_ptrs = tt.addptr %out_vector, %offsets
tt.store %out_ptrs, %values, %mask
有几个细节容易误读:
dense<128> : tensor<128xi32>表示 128 个i32元素,每个值都是 128。属性给出数值,结果类型给出形状和元素类型。tt.splat显式地将标量广播为逻辑张量。tt.addptr按元素大小进行指针运算。arith.cmpi打印的类型描述其操作数,结果元素则是i1。类似地,tt.load打印的是指针操作数的类型,结果类型从指向的元素类型推导。
此时,tensor<128xf32> 只说明有 128 个逻辑值,没有说明由哪些线程计算。
完整 IR:
#loc = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":8:1)
#loc11 = loc("x"(#loc))
#loc12 = loc("y"(#loc))
#loc13 = loc("out"(#loc))
module {
tt.func public @add_kernel(%x: !tt.ptr<f32> loc("x"(#loc)), %y: !tt.ptr<f32> loc("y"(#loc)), %out: !tt.ptr<f32> loc("out"(#loc))) attributes {noinline = false} {
%mask = arith.constant dense<128> : tensor<128xi32> loc(#loc14)
%c128_i32 = arith.constant 128 : i32 loc(#loc2)
%offsets = tt.get_program_id x : i32 loc(#loc15)
%offsets_0 = arith.muli %offsets, %c128_i32 : i32 loc(#loc15)
%offsets_1 = tt.make_range {end = 128 : i32, start = 0 : i32} : tensor<128xi32> loc(#loc16)
%offsets_2 = tt.splat %offsets_0 : i32 -> tensor<128xi32> loc(#loc15)
%offsets_3 = arith.addi %offsets_2, %offsets_1 : tensor<128xi32> loc(#loc15)
%mask_4 = arith.cmpi slt, %offsets_3, %mask : tensor<128xi32> loc(#loc14)
%values = tt.splat %x : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>> loc(#loc17)
%values_5 = tt.addptr %values, %offsets_3 : tensor<128x!tt.ptr<f32>>, tensor<128xi32> loc(#loc17)
%values_6 = tt.load %values_5, %mask_4 : tensor<128x!tt.ptr<f32>> loc(#loc18)
%values_7 = tt.splat %y : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>> loc(#loc19)
%values_8 = tt.addptr %values_7, %offsets_3 : tensor<128x!tt.ptr<f32>>, tensor<128xi32> loc(#loc19)
%values_9 = tt.load %values_8, %mask_4 : tensor<128x!tt.ptr<f32>> loc(#loc20)
%values_10 = arith.addf %values_6, %values_9 : tensor<128xf32> loc(#loc18)
%0 = tt.splat %out : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>> loc(#loc9)
%1 = tt.addptr %0, %offsets_3 : tensor<128x!tt.ptr<f32>>, tensor<128xi32> loc(#loc9)
tt.store %1, %values_10, %mask_4 : tensor<128x!tt.ptr<f32>> loc(#loc10)
tt.return loc(#loc)
} loc(#loc)
} loc(#loc)
#loc1 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":10:12)
#loc2 = loc(unknown)
#loc3 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":9:15)
#loc4 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":9:42)
#loc5 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":11:22)
#loc6 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":11:14)
#loc7 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":11:56)
#loc8 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":11:48)
#loc9 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":12:14)
#loc10 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":12:5)
#loc14 = loc("mask"(#loc1))
#loc15 = loc("offsets"(#loc3))
#loc16 = loc("offsets"(#loc4))
#loc17 = loc("values"(#loc5))
#loc18 = loc("values"(#loc6))
#loc19 = loc("values"(#loc7))
#loc20 = loc("values"(#loc8))
TTGIR:将值分配给线程
对于这个 kernel,TTGIR 与 TTIR 几乎相同。最重要的新增内容是布局编码:
#blocked = #ttg.blocked<{
sizePerThread = [1],
threadsPerWarp = [32],
warpsPerCTA = [4],
order = [0]
}>
没有编码的 TTIR 类型:
tensor<128xf32>
变为:
tensor<128xf32, #blocked>
tensor 是 MLIR 内置的有秩张量类型,它特意预留了可选的 encoding 字段。#blocked 是存储在该字段中的 TritonGPU 方言属性,属于编译期类型信息,并非 SSA 操作数或运行时值。
这个布局恰好覆盖整个张量:
1 element/thread × 32 threads/warp × 4 warps/CTA = 128 elements
因此,元素归属关系实际上是:
tensor element = warp_id × 32 + lane_id = threadIdx.x
TTGIR 还记录了执行配置:
module attributes {
"ttg.num-ctas" = 1 : i32,
"ttg.num-warps" = 4 : i32,
"ttg.threads-per-warp" = 32 : i32,
ttg.target = "cuda:80"
}
所有值都使用相同布局,线程也不需要其他线程持有的值,所以这里不需要 ttg.convert_layout 或共享内存操作。更复杂的 kernel,尤其是归约和矩阵乘法,其 TTGIR 与 TTIR 会有很大差异。
完整 IR:
#blocked = #ttg.blocked<{sizePerThread = [1], threadsPerWarp = [32], warpsPerCTA = [4], order = [0]}>
#loc = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":8:1)
#loc11 = loc("x"(#loc))
#loc12 = loc("y"(#loc))
#loc13 = loc("out"(#loc))
module attributes {"ttg.num-ctas" = 1 : i32, "ttg.num-warps" = 4 : i32, ttg.target = "cuda:80", "ttg.threads-per-warp" = 32 : i32, "ttng.two-ctas" = false} {
tt.func public @add_kernel(%x: !tt.ptr<f32> loc("x"(#loc)), %y: !tt.ptr<f32> loc("y"(#loc)), %out: !tt.ptr<f32> loc("out"(#loc))) attributes {noinline = false} {
%c128_i32 = arith.constant 128 : i32 loc(#loc1)
%cst = arith.constant dense<128> : tensor<128xi32, #blocked> loc(#loc1)
%offsets = tt.get_program_id x : i32 loc(#loc14)
%offsets_0 = arith.muli %offsets, %c128_i32 : i32 loc(#loc14)
%offsets_1 = tt.make_range {end = 128 : i32, start = 0 : i32} : tensor<128xi32, #blocked> loc(#loc15)
%offsets_2 = tt.splat %offsets_0 : i32 -> tensor<128xi32, #blocked> loc(#loc14)
%offsets_3 = arith.addi %offsets_2, %offsets_1 : tensor<128xi32, #blocked> loc(#loc14)
%mask = arith.cmpi slt, %offsets_3, %cst : tensor<128xi32, #blocked> loc(#loc16)
%values = tt.splat %x : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>, #blocked> loc(#loc17)
%values_4 = tt.addptr %values, %offsets_3 : tensor<128x!tt.ptr<f32>, #blocked>, tensor<128xi32, #blocked> loc(#loc17)
%values_5 = tt.load %values_4, %mask : tensor<128x!tt.ptr<f32>, #blocked> loc(#loc18)
%values_6 = tt.splat %y : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>, #blocked> loc(#loc19)
%values_7 = tt.addptr %values_6, %offsets_3 : tensor<128x!tt.ptr<f32>, #blocked>, tensor<128xi32, #blocked> loc(#loc19)
%values_8 = tt.load %values_7, %mask : tensor<128x!tt.ptr<f32>, #blocked> loc(#loc20)
%values_9 = arith.addf %values_5, %values_8 : tensor<128xf32, #blocked> loc(#loc18)
%0 = tt.splat %out : !tt.ptr<f32> -> tensor<128x!tt.ptr<f32>, #blocked> loc(#loc9)
%1 = tt.addptr %0, %offsets_3 : tensor<128x!tt.ptr<f32>, #blocked>, tensor<128xi32, #blocked> loc(#loc9)
tt.store %1, %values_9, %mask : tensor<128x!tt.ptr<f32>, #blocked> loc(#loc10)
tt.return loc(#loc)
} loc(#loc)
} loc(#loc)
#loc1 = loc(unknown)
#loc2 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":9:15)
#loc3 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":9:42)
#loc4 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":10:12)
#loc5 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":11:22)
#loc6 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":11:14)
#loc7 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":11:56)
#loc8 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":11:48)
#loc9 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":12:14)
#loc10 = loc("/home/qiping-pan/Documents/workspace/triton/triton_ir_walkthrough.py":12:5)
#loc14 = loc("offsets"(#loc2))
#loc15 = loc("offsets"(#loc3))
#loc16 = loc("mask"(#loc4))
#loc17 = loc("values"(#loc5))
#loc18 = loc("values"(#loc6))
#loc19 = loc("values"(#loc7))
#loc20 = loc("values"(#loc8))
LLVM IR:每个线程的标量程序
TTGIR 降级后,张量类型和布局都消失了。这个 kernel 需要 128 个线程:
attributes #0 = { nounwind "nvvm.reqntid"="128" }
前三个参数 x、y、out 都位于全局地址空间 1。Triton 还追加全局 scratch 和性能分析 scratch 指针,本例并未使用。
逻辑偏移变成标量索引运算:
%pid = call i32 @llvm.nvvm.read.ptx.sreg.ctaid.x()
%base = shl i32 %pid, 7
%tid = call i32 @llvm.nvvm.read.ptx.sreg.tid.x()
%local = and i32 %tid, 127
%offset = or disjoint i32 %base, %local
%mask = icmp slt i32 %offset, 128
等价于:
int offset = blockIdx.x * 128 + threadIdx.x;
bool mask = offset < 128;
乘以 128 变成左移 7 位。最后的加法变成 or disjoint,因为移位后的 program ID 与 7 位局部索引不会有重叠的置位位。
每个指针张量变成每线程一个地址:
%index64 = sext i32 %offset to i64
%x_ptr = getelementptr [4 x i8], ptr addrspace(1) %x, i64 %index64
以 [4 x i8] 为单位索引,每个元素前进 4 字节,正好是 f32 的大小。
Triton 通过内联 PTX 生成带谓词的全局访存。加载大致为:
mov.u32 result, 0;
@predicate ld.global.b32 result, [address];
初始化为零定义了被 mask 掉时的值。加载后的原始 i32 位模式被 bitcast 为 float,加法变成每线程一条标量指令,结果再转换回原始位模式用于存储:
%x_float = bitcast i32 %x_bits to float
%y_float = bitcast i32 %y_bits to float
%sum = fadd float %x_float, %y_float
%sum_bits = bitcast float %sum to i32
带 mask 的存储为:
@predicate st.global.b32 [address], value;
因此,每线程的完整行为是:
int offset = blockIdx.x * 128 + threadIdx.x;
bool mask = offset < 128;
float xv = mask ? x[offset] : 0.0f;
float yv = mask ? y[offset] : 0.0f;
float result = xv + yv;
if (mask)
out[offset] = result;
完整 IR:
; ModuleID = 'LLVMDialectModule'
source_filename = "LLVMDialectModule"
target datalayout = "e-p3:32:32-p4:32:32-p5:32:32-p6:32:32-p7:32:32-p101:32:32-i64:64-i128:128-i256:256-v16:16-v32:32-n16:32:64"
; Function Attrs: nounwind
define ptx_kernel void @add_kernel(ptr addrspace(1) %0, ptr addrspace(1) %1, ptr addrspace(1) %2, ptr addrspace(1) nofree readnone captures(none) %3, ptr addrspace(1) nofree readnone captures(none) %4) local_unnamed_addr #0 !dbg !4 {
%6 = tail call i32 @llvm.nvvm.read.ptx.sreg.ctaid.x(), !dbg !11
%7 = shl i32 %6, 7, !dbg !11
%8 = tail call i32 @llvm.nvvm.read.ptx.sreg.tid.x(), !dbg !12
%9 = and i32 %8, 127, !dbg !12
%10 = or disjoint i32 %7, %9, !dbg !11
%11 = icmp slt i32 %10, 128, !dbg !13
%12 = sext i32 %10 to i64, !dbg !14
%13 = getelementptr [4 x i8], ptr addrspace(1) %0, i64 %12, !dbg !14
%14 = tail call i32 asm sideeffect "mov.u32 $0, 0x0;\0A\09@$2 ld.global.b32 { $0 }, [ $1 + 0 ];", "=r,l,b"(ptr addrspace(1) %13, i1 %11) #2, !dbg !15
%15 = bitcast i32 %14 to float, !dbg !15
%16 = getelementptr [4 x i8], ptr addrspace(1) %1, i64 %12, !dbg !16
%17 = tail call i32 asm sideeffect "mov.u32 $0, 0x0;\0A\09@$2 ld.global.b32 { $0 }, [ $1 + 0 ];", "=r,l,b"(ptr addrspace(1) %16, i1 %11) #2, !dbg !17
%18 = bitcast i32 %17 to float, !dbg !17
%19 = fadd float %15, %18, !dbg !15
%20 = getelementptr [4 x i8], ptr addrspace(1) %2, i64 %12, !dbg !18
%21 = bitcast float %19 to i32, !dbg !19
tail call void asm sideeffect "@$2 st.global.b32 [ $1 + 0 ], { $0 };", "r,l,b"(i32 %21, ptr addrspace(1) %20, i1 %11) #2, !dbg !19
ret void, !dbg !20
}
; Function Attrs: mustprogress nocallback nofree nosync nounwind speculatable willreturn memory(none)
declare noundef range(i32 0, 2147483647) i32 @llvm.nvvm.read.ptx.sreg.ctaid.x() #1
; Function Attrs: mustprogress nocallback nofree nosync nounwind speculatable willreturn memory(none)
declare noundef range(i32 0, 1024) i32 @llvm.nvvm.read.ptx.sreg.tid.x() #1
attributes #0 = { nounwind "nvvm.reqntid"="128" }
attributes #1 = { mustprogress nocallback nofree nosync nounwind speculatable willreturn memory(none) }
attributes #2 = { nounwind }
!llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!2, !3}
!0 = distinct !DICompileUnit(language: DW_LANG_C, file: !1, producer: "triton", isOptimized: true, runtimeVersion: 0, emissionKind: LineTablesOnly)
!1 = !DIFile(filename: "triton_ir_walkthrough.py", directory: "/home/qiping-pan/Documents/workspace/triton")
!2 = !{i32 2, !"Debug Info Version", i32 3}
!3 = !{i32 4, !"nvvm-reflect-ftz", i32 1}
!4 = distinct !DISubprogram(name: "add_kernel", linkageName: "add_kernel", scope: !1, file: !1, line: 8, type: !5, scopeLine: 8, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0)
!5 = !DISubroutineType(cc: DW_CC_normal, types: !6)
!6 = !{null, !7, !7, !7, !9, !9}
!7 = !DIDerivedType(tag: DW_TAG_pointer_type, name: "pointer", baseType: !8, size: 64, dwarfAddressSpace: 1)
!8 = !DIBasicType(name: "float", size: 32, encoding: DW_ATE_float)
!9 = !DIDerivedType(tag: DW_TAG_pointer_type, name: "pointer", baseType: !10, size: 64, dwarfAddressSpace: 1)
!10 = !DIBasicType(name: "unknown_type", encoding: DW_ATE_signed)
!11 = !DILocation(line: 9, column: 15, scope: !4)
!12 = !DILocation(line: 9, column: 42, scope: !4)
!13 = !DILocation(line: 10, column: 12, scope: !4)
!14 = !DILocation(line: 11, column: 22, scope: !4)
!15 = !DILocation(line: 11, column: 14, scope: !4)
!16 = !DILocation(line: 11, column: 56, scope: !4)
!17 = !DILocation(line: 11, column: 48, scope: !4)
!18 = !DILocation(line: 12, column: 14, scope: !4)
!19 = !DILocation(line: 12, column: 5, scope: !4)
!20 = !DILocation(line: 8, column: 1, scope: !4)
各层之间的映射
| Triton 源码 | TTIR/TTGIR | LLVM/PTX |
|---|---|---|
tl.program_id(0) | tt.get_program_id x | ctaid.x |
tl.arange(0, 128) | tt.make_range, #blocked | tid.x |
pid * 128 + range | arith.muli, arith.addi | shl, or disjoint |
offsets < n | arith.cmpi slt | icmp slt / PTX 谓词 |
x + offsets | tt.splat, tt.addptr | 标量 getelementptr |
tl.load | tt.load | 带谓词的 ld.global.b32 |
| 张量加法 | arith.addf | 每线程标量 fadd |
tl.store | tt.store | 带谓词的 st.global.b32 |
关键边界在于 TTIR 到 TTGIR:逻辑张量获得布局编码。一旦确定元素归属,降级过程就能将这个 kernel 的每个张量操作替换成分布在 128 个 GPU 线程上的标量工作。