这篇文章讨论如何将 MLIR 逐步降级为 LLVM IR。
渐进式降级
渐进式降级是 MLIR 的另一项核心能力。它允许不同抽象层次的操作共存,使某个 pass 只关注需要处理的操作,同时保留其他操作的抽象。
本文将实现 print 之外所有操作的降级。
方言转换
我们使用 MLIR 的方言转换框架实现降级,也就是从一种方言转换为另一种方言。主要需要实现三部分:
- 转换目标:哪些操作需要被转换,哪些操作可以作为转换结果。
- 改写模式:具体如何转换。
- 类型转换器:本例尚未涉及。教程说明,它可用于转换基本块参数的类型,但这里不需要。
转换目标
降级过程实现为一个 pass。我们已经熟悉 pass 的定义方式。注意,它作用于 ModuleOp,并将 affine、func 和 memref 声明为依赖方言。我还不清楚 registry.insert 的底层实现。
namespace {
struct ToyToAffineLoweringPass
: public PassWrapper<ToyToAffineLoweringPass, OperationPass<ModuleOp>> {
MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(ToyToAffineLoweringPass)
StringRef getArgument() const override { return "toy-to-affine"; }
void getDependentDialects(DialectRegistry ®istry) const override {
registry.insert<affine::AffineDialect, func::FuncDialect,
memref::MemRefDialect>();
}
void runOnOperation() final;
};
} // namespace
接着实现 runOnOperation。先创建 ConversionTarget,再用 addLegalDialect 声明转换后允许存在的方言,也就是转换目标。同时声明非法方言,确保 pass 执行后,除 toy.print 外的所有 Toy 操作都被正确降级。toy.print 需要动态判定合法性,因为不希望它的类型仍停留在原来的张量表示。
然后实现改写模式。我们向模式集合中添加七条模式,例如 AddOpLowering,再执行改写。
void ToyToAffineLoweringPass::runOnOperation() {
// The first thing to define is the conversion target. This will define the
// final target for this lowering.
ConversionTarget target(getContext());
// We define the specific operations, or dialects, that are legal targets for
// this lowering. In our case, we are lowering to a combination of the
// `Affine`, `Arith`, `Func`, and `MemRef` dialects.
target.addLegalDialect<affine::AffineDialect, BuiltinDialect,
arith::ArithDialect, func::FuncDialect,
memref::MemRefDialect>();
// We also define the Toy dialect as Illegal so that the conversion will fail
// if any of these operations are *not* converted. Given that we actually want
// a partial lowering, we explicitly mark the Toy operations that don't want
// to lower, `toy.print`, as `legal`. `toy.print` will still need its operands
// to be updated though (as we convert from TensorType to MemRefType), so we
// only treat it as `legal` if its operands are legal.
target.addIllegalDialect<toy::ToyDialect>();
target.addDynamicallyLegalOp<toy::PrintOp>([](toy::PrintOp op) {
return llvm::none_of(op->getOperandTypes(),
[](Type type) { return llvm::isa<TensorType>(type); });
});
// Now that the conversion target has been defined, we just need to provide
// the set of patterns that will lower the Toy operations.
RewritePatternSet patterns(&getContext());
patterns.add<AddOpLowering, ConstantOpLowering, FuncOpLowering, MulOpLowering,
PrintOpLowering, ReturnOpLowering, TransposeOpLowering>(
&getContext());
// With the target and rewrite patterns defined, we can now attempt the
// conversion. The conversion will signal failure if any of our `illegal`
// operations were not converted successfully.
if (failed(
applyPartialConversion(getOperation(), target, std::move(patterns))))
signalPassFailure();
}
/// Create a pass for lowering operations in the `Affine` and `Std` dialects,
/// for a subset of the Toy IR (e.g. matmul).
std::unique_ptr<Pass> mlir::toy::createLowerToAffinePass() {
return std::make_unique<ToyToAffineLoweringPass>();
}
下面看看每条改写模式的具体实现。
二元操作的降级
二元操作是逐元素执行的,所以要将操作展开为循环,以遍历所有元素。lowerOpToLoops 完成以下工作:
- 接收一个操作和一个
LoopIterationFn,后者定义每次循环迭代执行的工作。 - 从操作的结果类型获取张量的秩。
- 根据张量维度生成步长、下界等循环参数。
- 使用这些参数构建 affine 方言的嵌套循环。
- 用分配结果缓冲区的
alloc替换当前操作。因为alloc返回的就是输出张量对应的内存引用。
static void lowerOpToLoops(Operation *op, PatternRewriter &rewriter,
LoopIterationFn processIteration) {
auto tensorType = llvm::cast<RankedTensorType>((*op->result_type_begin()));
auto loc = op->getLoc();
// Insert an allocation and deallocation for the result of this operation.
auto memRefType = convertTensorToMemRef(tensorType);
auto alloc = insertAllocAndDealloc(memRefType, loc, rewriter);
// Create a nest of affine loops, with one loop per dimension of the shape.
// The buildAffineLoopNest function takes a callback that is used to construct
// the body of the innermost loop given a builder, a location and a range of
// loop induction variables.
SmallVector<int64_t, 4> lowerBounds(tensorType.getRank(), /*Value=*/0);
SmallVector<int64_t, 4> steps(tensorType.getRank(), /*Value=*/1);
/*
for a tensor of shape (2, 3), the rank is 2
lowerBounds = [0, 0]
steps = [1, 1]
tensorType.getShape() = [2, 3]
*/
affine::buildAffineLoopNest(
rewriter, loc, lowerBounds, tensorType.getShape(), steps,
[&](OpBuilder &nestedBuilder, Location loc, ValueRange ivs) {
// Call the processing function with the rewriter and the loop
// induction variables. This function will return the value to store at
// the current index.
Value valueToStore = processIteration(nestedBuilder, ivs);
affine::AffineStoreOp::create(nestedBuilder, loc, valueToStore, alloc,
ivs);
});
// Replace this operation with the generated alloc.
rewriter.replaceOp(op, alloc);
}
对于二元操作,只需获取左右输入,并创建相应的 LoweredBinaryOp。加法对应 arith::AddFOp。
template <typename BinaryOp, typename LoweredBinaryOp>
struct BinaryOpLowering : public OpConversionPattern<BinaryOp> {
using OpConversionPattern<BinaryOp>::OpConversionPattern;
using OpAdaptor = typename OpConversionPattern<BinaryOp>::OpAdaptor;
LogicalResult
matchAndRewrite(BinaryOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const final {
auto loc = op->getLoc();
lowerOpToLoops(op, rewriter, [&](OpBuilder &builder, ValueRange loopIvs) {
// Generate loads for the element of 'lhs' and 'rhs' at the
// inner loop.
auto loadedLhs =
affine::AffineLoadOp::create(builder, loc, adaptor.getLhs(), loopIvs);
auto loadedRhs =
affine::AffineLoadOp::create(builder, loc, adaptor.getRhs(), loopIvs);
// Create the binary operation performed on the loaded
// values.
return LoweredBinaryOp::create(builder, loc, loadedLhs, loadedRhs);
});
return success();
}
};
using AddOpLowering = BinaryOpLowering<toy::AddOp, arith::AddFOp>;
using MulOpLowering = BinaryOpLowering<toy::MulOp, arith::MulFOp>;
常量操作的降级
常量操作主要需要完成两件事:
- 用
insertAllocAndDealloc为张量分配缓冲区,类似 add 和 mul。 - 把初始值逐元素写入缓冲区。
写入元素的过程很有意思,类似深度优先遍历:递归确定当前维度的索引,再进入下一维;到达最后一维时,完整索引已确定,就可以执行存储。
struct ConstantOpLowering : public OpConversionPattern<toy::ConstantOp> {
using OpConversionPattern<toy::ConstantOp>::OpConversionPattern;
LogicalResult
matchAndRewrite(toy::ConstantOp op, OpAdaptor adaptor,
ConversionPatternRewriter &rewriter) const final {
DenseElementsAttr constantValue = op.getValue();
Location loc = op.getLoc();
// When lowering the constant operation, we allocate and assign the constant
// values to a corresponding memref allocation.
auto tensorType = llvm::cast<RankedTensorType>(op.getType());
auto memRefType = convertTensorToMemRef(tensorType);
auto alloc = insertAllocAndDealloc(memRefType, loc, rewriter);
// We will be generating constant indices up-to the largest dimension.
// Create these constants up-front to avoid large amounts of redundant
// operations.
auto valueShape = memRefType.getShape();
SmallVector<Value, 8> constantIndices;
if (!valueShape.empty()) {
for (auto i : llvm::seq<int64_t>(0, *llvm::max_element(valueShape)))
constantIndices.push_back(
arith::ConstantIndexOp::create(rewriter, loc, i));
} else {
// This is the case of a tensor of rank 0.
constantIndices.push_back(
arith::ConstantIndexOp::create(rewriter, loc, 0));
}
// The constant operation represents a multi-dimensional constant, so we
// will need to generate a store for each of the elements. The following
// functor recursively walks the dimensions of the constant shape,
// generating a store when the recursion hits the base case.
SmallVector<Value, 2> indices;
auto valueIt = constantValue.value_begin<FloatAttr>();
std::function<void(uint64_t)> storeElements = [&](uint64_t dimension) {
// The last dimension is the base case of the recursion, at this point
// we store the element at the given index.
if (dimension == valueShape.size()) {
affine::AffineStoreOp::create(
rewriter, loc, arith::ConstantOp::create(rewriter, loc, *valueIt++),
alloc, llvm::ArrayRef(indices));
return;
}
// Otherwise, iterate over the current dimension and add the indices to
// the list.
// CORE DFS PART
for (uint64_t i = 0, e = valueShape[dimension]; i != e; ++i) {
indices.push_back(constantIndices[i]);
storeElements(dimension + 1);
indices.pop_back();
}
};
// Start the element storing recursion from the first dimension.
storeElements(/*dimension=*/0);
// Replace this operation with the generated alloc.
rewriter.replaceOp(op, alloc);
return success();
}
};
其他操作较为简单,这里略过。还要注意,为使转换后的程序合法,需要让 print 接受 MemRef,也就是张量降级后的表示。
def PrintOp : Toy_Op<"print"> {
...
// The print operation takes an input tensor to print.
// We also allow a F64MemRef to enable interop during partial lowering.
let arguments = (ins AnyTypeOf<[F64Tensor, F64MemRef]>:$input);
}