这篇文章讨论如何通过接口实现改写。
接口
接口是 MLIR 的另一项核心能力,用来解决 pass 实现中大量重复代码的问题。例如,add、mult、div 等操作具有共同的结构和语义,往往会被同一个 pass 处理。但按上一篇的做法,每种操作都要分别实现处理逻辑。能否只实现一次公共逻辑?
这就引出了接口和 trait。我们可以提取面向接口的公共逻辑,让各个操作实现对应接口。这样既分离了通用逻辑与定制逻辑,也更容易维护,抽象更加准确。
下面通过形状推导说明接口的用法。目前允许张量不带形状,但真正生成代码时,每个张量都需要形状信息,例如进行边界检查。因此要根据已知形状推导出所有张量的形状。
内联
首先要内联所有函数。用户定义的函数在形状上是泛型的,同一函数中的张量会因不同调用的输入而具有不同形状。另一种方法是函数特化,为每种可能的输入组合创建独立函数。
使用内置的 InlinerPass 可以方便地完成内联,只需为 Toy 实现对应接口:
struct ToyInlinerInterface : public DialectInlinerInterface {
using DialectInlinerInterface::DialectInlinerInterface;
/// All call operations within toy can be inlined.
bool isLegalToInline(Operation *call, Operation *callable, bool wouldBeCloned) const final {
return true;
}
/// All operations within toy can be inlined.
bool isLegalToInline(Operation *, Region *, bool, IRMapping &) const final {
return true;
}
// All functions within toy can be inlined.
bool isLegalToInline(Region *, Region *, bool, IRMapping &) const final {
return true;
}
/// Handle the given inlined terminator(toy.return) by replacing it with a new
/// operation as necessary.
void handleTerminator(Operation *op, ValueRange valuesToRepl) const final {
// Only "toy.return" needs to be handled here.
auto returnOp = cast<ReturnOp>(op);
// Replace the values directly with the return operands.
assert(returnOp.getNumOperands() == valuesToRepl.size());
for (const auto &it : llvm::enumerate(returnOp.getOperands()))
valuesToRepl[it.index()].replaceAllUsesWith(it.value());
}
};
从 AST 生成 MLIR 时,还要把 main 之外的函数标记为 private,以便内联器清理它们:
mlir::toy::FuncOp mlirGen(FunctionAST &funcAST) {
...
// If this function isn't main, then set the visibility to private.
if (funcAST.getProto()->getName() != "main")
function.setPrivate();
return function;
}
在初始化时添加接口:
/// Dialect initialization, the instance will be owned by the context. This is
/// the point of registration of types and operations for the dialect.
void ToyDialect::initialize() {
addOperations<
#define GET_OP_LIST
#include "toy/Ops.cpp.inc"
>();
addInterfaces<ToyInlinerInterface>();
}
接着让内联器处理 toy.generic_call 和 toy.func。因为这些是自定义方言中的操作,内联器并不知道它们分别表示调用和函数。修改操作声明即可告知它:
// in Opts.td
include "mlir/Interfaces/CallInterfaces.td"
def FuncOp : Toy_Op<"func",
[FunctionOpInterface, IsolatedFromAbove]> {
...
}
def GenericCallOp : Toy_Op<"generic_call",
[DeclareOpInterfaceMethods<CallOpInterface>]> {
...
}
最后还需要一个 cast 操作。函数输入没有确定形状,而 main 中的张量带有形状。我们显式将张量转换为无形状类型,保证函数可以正确内联;之后再处理这些转换。
注意,cast 的结果类型就是目标类型,因此操作的类型签名已经定义了其功能。
def CastOp : Toy_Op<"cast", [
DeclareOpInterfaceMethods<CastOpInterface>,
DeclareOpInterfaceMethods<ShapeInferenceOpInterface>,
Pure,
SameOperandsAndResultShape
]> {
let summary = "shape cast operation";
let description = [{
The "cast" operation converts a tensor from one type to an equivalent type
without changing any data elements. The source and destination types must
both be tensor types with the same element type. If both are ranked, then
shape is required to match. The operation is invalid if converting to a
mismatching constant dimension.
}];
let arguments = (ins F64Tensor:$input);
let results = (outs F64Tensor:$output);
let assemblyFormat = "$input attr-dict `:` type($input) `to` type($output)";
}
可以在 materializeCallConversion 中添加 cast,这是 MLIR 内联 pass 提供的内置钩子。
struct ToyInlinerInterface : public DialectInlinerInterface {
...
/// Attempts to materialize a conversion for a type mismatch between a call
/// from this dialect, and a callable region. This method should generate an
/// operation that takes 'input' as the only operand, and produces a single
/// result of 'resultType'. If a conversion can not be generated, nullptr
/// should be returned.
Operation *materializeCallConversion(OpBuilder &builder, Value input,
Type resultType,
Location conversionLoc) const final {
return CastOp::create(builder, conversionLoc, resultType, input);
}
};
到这里,添加 createInlinerPass 就能完成所有函数调用的内联。
形状推导
形状推导的核心是根据操作输入确定输出类型。我们继续使用 ODS 描述推导规则,之前也用它声明了操作。新建 ShapeInferenceInterface.td,在 method 部分声明名为 inferShapes 的接口方法。
#ifndef SHAPE_INFERENCE_INTERFACE
#define SHAPE_INFERENCE_INTERFACE
include "mlir/IR/OpBase.td"
def ShapeInferenceOpInterface : OpInterface<"ShapeInference"> {
let description = [{
Interface to access a registered method to infer the return types for an
operation that can be used during type inference.
}];
let methods = [
InterfaceMethod<"Infer and set the output shape for the current operation.",
"void", "inferShapes">
];
}
#endif // SHAPE_INFERENCE_INTERFACE
然后在 Ops.td 中引入接口,为关注的 transpose、add、mult 添加声明。
include "toy/ShapeInferenceInterface.td"
def MulOp : Toy_Op<"mul",
[..., DeclareOpInterfaceMethods<ShapeInferenceOpInterface>]> {
...
}
还要在 Dialect.cpp 中为这些操作实现 inferShapes。逐元素操作的输出形状与输入相同,因此很直接。注意,inferShapes 会直接修改操作。
void MulOp::inferShapes() { getResult().setType(getLhs().getType()); }
对于转置,需要交换两个维度。
void TransposeOp::inferShapes() {
auto arrayTy = llvm::cast<RankedTensorType>(getOperand().getType());
SmallVector<int64_t, 2> dims(llvm::reverse(arrayTy.getShape()));
getResult().setType(RankedTensorType::get(dims, arrayTy.getElementType()));
}
最后创建一个利用接口执行类型推导的 pass,新建 ShapeInferencePass.cpp:
struct ShapeInferencePass
: public mlir::PassWrapper<ShapeInferencePass, OperationPass<toy::FuncOp>> {
MLIR_DEFINE_EXPLICIT_INTERNAL_INLINE_TYPE_ID(ShapeInferencePass)
StringRef getArgument() const override { return "toy-shape-inference"; }
void runOnOperation() override {
auto f = getOperation();
// Populate the worklist with the operations that need shape inference:
// these are operations that return a dynamic shape.
llvm::SmallPtrSet<mlir::Operation *, 16> opWorklist;
f.walk([&](mlir::Operation *op) {
if (returnsDynamicShape(op))
opWorklist.insert(op);
});
// Iterate on the operations in the worklist until all operations have been
// inferred or no change happened (fix point).
while (!opWorklist.empty()) {
// Find the next operation ready for inference, that is an operation
// with all operands already resolved (non-generic).
auto nextop = llvm::find_if(opWorklist, allOperandsInferred);
if (nextop == opWorklist.end())
break;
Operation *op = *nextop;
opWorklist.erase(op);
// Ask the operation to infer its output shapes.
LDBG() << "Inferring shape for: " << *op;
if (auto shapeOp = dyn_cast<ShapeInference>(op)) {
shapeOp.inferShapes();
} else {
op->emitError("unable to infer shape of operation without shape "
"inference interface");
return signalPassFailure();
}
}
// If the operation worklist isn't empty, this indicates a failure.
if (!opWorklist.empty()) {
f.emitError("Shape inference failed, ")
<< opWorklist.size() << " operations couldn't be inferred\n";
signalPassFailure();
}
}
/// A utility method that returns if the given operation has all of its
/// operands inferred.
static bool allOperandsInferred(Operation *op) {
return llvm::all_of(op->getOperandTypes(), [](Type operandType) {
return llvm::isa<RankedTensorType>(operandType);
});
}
/// A utility method that returns if the given operation has a dynamically
/// shaped result.
static bool returnsDynamicShape(Operation *op) {
return llvm::any_of(op->getResultTypes(), [](Type resultType) {
return !llvm::isa<RankedTensorType>(resultType);
});
}
};
/// Create a Shape Inference pass.
std::unique_ptr<mlir::Pass> mlir::toy::createShapeInferencePass() {
return std::make_unique<ShapeInferencePass>();
}
注意以下几点:
ShapeInferencePass继承OperationPass。- 它作用于
toy::FuncOp,因为只应在函数内部执行,而不应跨越函数边界,即过程内分析。内联后通常只剩 main,但这样仍能保证形状推导不跨函数运行。
形状推导 pass 的工作步骤如下:
- 获取函数体中的全部操作。
- 收集返回动态形状的操作。
- 迭代调用这些操作的
inferShape,直到所有操作都完成类型推导。
最后将 pass 加入 pass manager 即可。接口是一种很有用的软件工程技术,如今在编译器中尤其出彩。