About the dot op codegen

[Superjomn]

Is the $c operand unnecessary in dot op?

According to dot's definition, its semantic is like $d = matrix_multiply($a, $b) + $c, and both the $c and $d have the same MMA layout.

I wonder whether the following two IR are the same:

 %86 = tt.dot %83, %84, %arg13 {allowTF32 = true} : tensor<64x64xf32, #shared> * tensor<64x64xf32, #shared> -> tensor<64x64xf32, #mma>

and

2. (comes from the ir snapshot for dot)

// cst_0 is 0
%85 = tt.dot %83, %84, %cst_0 {allowTF32 = true} : tensor<64x64xf32, #shared> * tensor<64x64xf32, #shared> -> tensor<64x64xf32, #mma>
%86 = arith.addf %arg13, %85 : tensor<64x64xf32, #mma>

In my understanding, the $c operand in dot is unnecessary, we can use a separate arith.addf to get the same result.

What's more, there are separate codegen-support for the binary ops, it seems that we can reuse them with convert_layout(MMA -> blocked), for example:

// cst_0 is 0
%85 = tt.dot %83, %84, %cst_0 {allowTF32 = true} : tensor<64x64xf32, #shared> * tensor<64x64xf32, #shared> -> tensor<64x64xf32, #mma>
%85_blocked = tt_gpu.convert_layout %85: (tensor<64x64xf32, #mma>) -> tensor<64x64xf32, #blocked0>
%86 = arith.addf %85_blocked, %arg13_blocked : tensor<64x64xf32, #block0> // this reuses the existing binary op codegen for blocked layout

Please correct me if I am wrong.

[yangjunpro]

@ptillet to hear your input for this.
My knee-jerking feeling is the same as Chunwei. I haven't understood the necessity for making tt.dot instruction to support c operand given that there is already a binary arith.addf instruction.

[ptillet]

These are the same IR, and indeed the frontend generates:

// cst_0 is 0
%85 = tt.dot %83, %84, %cst_0 {allowTF32 = true}
%86 = arith.addf %arg13, %85

And we have a rewrite rule that translates it into

%86 = tt.dot %83, %84, %arg13 {allowTF32 = true}

At the Triton-IR level, before the layouts are introduced.

The rationale for having tt.dot take a third operand is that it makes the codegen of mma ptx instruction much more natural, since IIUC the hardware does compute D = A * B + C ?

[yangjunpro]

Yes, the hardware supports D = A * B + C, it makes sense to me now. So we can combine
// cst_0 is 0
%85 = tt.dot %83, %84, %cst_0 {allowTF32 = true}
%86 = arith.addf %arg13, %85
as a single IR

%86 = tt.dot %83, %84, %arg13 {allowTF32 = true}

and directly map that IR to a native MMA instruction.

[Superjomn]

I see, this make sense.