05.srt

1
00:00:00,066 --> 00:00:03,833
字幕生成：qiaokai 字幕校对：mkwei

2
00:00:05,800 --> 00:00:09,566
大家好那个我是ZOMI

3
00:00:09,566 --> 00:00:10,133
那今天呢

4
00:00:10,133 --> 00:00:12,466
来到一个AI编译器系列之

5
00:00:12,466 --> 00:00:13,766
前端优化的

6
00:00:13,766 --> 00:00:14,733
布局转换

7
00:00:14,733 --> 00:00:15,866
这里面的布局转换呢

8
00:00:15,866 --> 00:00:17,566
主要是指数据布局

9
00:00:17,766 --> 00:00:19,399
那其实在上一个内容里面呢

10
00:00:19,400 --> 00:00:21,466
已经详细的去展开过了

11
00:00:21,466 --> 00:00:23,866
布局转换里面的前面的部分的内容

12
00:00:23,866 --> 00:00:26,766
第一个就是数据内存的排布

13
00:00:26,766 --> 00:00:27,466
数据呢

14
00:00:27,466 --> 00:00:29,999
在内存里面是怎么个排放的

15
00:00:30,266 --> 00:00:30,566
第二个呢

16
00:00:30,566 --> 00:00:33,999
讲了张量数据具体的排布方式

17
00:00:34,000 --> 00:00:36,266
第三个呢举了两个经典的例子

18
00:00:36,266 --> 00:00:38,666
NCHW和NHWC

19
00:00:38,733 --> 00:00:40,899
具体有什么区别

20
00:00:41,300 --> 00:00:43,300
接下来在这一节里面呢

21
00:00:43,300 --> 00:00:45,200
主要是讲下面这两个内容

22
00:00:45,200 --> 00:00:47,666
那第一个呢就是华为昇腾

23
00:00:47,666 --> 00:00:50,299
AI芯片的一个数据的排布方式

24
00:00:51,000 --> 00:00:53,000
这个内容我觉得是非常有意思的

25
00:00:53,000 --> 00:00:54,966
它不仅是代表华为昇腾

26
00:00:55,000 --> 00:00:56,966
也代表其他AI芯片厂商

27
00:00:56,966 --> 00:00:58,766
针对自己的芯片微架构呢

28
00:00:58,766 --> 00:01:00,266
提供特殊的数据存储和

29
00:01:00,266 --> 00:01:01,533
数据排布的方式

30
00:01:01,566 --> 00:01:02,466
那最后一个呢

31
00:01:02,466 --> 00:01:04,566
就是真正的回到今天的话题

32
00:01:04,566 --> 00:01:06,099
了解完基础知识之后呢

33
00:01:06,100 --> 00:01:07,100
来了解

34
00:01:07,266 --> 00:01:09,966
AI编译器对于数据布局

35
00:01:09,966 --> 00:01:11,799
转换优化的一个pass

36
00:01:11,800 --> 00:01:13,066
具体是怎么实现的

37
00:01:14,900 --> 00:01:17,333
接下来去了解一下华为昇腾

38
00:01:17,333 --> 00:01:19,399
处理器的一个数据排布

39
00:01:20,100 --> 00:01:22,100
在华为昇腾AI处理器当中呢

40
00:01:22,100 --> 00:01:25,000
为了提高整体的一个通用矩阵乘法

41
00:01:25,000 --> 00:01:26,900
即GEMM的一个运算速率

42
00:01:27,100 --> 00:01:28,466
和访存的效率

43
00:01:28,500 --> 00:01:30,733
所以呢所有的张量呢

44
00:01:30,733 --> 00:01:34,199
一般来说都不会使用NCHW这种方式

45
00:01:34,200 --> 00:01:39,200
而是把C进行切分采用的NCHWC0的方式

46
00:01:39,200 --> 00:01:42,200
那有时候呢内部呢叫做5HD的

47
00:01:42,200 --> 00:01:43,100
数据的方式

48
00:01:43,500 --> 00:01:44,366
那我现在呢

49
00:01:44,366 --> 00:01:47,166
以一个三维的数据作为例子

50
00:01:47,166 --> 00:01:48,799
就是HWC

51
00:01:49,000 --> 00:01:52,000
那H呢就是feature map的一个长和宽

52
00:01:52,100 --> 00:01:54,300
C呢就是Channel的大小

53
00:01:54,300 --> 00:01:55,666
那这个是三维的

54
00:01:55,800 --> 00:01:57,966
然后呢会把NHWC

55
00:01:57,966 --> 00:01:58,566
N呢

56
00:01:58,566 --> 00:02:01,133
其实还有一维 batch size的那个维度

57
00:02:01,133 --> 00:02:03,333
N呢现在先把它抛到一边

58
00:02:03,333 --> 00:02:05,066
因为它基本上没有变化

59
00:02:05,766 --> 00:02:06,333
接着呢

60
00:02:06,333 --> 00:02:09,299
对HWC里面的C channel进行切分

61
00:02:09,300 --> 00:02:12,866
里面呢切分成为c个C0

62
00:02:13,266 --> 00:02:14,399
所以可以看到

63
00:02:14,500 --> 00:02:16,100
传统的C这个channel

64
00:02:16,100 --> 00:02:19,800
把它切成多个C0 C1呢就等于4了

65
00:02:19,800 --> 00:02:23,766
1234分别由四个颜色去代替

66
00:02:23,766 --> 00:02:24,699
而这里面呢

67
00:02:24,700 --> 00:02:28,200
最后可以看到HW呢是正常的

68
00:02:28,766 --> 00:02:30,566
channel数呢就变成C0

69
00:02:30,566 --> 00:02:32,133
而C1呢有四个

70
00:02:32,133 --> 00:02:34,799
这个就是华为昇腾处理器的5HD

71
00:02:34,800 --> 00:02:36,066
的数据存储格式

72
00:02:38,766 --> 00:02:41,599
哎ZOMI老师我有一个疑问嘿

73
00:02:41,600 --> 00:02:44,933
为什么要把c这个轴切出来

74
00:02:45,000 --> 00:02:47,766
切成两个C0和C1的两个轴呢

75
00:02:49,566 --> 00:02:51,266
其实C0这个轴块

76
00:02:51,266 --> 00:02:53,199
C0这个数字呢是跟

77
00:02:53,400 --> 00:02:55,733
昇腾里面的达芬奇的微架构是相关的

78
00:02:55,733 --> 00:02:57,299
等于AI core里面

79
00:02:57,300 --> 00:02:59,200
矩阵计算单元的大小

80
00:02:59,400 --> 00:03:03,000
那对于FP16类型呢这个C0呢就为16

81
00:03:03,000 --> 00:03:05,200
对于INT8这个类型呢为32

82
00:03:05,200 --> 00:03:06,933
刚好是对应四个字节

83
00:03:06,933 --> 00:03:09,133
FP16呢对应两个字节

84
00:03:09,400 --> 00:03:12,100
这部分的数据呢需要连续的存储的

85
00:03:12,100 --> 00:03:14,533
所以呢C1等于C除以C0

86
00:03:14,533 --> 00:03:17,333
如果结果不整除呢就要向上取整了

87
00:03:17,333 --> 00:03:20,066
保证所有的数据都能够对齐

88
00:03:20,066 --> 00:03:22,066
那至于数据为什么需要对齐

89
00:03:22,066 --> 00:03:22,666
这个概念呢

90
00:03:22,666 --> 00:03:24,366
在前一个内容里面

91
00:03:24,366 --> 00:03:26,533
其实已经跟大家汇报过了

92
00:03:28,166 --> 00:03:31,799
接下来呢看一下NHWC或者NCHW呢

93
00:03:31,900 --> 00:03:35,133
是怎么变成NC1HWC0的

94
00:03:35,266 --> 00:03:38,366
以NHWC作为一个例子

95
00:03:38,366 --> 00:03:40,699
其实啊不是说非常难

96
00:03:41,533 --> 00:03:43,566
在这里面呢会分为两步

97
00:03:43,566 --> 00:03:46,299
那第一步呢就是把NHWC的数据

98
00:03:46,300 --> 00:03:48,366
在C轴的维度进行分割

99
00:03:48,366 --> 00:03:51,733
变成一个C1份NHWC0

100
00:03:51,866 --> 00:03:55,199
接着呢会把C1份的NHWC0呢

101
00:03:55,200 --> 00:03:57,566
在内存里面连续的排列

102
00:03:58,133 --> 00:04:01,166
最后呢就变成了NC1HWC0

103
00:04:01,166 --> 00:04:03,699
那具体呢在PyTorch里面呢可能

104
00:04:03,800 --> 00:04:06,933
进行一个reshape成NHWC1C0

105
00:04:06,966 --> 00:04:10,066
然后transpose成03124

106
00:04:12,333 --> 00:04:13,666
接下来分享一个

107
00:04:13,666 --> 00:04:15,533
在达芬奇微架构里面一个

108
00:04:15,533 --> 00:04:17,199
比较有意思的一个点

109
00:04:17,333 --> 00:04:19,999
会额外提供两个数据的格式

110
00:04:20,000 --> 00:04:23,300
一个呢叫做FRACTAL Z

111
00:04:23,300 --> 00:04:25,366
一个呢叫做FRACTAL NZ

112
00:04:25,400 --> 00:04:28,266
那FRACTAL呢它是分形的意思

113
00:04:28,600 --> 00:04:31,500
因为达芬奇呢它是一个SIMD的架构

114
00:04:31,566 --> 00:04:33,366
cube核呢就是AI core

115
00:04:34,466 --> 00:04:38,799
输出的格式呢其实是NW1H1H0W0

116
00:04:38,866 --> 00:04:39,866
嗯比较复杂

117
00:04:39,866 --> 00:04:41,899
但是呢从下面这个图里面呢

118
00:04:41,900 --> 00:04:43,166
简单的去理解一下

119
00:04:43,366 --> 00:04:44,733
假设呢现在一次过呢

120
00:04:44,733 --> 00:04:46,399
除以一行的数据

121
00:04:46,400 --> 00:04:47,100
那这里面呢

122
00:04:47,100 --> 00:04:49,533
我每一次丢进去AI core里面呢

123
00:04:49,533 --> 00:04:51,566
我就会直接去把这里面的

124
00:04:51,600 --> 00:04:53,700
一个小窗口的数据处理完

125
00:04:53,700 --> 00:04:55,466
那处理这个小窗口的数据呢

126
00:04:55,700 --> 00:04:57,533
有点类似于进行卷积计算的时候呢

127
00:04:57,533 --> 00:04:59,899
就把一个小窗口的数据算完

128
00:04:59,933 --> 00:05:01,466
让有非常多的cube核

129
00:05:01,466 --> 00:05:03,099
所以可以同时去

130
00:05:03,100 --> 00:05:05,166
计算非常多的这种小窗口

131
00:05:05,166 --> 00:05:06,699
而这种小窗口的数据呢

132
00:05:06,700 --> 00:05:08,100
具体是怎么排布的呢

133
00:05:08,366 --> 00:05:10,333
这里面呢有好几种不同的方式

134
00:05:10,366 --> 00:05:12,366
一个呢就是小z大z

135
00:05:12,466 --> 00:05:14,933
那小z呢就像这种我先按行排布

136
00:05:14,933 --> 00:05:15,899
然后下一个

137
00:05:16,166 --> 00:05:17,266
在这个小窗口里面呢

138
00:05:17,266 --> 00:05:18,599
在按行进行取数据

139
00:05:18,600 --> 00:05:20,533
然后再按行进行取数据

140
00:05:20,566 --> 00:05:21,699
这种就是小z

141
00:05:21,766 --> 00:05:22,466
那大z呢

142
00:05:22,466 --> 00:05:24,599
就是在整个大的feature map里面呢

143
00:05:24,700 --> 00:05:26,133
不断的按行进行取

144
00:05:26,133 --> 00:05:27,799
然后不断的按行进行取

145
00:05:27,800 --> 00:05:29,766
那这种呢就是大z的意思

146
00:05:29,800 --> 00:05:32,866
那所谓的小n大Z呢就在块列呢

147
00:05:32,866 --> 00:05:34,599
按列的方式进行排序

148
00:05:34,600 --> 00:05:36,533
我先取完这一列再取第二列

149
00:05:36,533 --> 00:05:37,533
再取第三列

150
00:05:37,600 --> 00:05:40,000
那块之间呢按行的方式去处理

151
00:05:40,000 --> 00:05:41,766
就是小n大Z

152
00:05:42,066 --> 00:05:43,466
例如权重的数据呢

153
00:05:43,466 --> 00:05:45,266
会以这种方式进行存储

154
00:05:45,600 --> 00:05:47,600
最后一个呢就是小z大N

155
00:05:47,700 --> 00:05:50,133
块列呢跟第一个的内容是相同的

156
00:05:50,133 --> 00:05:51,766
块列按行进行排列

157
00:05:51,800 --> 00:05:54,400
但是块间呢是按列进行排列的

158
00:05:54,600 --> 00:05:55,866
例如卷积输出结果呢

159
00:05:55,866 --> 00:05:58,299
都会以小z大N的方式进行排列

160
00:05:58,333 --> 00:06:00,799
那这这个呢只是额外新增的知识

161
00:06:00,800 --> 00:06:02,500
至于为什么要这么排列

162
00:06:02,500 --> 00:06:04,200
为什么要用小z大Z

163
00:06:04,366 --> 00:06:05,933
来去存储feature map

164
00:06:06,066 --> 00:06:07,766
为什么要用小n大Z

165
00:06:07,866 --> 00:06:09,333
来去存储卷积呢

166
00:06:09,566 --> 00:06:10,899
这个在后面的

167
00:06:10,966 --> 00:06:13,666
AI芯片去介绍达芬奇架构的时候

168
00:06:13,666 --> 00:06:15,066
会给大家去展开

169
00:06:15,066 --> 00:06:17,566
只是在内存数据转换这个内容里面呢

170
00:06:17,566 --> 00:06:19,333
简单给大家演示

171
00:06:19,366 --> 00:06:21,166
不同的芯片厂商不同的硬件

172
00:06:21,166 --> 00:06:23,299
他会有自己的一个数据存储的格式

173
00:06:23,300 --> 00:06:25,000
而数据存储的格式呢

174
00:06:25,000 --> 00:06:26,766
是根据硬件的读取方式

175
00:06:26,766 --> 00:06:29,466
或者硬件的计算方式所决定的

176
00:06:30,666 --> 00:06:33,066
接下来来到了一个最重要内容

177
00:06:33,066 --> 00:06:35,666
就是编译布局转换的一个优化

178
00:06:35,666 --> 00:06:37,933
就是在真正的AI编译器里面

179
00:06:37,933 --> 00:06:41,133
是如何对数据进行转换优化的

180
00:06:41,300 --> 00:06:43,266
那其实呢现在还是这个内容

181
00:06:43,266 --> 00:06:44,966
只是呢现在真正的

182
00:06:45,000 --> 00:06:46,400
了解完基础知识之后呢

183
00:06:46,400 --> 00:06:48,300
深入到AI编译器里面

184
00:06:48,333 --> 00:06:50,666
具体怎么做转换

185
00:06:51,900 --> 00:06:52,666
来看看

186
00:06:52,666 --> 00:06:55,099
实际上在AI编译器系里面的layout transformation

187
00:06:55,100 --> 00:06:56,166
的目的就是希望

188
00:06:56,166 --> 00:06:57,966
将内部的数据的布局

189
00:06:57,966 --> 00:06:59,133
转换成为后端

190
00:06:59,133 --> 00:07:00,266
就是硬件

191
00:07:00,333 --> 00:07:01,999
很友好的方式

192
00:07:02,133 --> 00:07:03,133
所以刚才说了

193
00:07:03,133 --> 00:07:04,499
数据怎么排布

194
00:07:04,500 --> 00:07:06,500
是根据硬件的执行方式

195
00:07:06,500 --> 00:07:07,600
相关联的

196
00:07:07,600 --> 00:07:08,933
不是说我随便排都行

197
00:07:08,933 --> 00:07:10,133
在CPU里面

198
00:07:10,133 --> 00:07:12,333
在GPU里面的排布方式是不一样的

199
00:07:12,500 --> 00:07:13,466
包括在昇腾

200
00:07:13,466 --> 00:07:14,933
处理器里面的排布方式

201
00:07:14,933 --> 00:07:15,866
也是不同的

202
00:07:15,866 --> 00:07:17,533
那最重要的方式呢

203
00:07:17,533 --> 00:07:19,133
就是找到在计算图当中

204
00:07:19,166 --> 00:07:22,366
存储这个张量最佳的数据布局

205
00:07:22,566 --> 00:07:23,966
大家注意一个概念呢

206
00:07:23,966 --> 00:07:26,699
这里面呢是谈的计算图哦

207
00:07:26,800 --> 00:07:28,000
所以会在图层IR

208
00:07:28,000 --> 00:07:29,400
里面去实现这个pass

209
00:07:29,400 --> 00:07:30,800
或者实现这个转换

210
00:07:30,800 --> 00:07:33,200
然后呢将数据布局呢转换成为

211
00:07:33,200 --> 00:07:35,800
具体的节点插到图中呢

212
00:07:35,900 --> 00:07:37,666
那值得注意的方式就是

213
00:07:37,666 --> 00:07:38,699
数据布局呢

214
00:07:38,700 --> 00:07:41,800
是最终对性能有非常大的影响

215
00:07:41,900 --> 00:07:45,000
而且转换的操作也有非常大的开销

216
00:07:45,000 --> 00:07:47,366
每一次数据的转换就涉及到IO

217
00:07:47,366 --> 00:07:48,766
涉及到访存

218
00:07:49,866 --> 00:07:50,899
回顾一些之前讲

219
00:07:50,900 --> 00:07:52,733
过的一些比较通用的概念

220
00:07:52,733 --> 00:07:55,399
那第一个呢就是NCHW这种格式呢

221
00:07:55,400 --> 00:07:58,166
在GPU上面是运行的比较快的

222
00:07:58,166 --> 00:07:59,266
所以GPU上面呢

223
00:07:59,266 --> 00:08:02,066
默认的去使用NCHW这种格式

224
00:08:02,333 --> 00:08:03,666
例如达芬奇架构呢

225
00:08:03,666 --> 00:08:06,499
就会提供一个CANN这个硬件的库呢

226
00:08:06,500 --> 00:08:09,700
芯片使能层呢去解决这些问题的

227
00:08:10,166 --> 00:08:13,099
还有第三点呢就是一些边缘的设备

228
00:08:13,100 --> 00:08:15,566
例如很简单的就是手机

229
00:08:15,566 --> 00:08:18,399
手机这块SOC呢里面有丰富的IP

230
00:08:18,533 --> 00:08:21,099
包括arm端侧的GPU还有ISP

231
00:08:21,100 --> 00:08:24,400
还有DPU等不同的这一些计算单元

232
00:08:24,400 --> 00:08:25,966
那不同的计算单元之间呢

233
00:08:25,966 --> 00:08:28,333
可能会有不同的数据排布

234
00:08:28,400 --> 00:08:30,000
所以呢针对边缘设备

235
00:08:30,000 --> 00:08:32,133
可能就会有异构的计算单元

236
00:08:32,133 --> 00:08:32,766
这个时候呢

237
00:08:32,766 --> 00:08:36,299
数据的转换呢就显得非常的重要了

238
00:08:36,300 --> 00:08:37,533
AI编译器呢

239
00:08:37,533 --> 00:08:38,999
就是希望能够提供一种

240
00:08:39,266 --> 00:08:42,366
跨硬件的一个数据排布转换的方式

241
00:08:42,366 --> 00:08:44,533
方便对接到不同的后端

242
00:08:45,000 --> 00:08:47,000
所以这个故事呢就告诉

243
00:08:47,200 --> 00:08:47,966
在写AI

244
00:08:47,966 --> 00:08:49,866
算法或者实现AI算法的时候呢

245
00:08:49,866 --> 00:08:52,299
其实AI框架或者AI编译器啊

246
00:08:52,300 --> 00:08:55,100
帮做了很多感知不到的工作

247
00:08:55,100 --> 00:08:57,733
不要以为AI框架只是一个简单的库

248
00:08:57,733 --> 00:08:59,666
直接调用算子就完了

249
00:08:59,700 --> 00:09:01,733
AI框架其实里面的

250
00:09:01,733 --> 00:09:03,566
技术含量还是非常高的

251
00:09:03,733 --> 00:09:04,399
那现在呢

252
00:09:04,400 --> 00:09:05,133
看一看

253
00:09:05,133 --> 00:09:07,866
具体的数据转换是怎么样操作的

254
00:09:07,900 --> 00:09:10,166
首先假设我输进去的数据呢

255
00:09:10,166 --> 00:09:12,333
是一个NHWC的数据

256
00:09:12,400 --> 00:09:15,333
那我输出的时候呢我是一个NCHW

257
00:09:15,533 --> 00:09:17,533
那这个时候呢我就进行了一个

258
00:09:17,533 --> 00:09:19,099
数据格式的转换了

259
00:09:19,266 --> 00:09:20,499
这个数据转换呢

260
00:09:20,500 --> 00:09:22,466
假设它是一个算子

261
00:09:22,466 --> 00:09:23,099
然后呢

262
00:09:23,100 --> 00:09:25,366
这个算子是对内存排布

263
00:09:25,366 --> 00:09:26,333
进行修改的

264
00:09:26,366 --> 00:09:30,966
这个算子叫做CASTDATA NHWC to NCHW

265
00:09:31,500 --> 00:09:32,933
那另外还有另外一种情况

266
00:09:32,933 --> 00:09:34,766
就把它们反过来

267
00:09:34,800 --> 00:09:37,300
输入是NCHW

268
00:09:37,533 --> 00:09:39,799
那我的输出呢是NHWC

269
00:09:39,933 --> 00:09:41,766
中间插一个CAST算子

270
00:09:41,766 --> 00:09:43,466
这个CASTDATA的算子呢

271
00:09:43,500 --> 00:09:45,866
是把NCHW的数据格式

272
00:09:45,900 --> 00:09:49,066
转换到NHWC的数据格式

273
00:09:49,266 --> 00:09:51,399
接下来看一个更复杂的例子

274
00:09:51,400 --> 00:09:54,900
就是常见的数据转换的一个节点

275
00:09:55,066 --> 00:09:56,533
那其实很简单

276
00:09:56,700 --> 00:09:58,800
嗯以左边的第一个图来看

277
00:09:58,800 --> 00:10:01,133
首先现在有两个算子

278
00:10:01,133 --> 00:10:04,566
第一个算子呢执行的是NCHW的格式

279
00:10:04,566 --> 00:10:06,399
它的输出也是NCHW

280
00:10:06,800 --> 00:10:07,733
那第二个算子呢

281
00:10:07,733 --> 00:10:10,966
它的输入和输出也是NCHW

282
00:10:11,166 --> 00:10:11,966
那这个时候呢

283
00:10:11,966 --> 00:10:13,966
在神经网络里面处理的时候呢

284
00:10:13,966 --> 00:10:15,366
其实是不感知的

285
00:10:15,400 --> 00:10:18,000
就我输入一个数据已经定义好了

286
00:10:18,066 --> 00:10:20,499
输出还是这种数据的格式

287
00:10:20,866 --> 00:10:22,499
但是呢有些情况下

288
00:10:22,500 --> 00:10:26,666
算子他处理的是一个NCHW

289
00:10:26,766 --> 00:10:28,699
输入呢是一个NHWC

290
00:10:28,733 --> 00:10:30,099
的数据的时候呢

291
00:10:30,100 --> 00:10:33,133
我这里面呢就需要做一个数据的转换

292
00:10:33,133 --> 00:10:36,199
然后输出的时候可能我还是变成NCHW

293
00:10:36,333 --> 00:10:38,566
就需要再插一个算子

294
00:10:38,566 --> 00:10:40,066
进行数据的转换

295
00:10:40,300 --> 00:10:40,700
当然呢

296
00:10:40,700 --> 00:10:42,900
还会遇到一个最复杂的方式

297
00:10:42,900 --> 00:10:45,200
就是我上一个算子的输入输出

298
00:10:45,266 --> 00:10:47,333
跟下一个算子的输入输出

299
00:10:47,333 --> 00:10:49,466
是完全不对等的

300
00:10:49,500 --> 00:10:52,166
这个时候AI框架或者AI编译器呢

301
00:10:52,166 --> 00:10:55,499
就需要去插入不同的数据转换的算子

302
00:10:55,500 --> 00:10:56,000
然后呢

303
00:10:56,000 --> 00:10:58,400
才能够使得整图能够跑得通

304
00:10:59,533 --> 00:11:01,499
那接下来呢讲两个例子

305
00:11:01,500 --> 00:11:04,133
第一个例子呢是在训练场景

306
00:11:04,133 --> 00:11:05,599
训练场景的AI编译器

307
00:11:05,600 --> 00:11:06,733
跟推理的场景呢

308
00:11:06,733 --> 00:11:09,099
AI编译器可能有一点点不同啊

309
00:11:09,100 --> 00:11:12,133
简单的来去从训练场景去看看

310
00:11:12,466 --> 00:11:14,199
假设呢现在有一个例子

311
00:11:14,200 --> 00:11:16,200
现在这里面呢有一个卷积

312
00:11:16,200 --> 00:11:18,066
然后下一层也有一个卷积

313
00:11:18,066 --> 00:11:19,466
但是上一层的卷积呢

314
00:11:19,466 --> 00:11:20,899
是一个1*1的卷积

315
00:11:20,900 --> 00:11:23,566
下一层的卷积是一个3*3的卷积

316
00:11:23,866 --> 00:11:27,299
例如1*1的卷积呢使用NHWC的格式

317
00:11:27,300 --> 00:11:29,900
但是下层的卷积是一个3*3的卷积

318
00:11:29,900 --> 00:11:32,166
会用NCHW这种格式

319
00:11:32,166 --> 00:11:34,366
这个时候呢就要像b图那样

320
00:11:34,466 --> 00:11:35,699
针对特定的格式呢

321
00:11:35,700 --> 00:11:38,266
插入具体的转换的算子

322
00:11:38,266 --> 00:11:40,199
AI编译器就会感知整个图的情况

323
00:11:40,200 --> 00:11:42,600
感知上下文就是上面的算子

324
00:11:42,733 --> 00:11:44,366
下面的算子的存储格式

325
00:11:44,366 --> 00:11:46,999
接着呢插入具体的转换算子

326
00:11:47,000 --> 00:11:48,766
这个就是第二种方式

327
00:11:48,766 --> 00:11:50,933
那当然呢还有第三种方式

328
00:11:51,066 --> 00:11:53,899
假设我是三个连续的1*1的卷积

329
00:11:53,900 --> 00:11:54,533
那这时候呢

330
00:11:54,533 --> 00:11:56,466
我有非常多的这种转换算子

331
00:11:56,466 --> 00:11:58,566
但实际上呢我卷积1*1的算子

332
00:11:58,566 --> 00:12:00,699
我的输入输出都是相同的

333
00:12:00,866 --> 00:12:01,366
这个时候呢

334
00:12:01,366 --> 00:12:04,099
AI编译器就会去取消转换的算子

335
00:12:04,100 --> 00:12:06,200
就是把一些转换的算子删掉

336
00:12:06,200 --> 00:12:09,400
重新变回a图就是1*1的卷机

337
00:12:09,400 --> 00:12:11,333
然后下一个1*1的卷机

338
00:12:11,333 --> 00:12:13,766
使用的都是NCHW这种格式

339
00:12:13,766 --> 00:12:15,766
这就是整体训练场景

340
00:12:15,866 --> 00:12:17,866
AI编译器会帮做的工作

341
00:12:17,866 --> 00:12:19,599
这也是华为昇腾

342
00:12:19,600 --> 00:12:22,533
CANN里面帮用户去解决的一些问题

343
00:12:25,266 --> 00:12:27,499
推理场景呢其实很重要的一点就

344
00:12:27,500 --> 00:12:30,533
是会对权重的布局进行转换

345
00:12:31,000 --> 00:12:31,766
这个时候呢

346
00:12:31,766 --> 00:12:33,966
小新可能会又想跳出来去问

347
00:12:33,966 --> 00:12:35,299
为什么要对

348
00:12:35,300 --> 00:12:37,900
权重的数据布局进行转换呢

349
00:12:38,366 --> 00:12:40,333
假设现在在GPU上面去

350
00:12:40,333 --> 00:12:41,299
训练神经网络

351
00:12:41,300 --> 00:12:43,566
但是呢在推理的时候呢

352
00:12:43,566 --> 00:12:45,966
就会到手机上面去进行推理

353
00:12:46,200 --> 00:12:48,566
手机上面呢主要是跑在CPU上面

354
00:12:48,666 --> 00:12:50,299
那权重的数据布局呢

355
00:12:50,300 --> 00:12:51,200
就跟GPU

356
00:12:51,200 --> 00:12:52,700
训练的时候的权重布局

357
00:12:52,700 --> 00:12:53,366
就会不一样

358
00:12:53,366 --> 00:12:55,499
所以会在AI推理编译器

359
00:12:55,500 --> 00:12:57,266
或者AI转换模块里面呢

360
00:12:57,266 --> 00:12:59,733
做一个权重布局的转换

361
00:13:01,866 --> 00:13:03,066
在这两节课里面呢

362
00:13:03,066 --> 00:13:04,366
主要是了解了

363
00:13:04,366 --> 00:13:06,333
数据内存的一个排布方式

364
00:13:06,333 --> 00:13:09,266
了解了张量的数据的整体的布局

365
00:13:09,266 --> 00:13:10,566
其实是非常复杂的

366
00:13:10,566 --> 00:13:12,266
另外呢还举了两个例子

367
00:13:12,266 --> 00:13:16,199
NCTW和NHWC两个具体的形态

368
00:13:16,266 --> 00:13:19,566
接着去看了一下华为昇腾达芬奇架构

369
00:13:19,566 --> 00:13:22,499
具体的数据排布的方式有分形的z

370
00:13:22,533 --> 00:13:26,766
分形的NZ NHC1WC0这种方式

371
00:13:26,766 --> 00:13:29,099
另外呢还了解了编译器

372
00:13:29,100 --> 00:13:30,733
特别是AI编译器里面的

373
00:13:30,733 --> 00:13:32,966
数据布局转换优化的一个具体的

374
00:13:32,966 --> 00:13:33,566
算法

375
00:13:33,566 --> 00:13:36,166
对计算图呢插入CASTDATA

376
00:13:36,166 --> 00:13:38,566
或者对CASTDATA呢进行一个消除

377
00:13:38,566 --> 00:13:40,299
使得整个计算图呢

378
00:13:40,300 --> 00:13:40,933
能够在

379
00:13:40,933 --> 00:13:43,499
对应的硬件上面真正的执行起来

380
00:13:44,466 --> 00:13:45,266
谢谢各位

381
00:13:45,366 --> 00:13:47,099
卷的不行了卷的不行了

382
00:13:47,100 --> 00:13:48,766
记得一键三连加关注哦

383
00:13:48,866 --> 00:13:50,199
所有的内容都会开源

384
00:13:50,200 --> 00:13:52,066
在下面这条链接里面

385
00:13:52,400 --> 00:13:53,466
拜了个拜