02.srt

1
00:00:00,066 --> 00:00:04,616
【字幕生成: 奔崩 字幕校对: 奔崩】

2
00:00:04,747 --> 00:00:07,120
哈喽，大家好，我是ZOMI

3
00:00:07,120 --> 00:00:08,840
今天的心情有点单

4
00:00:08,840 --> 00:00:11,120
但是为了讲课的效果

5
00:00:11,120 --> 00:00:12,680
我还是得激情满满

6
00:00:13,240 --> 00:00:15,640
今天还是来到AI编译器里面的

7
00:00:15,640 --> 00:00:17,520
后端优化第二个内容

8
00:00:17,520 --> 00:00:18,400
第二个内容是

9
00:00:18,400 --> 00:00:20,720
算子的计算和调度

10
00:00:20,720 --> 00:00:24,160
这里面强调算子的计算和调度

11
00:00:24,160 --> 00:00:25,360
就是分开两个

12
00:00:25,360 --> 00:00:27,080
一个是计算这一part

13
00:00:27,080 --> 00:00:28,840
一个是调度这个part

14
00:00:28,840 --> 00:00:31,920
来看看它跟后端编译有什么不一样

15
00:00:31,920 --> 00:00:34,600
现在还是在后端编译的模块

16
00:00:34,600 --> 00:00:37,880
大部分的时间在对算子进行一个优化

17
00:00:37,880 --> 00:00:39,000
对算子优化之前

18
00:00:39,000 --> 00:00:40,680
先了解一个概念

19
00:00:40,680 --> 00:00:42,520
就是算子的计算还有调度

20
00:00:43,800 --> 00:00:45,840
现在看一下所谓的算子

21
00:00:45,840 --> 00:00:46,560
什么叫算子

22
00:00:46,720 --> 00:00:50,040
深度学习其实有非常多的计算单元去组成的

23
00:00:50,040 --> 00:00:53,360
这些计算单元大部分都是一个函数的映射

24
00:00:53,360 --> 00:00:55,120
就是一个数学的映射关系

25
00:00:55,120 --> 00:00:57,600
所以把这个当做一个算子

26
00:00:57,800 --> 00:00:59,913
所谓的算法就是解决某个问题

27
00:00:59,913 --> 00:01:02,080
提出的方案的完整描述

28
00:01:02,080 --> 00:01:04,080
这个就是算子跟算法的

29
00:01:04,080 --> 00:01:06,080
一个确切的定义和区分

30
00:01:07,760 --> 00:01:09,760
现在看看算子的计算

31
00:01:09,760 --> 00:01:11,800
算子的计算 是算子的定义

32
00:01:11,800 --> 00:01:14,600
回答这个算子到底是什么

33
00:01:14,600 --> 00:01:16,080
长什么样子

34
00:01:16,080 --> 00:01:17,431
调度主要指的是

35
00:01:17,431 --> 00:01:21,160
算子的执行策略和具体的实现

36
00:01:21,160 --> 00:01:23,520
这么去理解可能还是有点抽象

37
00:01:23,520 --> 00:01:25,280
这么去理解

38
00:01:25,280 --> 00:01:26,840
假设现在有一个算子

39
00:01:26,840 --> 00:01:29,720
在不同的硬件平台有不同的实现

40
00:01:29,720 --> 00:01:31,920
虽然调度方式是不同的

41
00:01:31,920 --> 00:01:35,160
但是这个算子的定义肯定是相同的

42
00:01:35,160 --> 00:01:37,840
它有且只有一种定义的形态

43
00:01:37,840 --> 00:01:39,688
所以会说算子

44
00:01:39,688 --> 00:01:42,040
实际上是分开计算和调度两方面

45
00:01:42,040 --> 00:01:44,200
计算的实现是明确的

46
00:01:44,200 --> 00:01:47,680
但是具体的调度方式在不同的硬件是不同的

47
00:01:47,680 --> 00:01:49,800
所以这两块会分开

48
00:01:49,800 --> 00:01:51,960
下面来看一个比较具体的例子

49
00:01:51,960 --> 00:01:54,080
就是高斯滤波

50
00:01:54,080 --> 00:01:57,080
下面 底色这是原始输进去的图片

51
00:01:57,080 --> 00:01:59,600
这个就是滤波核 高斯滤波核

52
00:01:59,600 --> 00:02:02,560
最后得到高斯滤波后的图片

53
00:02:02,560 --> 00:02:05,560
这个就是整个算子的计算的逻辑

54
00:02:05,560 --> 00:02:07,920
具体的算法就如这里面所示

55
00:02:07,920 --> 00:02:09,400
当一个像素的时候

56
00:02:09,400 --> 00:02:11,360
当一张图片的时候

57
00:02:11,360 --> 00:02:13,120
它是这个计算公式

58
00:02:14,600 --> 00:02:16,720
当然了这个算子计算的效果

59
00:02:16,720 --> 00:02:18,378
就是为了让图片

60
00:02:18,378 --> 00:02:20,040
做一个高斯的模糊

61
00:02:20,040 --> 00:02:23,040
但是在实际上对图像进行模糊

62
00:02:23,320 --> 00:02:25,280
有很多的实现方式

63
00:02:25,280 --> 00:02:27,760
这种是最原始的实现方式

64
00:02:27,760 --> 00:02:29,760
我有一个X有一个Y

65
00:02:29,760 --> 00:02:32,200
我先对图像的横坐标进行迭代

66
00:02:32,200 --> 00:02:34,800
再对图像的纵坐标进行迭代

67
00:02:34,800 --> 00:02:37,800
接着去计算横向的高斯滤波

68
00:02:37,800 --> 00:02:40,600
同样的我对这张图片的X和Y

69
00:02:40,600 --> 00:02:43,040
就横轴和纵轴进行一个迭代

70
00:02:43,040 --> 00:02:45,880
接着去计算纵轴的高斯滤波

71
00:02:46,040 --> 00:02:47,640
这种方式是其中一种

72
00:02:47,640 --> 00:02:49,520
它的时间耗时是非常大的

73
00:02:49,520 --> 00:02:51,880
但是简单的看一看

74
00:02:51,920 --> 00:02:53,760
这里面把横轴和纵轴

75
00:02:53,760 --> 00:02:56,320
就是把X和Y反过来之后

76
00:02:56,320 --> 00:02:58,360
可以看到它的运算时间

77
00:02:58,360 --> 00:02:59,880
是直接增加了

78
00:03:00,880 --> 00:03:03,360
那这种把X和Y反转过来

79
00:03:03,360 --> 00:03:05,320
就是算子的调度方式

80
00:03:05,320 --> 00:03:07,840
而算子的实现或者算子的逻辑

81
00:03:08,240 --> 00:03:11,160
高斯滤波这个原理是没有变的

82
00:03:12,400 --> 00:03:15,640
上面两个操作的算子的实现的功能

83
00:03:15,880 --> 00:03:17,760
就是算法其实是一样

84
00:03:17,760 --> 00:03:20,760
但为什么速度会有不一样的区别呢

85
00:03:21,280 --> 00:03:23,880
这个原因主要是跟硬件设计

86
00:03:23,880 --> 00:03:25,320
是非常相关的

87
00:03:25,760 --> 00:03:28,320
为了更好地贴近硬件的设计

88
00:03:28,320 --> 00:03:29,751
所以这里面

89
00:03:29,751 --> 00:03:32,000
把算子的计算和调度分开

90
00:03:32,000 --> 00:03:33,960
那计算还是那个逻辑

91
00:03:33,960 --> 00:03:35,080
但是调度方式

92
00:03:35,200 --> 00:03:37,680
就会根据硬件的具体的设计

93
00:03:37,680 --> 00:03:39,139
然后用到了SIMD

94
00:03:39,139 --> 00:03:42,422
对数据进行平铺、展开、向量化等

95
00:03:42,422 --> 00:03:44,120
非常多的优化的手段

96
00:03:44,120 --> 00:03:47,160
为的就是充分地利用硬件的性能

97
00:03:47,160 --> 00:03:49,240
而不改变算法本身的设计

98
00:03:49,280 --> 00:03:51,000
从而提升运算的效率

99
00:03:51,000 --> 00:03:53,560
那右边的这个就是高斯滤波

100
00:03:53,560 --> 00:03:55,640
一个具体的展开的计算公式

101
00:03:55,640 --> 00:03:58,480
可以看到这里面做了很多的分片

102
00:03:58,480 --> 00:04:00,160
这里面确实做了很多的分片

103
00:04:00,160 --> 00:04:02,000
没有刚才两个大循环了

104
00:04:02,280 --> 00:04:04,960
而且还对数据进行一个展开

105
00:04:04,960 --> 00:04:06,640
展开成buffer 然后去计算

106
00:04:06,640 --> 00:04:09,400
那这种方式就是单独的做优化

107
00:04:09,400 --> 00:04:11,480
叫做调度的优化

108
00:04:12,840 --> 00:04:14,920
现在来看看一个新的概念

109
00:04:14,920 --> 00:04:16,640
叫做调度空间

110
00:04:16,680 --> 00:04:18,400
算子的调度 具体的执行

111
00:04:18,400 --> 00:04:20,000
所有的可能的调度方式

112
00:04:20,000 --> 00:04:21,360
称为调度空间

113
00:04:21,360 --> 00:04:24,920
但是调度空间是非常的大的

114
00:04:24,920 --> 00:04:26,640
AI编译器出现的目的

115
00:04:26,840 --> 00:04:28,647
就是为了给算子

116
00:04:28,647 --> 00:04:30,360
提供一种最优的调度方式

117
00:04:30,360 --> 00:04:31,600
使得这个算子

118
00:04:31,760 --> 00:04:35,280
在硬件上面运行的时间最少最优

119
00:04:35,280 --> 00:04:37,680
这个点就是AI编译器后端

120
00:04:37,680 --> 00:04:40,320
要实现的一个功能或者它的目标

121
00:04:41,440 --> 00:04:43,840
下面来看看新的概念

122
00:04:43,840 --> 00:04:45,560
叫做调度树

123
00:04:45,760 --> 00:04:48,000
假设现在有一个卷积算子

124
00:04:48,000 --> 00:04:49,387
但是这个卷积算子

125
00:04:49,387 --> 00:04:51,120
实现起来是非常复杂的

126
00:04:51,120 --> 00:04:52,920
它基于一个高维的张量

127
00:04:52,920 --> 00:04:54,000
进行一个迭代

128
00:04:54,000 --> 00:04:57,400
所以会有非常（多）的for去进行嵌套

129
00:04:59,040 --> 00:05:01,200
在神经网络里面这种方式

130
00:05:01,360 --> 00:05:04,160
叫做多重循环这种特征

131
00:05:04,160 --> 00:05:05,684
第二个特征

132
00:05:05,684 --> 00:05:07,320
就是神经网络里面的算子

133
00:05:07,320 --> 00:05:09,120
是没有复杂的控制流的

134
00:05:09,120 --> 00:05:12,404
更多的是对高维张量的数据

135
00:05:12,404 --> 00:05:13,280
进行计算

136
00:05:13,320 --> 00:05:16,400
所以说它一共有三个主要的特征

137
00:05:16,400 --> 00:05:18,440
针对这三个主要的特征

138
00:05:19,000 --> 00:05:22,000
又把它进行进一步的抽象

139
00:05:22,000 --> 00:05:24,589
假设下面还是刚才

140
00:05:24,589 --> 00:05:26,920
高斯滤波的一个程序

141
00:05:26,920 --> 00:05:30,240
现在把它抽象起来三种结构代码

142
00:05:30,240 --> 00:05:31,880
第一种就是内存的分配

143
00:05:31,880 --> 00:05:33,640
就定义了一个内存

144
00:05:33,640 --> 00:05:35,815
接着像这种for这种

145
00:05:35,815 --> 00:05:37,120
叫做循环

146
00:05:37,120 --> 00:05:39,583
最底层的这种等号 赋值的

147
00:05:39,583 --> 00:05:40,720
叫做计算

148
00:05:40,760 --> 00:05:44,800
可以看到其实大部分的算子的调度方式

149
00:05:44,960 --> 00:05:46,589
都是由

150
00:05:46,589 --> 00:05:50,080
内存分配 循环 计算 去实现的

151
00:05:50,840 --> 00:05:52,640
有了这个基础概念之后

152
00:05:52,800 --> 00:05:55,400
现在就定义了一个Schedule Trees

153
00:05:55,400 --> 00:05:56,520
就是调度树

154
00:05:56,520 --> 00:05:58,960
调度树有四种节点去组成

155
00:05:58,960 --> 00:06:00,160
第一种是根节点

156
00:06:00,160 --> 00:06:01,160
根节点很明确

157
00:06:01,480 --> 00:06:02,360
一棵树

158
00:06:02,360 --> 00:06:03,520
一棵语法树

159
00:06:03,520 --> 00:06:04,560
一棵抽象树

160
00:06:04,560 --> 00:06:05,880
一棵数据结构的树

161
00:06:05,880 --> 00:06:07,200
肯定会有一个根

162
00:06:07,200 --> 00:06:08,920
接着会有一个loop的节点

163
00:06:08,920 --> 00:06:10,880
就是刚才的循环节点

164
00:06:10,880 --> 00:06:12,880
接着会有一个存储的节点

165
00:06:12,880 --> 00:06:15,000
刚才的内存分配

166
00:06:15,000 --> 00:06:17,400
然后还有一个计算的节点

167
00:06:17,400 --> 00:06:18,520
四个节点

168
00:06:18,520 --> 00:06:19,320
四个节点之后

169
00:06:19,480 --> 00:06:21,760
就可以组成调度树了

170
00:06:21,760 --> 00:06:22,880
就Schedule Tree

171
00:06:25,480 --> 00:06:28,320
下面根据代码来去看看

172
00:06:28,320 --> 00:06:31,000
四个节点具体是怎么相关的

173
00:06:31,040 --> 00:06:32,440
首先在执行的时候

174
00:06:32,560 --> 00:06:34,720
肯定需要分配一个内存

175
00:06:34,880 --> 00:06:36,920
这种以正方形来去确认

176
00:06:36,960 --> 00:06:39,600
接着会有很多的循环迭代

177
00:06:39,600 --> 00:06:40,960
叫做loop的节点

178
00:06:40,960 --> 00:06:41,680
那loop节点

179
00:06:41,800 --> 00:06:43,800
用圆形来去代替

180
00:06:44,080 --> 00:06:45,560
真正的loop完之后

181
00:06:45,560 --> 00:06:46,560
就是循环完之后

182
00:06:46,840 --> 00:06:49,200
就会来到了计算节点

183
00:06:49,400 --> 00:06:50,160
这种计算

184
00:06:50,320 --> 00:06:52,120
就是对数据实际的计算

185
00:06:52,120 --> 00:06:55,160
计算用菱形去代替 菱形

186
00:06:55,160 --> 00:06:57,000
所以说通过这个调度树

187
00:06:57,160 --> 00:06:58,640
可以很好的去表达

188
00:06:58,640 --> 00:07:01,200
算子的原理的概念

189
00:07:01,520 --> 00:07:03,240
现在提一个问题

190
00:07:03,240 --> 00:07:04,000
就是

191
00:07:04,280 --> 00:07:06,760
如何理解调度树的语义

192
00:07:06,800 --> 00:07:08,280
就怎么解析调度树

193
00:07:08,480 --> 00:07:09,600
其实很简单

194
00:07:10,240 --> 00:07:11,600
对整个调度树

195
00:07:11,760 --> 00:07:13,320
进行一个优先的遍历

196
00:07:13,320 --> 00:07:14,120
不管是广度优先（BFS）

197
00:07:14,120 --> 00:07:14,920
还是深度优先（DFS）

198
00:07:15,320 --> 00:07:16,480
都可以很方便的

199
00:07:16,480 --> 00:07:18,840
把它转成对应的程序代码

200
00:07:19,120 --> 00:07:20,960
有了调度树之后

201
00:07:21,200 --> 00:07:24,360
就能够把算子的调度的方式

202
00:07:24,360 --> 00:07:25,280
把它拆出来

203
00:07:25,280 --> 00:07:27,960
然后用一个调度树来去表示

204
00:07:28,880 --> 00:07:31,320
现在AI编辑器的后端的目的

205
00:07:31,480 --> 00:07:33,440
就是去优化这个调度树

206
00:07:33,440 --> 00:07:35,800
就优化算子的调度的方式

207
00:07:35,960 --> 00:07:38,160
而算子的原理是不变的

208
00:07:39,240 --> 00:07:41,360
在算子原理不变的前提下

209
00:07:41,560 --> 00:07:43,400
去优化调度树

210
00:07:45,400 --> 00:07:47,480
而对调度树怎么优化

211
00:07:47,600 --> 00:07:50,080
这里简单的抛出几个概念

212
00:07:50,080 --> 00:07:51,680
将会在下一节里面

213
00:07:51,960 --> 00:07:53,440
详细的展开的

214
00:07:53,440 --> 00:07:55,200
就是调度的转换

215
00:07:55,600 --> 00:07:56,360
有了调度树

216
00:07:56,520 --> 00:07:58,560
刚才就是schedule trees

217
00:07:58,960 --> 00:08:00,680
现在需要去了解一下

218
00:08:00,680 --> 00:08:02,360
调度的一些转换的方式

219
00:08:02,600 --> 00:08:03,880
首先它有非常多种

220
00:08:04,120 --> 00:08:05,480
这里面有 Reorder

221
00:08:05,680 --> 00:08:07,920
就是为相同功能的计算

222
00:08:08,200 --> 00:08:10,160
去切换循环节点

223
00:08:10,520 --> 00:08:13,520
圆圆的刚才就是循环节点

224
00:08:13,520 --> 00:08:15,200
可以看到循环的方式

225
00:08:15,400 --> 00:08:17,280
或者循环的结构

226
00:08:17,440 --> 00:08:18,960
其实是可以转换的

227
00:08:18,960 --> 00:08:21,640
先遍历X 还是先遍历Y

228
00:08:21,640 --> 00:08:23,360
其实是相同的

229
00:08:23,800 --> 00:08:25,720
接着再看看另外一个例子

230
00:08:25,720 --> 00:08:27,760
就是更改计算中间的结果

231
00:08:27,760 --> 00:08:29,480
叫做hoist compute

232
00:08:29,480 --> 00:08:30,800
或者lower compute

233
00:08:31,160 --> 00:08:32,520
像右边的调度树

234
00:08:32,680 --> 00:08:35,280
可以对它的计算中间的结果的粒度

235
00:08:35,400 --> 00:08:36,400
进行控制

236
00:08:36,400 --> 00:08:38,600
就把f.x往前提了

237
00:08:38,600 --> 00:08:41,800
把g.x变成了跟f.x平级的关系

238
00:08:42,080 --> 00:08:43,480
这种其实是不影响的

239
00:08:43,480 --> 00:08:45,680
因为后面也没有用到g.x了

240
00:08:46,080 --> 00:08:48,160
所以这种更改计算的中间结果

241
00:08:48,320 --> 00:08:50,680
是不影响整个调度树的调度的

242
00:08:50,960 --> 00:08:52,760
接下来再看另外一个

243
00:08:52,760 --> 00:08:55,200
就是将内联函数的一个删除

244
00:08:55,200 --> 00:08:56,960
内存开辟了一个g

245
00:08:56,960 --> 00:08:58,320
内存开辟了一个f

246
00:08:58,320 --> 00:08:59,600
然后有三个loop

247
00:08:59,600 --> 00:09:00,720
就三个循环

248
00:09:00,720 --> 00:09:02,120
或者三个嵌套的迭代

249
00:09:02,600 --> 00:09:04,520
从左边的这个图可以看到了

250
00:09:04,560 --> 00:09:07,880
f.x把它变成了这里面的一个计算

251
00:09:07,880 --> 00:09:09,520
g.x跟h.x这个loop

252
00:09:09,680 --> 00:09:11,240
把它往前提

253
00:09:11,520 --> 00:09:14,600
这种叫做inline或者deinline

254
00:09:15,040 --> 00:09:16,920
把内联函数直接展开

255
00:09:16,920 --> 00:09:18,720
放到具体的计算里面

256
00:09:20,240 --> 00:09:21,400
基于调度树

257
00:09:21,400 --> 00:09:23,440
去了解这些不同的调度方式

258
00:09:23,440 --> 00:09:24,320
有什么用呢

259
00:09:24,560 --> 00:09:26,320
目的就是为调度

260
00:09:26,480 --> 00:09:28,760
寻找一种最优的算法

261
00:09:28,760 --> 00:09:30,880
使得运算效率更高

262
00:09:31,320 --> 00:09:33,080
而刚才有调度树之后

263
00:09:33,160 --> 00:09:34,440
就可以对调度树的

264
00:09:34,440 --> 00:09:37,560
每一个节点的耗时算出来

265
00:09:37,560 --> 00:09:39,760
然后通过建立一个cost function

266
00:09:39,760 --> 00:09:41,000
就是代价函数

267
00:09:41,000 --> 00:09:42,360
通过建模的方式

268
00:09:42,360 --> 00:09:43,960
去寻找一条路径

269
00:09:43,960 --> 00:09:46,000
或者调度树的装配方式

270
00:09:46,000 --> 00:09:47,360
使得这个调度树

271
00:09:47,480 --> 00:09:49,160
满足cost function

272
00:09:49,160 --> 00:09:51,120
就是代价函数最优

273
00:09:51,280 --> 00:09:53,960
这种就找到一个最好的调度策略了

274
00:09:55,520 --> 00:09:57,440
当然了调度策略没有这么简单

275
00:09:57,440 --> 00:09:58,760
可能实际的调度策略

276
00:09:58,880 --> 00:10:01,360
auto scheduler是非常复杂的一个算法

277
00:10:01,480 --> 00:10:04,480
刚才只是简单的讲了一些它的原理

278
00:10:06,000 --> 00:10:07,960
今天的内容就到这里为止

279
00:10:07,960 --> 00:10:08,560
好了

280
00:10:08,560 --> 00:10:09,320
谢谢各位

281
00:10:10,080 --> 00:10:10,880
卷的不行了

282
00:10:10,880 --> 00:10:11,680
卷的不行了

283
00:10:11,680 --> 00:10:13,360
记得一键三连加关注哦

284
00:10:13,640 --> 00:10:15,160
所有的内容都会开源在

285
00:10:15,160 --> 00:10:16,560
下面这条链接里面

286
00:10:17,160 --> 00:10:17,840
拜了个拜