【文件属性】:
文件名称:AI技术论文
文件大小:1.06MB
文件格式:PDF
更新时间:2021-09-03 10:36:18
Network AI
Network Morphism
人工智能论文2018
We present in this paper a systematic study on
how to morph a well-trained neural network to
a new one so that its network function can be
completely preserved. We define this as network
morphism in this research. After morphing
a parent network, the child network is expected
to inherit the knowledge from its parent network
and also has the potential to continue growing
into a more powerful one with much shortened
training time. The first requirement for this network
morphism is its ability to handle diverse
morphing types of networks, including changes
of depth, width, kernel size, and even subnet.
To meet this requirement, we first introduce the
network morphism equations, and then develop
novel morphing algorithms for all these morphing
types for both classic and convolutional neural
networks. The second requirement for this
network morphism is its ability to deal with nonlinearity
in a network. We propose a family of
parametric-activation functions to facilitate the
morphing of any continuous non-linear activation
neurons. Experimental results on benchmark
datasets and typical neural networks demonstrate
the effectiveness of the proposed network morphism
scheme.