Automatic activation value check point searching method based on meta-information estimation

The invention discloses an automatic activation value check point searching method based on meta-information estimation, and particularly relates to the field of deep large model optimization, comprising the following steps: S1, initializing a model; s2, establishing a linearization model; s3, extra...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: YAO BOYUAN, BIAN ZHENGDA, SHAO YANJUN, MAI SIQI, LI YONGBIN, LEE, SEUNG-GYE, HUANG HAICHEN, LOU YUXUAN, LIU YULIANG, LU GUANGYANG, WU JUNMING, CHEN WEIWEN, LIU HONGXIN, FANG JIARUI
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:The invention discloses an automatic activation value check point searching method based on meta-information estimation, and particularly relates to the field of deep large model optimization, comprising the following steps: S1, initializing a model; s2, establishing a linearization model; s3, extracting meta-information; s4, dynamic planning solution is carried out; according to the method, a propagation mode of universal node characteristics is designed, part of dependence is relieved, nodes which can be regarded as linearization segmentation points in the model are increased, the chain length of a linearization network is increased, and therefore the search space is increased, meanwhile, linearization is full-automatic, and the model does not need to be rewritten. 本发明公开了基于元信息估计的自动激活值检查点搜索方法,具体涉及深度大模型优化领域,包括以下步骤:S1、初始化模型;S2、线性化模型的建立;S3、元信息抽取;S4、动态规划求解;S5、计算图优化:S6、执行,本发明通过设计通用节点特性的传播方式,解除了部分依赖,使得模型内可视为线性化分割点的节点增多,增加了线性化网络的链长,从而增加了搜索空间,同时,本发明的线性化是全自动的,不需要对模型进行改写。