Big data DAG task flow scheduling method and system and storage medium

The invention provides a big data DAG task flow scheduling method. The method specifically comprises the steps that visual scheduling configuration is conducted on a DAG process based on nginx; an external system is accessed to the scheduling platform through a cross-platform interface schema-api, a...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: LIAN HUIQI, WU SHAOHUA, WU JIANGHUANG, ZHUANG XIAOMING, XU JIAYU, SONG ZHENGCHEN
Format: Patent
Sprache:chi ; eng
Schlagworte:
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page
container_issue
container_start_page
container_title
container_volume
creator LIAN HUIQI
WU SHAOHUA
WU JIANGHUANG
ZHUANG XIAOMING
XU JIAYU
SONG ZHENGCHEN
description The invention provides a big data DAG task flow scheduling method. The method specifically comprises the steps that visual scheduling configuration is conducted on a DAG process based on nginx; an external system is accessed to the scheduling platform through a cross-platform interface schema-api, and an instruction is transmitted to a scheduling center schema-master for further processing; in response to a scheduling algorithm preset by a scheduling center super-master, completing a DAG process, and carrying out splitting, task storage, result callback and execution notification operation; and the execution center schema-worker loads the required task engine to further complete the execution of the task and the progress tracking record, and stores the task and the progress tracking record. Through the design of each link, high performance and high expansion are brought into full play, along with the development of the Internet, the problem that new and old data, generally super-large data, are processed, int
format Patent
fullrecord <record><control><sourceid>epo_EVB</sourceid><recordid>TN_cdi_epo_espacenet_CN116069462A</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>CN116069462A</sourcerecordid><originalsourceid>FETCH-epo_espacenet_CN116069462A3</originalsourceid><addsrcrecordid>eNrjZHBzykxXSEksSVRwcXRXKEkszlZIy8kvVyhOzkhNKc3JzEtXyE0tychPUUjMS1EoriwuSc2FMEvyixLTU4GyKZmluTwMrGmJOcWpvFCam0HRzTXE2UM3tSA_PrW4IDE5NS-1JN7Zz9DQzMDM0sTMyNGYGDUAnL4yng</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>patent</recordtype></control><display><type>patent</type><title>Big data DAG task flow scheduling method and system and storage medium</title><source>esp@cenet</source><creator>LIAN HUIQI ; WU SHAOHUA ; WU JIANGHUANG ; ZHUANG XIAOMING ; XU JIAYU ; SONG ZHENGCHEN</creator><creatorcontrib>LIAN HUIQI ; WU SHAOHUA ; WU JIANGHUANG ; ZHUANG XIAOMING ; XU JIAYU ; SONG ZHENGCHEN</creatorcontrib><description>The invention provides a big data DAG task flow scheduling method. The method specifically comprises the steps that visual scheduling configuration is conducted on a DAG process based on nginx; an external system is accessed to the scheduling platform through a cross-platform interface schema-api, and an instruction is transmitted to a scheduling center schema-master for further processing; in response to a scheduling algorithm preset by a scheduling center super-master, completing a DAG process, and carrying out splitting, task storage, result callback and execution notification operation; and the execution center schema-worker loads the required task engine to further complete the execution of the task and the progress tracking record, and stores the task and the progress tracking record. Through the design of each link, high performance and high expansion are brought into full play, along with the development of the Internet, the problem that new and old data, generally super-large data, are processed, int</description><language>chi ; eng</language><subject>CALCULATING ; COMPUTING ; COUNTING ; ELECTRIC DIGITAL DATA PROCESSING ; PHYSICS</subject><creationdate>2023</creationdate><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://worldwide.espacenet.com/publicationDetails/biblio?FT=D&amp;date=20230505&amp;DB=EPODOC&amp;CC=CN&amp;NR=116069462A$$EHTML$$P50$$Gepo$$Hfree_for_read</linktohtml><link.rule.ids>230,308,776,881,25542,76289</link.rule.ids><linktorsrc>$$Uhttps://worldwide.espacenet.com/publicationDetails/biblio?FT=D&amp;date=20230505&amp;DB=EPODOC&amp;CC=CN&amp;NR=116069462A$$EView_record_in_European_Patent_Office$$FView_record_in_$$GEuropean_Patent_Office$$Hfree_for_read</linktorsrc></links><search><creatorcontrib>LIAN HUIQI</creatorcontrib><creatorcontrib>WU SHAOHUA</creatorcontrib><creatorcontrib>WU JIANGHUANG</creatorcontrib><creatorcontrib>ZHUANG XIAOMING</creatorcontrib><creatorcontrib>XU JIAYU</creatorcontrib><creatorcontrib>SONG ZHENGCHEN</creatorcontrib><title>Big data DAG task flow scheduling method and system and storage medium</title><description>The invention provides a big data DAG task flow scheduling method. The method specifically comprises the steps that visual scheduling configuration is conducted on a DAG process based on nginx; an external system is accessed to the scheduling platform through a cross-platform interface schema-api, and an instruction is transmitted to a scheduling center schema-master for further processing; in response to a scheduling algorithm preset by a scheduling center super-master, completing a DAG process, and carrying out splitting, task storage, result callback and execution notification operation; and the execution center schema-worker loads the required task engine to further complete the execution of the task and the progress tracking record, and stores the task and the progress tracking record. Through the design of each link, high performance and high expansion are brought into full play, along with the development of the Internet, the problem that new and old data, generally super-large data, are processed, int</description><subject>CALCULATING</subject><subject>COMPUTING</subject><subject>COUNTING</subject><subject>ELECTRIC DIGITAL DATA PROCESSING</subject><subject>PHYSICS</subject><fulltext>true</fulltext><rsrctype>patent</rsrctype><creationdate>2023</creationdate><recordtype>patent</recordtype><sourceid>EVB</sourceid><recordid>eNrjZHBzykxXSEksSVRwcXRXKEkszlZIy8kvVyhOzkhNKc3JzEtXyE0tychPUUjMS1EoriwuSc2FMEvyixLTU4GyKZmluTwMrGmJOcWpvFCam0HRzTXE2UM3tSA_PrW4IDE5NS-1JN7Zz9DQzMDM0sTMyNGYGDUAnL4yng</recordid><startdate>20230505</startdate><enddate>20230505</enddate><creator>LIAN HUIQI</creator><creator>WU SHAOHUA</creator><creator>WU JIANGHUANG</creator><creator>ZHUANG XIAOMING</creator><creator>XU JIAYU</creator><creator>SONG ZHENGCHEN</creator><scope>EVB</scope></search><sort><creationdate>20230505</creationdate><title>Big data DAG task flow scheduling method and system and storage medium</title><author>LIAN HUIQI ; WU SHAOHUA ; WU JIANGHUANG ; ZHUANG XIAOMING ; XU JIAYU ; SONG ZHENGCHEN</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-epo_espacenet_CN116069462A3</frbrgroupid><rsrctype>patents</rsrctype><prefilter>patents</prefilter><language>chi ; eng</language><creationdate>2023</creationdate><topic>CALCULATING</topic><topic>COMPUTING</topic><topic>COUNTING</topic><topic>ELECTRIC DIGITAL DATA PROCESSING</topic><topic>PHYSICS</topic><toplevel>online_resources</toplevel><creatorcontrib>LIAN HUIQI</creatorcontrib><creatorcontrib>WU SHAOHUA</creatorcontrib><creatorcontrib>WU JIANGHUANG</creatorcontrib><creatorcontrib>ZHUANG XIAOMING</creatorcontrib><creatorcontrib>XU JIAYU</creatorcontrib><creatorcontrib>SONG ZHENGCHEN</creatorcontrib><collection>esp@cenet</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>LIAN HUIQI</au><au>WU SHAOHUA</au><au>WU JIANGHUANG</au><au>ZHUANG XIAOMING</au><au>XU JIAYU</au><au>SONG ZHENGCHEN</au><format>patent</format><genre>patent</genre><ristype>GEN</ristype><title>Big data DAG task flow scheduling method and system and storage medium</title><date>2023-05-05</date><risdate>2023</risdate><abstract>The invention provides a big data DAG task flow scheduling method. The method specifically comprises the steps that visual scheduling configuration is conducted on a DAG process based on nginx; an external system is accessed to the scheduling platform through a cross-platform interface schema-api, and an instruction is transmitted to a scheduling center schema-master for further processing; in response to a scheduling algorithm preset by a scheduling center super-master, completing a DAG process, and carrying out splitting, task storage, result callback and execution notification operation; and the execution center schema-worker loads the required task engine to further complete the execution of the task and the progress tracking record, and stores the task and the progress tracking record. Through the design of each link, high performance and high expansion are brought into full play, along with the development of the Internet, the problem that new and old data, generally super-large data, are processed, int</abstract><oa>free_for_read</oa></addata></record>
fulltext fulltext_linktorsrc
identifier
ispartof
issn
language chi ; eng
recordid cdi_epo_espacenet_CN116069462A
source esp@cenet
subjects CALCULATING
COMPUTING
COUNTING
ELECTRIC DIGITAL DATA PROCESSING
PHYSICS
title Big data DAG task flow scheduling method and system and storage medium
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-11T06%3A17%3A19IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-epo_EVB&rft_val_fmt=info:ofi/fmt:kev:mtx:patent&rft.genre=patent&rft.au=LIAN%20HUIQI&rft.date=2023-05-05&rft_id=info:doi/&rft_dat=%3Cepo_EVB%3ECN116069462A%3C/epo_EVB%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true