Jupyterhub distributed deployment-based data reading method
The invention relates to a data reading method based on jupyterhub distributed deployment, and the method comprises the following steps: 1) making installation packages of jupyterlab and jupyterhub into mirror images, and deploying the mirror images on k8s to realize containerized distributed deploy...
Gespeichert in:
Hauptverfasser: | , , |
---|---|
Format: | Patent |
Sprache: | chi ; eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The invention relates to a data reading method based on jupyterhub distributed deployment, and the method comprises the following steps: 1) making installation packages of jupyterlab and jupyterhub into mirror images, and deploying the mirror images on k8s to realize containerized distributed deployment; (2) unified user management is carried out by adopting jupyterhub, and resource management is carried out by adopting Ks8; 3) starting a pod service, performing data modeling, judging whether remote data needs to be queried or not, if not, entering a step 4), and if yes, entering a step 5); 4) reading local modeling data; and (5) loading the distributed file by using the alluxio-fuse, loading different far-end storage systems and the data of the hdfs by using the alluxio, and carrying out data return. Compared with the prior art, the method has the advantages of being low in data reading time consumption, unified in naming management, capable of achieving resource control and isolation and the like.
本发明涉及一种基于 |
---|