End-to-end neural network based video coding

Systems and techniques for processing video data using a neural network system are described herein. For example, a process may include generating, by a first convolutional layer of an encoder sub-network of a neural network system, an output value associated with a luma channel of a frame. The process may include generating, by a second convolutional layer of the encoder sub-network, an output value associated with at least one chroma channel of the frame. The process may include generating a combined representation of the frame by combining an output value associated with a luma channel of the frame and an output value associated with at least one chroma channel of the frame. The process may include generating encoded video data based on the combined representations of the frames. 本文描述了用于使用神经网络系统处理视频数据的系统和技术。例如，过程可以包括：由神经网络系统的编码器子网络的第一卷积层生成与帧的亮度通道相关联的输出值。该过程可以包括由编码器子网络的第二卷积层生成与帧的至少一个色度通道相关联的输出值。该过程可以包括通过对与帧的亮度通道相关联的输出值和与帧的至少一个色度通道相关联的输出值进行组合来生成帧的经组合表示。该过程可以包括基于帧的经组合表示生成经编码视频数据。