基于EdgeX的舰艇大数据处理平台架构

Research on the architecture of ship big data processing platform based on EdgeX

作者单位:

中国舰船研究院，北京 100101China Ship Research and Development Academy, Beijing 100101, China

EdgeX;物联网;舰艇大数据;数据处理EdgeX; internet of things; ship big data; data processing

随着新一代信息技术和我国舰艇信息化的飞速发展，舰艇执行任务过程中产生的数据量也呈爆炸式增长，为了更好对这些数据进行收集、挖掘、分析、运用，本文在分析主流大数据平台与数据挖掘平台的优缺点基础上，结合舰艇信息化现状与国内外大数据与数据挖掘技术发展情况，对舰艇大数据处理平台架构进行研究，划分舰艇大数据平台的层次架构，提出基于EdgeX的舰艇大数据处理平台。本文所提出的舰艇大数据处理平台可以实时采集舰艇雷达、声呐信号与传感器数据，在不同的数据挖掘计算场景中，可以使用不同的计算处理引擎进行数据分析。最后通过模拟实验验证了舰艇大数据处理平台架构的可行性，为舰艇物联网技术和舰艇智能化提供平台技术支撑。The scale of data generated by ships executing tasks grows explosively with the rapid development of new generation of information technology and native ship Informationization. This article focuses on the structure of ship big data processing platform and proposes the structure of big data processing platform based on EdgeX, analysing the advantages and disadvantages of main big data platform and data mining platform, combining with the current situation of ship Informationization and big data and data mining technology.This structure of big data is used to collect the sonar signal of ship radar and the data of sensor in real time. It supports diverse computing processing engines to analyse the data dealing with different data mining computing scenarios. This article verify the feasibility of the structure of big data processing platform on the ship by a series of simulation experiments, making the progress of intelligent ship in terms of platform.

参考文献：

[1] WANG Y P E, LIN X, ADHIKARY A, et al. A primer on 3GPP narrowband internet of things (NB-IoT)[J]. IEEE Communications Magazine, 2016, 55(3)

[2] 李学龙, 龚海刚. 大数据系统综述[J]. 中国科学: 信息科学, 2015, 45(1): 1–44

[3] 赵梓铭, 刘芳, 蔡志平, 等. 边缘计算: 平台、应用与挑战[J]. 计算机研究与发展, 2018, 55(002): 327–337

[4] 温华斌. 基于Cloudlet三层结构模型的移动协同计算平台的研究与实现[D]. 哈尔滨: 哈尔滨工业大学, 2015.

[5] 潘卫军, 刘铠源, 王润东, 等. 民航空管大数据处理平台架构研究[J]. 计算机应用与软件, 2020, 37(6): 48–52, 113

[6] 戚红雨. 流式处理框架发展综述[J]. 信息化研究, 2019

[7] 郭乔进, 胡杰, 宫世杰, 等. 深度学习计算平台发展综述[J]. 信息化研究, 2019(3)

[8] ESCAMILLA-AMBROSIO P. J., RODR GUEZ-MOTA A., et al. Distributing computing in the internet of things: cloud, fog and edge computing overview[J]. Studies in Computational Intelligence, 2018: 87–115

[9] ZAHARIA M, CHOWDHURY M, DAS T, et al. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing[C]//In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, Berkeley, 2012.2.

[10] DEAN J, GHEMAWAT S. MapReduce: simplified data processing on large clusters. Commun ACM, 2008, 51: 107–113.

[11] WHITE T. Hadoop: The Definitive Guide[J]. O'rlly Media Inc Gravenstn Highway North, 2012, 215(11): 1–4

[12] GOODHOPE K, KOSHY J, KREPS J, et al. Building LinkedIn’s real-time activity data pipeline[J]. Data Engineering, 2012, 35: 33–45