基于Hadoop技术的船舶柴油机大数据处理平台

Construction of big data processing platform for marine diesel engine based on Hadoop technology

大数据;Hadoop;HDFS;柴油机;船舶;故障诊断big data; Hadoop; HDFS; diesel engine; ship; fault diagnosis

柴油机是船舶推进系统的重要组成部分，随着柴油机技术的不断发展，对柴油机相关参数进行监测并分析是必然发展趋势。柴油机的故障监测需要依赖大量的柴油机监测数据，而嵌入式处理器无法实现大量监测数据的实时采集和分析。本文提出一种基于Hadoop技术的船舶柴油机大数据处理平台，对Hadoop技术的HDFS和MapReduce的实现原理进行详细分析。在此基础上设计系统的整体结构框图，并对多文件的写入速度和数据排序进行测试。本文基于Hadoop技术的船舶柴油机大数据平台可以裸船舶柴油机数据的存储和分析需求，并且具有较高的可靠性和扩展性。Diesel engine is an important part of marine propulsion system. With the continuous development of diesel engine technology, it is an inevitable trend to monitor and analyze related parameters of diesel engine in the future. The fault monitoring of diesel engine depends on a large number of diesel engine monitoring data, and embedded processor can not realize the real-time collection and analysis of a large number of monitoring data. In this paper, a big data processing platform for marine diesel engine based on Hadoop technology is proposed, and the implementation principle of HDFS and MapReduce of Hadoop technology is analyzed in detail. On this basis, the overall structure diagram of the system is designed, and the writing speed of multiple files and data sorting are tested. The big data platform of marine diesel engine with Hadoop technology designed in this paper can meet the storage and analysis requirements of marine diesel engine data, and has high reliability and scalability.

参考文献：

[1] 张新宇, 蒋茂玎, 吴磊, 等. 基于远程监控大数据处理的柴油机SCR系统研究[J]. 内燃机, 2021(6): 10–14

ZHANG Xin-yu, JIANG Mao-qing, WU Lei, et al. Research on SCR system of diesel engine based on remote monitoring and big data processing[J]. Internal Combustion Engine, 2021(6): 10–14

[2] 石利勇. 基于船舶柴油机监测数据的能耗研究[D]. 重庆: 重庆交通大学, 2021.

[3] 赵海朋. 柴油机典型故障特征提取与智能诊断方法研究[D]. 北京: 北京化工大学, 2020.

[4] 张其俊. 船用柴油机故障诊断技术研究[J]. 中国水运, 2020(7): 128–129

ZHANG Qi-jun. Research on fault diagnosis technology of marine diesel engines[J]. China Water Transport, 2020(7): 128–129

[5] 仲国强. 基于数据驱动的船舶柴油机智能故障诊断研究[D]. 大连: 大连海事大学, 2020.

[6] 熊威, 金华标. 船舶柴油机大数据处理平台的研究与设计[J]. 内燃机, 2018(2): 9–12

XIONG Wei, JIN Hua-biao. Research and design of a big data processing platform for marine diesel engines[J]. Internal Combustion Engine, 2018(2): 9–12

[7] 孙峰, 黄连忠, 刘伊凡, 等. 一种应用数据挖掘技术评估柴油机性能的方法[J]. 大连海事大学学报, 2017, 43(3): 83–88

SUN Feng, HUANG Lian-zhong, LIU Yi-fan, et al. A method of applying data mining technology to evaluate the performance of diesel engines[J]. Journal of Dalian Maritime University, 2017, 43(3): 83–88