Addressing data gaps in environmental sensors: a case study on temporal patterns and restoration strategies in East Kazakhstan region

2024

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.1016/j.procs.2024.08.062
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Nowadays, the monitoring systems embedded with diverse sensors within the Internet of Things play an important role in how we collect data and provide in-depth insights into human behavior, environmental trends, and different industrial processes. These sensors can gather data in real-time and due to their ability to connect through various communication interfaces, it is possible to integrate such devices into existing networks ensuring that data can be transmitted in real-time to central systems or mobile devices. However, many factors such as connectivity issues, severe weather, and sensor faults, might cause missing data, thus leading to poor decision-making. In this study, an in-depth analysis was performed on environmental meteorological data collected from May 2022 to March 2023 in the East Kazakhstan region using Vaisala WXT536 multi-parameter weather sensor. The data restoration strategy that is based on the duration of data gaps is evaluated. For short-term gaps under 30 minutes in duration, the mean and median imputation achieved higher results with a root mean squared error of 0.25 for temperature than training Random Forest algorithm which achieved 3.59, while for the extended periods of missing values, the Long Short-Term Memory model was utilized with 2.47 for the temperature. The main goal of the research is to analyze the temporal patterns of missing data and potential reasons for these failures as well as to evaluate the data restoration strategies.