Big data processing technology in a distributed information system

2021

Full text Open access: Hybrid Metadata (CSL-JSON)
DOI 10.51301/vest.su.2021.i2.35
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

Digital technologies exist in almost all areas of human life. The volume of data recorded in global warehouses increases every second, which means that the conditions for storing information change at the same rate, and new opportunities must appear to increase its volume. The introduction of big data technology is a new step in scientific and technical development in various sectors of society. Despite the fact that big data technology is at the initial stage of development, it has achieved high results in the field of Information Technology. There are no universal methods for processing big data, but they allow you to use different methods to partially solve the problem. Successful use of Big Data Increases work efficiency several times. Currently, Big Data Processing is focused on the effectiveness of using new technologies. The article discusses methods and approaches to working with big data, emerging complex problems and the importance of using big data in a distributed Information System, Functional Requirements for a distributed information system that supports scientific and educational activities, and the big data architecture that supports scientific and educational activities of a distributed information system.