Modeling the processing of a large amount of data

Journal of Mathematics Mechanics and Computer Science, 2018

Full text Open access: Gold Metadata (CSL-JSON)
DOI 10.26577/jmmcs-2018-1-490
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

The definition of large amounts of data, Big Data, is used to refer to technologies such as storing and analyzing a significant amount of data that requires high speed and real-time decision-making when processing. Typically, when serious analysis is said, especially if the term DataMining is used, hat there is a huge amount of data. There are no universal methods of analysis or algorithms suitable for any cases and any volumes of information. Data analysis methods differ significantly in performance, quality of results, usability and data requirements. Optimization can be carried out at various levels: equipment, databases, analytical platform, preparation of source data, specialized algorithms. Big data is a set of technologies that are designed to perform three operations. First, to process large amounts of data compared to "standard" scenarios. Secondly, be able to work with fast incoming data in very large volumes. That is, the data is not just a lot, but they are constantly becoming more and more. Thirdly, they must be able to work with structured and poorly structured data in parallel in different aspects. Large data suggest that the input algorithms receive a stream of not always structured information and that more can be extracted from it than any one idea. The results of the study are used by the authors in modeling large data and developing a web application.