Development of methods for receiving analytical data from different SQL/ NoSQL databases and leading to uniform format

А.А. Куандыков, Raissa Uskenbayeva, Нуржан Мукажанов

2015

Closed access Metadata (CSL-JSON)
DOI 10.1109/iccas.2015.7364840
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

In this paper unified structure formation method (ETL) development for operative analysis of data stored in different SQL and NoSQL databases are considered. Unified structure formation method (ETL) is based on work with Map Reduce parallel model. In order to perform the method following steps should be considered: development of searching forms (pattern) for data searching in different database, transforming results of searching through Map Reduce parallel model, taking pre-aggreagtional data during transforming process and preparing data for downloading to data warehouse.