Optimization of Database Operations in the Application for Text Corpus Analysis

2024

Closed access Metadata (CSL-JSON)
DOI 10.1109/opcs63516.2024.10720387
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

The user's comfortable work with large volumes of text is ensured by optimizing the web application. This optimization is primarily focused on optimizing work with the database. The SQL query language is an example of a declarative language that allows for effective query optimization.The article formalizes the optimization problem as improving the quality of application usage by increasing the user's productivity in the system. This is achieved by reducing the reaction time of the system in response to user actions.There are three main strategies for query optimization: modifying the source code of the program that interacts with the database, modifying the structure of queries, and modifying the structure of the database itself. The technologies used to create the optimized software system include: the application was written in Python using the Flask web framework. Work with the database was done using the SQLAlchemy library and PostgreSQL was chosen as the DBMS (Database Management System). To test the functionality of the optimized queries, the pytest library was used.Specific optimization examples were presented for the following situations: long execution time for any stage of a data request can be solved by changing the query structure by adding an index. Long execution time for any stage of data change can be solved by deleting the index from the query. A large number of queries within a cycle can be replaced with an equivalent query outside the loop to improve performance.The results of the computational experiments demonstrate the high efficiency of the optimization technique.