Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “data preprocessing”

Schematic diagram of data preprocessing.

Hongli Liu (493057), Xinmei Mao (14672261), Xuechen Wang (5831735) et al. · Figshare · 2025

<div> Background Triglyceride and glucose (TyG) indices have been used as predictors of several chronic diseases. However, there is currently a lack of research that can comprehensively reflect the impact of TyG-related indicators on chronic diseases in middle-aged and elderly populations. The aim of this study was to investigate the relationship of TyG and its related indicators with chronic diseases and their time-dependent predictive ability in the elderly. Study design Retrospective observational cohort study using China Health and Retirement Longitudinal Study (CHARLS) 2011–2020 data. Methods Based on longitudinal data obtained from the CHARLS from 2011 to 2020, a total of 12,966 participants were included in the study. Participants were stratified into three groups according to their TyG index. Pearson correlation coefficient and Cox model are used to assess the relationship between the TyG index, its parameters, and common chronic diseases, while Harrell’s C-index is used to evaluate their risk prediction capability. Results The TyG index and its related indicators exhibit a positive dose-response relationship with the risk of diabetes, heart disease, dyslipidemia, hypertension, and stroke, while demonstrating a negative dose-response relationship with digestive system diseases. Harrell’s C-index results indicated that TyG-WC demonstrates superior predictive performance overall. Conclusion The TyG index and its related indicators are significantly correlated with newly onset emerging chronic diseases, with TyG-WC exhibits superior risk prediction performance. </div>

Cited by 0Open access

An instance of data preprocessing.

Handi Sun (15332670), Liang Wan (119472), Mengying Liu (2149954) et al. · Figshare · 2023

&lt;div&gt;&lt;p&gt;Network intrusion detection plays a crucial role in ensuring network security by distinguishing malicious attacks from normal network traffic. However, imbalanced data affects the performance of intrusion detection system. This paper utilizes few-shot learning to solve the data imbalance problem caused by insufficient samples in network intrusion detection, and proposes a few-shot intrusion detection method based on prototypical capsule network with the attention mechanism. Our method is mainly divided into two parts, a temporal-spatial feature fusion method using capsules for feature extraction and a prototypical network classification method with attention and vote mechanisms. The experimental results demonstrate that our proposed model outperforms state-of-the-art methods on imbalanced datasets.&lt;/p&gt;&lt;/div&gt;

Cited by 0Open access

Ground-Penetrating Radar Data Preprocessing

O. V. Sytnik · Telecommunications and Radio Engineering · 2006

An algorithm for dynamic selection of clutter components in a signal of a videopulse ground-penetrating radar (GPR) is proposed. Clutter-resistant estimates of radio contracts are made. The robustness of these estimates has been achieved by optimal data weighting. An algorithm of suppressing anomalous bursts in a signal is proposed. The ways to subsequently elaborate the techniques for preprocessing these types of signals are discussed.

Cited by 1

Preprocessing and data analysis scripts

Graf-Vlachy, Lorenz, Wagner, Stefan, Graziotin, Daniel · Figshare · 2022

Preprocessing and data analysis scripts for: Text and Team: What Article Metadata Characteristics Drive Citations in Software Engineering? L. Graf-Vlachy, D. Graziotin, S. Wagner 25th International Conference on Evaluation and Assessment in Software Engineering (EASE) https://doi.org/10.1145/3530019.3530022

Cited by 0Open access

Preprocessing of Remotely‐Sensed Data

Paul M. Mather, Magaly Koch · 2010

This chapter contains sections titled: Introduction Cosmetic Operations Geometric Correction and Registration Atmospheric Correction Illumination and View Angle Effects Sensor Calibration Terrain Effects Summary

Cited by 4

Data Preprocessing For Supervised Leaning

Sotiris Kotsiantis, Dimitris Kanellopoulos, Panagiotis Pintelas · Zenodo (CERN European Organization for Nuclear Research) · 2007

Many factors affect the success of Machine Learning (ML) on a given task. The representation and quality of the instance data is first and foremost. If there is much irrelevant and redundant information present or noisy and unreliable data, then knowledge discovery during the training phase is more difficult. It is well known that data preparation and filtering steps take considerable amount of processing time in ML problems. Data pre-processing includes data cleaning, normalization, transformation, feature extraction and selection, etc. The product of data pre-processing is the final training set. It would be nice if a single sequence of data pre-processing algorithms had the best performance for each data set but this is not happened. Thus, we present the most well know algorithms for each step of data pre-processing so that one achieves the best performance for their data set.

Cited by 643Open access

Preprocessing of Field Spectrometric Data

D. Borisova, Doyno Petkov, V. Atanassov et al. · Proceedings · 2013

Remote sensing using geophysical principles begins with the development and design of equipment for performing research of objects remotely and without disturbing their integrity. In geological remote sensing studies the determination of the chemical/mineral composition and the structure of the objects within the field of view of the instrument either obtained in the lab, with a field spectrometer, or with a remote sensor is a main goal. The aim of this study is to present and test the procedure of preprocessing of the field spectrometric data. Field spectrometric studies of rocks were made to collect spectral signatures of different rock types for the reliable detection and identification of their mineral and chemical composition. The experiments are based on major physical principles such as light scattering, absorption of light, and reflection of light in the electromagnetic spectrum (EMS). Field spectral measurements were made with Thematically Oriented Multi-channel Spectrometer designed and constructed in Remote Sensing Systems Department at SRTI-BAS. The spectrometer with increased spectral resolution works in (400-900) nm range of EMS. The results are compared with similar data from spectral libraries. They correspond to the shape of reflectance spectra in the same range of EMS obtained with other spectrometers.

Cited by 0

Data set after experimental preprocessing

Qian, Lu · ScienceDB · 2023

ACE2005 English dataset contains various types of data composed of entity, relationship, and event annotation, and supports English, Arabic, and Chinese. The dataset annotates 16,375 sentences in 599 documents. The data sources of the documents mainly come from the following six fields: Newswire, radio news, radio talks, blogs, newsgroups, and telephone conversations. The submitted data preprocessed ACE2005 data set, added dependency analysis results, and divided it into three data sets: training set, test set, and verification set.

Cited by 0Open access

fPrep: Preprocessing of fMRI data.

Morton, Neal W · Zenodo (CERN European Organization for Nuclear Research) · 2022

This version adds a Zenodo configuration for easier archiving.

Cited by 0Open access

The data after experimental preprocessing

Yao, Liu, Yani, Wu, xiao zheng · ScienceDB · 2025

Preprocessed experimental data of the paper "Research on the Construction of Cognitive Graph and Generation of Vector Knowledge Base in the Medical Field", including knowledge graphs, knowledge graph term matrices, entity-attribute classification relationships, and results of concept indexing and entity extraction.

Cited by 0Open access

Data collection and preprocessing flow.

Ting Liang (506921), Shuang Li (146392) · Figshare · 2025

<div> Mobile payment systems have experienced rapid growth, but accurate forecasting remains challenging due to market dynamics and complex adoption factors. This paper proposes a Hybrid ARIMA-LSTM-Transformer model that combines time series forecasting, sequential learning, and attention mechanisms to address these challenges. Experimental results across five datasets demonstrate our model’s superior performance with MAE of 0.075, RMSE of 0.121, and R2 score of 0.948, outperforming traditional approaches. The model’s high accuracy and adaptability make it valuable for real-world applications in digital economy planning and mobile payment market analysis. </div>

Cited by 0Open access

Data Preprocessing for Supervised Learning

S. B. Kotsiantis, et al. · 2006

Many factors affect the success of Machine Learning (ML) on a given task. The representation and quality of the instance data is first and foremost. If there is much irrelevant and redundant information present or noisy and unreliable data, then knowledge discovery during the training phase is more difficult. It is well known that data preparation and filtering steps take considerable amount of processing time in ML problems. Data pre-processing includes data cleaning, normalization, transformation, feature extraction and selection, etc. The product of data pre-processing is the final training set. It would be nice if a single sequence of data pre-processing algorithms had the best performance for each data set but this is not happened. Thus, we present the most well know algorithms for each step of data pre-processing so that one achieves the best performance for their data set.

Cited by 0

Data sources and preprocessing procedures.

Yuan Han (1267323), Wenhu Jin (22608205), Heng Liu (293917) et al. · Figshare · 2025

<div> This study addresses the inefficiencies in ecological restoration on the Qinghai-Tibet Plateau (QTP), particularly concerning prolonged vegetation restoration cycles, slow soil quality improvement, and difficulties in quantifying manual intervention measures. An integrated Cubist regression tree model is developed using ecological environment data from the QTP and multi-source environmental monitoring data from 2019 to 2023. The model combines a lightweight self-attention mechanism (SA) with bidirectional gated recurrent units (BiGRU) to enhance the accuracy and adaptability of restoration efficiency prediction. The SA mechanism dynamically adjusts environmental factor weights to strengthen nonlinear relationship capture capabilities, while the BiGRU learner optimizes temporal feature representation to accommodate spatiotemporal variability in restoration processes. Input factors include fractional vegetation cover (FVC), temperature, precipitation, soil moisture, and manual intervention measures (irrigation volume, planting density), with outputs being vegetation restoration rate and soil quality improvement effects. Experimental results demonstrate that the model achieves less than 5% error in vegetation restoration rate prediction, with correlation coefficients exceeding 0.90, and 96% accuracy in soil improvement prediction. Temperature and precipitation show contribution rates of 32% and 25%, respectively, while soil moisture and NDVI jointly contribute 25%. Prediction accuracy remains above 90% across different altitude zones, indicating strong regional adaptability. Notably, in areas with annual precipitation below 200 millimeters, every 10% increase in irrigation volume leads to approximately 15% improvement in vegetation survival rate. This study provides quantitative and operational intervention guidelines for plateau ecological restoration, enhances the evaluation efficiency of manual intervention measures, and has significant practical application value. </div>

Cited by 0Open access

Preprocessing of Alarm Data for Data Mining

Zahra Mannani, Iman Izadi, Nasser Ghadiri · Industrial & Engineering Chemistry Research · 2019

Many industries, including process industry, face an increased number of alarms every day. This is due to advanced computer-based monitoring and control technologies that are widely available in all industrial plants. On the other hand, data mining, as a method of finding patterns in data, has been widely used to discover patterns and relationships in alarm data, in hopes of reducing the volume of alarms and operators’ workload. One of the first steps in data mining is to prepare and cleanup raw data for better mining, also known as preprocessing. In this paper, we focus on preprocessing of alarm data and investigate the steps required for data preparation. Two steps—namely, removing chattering alarms and reconstruction of missing alarms—are more challenging. For chattering alarms, many algorithms are proposed with a discussion on the time frame that should be selected for removing chattering alarms. As for the reconstruction of missing alarms, two methods are presented, using information from the same alarm tag or other related alarms. A case study shows the efficiency of the proposed methods.

Cited by 18