Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “data preprocessing”

DNA microarray data preprocessing

Sampsa Hautaniemi, Antti Lehmussola, Olli Yli‐Harja · 2004

Microarray technology is an essential part of modern biomedical research. At the moment, the most widely utilized microarray platform in academic biomedical research is DNA microarray due to its flexibility and low costs. One of the biggest drawback of the DNA microarray technology is high variation in data quality. Thus, in order to obtain reliable results, sophisticated image analysis and extensive preprocessing steps should be applied before actual microarray data analysis can take place. In this study we highlight some essential microarray preprocessing steps.

Cited by 4

Data preprocessing flow-chart.

Anant Shinde (3954662), Sovesh Mohapatra (9635810), Gottfried Schlaug (343052) · Figshare · 2023

<div><p>Transcranial direct current stimulation (tDCS) can noninvasively modulate behavior, cognition, and physiologic brain functions depending on polarity and dose of stimulation as well as montage of electrodes. Concurrent tDCS-fMRI presents a novel way to explore the parameter space of non-invasive brain stimulation and to inform the experimenter as well as the participant if a targeted brain region or a network of spatially separate brain regions has been engaged and modulated. We compared a multi-electrode (ME) with a single electrode (SE) montage and both active conditions with a no-stimulation (NS) control condition to assess the engagement of a brain network and the ability of different electrode montages to modulate network activity. The multi-electrode montage targeted nodal regions of the right Arcuate Fasciculus Network (AFN) with anodal electrodes placed over the skull position of the posterior superior temporal/middle temporal gyrus (STG/MTG), supramarginal gyrus (SMG), posterior inferior frontal gyrus (IFG) and a return cathodal electrode over the left supraorbital region. In comparison, the single electrode montage used only one anodal electrode over a nodal brain region of the AFN, but varied the location between STG/MTG, SMG, and posterior IFG for different participants. Whole-brain rs-fMRI was obtained every three seconds. The tDCS-stimulator was turned on at 3 minutes after the scanning started. A 4D rs-fMRI data set was converted to dynamic functional connectivity (DFC) matrices using a set of ROI pairs belonging to the AFN as well as other unrelated brain networks. In this study, we evaluated the performance of five algorithms to classify the DFC matrices from the three conditions (ME, SE, NS) into three different categories. The highest accuracy of 0.92 was obtained for the classification of the ME condition using the K Nearest Neighbor (KNN) algorithm. In other words, applying the classification algorithm allowed us to identify the engagement of the AFN and the ME condition was the best montage to achieve such an engagement. The top 5 ROI pairs that made a major contribution to the classification of participant’s rs-fMRI data were identified using model performance parameters; ROI pairs were mainly located within the right AFN. This proof-of-concept study using a classification algorithm approach can be expanded to create a near real-time feedback system at a participant level to detect the engagement and modulation of a brain network that spans multiple brain lobes.</p></div>

Cited by 0Open access

The data preprocessing steps.

Tehnan I. A. Mohamed (16846175), Olaide N. Oyelade (14047002), Absalom E. Ezugwu (4031198) · Figshare · 2023

<div><p>Recently, research has shown an increased spread of non-communicable diseases such as cancer. Lung cancer diagnosis and detection has become one of the biggest obstacles in recent years. Early lung cancer diagnosis and detection would reliably promote safety and the survival of many lives globally. The precise classification of lung cancer using medical images will help physicians select suitable therapy to reduce cancer mortality. Much work has been carried out in lung cancer detection using CNN. However, lung cancer prediction still becomes difficult due to the multifaceted designs in the CT scan. Moreover, CNN models have challenges that affect their performance, including choosing the optimal architecture, selecting suitable model parameters, and picking the best values for weights and biases. To address the problem of selecting optimal weight and bias combination required for classification of lung cancer in CT images, this study proposes a hybrid metaheuristic and CNN algorithm. We first designed a CNN architecture and then computed the solution vector of the model. The resulting solution vector was passed to the Ebola optimization search algorithm (EOSA) to select the best combination of weights and bias to train the CNN model to handle the classification problem. After thoroughly training the EOSA-CNN hybrid model, we obtained the optimal configuration, which yielded good performance. Experimentation with the publicly accessible Iraq-Oncology Teaching Hospital / National Center for Cancer Diseases (IQ-OTH/NCCD) lung cancer dataset showed that the EOSA metaheuristic algorithm yielded a classification accuracy of 0.9321. Similarly, the performance comparisons of EOSA-CNN with other methods, namely, GA-CNN, LCBO-CNN, MVO-CNN, SBO-CNN, WOA-CNN, and the classical CNN, were also computed and presented. The result showed that EOSA-CNN achieved a specificity of 0.7941, 0.97951, 0.9328, and sensitivity of 0.9038, 0.13333, and 0.9071 for normal, benign, and malignant cases, respectively. This confirms that the hybrid algorithm provides a good solution for the classification of lung cancer.</p></div>

Cited by 0Open access

Preprocessing of GPR data

M. Szymczyk, P. Szymczyk · Image Processing & Communications · 2013

Abstract In this article a set of procedures for data preprocessing of GPR radargrams are presented. Raw data taken from GPR are affected by different noises and instability of equipment. The data in this form, are not suitable for the further analysis. They must undergo a set of transformations in order to obtain indispensable information.

Cited by 28Open access

Data Acquisition and Preprocessing

Riitta Hari, Aina Puce · 2017

Abstract This chapter introduces principles of MEG/EEG data acquisition and preprocessing. MEG and EEG are recorded as analog (continuous-value) signals, amplified, and then filtered to remove undesirable components with lower and higher frequencies relative to the signals of interest. The analog signals are transformed to digital form by sampling (also called digitization) at rates that avoid aliasing, a very serious signal distortion described in this chapter; the appropriate sampling frequencies are at least 2 times the highest frequency in the sampled data (so-called Nyquist criterion). Different filter types (lowpass, highpass, bandpass, and notch) and their cut-off frequencies and roll-offs are examined. Examples of pitfalls that can occur with incorrectly selected filters and sampling rates are presented.

Cited by 3

Data Exploration and Preprocessing

2020

This chapter explains how to use NumPy, Pandas, and Scikit-learn to explore data, perform common feature engineering tasks, and select the features that a person will use to train his/her models. Information on the correlation between input features and the target can be helpful in picking out the best features from the data to use for model building and predictions. Information on the correlation between the features themselves can be helpful in reducing the number of features and the general risk of overfitting. Principal component analysis is a statistical technique that can be used to reduce the number of features in the dataset yet maintain a large percentage of the variance of the original data. Finally, the chapter deals with a brute-force technique called recursive feature elimination, which can be used to assess the importance of features in the performance of a machine-learning model.

Cited by 1

Data Preprocessing and Cleaning

Scognamiglio, Fulvio · Open Science Framework · 2025

This stage covers the preprocessing and cleaning of all raw OpenAlex metadata retrieved in Stage 1. The objective is to create a unified, high-quality, and analysis-ready dataset by merging journal-level files, removing paratext, diagnosing and resolving missing or inconsistent metadata, and performing rigorous deduplication using DOIs, exact and fuzzy titles, and metadata cross-checking. This stage also includes the construction and standardisation of two relational tables—Authors and Institutions—derived from OpenAlex identifiers, as well as the harmonisation of all related metadata across the dataset. The workflow follows a preregistered, transparent, and literature-informed methodology to ensure reproducibility and provide a clean, consistent foundation for the subsequent data enrichment stage.

Cited by 0Open access

Data sources and preprocessing.

Di Wang (329735), Haizhong Qian (21370128) · Figshare · 2025

<div> Scenic area attractiveness is a core factor in urban tourism development. Developments in social media and multi-source spatiotemporal data provide a basis for studying complex tourist behaviors, overcoming the limitations of traditional interview survey data. This study combines point of interest (POI), mobile signaling, and microblog check-in data to analyze scenic area popularity in Dali using kernel density analysis, hotspot analysis, and gravity models. It also uses ROST-CM6 to perform sentiment analysis on microblog check-in and text data to obtain tourist satisfaction, and combines the popularity and satisfaction to assess scenic area attractiveness. Additionally, GeoDetector is used to examine the impact of subjective human factors, objective factors of the attractions themselves, and the number of POI facilities around the attractions on the scenic area attractiveness in Dali. We obtained several key findings. First, the distribution of scenic areas in Dali City showed a two-center, multi-point pattern, including two core scenic areas (i.e., Dali Ancient City and Xizhou Ancient Town) and numerous scattered areas. Second, the majority of scenic areas in Dali City were more active in the daytime than at night, whereas Dali Ancient City was most active at night. Tourists in Dali City mostly came from Yunnan Province, neighboring provinces, and economically developed coastal regions. Third, a text-based sentiment analysis revealed numerous high-frequency adjectives reflecting positive sentiment, indicating high scenic area satisfaction. Fourth, the number of internal POIs had the greatest effects on scenic area popularity and attractiveness. Specifically, the more POIs, the more popular and attractive the scenic area. The interactive decision-making power of various factors was greater than the decision-making power of individual factors. These findings provide insight into the determinants of scenic area satisfaction, providing a basis for the development of urban tourism. </div>

Cited by 0Open access

Preprocessing and data augmentation.

Yunrui Shang (19923705), Qi Peng (702503), Zixuan Wu (4284007) et al. · Figshare · 2024

&lt;div&gt;&lt;p&gt;The relation between emotions and music is substantial because music as an art can evoke emotions. Music emotion recognition (MER) studies the emotions that music brings in the effort to map musical features to the affective dimensions. This study conceptualizes the mapping of music and emotion as a multivariate time series regression problem, with the aim of capturing the emotion flow in the Arousal-Valence emotional space. The Efficient Net-Music Informer (ENMI) Network was introduced to address this phenomenon. The ENMI was used to extract Mel-spectrogram features, complementing the time series data. Moreover, the Music Informer model was adopted to train on both time series music features and Mel-spectrogram features to predict emotional sequences. In our regression task, the model achieved a root mean square error (RMSE) of 0.0440 and 0.0352 in the arousal and valence dimensions, respectively, in the DEAM dataset. A comprehensive analysis of the effects of different hyperparameters tuning was conducted. Furthermore, different sequence lengths were predicted for the regression accuracy of the ENMI Network on three different datasets, namely the DEAM dataset, the Emomusic dataset, and the augmented Emomusic dataset. Additionally, a feature ablation on the Mel-spectrogram features and an analysis of the importance of the various musical features in the regression results were performed, establishing the effectiveness of the model presented herein.&lt;/p&gt;&lt;/div&gt;

Cited by 0Open access

Preprocessing of clinical data

Broad Institute TCGA Genome Data Analysis Center · The Broad Institute - Data Science Platform · 2013

Cited by 0Open access