Ibnu Mansyur Hamdani, Nurhidayat Nurhidayat, Andi Sumar Karman et al. · Intisari Jurnal Inovasi Pengabdian Masyarakat · 2024
Pengabdian kepada masyarakat merupakan salah satu bentuk tridarma perguruan tinggi yang bertujuan memberikan kontribusi nyata dalam peningkatan kualitas sumber daya manusia. Program Studi Teknik Komputer, Institut Teknologi dan Sains Muhammadiyah Kolaka Utara melaksanakan kegiatan pengabdian masyarakat secara daring melalui Zoom. Kegiatan ini difokuskan pada teknologi, mengangkat tema Data Science dan Data Preprocessing, relevan dengan mata kuliah Big Data. Pemateri berasal dari Akademi Komunitas Industri Manufaktur Bantaeng dengan keahlian di bidang Artificial Intelligence dan Cloud Computing. Tujuan kegiatan ini adalah meningkatkan pemahaman mahasiswa tentang konsep Data Science dan Data Preprocessing dalam pengelolaan dan analisis data di era teknologi informasi yang pesat. Metode pelaksanaan mencakup edukasi, pelatihan, dan interaksi aktif antara pemateri dan peserta. Hasil kegiatan menunjukkan peningkatan pemahaman mahasiswa, dengan nilai rata-rata pretest 67,87 meningkat menjadi 81,73 pada posttest. Kegiatan ini tidak hanya memperkaya pengetahuan mahasiswa tetapi juga mempersiapkan mereka untuk menghadapi tantangan di dunia kerja yang semakin kompleks dan berbasis data, serta mendukung perkembangan ilmu pengetahuan dan teknologi. Dengan demikian, kegiatan ini memberikan manfaat signifikan dalam meningkatkan kompetensi mahasiswa dan berkontribusi pada kemajuan pendidikan tinggi.
Cited by 2Open access
Luyao Wang (3958562), Hong Fan (146238), Yankun Wang (334944) · Figshare · 2018
<p>(a) the original night-time light data in Guiyang during September, 2016; (b) the original data with abnormal values; (c) the data after data preprocessing and the abnormal points in a and b were eliminated.</p>
Cited by 0Open access
Peiyu Li, Tian Zhao, Yanfang Yang et al. · Electronics · 2025
Trajectory data from GPS and sensors are increasingly available, necessitating effective preprocessing techniques for data mining. To systematically review the methods and models for trajectory data preprocessing, we conducted a systematic literature search in IEEE Xplore, Association for Computing Machinery Digital Library (ACM DL), Scopus, Web of Science, and Transport Research International Documentation published over the past several decades, using keywords related to trajectory data preprocessing. The studies were screened and selected based on predefined inclusion and exclusion criteria. We included 138 studies, summarizing techniques in data cleaning, compression, segmentation, and map matching. Key algorithms and their performance are compared. This review synthesizes current preprocessing methods and identifies future research directions, including real-time processing, semantic labeling, and privacy protection.
Cited by 0Open access
Huiyi Wu · 2025
Cited by 0Open access
Hui Liu, Yanfei Li, Zhu Duan · Engineering Applications of Computational Methods · 2025
Cited by 0
Fiona Draxler · Zenodo (CERN European Organization for Nuclear Research) · 2025
This presentation presents examples of data matching in practice. As a first example, it presents a case study on the effectiveness of COVID-19 vaccinations. It covers technical and privacy considerations for matching spatial data with surveys, challenges and potential applications. It also discusses how traditional data matching is changing with larger volumes of (unstructured) data (Big Data Perspective), and Language Models for data matching. This presentation is the fourth part of a set of presentations on the topic of "Tools and Workflows (Linked Data)". Presentation available after free registration: https://elearning.gesis.org/course/section.php?id=1274
Cited by 0Open access
Hui Liu, Yanfei Li, Zhu Duan · Engineering Applications of Computational Methods · 2025
Cited by 0
Shuvo Biswas (21728782), Sajeeb Saha (15840170), Muhammad Shahin Uddin (21728785) et al. · Figshare · 2025
<div> Skin cancer (SC) is the most prominent form of cancer in humans, with over 1 million new cases reported worldwide each year. Early identification of SC plays a crucial role in effective treatment. However, protecting patient data privacy is a major concern in medical research. Therefore, this study presents a smart framework for classifying SC leveraging deep learning (DL), federated learning (FL) and explainable AI (XAI). We tested the presented framework on two well-known datasets, ISBI2016 and ISBI2017. The data was first preprocessed by several techniques: resizing, normalization, balancing, and augmentation. Six advanced DL algorithms—VGG16, Xception, DenseNet169, InceptionV3, MobileViT, and InceptionResNetV2—were applied for classification tasks. Among these, the DenseNet169 algorithm obtained the highest accuracy of 83.3% in ISBI2016 and 92.67% in ISBI2017. All models were then tested in an FL platform to maintain data privacy. In the FL platform, the VGG16 algorithm showed the best results, with 92.08% accuracy on ISBI2016 and 94% on ISBI2017. To ensure model interpretability, an XAI-based algorithm named Local Interpretable Model-Agnostic Explanations (LIME) was used to explain the predictions of the proposed framework. We believe the proposed framework offers a dependable tool for SC diagnosis while protecting sensitive medical data. </div>
Cited by 0Open access
Pamela N. Luna (9767753), Jonathan M. Mansbach (4347571), Chad A. Shaw (9148615) · Figshare · 2020
Cited by 0Open access
Gábor Makrai, Zoltán Prekopcsák · 2011
We introduce a user-friendly graphical data preprocessing application based on Hive, one of the well known opensource distributed warehouse systems. It is comfortable, and easy to use for preprocessing purposes, but to prove usability of this application, we created measurement a framework to ensure precise results. These results show that our application has outstanding scaling capability in the case of increasing data amount and increasing number of applied computers. We conclude that this is a good solution for medium and large scale data preprocessing.
Cited by 0
Takayuki Hiraoka (12369473), Takashi Kirimura (21492105), Naoya Fujiwara (2176900) · Figshare · 2025
Definition of relative entropy, data preprocessing
Cited by 0Open access
Yun Mao (494759), Yang Xiao (146335), Xia Hu (767914) · Figshare · 2025
<b>A.</b> Box plot of gene expression before normalization. The x-axis represents the samples, and the y-axis represents the gene expression values. <b>B.</b> Boxplots of gene expression after normalization. <b>C.</b> PCA dimension reduction diagram before the calibration. The x- and y-axes represent two dimensionality reductions, and the dots in the plot represent samples, with early samples in blue and late samples in red. <b>D.</b> PCA dimension reduction map after normalization. <b>E.</b> Volcano plots of the DEGs. The x-axis is log2 (fold change), the y-axis is log10 (p value), and each dot represents one gene, with downregulated genes in purple, upregulated genes in red, and black for genes whose expression did not significantly change. <b>F.</b> Heatmap of the DEGs. The top color bars represent both groups, with red indicating early expression, green indicating late expression, blue indicating low expression and red indicating high expression.
Cited by 0Open access
Adam Grabell, Manasa Kalanadhabhatta, Alex Dhima · OSF Preprints (OSF Preprints) · 2024
Cited by 0
Xu Jia (187142), Jiaojiao Peng (9891450), Junhong Lv (19162705) et al. · Figshare · 2025
Sample: Sample Name; Insert Size(bp): Insert Fragment Length (default 350 bp library); Raw Data: Raw Sequencing Data; Clean Data: Filtered Valid Data; Clean_Q20: Percentage of bases in Clean Data with a sequencing error rate < 0.01 (quality score > 20); Clean_Q30: Percentage of bases in Clean Data with a sequencing error rate < 0.001 (quality score > 30); Clean_GC(%): GC Content in Clean Data; Effective(%): Percentage of Valid Data (Clean Data) relative to Raw Data (Raw Data). (DOCX)
Cited by 0Open access
Apichat Suratanee (634281), Panita Chutimanukul (11334306), Tanapon Saelao (19787028) et al. · Figshare · 2024
<p>(A) The original raw hyperspectral data. (B) The spectral data after applying multiplicative scatter correction to reduce multiplicative effects. (C) The corrected spectral data with outlier samples removed.</p>
Cited by 0Open access
Jean Simon · HAL (Le Centre pour la Communication Scientifique Directe) · 2014
International audience
Cited by 0Open access
Кучеренко, О. А., Кучеренко, О. О., Kucherenko, O. A. et al. · ELARTU (Ternopil National Technical University) · 2023
Cited by 0
Engy Fouda · Apress eBooks · 2025
In the preface, we strive to strike a balance between utilizing the integrated development environment (IDE) and the code. In Chapters 5 and 6, we focus on the code. This chapter presents advanced data cleaning and querying methods, including the use of PROC SQL, SELECT, WHERE, WHEN, and DO loops.
Cited by 0
Dragana Stojnev, Aleksandra Stojnev Ilić · 2020
Neural networks require big amount of input data in order to be properly trained, and the output and its accuracy depend on the quality of the input dataset. Most of the images used to train these networks either contain too much or not enough information, and therefore need to be preprocessed so as to reduce or even remove the noise from them, extract useful information and remove the useless ones, or apply other techniques that improve input quality for a neural network, such as super-resolution. With suitable input provided, it will be possible to create prediction models with higher precision and better accuracy. This paper gives an overview of state-of-the-art techniques for image preprocessing for different convolutional neural networks, and describes an application that demonstrates one of them.
Cited by 3Open access
Rong-qiang Gao, Shifu Fan, Yan-lu Yan et al. · PubMed · 2004
Spectroscopic data of samples recorded by spectroscopic instruments are confused by a series of noises, and interferences, therefore the proper data preprocessing is the basis of the following spectroscopic calibration, model establishment and transference, which is very important for the achievement of accurate analytical results. This paper reports our research work, combined with NIR spectroscopic analyses of the protein contents of wheat, that is the preprocessing of NIR spectral data recorded for 66 different wheat samples by a NIR grating spectrophotometer and a NIR Fourier transform spectrometer, respectively. The preprocessing algorithm is wavelet transform with the Gaussian first and second order derivatives. Compared with the result of preprocessing by normal first order difference algorithm, the wavelet transform algorithm by Gaussian derivatives was proved to be very effective and applicable, the spectra were smoothed perfectly, noises were eliminated obviously, and the spectral sections, which include all useful information for spectral analyses, were displayed clearly. So, it is very beneficial to the following spectral calibration, model establishment and transference.
Cited by 10