Kenny Daily, et al. · 2010
Background: High-throughput sequencing (HTS) technologies play important roles in the life sciences by allowing the rapid parallel sequencing of very large numbers of relatively short nucleotide sequences, in applications ranging from genome sequencing and resequencing to digital microarrays and ChIP-Seq experiments. As experiments scale up, HTS technologies create new bioinformatics challenges for the storage and sharing of HTS data. Results: We develop data structures and compression algorithms for HTS data. A processing stage maps short sequences to a reference genome or a large table of sequences. Then the integers representing the short sequence absolute or relative addresses, their length, and the substitutions they may contain are compressed and stored using various entropy coding algorithms, including both old and new fixed codes (e.g Golomb, Elias Gamma, MOV) and variable codes (e.g. Huffman). The general methodology is illustrated and applied to several HTS data sets. Results show that the information contained in HTS files can be compressed by a factor of 10 or more, depending on the statistical properties of the data sets and various other choices and constraints. Our algorithms fair well against general purpose compression programs such as gzip, bzip2 and 7zip; timing results show that our algorithms are consistently faster than the best general purpose compression programs. Conclusions: It is not likely that exactly one encoding strategy will be optimal for all types of HTS data. Different experimental conditions are going to generate various data distributions whereby one encoding strategy can be more effective than another. We have implemented some of our encoding algorithms into the software package GenCompress which is available upon request from the authors. With the advent of HTS technology and increasingly new experimental protocols for using the technology, sequence databases are expected to continue rising in size. The methodology we have proposed is general, and these advanced compression techniques should allow researchers to manage and share their HTS data in a more timely fashion.
Cited by 0
Zipstein, Marc · OpenGrey (Institut de l'Information Scientifique et Technique) · 1990
Cited by 0
Ayoub Yahiaoui, Abderrazak Khediri, Hama Soltani et al. · Lecture notes in networks and systems · 2024
Cited by 0
Hari Mukka · 2014
Cited by 0
De la Puente Gimena, Ignacio · Academica-e (Universidad Pública de Navarra) · 2019
De acuerdo a la normativa de TFEs el repositorio no puede dar acceso a este trabajo. Para consultarlo póngase en contacto con el tutor del trabajo. Puede acceder al resumen del mismo pinchando en el pdf adjunto.
Cited by 0
Shrenik Nambiar, H. Srikanth Kamath · International Journal of Engineering & Technology · 2019
The amount of data being shared over the internet is increasing exponentially. In this digital age, where even devices like refrigerators are connected, data needs to be stored in compressed form. The compressed data should be retrieved without loss of information else the data will be deemed corrupt. As we are approaching 5G communication, the data need to be transferred over the internet at a higher rate. This cannot be achieved by older compression algorithms which has lesser compression ratio and even lesser compression and decompression speed. In this paper, an analysis of modern compression algorithms along with some older compression algorithms have been conducted. Also the implementation and comparison is conducted. The comparison was done with the help of graphs plotted using MATLAB soft- ware. The compression algorithms compared were Deflate, bzip2, Zstandard, Brotli, LZ4 and LZO. The files used for compression were taken from Canterbury and Silesia Corpus.
Cited by 0Open access
Kuo, May-chen (author) · University of Southern California Digital Library · 2015
The mocap data has been widely used in many motion synthesis applications for education, medical diagnosis, entertainment, etc. In the entertainment business, the synthesized motion can be easily ported to different models to animate virtual creatures. The richness of a mocap database is essential to motion synthesis applications. In general, the richer the collection, the higher quality the synthesized motion. Since there exists limitation on network bandwidth or storage capacity, there are constraints on the size of the mocap collection to be used. It is desirable to develop an effective compression scheme to accommodate a larger mocap data collection for higher quality motion synthesis. In order to synthesize natural and realistic motion from existing motion capture database particularly in the context of video game applications, the compression procedure enables efficient management.
Cited by 0Open access
Puente Gimena, Ignacio de la · Academica-e (Universidad Pública de Navarra)
De acuerdo a la normativa de TFEs el repositorio no puede dar acceso a este trabajo. Para consultarlo póngase en contacto con el tutor del trabajo. Puede acceder al resumen del mismo pinchando en el pdf adjunto.
Cited by 0
Marty C. Brandon, Douglas C. Wallace, Pierre Baldi · Bioinformatics · 2009
Data are publicly available from GenBank, the HapMap web site, and the MITOMAP database. Supplementary materials with additional results, statistics, and software implementations are available from http://mammag.web.uci.edu/bin/view/Mitowiki/ProjectDNACompression.
Cited by 99Open access
Paj Paul Volf · TU/e Research Portal (Eindhoven University of Technology) · 2002
Cited by 37Open access
Andrew D'Arms, Lexi Dunsmore, Samuel J. Ferguson et al. · 2024
Efficient data compression methods are critical for optimizing storage, accelerating data transfer, and improving system performance. This study gives a comprehensive comparison of three data compression algorithms: Run-Length Encoding (RLE), Lempel-Ziv-Welch (LZW), and Burrows-Wheeler Transform (BWT). Each technique was assessed in terms of time efficiency, geographical complexity, and feasibility. According to our findings, LZW typically outperforms other approaches, although RLE and BWT perform very well in some cases. We thoroughly evaluate each algorithm to identify its merits, limitations, and potential areas for development.
Cited by 1
BOTERO, Beatriz, HIFDI, Nourredine · OpenGrey (Institut de l'Information Scientifique et Technique) · 2000
Les systèmes SAR tirent profit du caractère tout temps du radar et de la capacité d'imagerie de haute résolution pour s'imposer dans un grand spectre d'applications aéronautiques ou spatiales. Une caractéristique importante inhérente à ces imageurs réside dans la génération d'un grand volume de données, imposant des contraintes sévères sur les débits de transmission ou sur les capacités de stockage embarquées. Les systèmes traitant ce type de problèmes doivent être soigneusement dimensionnés pour satisfaire les contraintes relatives à la charge de traitement et celles relatives aux capacités de mémorisation et de transmission. Cette thèse aborde ce thème dans le contexte des systèmes de compression des données brutes SAR. Les problèmes traités relèvent aussi bien de l'algorithmie que du système de traitement. Nous nous sommes donc intéressés aux différents types d'algorithmes de compression et aux modèles d'exécution associés. Les algorithmes évalués sont de deux types : les méthodes de compression directes (BAQ, UPQ, ...) et les méthodes de compression transformées (DCT, FFT, WT). Nous nous sommes ensuite intéressés à l'optimisation séquentielle et parallèle des algorithmes. La mise en évidence du parallélisme inhérent à ces algorithmes est exploitée pour une meilleure efficacité d'exécution sur des sytèmes cibles. Les différentes techniques de parallélisation sont appliquées aux algorithmes aussi bien pour les méthodes directes que pour les méthodes transformées. Les systèmes cibles incluent des systèmes massivement parallèles, comme le système CRAY-T3D/T3E, et des systèmes composés de processeurs dédiés au traitement du signal comme le TMS320C40. L'évaluation et l'analyse des performances que nous avons effectuées sont dirigées par deux objectifs. L'étude du comportement opérationnel mesuré par le taux de compression obtenu et la qualité de l'image restituée basée sur l'évaluation des critères standard et spécifiques aux images SAR. L'étude de l'efficacité d'exécution où nous adressons le problème de la diminution du temps global d'exécution, de l'exploitation du parallélisme et de la minimisation des communications. Les résultats obtenus permettent de statuer sur la recherche de l'adéquation entre les algorithmes de compression et les architectures de traitement associées.
Cited by 0Open access
Bouza M.K. · Artificial Intelligence · 2020
The article examines the algorithms for JPEG and JPEG-2000 compression of various graphic images. The main steps of the operation of both algorithms are given, their advantages and disadvantages are noted. The main differences between JPEG and JPEG-2000 are analyzed. It is noted that the JPEG-2000 algorithm allows re-moving visually unpleasant effects. This makes it possible to highlight important areas of the image and improve the quality of their compression. The features of each step of the algorithms are considered and the difficulties of their implementation are compared. The effectiveness of each algorithm is demonstrated by the example of a full-color image of the BSU emblem. The obtained compression ratios were obtained and shown in the corresponding tables using both algorithms. Compression ratios are obtained for a wide range of quality values from 1 to ten. We studied various types of images: black and white, business graphics, indexed and full color. A modified LZW-Lempel-Ziv-Welch algorithm is presented, which is applicable to compress a variety of information from text to images. The modification is based on limiting the graphic file to 256 colors. This made it possible to index the color with one byte instead of three. The efficiency of this modification grows with increasing image sizes. The modified LZW-algorithm can be adapted to any image from single-color to full-color. The prepared tests were indexed to the required number of colors in the images using the FastStone Image Viewer program. For each image, seven copies were obtained, containing 4, 8, 16, 32, 64, 128 and 256 colors, respectively. Testing results showed that the modified version of the LZW algorithm allows for an average of twice the compression ratio. However, in a class of full-color images, both algorithms showed the same results. The developed modification of the LZW algorithm can be successfully applied in the field of site design, especially in the case of so-called flat design. The comparative characteristics of the basic and modified methods are presented.
Cited by 0
Mohammad Hosseini · ArXiv.org · 2025
Today, with the growing demands of information storage and data transfer, data compression is becoming increasingly important. Data Compression is a technique which is used to decrease the size of data. This is very useful when some huge files have to be transferred over networks or being stored on a data storage device and the size is more than the capacity of the data storage or would consume so much bandwidth for transmission in a network. With the advent of the Internet and mobile devices with limited resources, data compression has gained even more importance. It can be effectively used to save both storage and bandwidth, thus to decrease download duration. Data compression can be achieved by a host of techniques. During this survey, I'm going to thoroughly discuss some of important data compression algorithms, their performance evaluation, and their major applications along with today's issues and recent research approaches.
Cited by 0Open access
Thakar, Kobie, Tuohy, Sylvester · Pace Digital Repository (Pace University) · 1993
Cited by 0
Dufort y Álvarez Zorrilla de San Martín, Guillermo · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · 2021
The massive generation of biological digital information creates various computing challenges such as its storage and transmission. For example, biomedical signals, such as electroencephalograms (EEG), are recorded by multiple sensors over long periods of time, resulting in large volumes of data. Another example is genome DNA sequencing data, where the amount of data generated globally is seeing explosive growth, leading to increasing needs for processing, storage, and transmission resources. In this thesis we investigate the use of data compression techniques for this problem, in two different scenarios where computational efficiency is crucial. First we study the compression of multi-channel biomedical signals. We present a new lossless data compressor for multi-channel signals, GSC, which achieves compression performance similar to the state of the art, while being more computationally efficient than other available alternatives. The compressor uses two novel integer-based implementations of the predictive coding and expert advice schemes for multi-channel signals. We also develop a version of GSC optimized for EEG data. This version manages to significantly lower compression times while attaining similar compression performance for that specic type of signal. In a second scenario we study the compression of DNA sequencing data produced by nanopore sequencing technologies. We present two novel lossless compression algorithms specifically tailored to nanopore FASTQ files. ENANO is a reference-free compressor, which mainly focuses on the compression of quality scores. It achieves state of the art compression performance, while being fast and with low memory consumption when compared to other popular FASTQ compression tools. On the other hand, RENANO is a reference-based compressor, which improves on ENANO, by providing a more efficient base call sequence compression component. For RENANO two algorithms are introduced, corresponding to the following scenarios: a reference genome is available without cost to both the compressor and the decompressor; and the reference genome is available only on the compressor side, and a compacted version of the reference is included in the compressed le. Both algorithms of RENANO significantly improve the compression performance of ENANO, with similar compression times, and higher memory requirements.
Cited by 0Open access
Annett Ungethüm, Patrick Damme, Johannes Pietrzyk et al. · Communications in computer and information science · 2017
Cited by 2
CUHK Digital Repository (Chinese University of Hong Kong) · 1997
Cited by 0
Juliana Hildebrandt, Dirk Habich, Thomas Kühn et al. · 2017
Cited by 3
Özkan Ufuk Nalbantoğlu, David J. Russell, Khalid Sayood · Entropy · 2009
Data compression at its base is concerned with how information is organized in data. Understanding this organization can lead to efficient ways of representing the information and hence data compression. In this paper we review the ways in which ideas and approaches fundamental to the theory and practice of data compression have been used in the area of bioinformatics. We look at how basic theoretical ideas from data compression, such as the notions of entropy, mutual information, and complexity have been used for analyzing biological sequences in order to discover hidden patterns, infer phylogenetic relationships between organisms and study viral populations. Finally, we look at how inferred grammars for biological sequences have been used to uncover structure in biological sequences.
Cited by 45Open access