Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “Generative Adversarial Networks and Image Synthesis”

Generative Adversarial Networks (GANs) for Image Synthesis and Augmentation

Yuxin Wen, Zhenhao Mei, Chuyue Qi · 2024

In the realm of medical imaging, a scarcity of reliable, sizable datasets for training supervised deep learning models persists. One solution involves leveraging Generative Adversarial Networks (GANs) to fabricate synthetic deepfake images, bridging this gap and enriching available data. Deepfake technology seamlessly transfers crucial features from source to target images or videos, mimicking the source with remarkable fidelity. Over the past decade, propelled by cutting-edge deep learning techniques and vast datasets, medical image processing has made monumental strides. Supervised deep learning models now achieve superhuman performance across diverse medical imaging applications. Synthetic deepfake images, produced through techniques like GANs, serve various purposes, including dataset balancing, modal translation, and expansion. To efficiently augment training data for deep learning, a hierarchical GAN (HGAN) framework is proposed, focusing on generating high-quality knee images. HGAN integrates spectral normalization in discrimination and pixel normalization in generation, enhancing training stability. Evaluation metrics such as AM Score and Mode Score are employed to benchmark HGAN against alternatives like PGGAN, reflecting its efficacy in advancing medical imaging research and application.

Cited by 2

Generative adversarial networks in computer vision: image synthesis and manipulation

Lingfeng Dong, Yi Zhang, Yuanyang Zou · Applied Mathematics and Nonlinear Sciences · 2023

Abstract In this paper, we first use the hidden variable loss to generate an adversarial network to optimize the efficiency of the original GAN operation. Secondly, the optimized GAN algorithm is used to train the encoder and decoder to form a new image-processing GAN by combining with the self-encoder and to standardize the operation process of image synthesis. On this basis, the performance of the self-encoder GAN is compared with the original GAN algorithm, and the quality of images generated by the self-encoder GAN and other ways is tested. The test results show that compared with the original GAN framework, the self-encoder GAN has a 50% improvement in operational efficiency in terms of root-mean-square error and an 80% reduction in synthesized global relative error. The self-coder GAN based on this algorithm is 22.5% higher than the traditional BiGAN framework and 2.5% higher than the OGAN framework in terms of FID quality criteria of the generated images and produces 30% less data capacity than BiGAN. The generated images have an average IS score of 3.435, which is superior to other base algorithms.

Cited by 0Open access

Medical Image Synthesis Using Variational Autoencoder and Generative Adversarial Networks

S. Hari Ganesh, B. Madhushree, K. Sowmya et al. · International Journal of Engineering and Manufacturing · 2025

Nowadays, image synthesis has become essential in the medical field for lever-aging deep learning technique to improve decision-making.Our proposed research work combines Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) to synthesize medical im-ages, enhancing diagnostics, medical training, and image analysis.The model presented combines a Discriminator, and a Variational Autoencoder to capitalize on the strengths of both VAEs and GANs.The Decoder is tasked with generating synthetic medical images, the Discriminator evaluates their distinguishing factor, and the VAE learns a probabilistic mapping from input to latent space, ensuring a structured representation of underlying medical features.The training process involves a decoder creating realistic medical images, a discriminator distinguishing real from synthetic ones, and a VAE capturing meaningful data variations in the latent space.Verified on the dataset sourced from the Kaggle.The model refines its parameters iteratively using a training loop, resulting in enhanced quality and variety of generated medical images.The proposed VAE-GAN model demonstrates its efficacy by generating diverse and realistic medical images.The structured latent space contributes to interpretability, making the images suitable for purposes like data augmentation, anomaly detection, and machine learning model training.

Cited by 0Open access

GENERATIVE ADVERSARIAL NETWORKS FOR IMAGE SYNTHESIS AND STYLE TRANSFER IN VIDEOS

Kokila Ramesh, B. Muni Lavanya, B. Rajesh Kumar et al. · ICTACT Journal on Image and Video Processing · 2023

In computer vision and artistic expression, the synthesis of visually compelling images and the transfer of artistic styles onto videos have gained significant attention. This research addresses the challenges in achieving realistic image synthesis and style transfer in the dynamic context of videos. Existing methods often struggle to maintain temporal coherence and fail to capture intricate details, prompting the need for innovative approaches. The conventional methods for image synthesis and style transfer in videos encounter difficulties in preserving the natural flow of motion and consistency across frames. This research aims to bridge this gap by leveraging the power of Generative Adversarial Networks (GANs) to enhance the quality and temporal coherence of synthesized images in video sequences. While GANs have demonstrated success in image generation, their application to video synthesis and style transfer remains an underexplored domain. The research seeks to address this gap by proposing a novel methodology that optimizes GANs for video-challenges, aiming for realistic, high-quality, and temporally consistent results. Our approach involves the development of a specialized GAN architecture tailored for video synthesis, incorporating temporal-aware modules to ensure smooth transitions between frames. Additionally, a style transfer mechanism is integrated, enabling the transfer of artistic styles onto videos seamlessly. The model is trained on diverse datasets to enhance its generalization capabilities. Experimental results showcase the efficacy of the proposed methodology in generating lifelike images and seamlessly transferring styles across video frames. Comparative analyses demonstrate the superiority of our approach over existing methods, highlighting its ability to address the temporal challenges inherent in video synthesis and style transfer.

Cited by 0Open access

Generative Adversarial Networks for Semantic Image Synthesis and Unconditional Synthesis with Limited Data

Vadim Sushko · bonndoc (University of Bonn) · 2024

Generative modeling of images is an important task aimed at synthesizing new images that are indistinguishable from real samples. The ability to generate diverse, realistic- looking images holds numerous applications, ranging from artistic content creation to data augmentation for other computer vision tasks. In this thesis, we study generative adversarial networks (GANs), which have gained popularity as the leading image synthesis framework due to their exceptional performance. GANs consist of a generator and a discriminator that are engaged in an adversarial game. In this game, the discriminator learns to correctly classify incoming real and fake images, while the generator is trained to fool the discriminator by producing more realistic samples. Although GANs have made significant advancements in recent years, more research is needed to further enhance their scalability across diverse datasets, as well as the quality and diversity of their synthesis. To this end, this thesis presents several new GAN methods that expand the literature on GANs in various aspects. 
\n <br /> 
\nFirstly, we address the task of semantic image synthesis, generating realistic images from semantic label maps. For this task, our work introduces a new segmentation-based discriminator that provides a strong training signal for the generator, eliminating the need for additional losses and tricks used in prior models. Compared to previous GAN approaches, our proposed method achieves a synthesis with higher image quality and diversity, while showing much better scalability to datasets with severe class imbalances. 
\n <br /> 
\nSecondly, we explore the training of unconditional GAN models in low data regimes. Previously, under limited data, GAN models suffered from training instabilities and memorization issues, which limited their application in restricted image domains. To address this, we propose new GAN approaches for various limited-data scenarios, including traditional one-shot and few-shot learning regimes. The advancements of our methods include new training schemes, improved architectures of discriminators, and novel regularization terms for generators. Our proposed methods enable high-quality and diverse synthesis from extremely small datasets, on which prior GAN models could not be trained successfully. 
\n <br /> 
\nOverall, this thesis advances the field of generative adversarial networks by introducing new GAN models that improve over prior work in the image synthesis quality, diversity, and applicability across various image domains. The proposed approaches demonstrate superior performance in semantic image synthesis and unconditional training with limited data, making GANs more powerful and effective for a wide range of computer vision applications.

Cited by 45Open access

Exploring Medical Image Data Augmentation and Synthesis using conditional Generative Adversarial Networks

Doncenco, Dorin · Theseus (Ammattikorkeakoulujen) · 2022

Obtaining healthcare data such as magnetic resonance imaging data for medical diagnosis is expensive and time-consuming. In this thesis, a method using generative adversarial networks is explored for synthesizing data of brain gliomas to improve the performance of image segmentation algorithms. The network was trained to create subjects with gliomas from a given label, and the network is able to synthesize visible tumors. The data was evaluated using DeepMedic, an image segmentation convolutional neural network. The performance of the model on the augmented dataset was benchmarked against the unaugmented dataset, and its performance was not improved. An analysis on the data is presented, and a future direction is given for how the generative adversarial network can be improved.

Cited by 0Open access

Generative Adversarial Networks with Data Augmentation and Multiple Penalty Areas for Image Synthesis

Li Chen, Huah Yong Chan · The International Arab Journal of Information Technology · 2023

The quality of generated images is one of the significant criteria for Generative Adversarial Networks (GANs) evaluation in image synthesis research. Previous researches proposed a great many tricks to modify the model structure or loss functions. However, seldom of them consider the effect of combination of data augmentation and multiple penalty areas on image quality improvement. This research introduces a GAN architecture based on data augmentation, in order to make the model fulfill 1-Lipschitz constraints, it proposes to consider these additional data included into the penalty areas which can improve ability of discriminator and generator. With the help of these techniques, compared with previous model Deep Convolutional GAN (DCGAN) and Wasserstein GAN with gradient penalty (WGAN-GP), the model proposed in this research can get lower Frechet Inception Distance score (FID) score 2.973 and 2.941 on celebA and LSUN towers at 64×64 resolution respectively which proves that this model can produce high visual quality results.

Cited by 2Open access

A Primary Comparison of Diffusion Models and Generative Adversarial Networks for Image Synthesis

Zhuyu Shen, Maoyu Mao, Pengfei Fan · 2024

The aim of this paper is to investigate the application of different types of datasets on image generation models, specifically the MINIST dataset and the CIFAR-10 dataset, and experiments were conducted using Diffusion Models and Generative Adversarial Networks (GANs) models. The performance and training process are evaluated and analyzed by comparing the two generative models for image synthesis. Through these comparison experiments, we find that both models have impressive performance in image generation. Specifically, Diffusion Models have a more stable training process and perform better in the later stages of training, while GANs have a shorter training time but are relatively less stable due to their adversarial training approach, and have more prominent generation results in the early stages but are slightly weaker than Diffusion Models in the later stages. These findings help me better understand and compare the characteristics and applicability scenarios of different generative models and applicable scenarios.

Cited by 1

Text to Image Synthesis using Stacked Conditional Variational Autoencoders and Conditional Generative Adversarial Networks

Tibebu, Haileleol, Malik, Aadil, De Silva, Varuna · arXiv (Cornell University) · 2022

Synthesizing a realistic image from textual description is a major challenge in computer vision. Current text to image synthesis approaches falls short of producing a highresolution image that represent a text descriptor. Most existing studies rely either on Generative Adversarial Networks (GANs) or Variational Auto Encoders (VAEs). GANs has the capability to produce sharper images but lacks the diversity of outputs, whereas VAEs are good at producing a diverse range of outputs, but the images generated are often blurred. Taking into account the relative advantages of both GANs and VAEs, we proposed a new stacked Conditional VAE (CVAE) and Conditional GAN (CGAN) network architecture for synthesizing images conditioned on a text description. This study uses Conditional VAEs as an initial generator to produce a high-level sketch of the text descriptor. This high-level sketch output from first stage and a text descriptor is used as an input to the conditional GAN network. The second stage GAN produces a 256x256 high resolution image. The proposed architecture benefits from a conditioning augmentation and a residual block on the Conditional GAN network to achieve the results. Multiple experiments were conducted using CUB and Oxford-102 dataset and the result of the proposed approach is compared against state-ofthe-art techniques such as StackGAN. The experiments illustrate that the proposed method generates a high-resolution image conditioned on text descriptions and yield competitive results based on Inception and Frechet Inception Score using both datasets

Cited by 0Open access

Synthesis of high-resolution fundus fluorescein angiography images using generative adversarial networks and image stitching

Yiwei Chen, Yi He, Wanyue Li et al. · 2nd International Conference on Laser, Optics and Optoelectronic Technology (LOPET 2022) · 2022

Although fundus fluorescein angiography is an imaging modality that supports ophthalmic diagnosis, it requires the intravenous injection of harmful fluorescein dye. We propose the synthesis of fluorescein angiography images from fundus structure images to avoid injection. Specifically, we automatically synthesize high-resolution fundus fluorescein angiography images through an algorithm that integrates a generative adversarial networks and image stitching and enhancement. By evaluating the peak signal-to-noise ratio and structural similarity index of the proposed algorithm, pix2pix, and cycleGAN, we confirmed the superior performance of our proposal. To further validate the proposed algorithm, we compared the fundus fluorescein angiography images synthesized by our algorithm, pix2pix, and cycleGAN. The experimental results show that our algorithm provides the highest resolution and quality in the synthesis of fluorescein angiography images from fundus structure images among the evaluated methods.

Cited by 0

Auxiliary Generative Adversarial Networks with Iliustration2Vec and Q-Learning based Hyperparameter Optimisation for Anime Image Synthesis

Vivian Sedov, Li Zhang · 2024

Harnessing the power of Generative Adversarial Networks (GANs) for the specialised task of anime face generation, this study introduces enhanced models of Auxiliary Classifier GAN (AC-GAN) and Wasserstein Auxiliary Classifier GAN (WAC-GAN) with modified network architectures and reinforcement learning-based hyperparameter optimisation. These models are uniquely adapted to handle the distinct nuances of anime-style imagery, a domain where conventional GANs often stumble due to complex stylistic variations and a heightened risk of mode collapse. Novel elements of our approach include, (1) modification of existing generator and discriminator architectures of both AC-GAN and WAC-GAN, (2) Q-learning based optimal hyperparameter selection, and (3) Illustration2Vec (I2V)-based automated attribute label extraction. Specifically, the Q-learning method is employed for hyperparameter search which effectively explores the search space of key network configurations by fulfilling the principles of Bellman optimality. Besides that, a deep learning-based 12V's method is utilised to generate attribute class labels and latent vectors to inform the generation process. Furthermore, we augment AC-GAN and WAC-GAN with additional layers to enhance their feature learning and generative capabilities. The insertion of these additional layers is calibrated based on the optimised network learning settings as well as the class labels derived from 12V, to fine-tune model scalability and diversity. Our experimental studies indicate that the conjunction of these techniques has led to a significant improvement in generating high-fidelity anime faces, adeptly handling the diverse and complex attributes inherent in anime-style imagery. The proposed strategies also showcase the potential of our customised AC-GAN and WAC-GAN models to master the nuanced art of anime face generation.

Cited by 1

Enhanced Colon Cancer Segmentation and Image Synthesis Through Advanced Generative Adversarial Networks Based-Sine Cosine Algorithm

Alawi Alqushaibi, Mohd Hilmi Hasan, Said Jadid Abdulkadir et al. · IEEE Access · 2024

Colorectal cancer (CRC) is a prevalent and life-threatening malignancy, demanding early diagnosis and effective treatment for improved patient outcomes. Accurate segmentation of colon cancer in medical images is a challenging task due to the complexity of its morphology and limited annotated data availability. This paper presents an efficient approach for colon cancer segmentation and image synthesis, combining an Attention U-Net and Pix2Pix Generative Adversarial Network (Pix2Pix-GAN) guided by Sine Cosine Algorithm (SCA) for hyperparameter tuning within the GAN framework. The utilization of SCA plays a pivotal role in optimizing the delicate balance between generator and discriminator dynamics, resulting in enhanced convergence and stability. Our method achieved state-of-the-art results with a mean Dice score of 0.9514, mean Intersection over Union of 0.9123, F beta score of 0.9636, and similarity index of 0.9430 outperforming existing methods. Moreover, the Mean Absolute Error reached a minimal value of 0.01583. This proposed approach shows promise in enhancing the accuracy and robustness of colon cancer diagnosis and treatment which could lead to better diagnosis and treatment of colon cancer.

Cited by 9Open access

Shadow-aware terrain classification: advancing hyperspectral image sensing through generative adversarial networks and correlated sample synthesis

Ali Gokalp Peker, Seniha Esen Yüksel, Ramazan Gökberk Cinbiş et al. · Journal of Applied Remote Sensing · 2024

In recent years, the utilization of hyperspectral sensors for remote sensing has marked a profound advancement due to the success of machine learning techniques. Nevertheless, difficulties still exist, especially in locations with shadows. The heterogeneity in spectral data due to different shadow origins, such as different types of clouds and different building designs, poses a significant obstacle to the advancement of shadow-aware classification algorithms. Furthermore, precisely labeling the underlying structures in shadowed areas is a very cumbersome effort. We present a loss function-based strategy based on generative adversarial networks to address this problem. Using the context of correlated samples, our loss function combines unpaired matchings and transitive style modifications via the fusion of contrastive learning, dual learning, cycle consistency, and curriculum learning algorithms. Our work transforms the non-shadowed training instances into the shadowed counterparts for use as synthetic training samples, as opposed to the conventional method of correcting shadowed pixels to their non-shadowed counterparts. We propose learning this transformation model with unpaired data samples, which is particularly advantageous compared with the collection process of the same samples with and without shadow. Synthetic samples for shadow-obscured regions can be produced when this method is used, and these samples improve the model’s performance in classification tasks. Rigorously tested through a combination of qualitative and quantitative evaluations, the introduced data augmentation technique improves the performance of terrain classification models, especially with limited data samples.

Cited by 0Open access