Skip to content
Free planNo credit card requiredStart writing
AI for Research home
AI for Research home
Dashboard
Sign inGet started

Search 250 million research papers

Read the abstract, see who wrote it and who cites it, and get the citation in APA, MLA, Chicago or Harvard.

About 10,000 papers for “Activation function”

Activation Function

Alessandro Migliaccio, Giovanni Iannone · 2023

At this point, to ensure that the neural network can build a valid mathematical model, it is necessary to introduce some specific functions: the activation functions. The chapter starts to explain them using an interesting analogy with the field of particle physics to explain the different functions and their advantages.

Cited by 0

Activation function .

Thibault Delobel (17420011), Luis E. Ayala-Hernández (6954404), Jesús J. Bosque (17420014) et al. · Figshare · 2023

<div><p>Low-grade gliomas are primary brain tumors that arise from glial cells and are usually treated with temozolomide (TMZ) as a chemotherapeutic option. They are often incurable, but patients have a prolonged survival. One of the shortcomings of the treatment is that patients eventually develop drug resistance. Recent findings show that persisters, cells that enter a dormancy state to resist treatment, play an important role in the development of resistance to TMZ. In this study we constructed a mathematical model of low-grade glioma response to TMZ incorporating a persister population. The model was able to describe the volumetric longitudinal dynamics, observed in routine FLAIR 3D sequences, of low-grade glioma patients acquiring TMZ resistance. We used the model to explore different TMZ administration protocols, first on virtual clones of real patients and afterwards on virtual patients preserving the relationships between parameters of real patients. In silico clinical trials showed that resistance development was deferred by protocols in which individual doses are administered after rest periods, rather than the 28-days cycle standard protocol. This led to median survival gains in virtual patients of more than 15 months when using resting periods between two and three weeks and agreed with recent experimental observations in animal models. Additionally, we tested adaptive variations of these new protocols, what showed a potential reduction in toxicity, but no survival gain. Our computational results highlight the need of further clinical trials that could obtain better results from treatment with TMZ in low grade gliomas.</p></div>

Cited by 0Open access

Active Identity Function as Activation Function

Ali Akbar Kiaei, Mahnaz Boush, Danial Safaei et al. · Preprints.org · 2023

Selecting the optimal activation function for training deep neural networks has always been challenging because it significantly impacts the neural network’s performance and training speed. At this point, researchers are more likely to employ RELU than other well-known activation functions. After RELU, researchers have proposed many activation functions to improve RELU. None of them was capable of surpassing RELU as their most significant rival. SWISH outperformed RELU in several challenging experiments like classification, object detection, and tracking. Replacing RELU units with SWISH, which improves performance in many tasks. This paper proposed a new activation function surpassing Google’s brain’s SWISH function, Which we named AIF. Experiments indicate that our proposed activation function outperforms SWISH in various tasks.

Cited by 8Open access

Stochastic Adaptive Activation Function

Kyungsu Lee, Jaeseung Yang, Haeyun Lee et al. · arXiv (Cornell University) · 2022

The simulation of human neurons and neurotransmission mechanisms has been realized in deep neural networks based on the theoretical implementations of activation functions. However, recent studies have reported that the threshold potential of neurons exhibits different values according to the locations and types of individual neurons, and that the activation functions have limitations in terms of representing this variability. Therefore, this study proposes a simple yet effective activation function that facilitates different thresholds and adaptive activations according to the positions of units and the contexts of inputs. Furthermore, the proposed activation function mathematically exhibits a more generalized form of Swish activation function, and thus we denoted it as Adaptive SwisH (ASH). ASH highlights informative features that exhibit large values in the top percentiles in an input, whereas it rectifies low values. Most importantly, ASH exhibits trainable, adaptive, and context-aware properties compared to other activation functions. Furthermore, ASH represents general formula of the previously studied activation function and provides a reasonable mathematical background for the superior performance. To validate the effectiveness and robustness of ASH, we implemented ASH into many deep learning models for various tasks, including classification, detection, segmentation, and image generation. Experimental analysis demonstrates that our activation function can provide the benefits of more accurate prediction and earlier convergence in many deep learning applications.

Cited by 0Open access

Static Activation Function Normalization

Pierre H. Richemond, Yike Guo · arXiv (Cornell University) · 2019

Recent seminal work at the intersection of deep neural networks practice and random matrix theory has linked the convergence speed and robustness of these networks with the combination of random weight initialization and nonlinear activation function in use. Building on those principles, we introduce a process to transform an existing activation function into another one with better properties. We term such transform \emph{static activation normalization}. More specifically we focus on this normalization applied to the ReLU unit, and show empirically that it significantly promotes convergence robustness, maximum training depth, and anytime performance. We verify these claims by examining empirical eigenvalue distributions of networks trained with those activations. Our static activation normalization provides a first step towards giving benefits similar in spirit to schemes like batch normalization, but without computational cost.

Cited by 0Open access

SELU Variation Activation Function

Apoorv, Aakash · Zenodo (CERN European Organization for Nuclear Research) · 2024

The SELU Variation is designed to enhance the self-normalizing properties of the standard SELU function by introducing a non-monotonic behavior for negative inputs through a sinusoidal component. Mathematical Formula The SELU Variation activation function is defined as: f(x) = λ⋅{x if x> 0 α⋅(eβ⋅x−1)+γ⋅sin⁡(ω⋅x) if x ≤ 0 Where: ( λ) is a scaling parameter to ensure self-normalization. ( α) and ( β ) adjust the exponential growth for negative inputs. ( γ) and ( ω ) control the amplitude and frequency of the sinusoidal component.

Cited by 0Open access

SiLU activation function diagram.

Wei Liu (20030), Xingxin Guo (21553301) · Figshare · 2025

<div> Advancements in medical imaging technology have facilitated the acquisition of high-quality brain images through computed tomography (CT) or magnetic resonance imaging (MRI), enabling professional brain specialists to diagnose brain tumors more effectively. However, manual diagnosis is time-consuming, which has led to the growing importance of automatic detection and classification through brain imaging. Conventional object detection models for brain tumor detection face limitations in brain tumor detection owing to the significant differences between medical images and natural scene images, as well as challenges such as complex backgrounds, noise interference, and blurred boundaries between cancerous and normal tissues. This study investigates the application of deep learning to brain tumor detection, analyzing the effect of three factors, the number of model parameters, input data batch size, and the use of anchor boxes, on detection performance. Experimental results reveal that an excessive number of model parameters or the use of anchor boxes may reduce detection accuracy. However, increasing the number of brain tumor samples improves detection performance. This study, introduces a backbone network built using RepConv and RepC3, along with FGConcat feature map splicing module to optimize the brain tumor detection model. The experimental results show that the proposed RepConv-RepC3-FGConcat Network (RRFNet) can learn underlying semantic information about brain tumors during training stage, while maintaining a low number of parameters during inference, which improves the speed of brain tumor detection. Compared with YOLOv8, RRFNet achieved a higher accuracy in brain tumor detection, with a mAP value of 79.2%. This optimized approach enhances both accuracy and efficiency, which is essential in clinical settings where time and precision are critical. </div>

Cited by 0Open access

List of activation function.

Ruhua Chen (10258912), Mohammad Reza Ghavidel Aghdam (20323852), Mohammad Khishe (7003436) · Figshare · 2024

&lt;div&gt;&lt;p&gt;Fine art recognition, traditionally dependent on human expertise, is undergoing a significant transformation with the integration of Artificial Intelligence (AI) and deep learning. This article introduces a novel AI-based approach for fine art recognition, utilizing Convolutional Neural Networks (CNNs) and advanced feature extraction techniques. Addressing the inherent challenges within this domain, we present a systematic methodology to enhance automated fine art recognition. By leveraging critical dataset characteristics such as objective type, genre, material, technique, and department, our method exhibits exceptional performance in classifying fine art pieces across diverse attributes. Our approach significantly improves accuracy and efficiency by integrating advanced feature extraction techniques with a customized CNN architecture. Experimental validation on a benchmark dataset highlights the efficacy of our method, indicating substantial contributions to the interdisciplinary field of fine art analysis.&lt;/p&gt;&lt;/div&gt;

Cited by 0Open access

Brain function activation map.

Tao Zhu (118644), Yihao Zhou (3307839), Anhong Dai (14204314) et al. · Figshare · 2024

&lt;div&gt;&lt;p&gt;Background&lt;/p&gt;&lt;p&gt;The probability of motor deficits after stroke is relatively high. At the same time many studies have reported that acupuncture and rehabilitation therapy have a significant effect on the treatment of stroke.&lt;/p&gt;&lt;p&gt;Objective&lt;/p&gt;&lt;p&gt;This systematic review and meta-analysis aimed to evaluate the clinical value of acupuncture and rehabilitation therapy on brain eloquent areas and neurological function in ischemic stroke.&lt;/p&gt;&lt;p&gt;Methods&lt;/p&gt;&lt;p&gt;Seven databases were electronically searched to screen randomized controlled trials (RCTs) of different intervention methods (acupuncture, rehabilitation) in the treatment of ischemic stroke. The search time is from January 1, 2000 to April 20, 2023, and the search languages are limited to Chinese and English. Two researchers independently screened literature and extracted data. The methodological quality of the studies was assessed using the Cochrane Handbook for Systematic Reviews of Interventions.&lt;/p&gt;&lt;p&gt;Results&lt;/p&gt;&lt;p&gt;A total of 17 randomized controlled studies were included, including 699 patients, with a maximum sample size of 144 cases and a minimum sample size of 11 cases. Among them, 3 studies reported the brain function in SM1 area. The effective rate of the experimental group was higher than that of the control group [relative risk (OR) = 3.24, 95%CI: 1.49 to 7.05, &lt;i&gt;P&lt;/i&gt; &lt; 0.05]. The FMA score of patients in the experimental group was higher than that in the control group [mean difference (MD) = 4.79, 95% CI: 3.86 to 5.71, &lt;i&gt;P&lt;/i&gt; &lt; 0.00001]. The NIHSS score of patients in the experimental group was lower than that in the control group [mean difference (MD) = -4.12, 95% CI: -6.99 to -1.26, &lt;i&gt;P&lt;/i&gt; &lt; 0.05].None of studies reported adverse events.&lt;/p&gt;&lt;p&gt;Conclusions&lt;/p&gt;&lt;p&gt;Acupuncture rehabilitation for ischemic stroke can activate corresponding brain functional areas and improve neurological deficits. The therapeutic effect of acupuncture rehabilitation treatment is better than that of basic western medicine treatment, and it is more effective in improving neurological deficits. At the same time, clinical research needs to use high-quality randomized double-blind controlled trials with more detailed and larger sample designs, long-term efficacy evaluation and evidence-based research methods.&lt;/p&gt;&lt;/div&gt;

Cited by 0Open access

Activation Function: Absolute Function, One Function Behaves more Individualized

Jinxin Wei, Zhé Hóu · 2022

&lt;p&gt;Inspire by nature world mode, a activation function is proposed. It is absolute function.Through test on mnist dataset and fully-connected neural network and convolutional neural network, some conclusions are put forward. The line of accuracy of absolute function is a little shaken that is different from the line of accuracy of relu and leaky relu. The absolute function can keep the negative parts as equal as the positive parts, so the individualization is more active than relu and leaky relu function. In order to generalization, the individualization is the reason of shake, the accuracy may be good in some set and may be worse in some set. The absolute function is less likely to be over-fitting. The batch size is small, the individualization is clear, vice versa. Through test on mnist and autoencoder, It is that the leaky relu function can do classification task well, while the absolute function can do generation task well. Because the classification task need more universality and generation task need more individualization. The pleasure irritation and painful irritation is not only the magnitude differences, but also the sign differences, so the negative part should keep as a part. Stimulation which happens frequently is low value, it is showed around zero in figure 1 . Stimulation which happens accidentally is high value, it is showed far away from zero in figure 1. So the high value is the big stimulation, which is individualization.&lt;/p&gt;

Cited by 6Open access