Reginald D. Smith · arXiv (Cornell University) · 2020
Two estimates for the inverse binary entropy function are derived using the property of information entropy to estimate combinatorics of sequences as well as related formulas from population genetics for the effective number of alleles. The second estimate shows close correspondence to the actual value of the inverse binary entropy function and can be seen as a close approximation away from low values of binary entropy where $p$ or $1-p$ are small.
Cited by 0Open access
Yang Li (7082), Chengxin Zhang (685159), Eric W. Bell (8614323) et al. · Figshare · 2021
Cited by 0Open access
Rohit Waddar, Vishwanath Rathod, H Netravati et al. · 2024
Stuttering is a condition where people have difficulty speaking smoothly. This includes repeating certain sounds or phrases in their speech. Autism, on the other hand, is a condition that affects communication and social interactions, leading to challenges in expressive speech, social cues, and interpersonal engagement. Stuttering is not a psychological issue; it can <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{con}_{\gamma}$</tex> tribute to social anxiety and low self-esteem. Currently, there is no known cure for stuttering; however, it can be effectively treated using speech-based stimulation to improve fluency. This study introduces a approach to stutter detection using a Convolutional Neural Network (CNN) with Mel-frequency cepstral coefficients (MFCC) feature extraction on the LibriStutter dataset. The research rigorously evaluates the performance of three loss functions binary cross-entropy, categorical cross-entropy, and log cosh highlighting their impact on CNN model accuracy. Notably, binary cross-entropy emerges as the most effective, achieving a remarkable testing accuracy of 86.32%. The study includes detailed insights into the methodology, specifically emphasizing the utilization of a 1D Convolutional Layer with 32 filters, max pooling layer (2), dropout layer (0.2), ReLu activation and other components in the CNN architecture. Additionally, the results underscore the superior performance optimization achieved by the binary cross-entropy loss function. This research not only advances stutter detection methodologies but also highlights the pivotal role of the chosen loss function in achieving superior accuracy.
Cited by 5
Owen M. O’Connor (11965460), Razan N. Alnahhas (1944835), Jean-Baptiste Lugagne (1353195) et al. · Figshare · 2022
<p>The inputs and outputs for the loss function are shown. The inputs include the ground truth (GT), the prediction made by the model (Pred), and the associated weight maps. The output is the pixelwise-weighted loss, which is a greyscale image. In step 1, the ground truth and prediction are used to calculate the binary cross-entropy loss. The first half of the equation measures the pixelwise loss associated with the model predicting background when the ground truth is a cell. The second half of the equation measures the pixelwise loss associated with the model predicting cell when the ground truth is background. In step 2, the weighted loss is calculated by performing an elementwise multiplication of the weight map and the loss calculated in step 1. The weight map helps the model learn the more important features, such as cell borders. This schematic of the loss function is simplified for visualization purposes. For clarity, we also show how the pixelwise-weighted loss maps onto the prediction, where cyan regions highlight areas where the loss is emphasized in order to improve the model’s performance for these regions of the example image.</p> <p>(EPS)</p>
Cited by 0Open access
V. V. Prelov · Moscow University Mathematics Bulletin · 2024
New inequalities for the binary entropy function are proved.
Cited by 0
Masoomeh Gomroki, Mahdi Hasanlou, Peter Reinartz · Research Square · 2022
<title>Abstract</title> Detecting changes in urban areas have always been an important element of urban planning and resource management. With the widening impacts of human activities on the ground terrain and landscapes, in recent decades the analysis of remote sensing data, including satellite images, has become a method of choice for rapid tracking of changes in the ground surface over wide geographical areas. However, this approach requires automated and semi-automated methods with the ability to detect changes in remote sensing data. In this study, Sentinel-2 satellite images taken in 2017 and 2021 were used for binary change detection in urban areas of northern Iran. This change detection was performed with a supervised deep learning method, which was used to generate a detailed change map of the area. Considering the generally good performance of convolutional neural networks in change detection applications, this study used an improved U-net with a weighted binary cross-entropy loss function, this function considers unequal weights for positive and negative targets so it performs better in detecting changes in complex urban areas especially small changes rather than binary cross-entropy. In addition to Sentinel-2 satellite images of northern Iran, Onera Satellite Change Detection (OSCD) dataset was also used to evaluate our proposed method. Experimental results showed the good performance of the method on both datasets, with the accuracy and F1score of final change maps reaching above 97%.
Cited by 4Open access
Parismita Sarma, Joseph Kiran Chayengia, Gitarthi Gogoi · 2023
In this paper authors tries to show success of using ensemble of loss function on GAN network model. The model was able to generate quality synthetic data on MNIST handwritten dataset. The performance of the simple vanilla GAN model is evaluated for binary entropy loss and least square loss combined and compared for both of these loss functions separately. Instead of using only one single activation function we have used multiple activation functions such as Sigmoid., Tanh and ReLU in the last layer of the discriminator. Minimum loss value is counted among the pairs of transformed output vectors that are changed using different activation functions. In this case study we have used MNIST handwritten dataset. At last observations are made that training a model using ensemble of L2 loss and binary cross entropy loss functions produced significantly better results compared to using only L2 and only binary cross entropy loss function.
Cited by 0
Sulekha Saxena, Vijay Gupta, P.N. Hrisheekesha · Biomedical Engineering Applications Basis and Communications · 2019
In this paper, we propose an automated approach that combines the generalized discriminant analysis (GDA) as feature reduction scheme with radial basis function (RBF) kernel and the online sequential extreme learning machine (OSELM) having Sigmoid, Hardlim, RBF and Sine activation function as binary classifier for detection of congestive heart failure (CHF) and coronary artery disease (CAD). For this analysis, 13 nonlinear features as Correlation Dimension (CD), Detrended Fluctuation Analysis (DFA) as DFA-[Formula: see text]1 and DFA-[Formula: see text]2, Bubble Entropy (BBEn), Sample Entropy (SampEn), Dispersion Entropy (DISEn), Lempel–Ziv Complexity (LZ), Sinai Entropy (SIEn), Improved Multiscale Permutation Entropy (IMPE), Hurst Exponent (HE), Permutation Entropy (PE), Approximate Entropy (ApEn) and Standard Deviation (SD1/SD2) were extracted from Heart Rate Variability (HRV) signals. For validation of proposed method, HRV data were obtained from standard database of normal sinus rhythm (NSR), CHF and CAD subjects. Numerical experiments were done on the combination of database sets such as NSR-CAD, CHF-CAD and NSR-CHF subjects. The simulation results show a clear difference in combination of database sets by using GDA having RBF, Gaussian kernel function and OSELM binary classifier having Sigmoid, RBF and Sine activation function and achieved an accuracy of 98.17% for NSR-CAD, 100% for NSR-CHF and CAD-CHF subjects.
Cited by 7
Mohammad Reza Rezaei-Dastjerdehei, Amirmohammad Mijani, Emad Fatemizadeh · 2020
Training a model and network on an imbalanced dataset always has been a challenging problem in the machine learning field that has been discussed by researchers. In fact, available machine learning algorithms are designed moderately imbalanced datasets and mainly do not consider the dataset's imbalanced problem. In the machine learning algorithm, the imbalance problem appears when the number of one class samples are significantly minor than another class. In order to solve the imbalance problem of a dataset, multiple algorithms are proposed in the field of machine learning and especially in deep learning. In this study, we have benefited from weighted binary cross-entropy in the learning process as a loss function instead of ordinary cross-entropy (binary cross-entropy). This model allocates more penalty to minority class samples during the learning process, and it makes that minority class samples are detected more accurately. Finally, we could improve Recall with preserving Accuracy. In fact, results show that using weighted binary cross-entropy recall increases about 10%, and precision does not decrease more than 3% in comparison to binary cross-entropy.
Cited by 77
Robert Dunne · 2006
This chapter contains sections titled: Introduction Interpreting Outputs as Probabilities The “Universal Approximator” and Consistency Variance and Bias Binary Variables and Logistic Regression MLP Models and Cross—Entropy A Derivation of the Softmax Activation Function The “Natural” Pairing and ΔQ A Comparison of Least Squares and Cross—Entropy Conclusion Complements and Exercises
Cited by 1
Carey Johnson · Mendeley Data · 2026
Raw binary data files (*.sdt) were converted to ascii fluorescence decay files (*.am) and ascii instrument response function files (*.ai). Settings for maximum entropy fits are contained in *.go files.
Cited by 0Open access
Carey Johnson · Mendeley Data · 2026
Raw binary data files (*.sdt) were converted to ascii fluorescence decay files (*.am) and ascii instrument response function files (*.ai). Settings for maximum entropy fits are contained in *.go files.
Cited by 0Open access
Lingfeng Liu, Suoxia Miao · Complexity · 2015
The information sources using logistic maps have been studied in this article. We use a particular symmetric binary function to generate the binary sequences due to the generating partition of the logistic maps, which can reflect the dynamics of chaotic real‐valued sequences. Using this binary function, we will establish a binary description of the logistic maps, and prove that this description is equivalent with the logistic maps. The probability of all the possible binary sequences generated by this symmetric binary function has been calculated according to this description. Furthermore, the cryptographic complexity of these binary sequences is evaluated theoretically and experimentally, including the correlation functions, the linear complexity, the nonlinear complexity, the Lempel–Ziv complexity and the approximate entropy. Such results show that the logistic binary sequences have good properties of cryptography. © 2015 Wiley Periodicals, Inc. Complexity 21: 121–129, 2016
Cited by 10
Oliver Baker, Carl P. Dettmann · ArXiv.org · 2026
We study the effect of the choice of embedding geometry on the entropy of random geometric graph ensembles with soft connection functions. First we show that when the connection range is small, the entropy is dependent only on the dimension of the geometry and not the shape, but for large connection ranges the boundaries of the domain matter. Next, we formulate the problem of estimating entropy as a problem of estimating the average degree of a graph with the binary entropy function as its connection function. We use this formulation to study the effect of boundaries on the entropy, and to estimate the entropy of soft random geometric graphs in complicated geometries where a closed form pair distance density is not available.
Cited by 0Open access
Yoshiaki Itoh, Joel E. Cohen · Journal of Physics A Mathematical and General · 1994
For conservative linear systems (finite-state Markov processes in discrete or continuous time), the relative entropy of two distinct trajectories is a monotonically decreasing function of time. These results naturally raise the question whether distinct trajectories of nonlinear conservative systems also display monotonically decreasing relative entropy. For binary interacting Lotka-Volterra systems with anti-symmetry, the relative entropy oscillates under the motion. The main new result of this paper is that, for ternary interacting Lotka-Volterra systems with anti-symmetry, the relative entropy of two distinct trajectories is a monotonically decreasing function of time near equilibrium. Far from equilibrium, distinct trajectories of ternary Lotka-Volterra systems with anti-symmetry need not have monotonically decreasing relative entropy.
Cited by 0
Oliver Baker, Carl P. Dettmann · arXiv (Cornell University) · 2026
We study the effect of the choice of embedding geometry on the entropy of random geometric graph ensembles with soft connection functions. First we show that when the connection range is small, the entropy is dependent only on the dimension of the geometry and not the shape, but for large connection ranges the boundaries of the domain matter. Next, we formulate the problem of estimating entropy as a problem of estimating the average degree of a graph with the binary entropy function as its connection function. We use this formulation to study the effect of boundaries on the entropy, and to estimate the entropy of soft random geometric graphs in complicated geometries where a closed form pair distance density is not available.
Cited by 0Open access
A.R. Villagómez-Manrique, Marcelo del Castillo‐Mussot · Revista Mexicana de Física · 2018
We impose an upper bound to the amount of agents wealth in binary exchanges, but when the upper bound is continuously lowered, thenthe entropy S decreases because the system exhibits more order. Furthermore, negative temperatures can mathematically obtained when the slope of the entropy as a function of the total energy (T = $(dS/dE)^{-1}$) is negative.
Cited by 1Open access
Mahmudul Hasan Rabbi · Preprints.org · 2025
Binary classification is a fundamental task in machine learning with applications in financial fraud detection, robotics, sentiment analysis, health care, and autonomous systems. The binary cross-entropy loss function is more widely used for optimizing classification models. However, sometimes it demonstrates numerical instability and high variance during the training leading to inconsistent convergence. To solve this issue, We propose an alternative loss function designed for binary classification to improve numerical stability and reduce training fluctuations. Experimental results show that the proposed loss function achieves smoother training dynamics and lower variance in loss curves compared to conventional approaches. These findings suggest that it serves as a robust alternative for binary classification tasks, improving model reliability in real-world applications.
Cited by 0Open access
S.S. Melnik, O. V. Usatenko · Physical Review E · 2014
We study the N-step binary stationary ergodic Markov chain and analyze its differential entropy. Supposing that the correlations are weak we express the conditional probability function of the chain through the pair correlation function and represent the entropy as a functional of the pair correlator. Since the model uses the two-point correlators instead of the block probability, it makes it possible to calculate the entropy of strings at much longer distances than using standard methods. A fluctuation contribution to the entropy due to finiteness of random chains is examined. This contribution can be of the same order as its regular part even at the relatively short lengths of subsequences. A self-similar structure of entropy with respect to the decimation transformations is revealed for some specific forms of the pair correlation function. Application of the theory to the DNA sequence of the R3 chromosome of Drosophila melanogaster is presented.
Cited by 8Open access
Guangyue Han, Brian Marcus · 2005
We prove that under a mild positivity assumption the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. We give examples to show how this can fail in some cases. And we study two natural special classes of hidden Markov chains in more detail: binary hidden Markov chains with an unambiguous symbol and binary Markov chains corrupted by binary symmetric noise. Finally, we show that under the positivity assumption the hidden Markov chain itself varies analytically, in a strong sense, as a function of the underlying Markov chain parameters. 1
Cited by 0