Bernardo Pulido-Gaytan (11979279), Andrei Tchernykh (11979273) · Figshare · 2024
<div><p>The widespread adoption of cloud computing necessitates privacy-preserving techniques that allow information to be processed without disclosure. This paper proposes a method to increase the accuracy and performance of privacy-preserving Convolutional Neural Networks with Homomorphic Encryption (CNN-HE) by Self-Learning Activation Functions (SLAF). SLAFs are polynomials with trainable coefficients updated during training, together with synaptic weights, for each polynomial independently to learn task-specific and CNN-specific features. We theoretically prove its feasibility to approximate any continuous activation function to the desired error as a function of the SLAF degree. Two CNN-HE models are proposed: CNN-HE-SLAF and CNN-HE-SLAF-R. In the first model, all activation functions are replaced by SLAFs, and CNN is trained to find weights and coefficients. In the second one, CNN is trained with the original activation, then weights are fixed, activation is substituted by SLAF, and CNN is shortly re-trained to adapt SLAF coefficients. We show that such self-learning can achieve the same accuracy 99.38% as a non-polynomial ReLU over non-homomorphic CNNs and lead to an increase in accuracy (99.21%) and higher performance (6.26 times faster) than the state-of-the-art CNN-HE CryptoNets on the MNIST optical character recognition benchmark dataset.</p></div>
Cited by 0Open access
Travis Pence, Daisuke Yamada, Vikas Singh · arXiv (Cornell University) · 2025
Contemporary large models often exhibit behaviors suggesting the presence of low-level primitives that compose into modules with richer functionality, but these fundamental building blocks remain poorly understood. We investigate this compositional structure in linear layers by asking: can we identify/synthesize linear transformations from a minimal set of geometric primitives? Using Clifford algebra, we show that linear layers can be expressed as compositions of bivectors -- geometric objects encoding oriented planes -- and introduce a differentiable algorithm that decomposes them into products of rotors. This construction uses only O(log^2 d) parameters, versus O(d^2) required by dense matrices. Applied to the key, query, and value projections in LLM attention layers, our rotor-based layers match the performance of strong baselines such as block-Hadamard and low-rank approximations. Our findings provide an algebraic perspective on how these geometric primitives can compose into higher-level functions within deep models.
Cited by 0Open access
Anubhab Baksi, Sumanta Chakraborty, Anupam Chattopadhyay et al. · 2024
In this paper, we consider the problem of quantum implementation of the symmetric-key ciphers. The typical ciphers in this category have two main components, the linear and the non-linear layers, we consider both. The linear layer can be described as a non-singular matrix over binary operations. The in-place implementation is one of the main considerations for implementing such layer to minimize the number of qubits. We discuss the research works that have been done to this point on this subject matter and report the improved implementation of some of the matrices. Lists of our modifications include; making it more efficient (specially for the larger matrices), consideration for quantum depth (along with a randomized algorithm for optimization), etc. We report benchmarks for the ASCON and SHA-2 linear matrices. As for the non-linear layer, the constituent block is called an substitution box (an S-box, for short). Our “DORCIS” tool presented in this paper, finds a quantum circuit with an optimized depth for given S-boxes of size 3- and 4-bit. It is a follow-up work on LIGHTER-R (which is applicable for 4-bit S-boxes only) with multiple extensions. Unlike LIGHTER-R, our DORCIS takes a quantum decomposition based on Clifford and T gates. Also, both the full quantum depth and the T depth can be optimized by DORCIS. We compare our implementation with other optimized quantum circuits shown in the other research works and show that we find an implementation with the same cost metric, or find an implementation with lower cost metric, compared other tools proposed in the literature, apart from being simpler and more efficient.
Cited by 7
Victor Cauchois · theses.fr (ABES) · 2018
Cette thèse s’intéresse à deux aspects de la cryptologie symétrique liés à l’utilisation de matrices MDS dans les couches de diffusion linéaires de primitives. Une première partie se fonde sur les conceptions de couches de diffusion linéaires de schémas de chiffrement symétrique à partir de matrices MDS. Les associations entre matrices récursives, respectivement circulantes, et polynômes sont calquées pour construire de nouvelles associations entre d’autres structures de matrices et des éléments d’anneaux de polynômes non commutatifs de Ore. À l’instar des matrices récursives et circulantes, ces structures bénéficient d’implémentations matérielles légères. Des codes de Gabidulin dérivent des méthodes de construction directe de telles matrices, optimales en termes de diffusion, proches d’involutions pour l’implémentation. La seconde partie développe une attaque par différenciation de permutations dont l’architecture s’inspire de l’AES. L’utilisation d’une couche de diffusion linéaire locale avec une matrice MDS induit une description macroscopique de la propagation de valeurs de différences à travers les étapes du chiffrement. Des chemins différentiels tronqués apparaissent, qui servent de point de départ à la conception d’attaques rebond. Les travaux présentés généralisent les attaques rebond connues à l’exploitation de chemins différentiels tronqués structurés non issus d’avalanches libres. Cette structure permet de ne pas consommer tous les degrés de libertés au cours d’une seule étape algorithmique mais de les répartir en trois étapes. Une attaque sur 11 tours d’une permutation de Grostl-512 est alors déployée.
Cited by 0Open access
R.M. Prokop · University of Alberta Library · 1993
Cited by 0Open access
Frédéric Lebon, Raffaella Rizzoni, R. Ronel-Idrissi · HAL (Le Centre pour la Communication Scientifique Directe) · 2005
Cited by 0
Yanghai Tsin, Sing Bing Kang, Richard Szeliski · IEEE Transactions on Pattern Analysis and Machine Intelligence · 2006
In this paper, we address stereo matching in the presence of a class of non-Lambertian effects, where image formation can be modeled as the additive superposition of layers at different depths. The presence of such effects makes it impossible for traditional stereo vision algorithms to recover depths using direct color matching-based methods. We develop several techniques to estimate both depths and colors of the component layers. Depth hypotheses are enumerated in pairs, one from each layer, in a nested plane sweep. For each pair of depth hypotheses, matching is accomplished using spatial-temporal differencing. We then use graph cut optimization to solve for the depths of both layers. This is followed by an iterative color update algorithm which we proved to be convergent. Our algorithm recovers depth and color estimates for both synthetic and real image sequences.
Cited by 48
Ruo Li, Yichen Yang · arXiv (Cornell University) · 2022
We propose a well-posed Maxwell-type boundary condition for the linear moment system in half-space. As a reduction of the Boltzmann equation, the moment equations are available to model Knudsen layers near a solid wall, where proper boundary conditions should be prescribed. Utilizing an orthogonal decomposition, we separate the part with a damping term from the system and then impose a new class of Maxwell-type boundary conditions on it. Due to the new boundary condition, we show that the half-space boundary value problem admits a unique solution with explicit expressions. Instantly, the well-posedness of the linear moment system is achieved. We apply the procedure to classical flow problems with the Shakhov collision term, such as the velocity slip and temperature jump problems. The model can capture Knudsen layers with very high accuracy using only a few moments.
Cited by 0Open access
Sing Bing Kang, Richard Szeliski, Yanghai Tsin et al. · 2006
Cited by 0
Philip Hall · Proceedings of the Royal Society of London A Mathematical and Physical Sciences · 1978
Abstract The linear stability of a flat Stokes layer is investigated. The results obtained show that, in the parameter range investigated, the flow is stable. It is shown that the Orr-Sommerfield equation for this flow has a continuous spectrum of damped eigenvalues at all values of the Reynolds number. In addition, a set of discrete eigenvalues exists for certain values of the Reynolds number. The eigenfunctions associated with this set are confined to the Stokes layer while those corresponding to the continuous spectrum persist outside the layer. The effect of introducing a second boundary a long way from the Stokes layer is also considered. It is shown that the least stable disturbance of this flow does not correspond to the least stable discrete eigenvalue of the infinite Stokes layer when this boundary tends to infinity.
Cited by 101
H. L Reed · Annual Review of Fluid Mechanics · 1996
The field of fluid mechanics is rapidly advancing, driven by unprecedented volumes of data from experiments, field measurements, and large-scale simulations at multiple spatiotemporal scales. Machine learning (ML) offers a wealth of techniques to extract ...Read More
Cited by 26
Xinping Yi · IEEE Transactions on Signal Processing · 2022
By stacking a number of convolutional layers, convolutional neural networks (CNNs) have made remarkable performance boosts in many artificial intelligence applications. While the convolution operation is well-understood, it is still a mystery why repeated convolutions yield so good expressive power and generalization performance. Noting that the linear convolution operation can be represented as a matrix-vector product with the matrix being of a Toeplitz structure, we propose to inspect the individual convolutional layer through its asymptotic spectral representation - the spectral density matrix - by leveraging Toeplitz matrix theory. Thanks to such spectral representation, we are able to develop a simple singular value approximation method with improved accuracy, and spectral norm upper bounds with reduced computational complexity, compared with the state-of-the-art methods. Both the improved approximation and upper bounds can be employed as regularization techniques to further enhance the generalization performance of CNNs. By extensive experiments on well-deployed CNN models (e.g., ResNets), we also demonstrate that the approximation approach achieves higher accuracy and the upper bounds are effective spectral regularizers for generalization.
Cited by 7
Gomez De La Cruz, Salvador Rey · CaltechTHESIS (California Institute of Technology) · 2023
Resolvent analysis is applied to nonequilibrium incompressible adverse pressure gradient (APG) turbulent boundary layers (TBL) and hypersonic boundary layers with high temperature real gas effects, including chemical nonequilibrium. Resolvent analysis is an equation-based, scale-dependent decomposition of the Navier Stokes equations, linearized about a known mean flow field. The decomposition identifies the optimal response and forcing modes, ranked by their linear amplification. To treat the nonequilibrium APG TBL, a biglobal resolvent analysis approach is used to account for the streamwise and wall-normal inhomogeneities in the streamwise developing flow. For the hypersonic boundary layer in chemical nonequilibrium, the resolvent analysis is constructed using a parallel flow assumption, incorporating N₂, O₂, NO, N, and O as a mixture of chemically reacting gases. Biglobal resolvent analysis is first applied to the zero pressure gradient (ZPG) TBL. Scaling relationships are determined for the spanwise wavenumber and temporal frequency that admit self-similar resolvent modes in the inner layer, mesolayer, and outer layer regions of the ZPG TBL. The APG effects on the inner scaling of the biglobal modes are shown to diminish as their self-similarity improves with increased Reynolds number. An increase in APG strength is shown to increase the linear amplification of the large-scale biglobal modes in the outer region, similar to the energization of large scale modes observed in simulation. The linear amplification of these modes grows linearly with the APG history, measured as the streamwise averaged APG strength, and relates to a novel pressure-based velocity scale. Resolvent analysis is then used to identify the length scales most affected by the high-temperature gas effects in hypersonic TBLs. It is shown that the high-temperature gas effects primarily affect modes localized near the peak mean temperature. Due to the chemical nonequilibrium effects, the modes can be linearly amplified through changes in chemical concentration, which have non-negligible effects on the higher order modes. Correlations in the components of the small-scale resolvent modes agree qualitatively with similar correlations in simulation data. Finally, efficient strategies for resolvent analysis are presented. These include an algorithm to autonomously sample the large amplification regions using a Bayesian Optimization-like approach and a projection-based method to approximate resolvent analysis through a reduced eigenvalue problem, derived from calculus of variations.
Cited by 0Open access
Liou, William W., United States. National Aeronautics and Space Administration. · The Journal of the Abraham Lincoln Association
Cited by 0Open access
William W. Liou · Physics of Fluids · 1994
The linear inviscid hydrodynamic stability of slightly curved free mixing layers is studied in this paper. The disturbance equation is solved numerically using a shooting technique. Two mean velocity profiles that represent stably and unstably curved free mixing layers are considered. Results are shown for cases of five curvature Richardson numbers. The stability characteristics of the shear layer are found to vary significantly with the introduction of the curvature effects. The results also indicate that, in a manner similar to the Görtler vortices observed in a boundary layer along a concave wall, instability modes of spatially developing streamwise vortex pairs may appear in centrifugally unstable curved mixing layers.
Cited by 36
Michéle G. Macaraeg, Craig L. Streett · Applied Numerical Mathematics · 1991
Cited by 38
A. D. D. Craik · Journal of Fluid Mechanics · 1971
An investigation is made of resonant triads of Tollmien-Schlichting waves in an unstable boundary layer. The triads considered are those comprising a two-dimensional wave and two oblique waves propagating at equal and opposite angles to the flow direction and such that all three waves have the same phase velocity in the downstream direction. For such a resonant triad remarkably powerful wave interations take place, which may cause a continuous and rapid transfer of energy from the primary shear flow to the disturbance. It appears that the oblique waves can grow particularly rapidly and it is suggested that such preferential growth may be responsible for the rapid development of three-dimensionality in unstable boundary layers. The non-linear energy transfer primarily takes place in the vicinity of the critical layer where the downstream propagation velocity of the waves equals the velocity of the primary flow. The theoretical analysis is initially carried out for a general primary velocity profile; then, in order to demonstrate the essential features of the results, precise interaction equations are derived for a particular profile consisting of a layer of constant shear bounded by a uniform flow. Some exact solutions of the general interaction equations are presented, one of which has the property that the wave amplitudes become indefinitely large at a finite time. The possible relevance of the present theoretical model to the experiments of Klebanoff, Tidstrom & Sargent (1962) is examined.
Cited by 475
Jing Wu, Foluso Ladeinde, Edward E. O’Brien et al. · 38th Aerospace Sciences Meeting and Exhibit · 2000
In this paper, we study the non-linear spatial stabi1it.y behavior of compressible mising layers, based on Landau’s pert,urbation series and Tollmien-Schlichting waves. It is wellknown from linear theory that, for spatially-developing perturbation waves, whether the disturbance is amplified, neutral, or damped is dependent on the imaginary part of the complex wavenumber, which could be negat.ive, zero, or positive. The non-linear theory goes further and predicts the possibilit,y for amplified wave to develop into supercritical equilibrium state and the possibility for damped waves to diverge into subcritical instability. The present study deals with the first of these possible situations since the non-linear behavior of the most amplified wave is being studied. It is found that the constants in Landau’s series determine the exist,ence of supercritical equilibrium state. If it exists, the constants will also determine the value of the equilibrium amplitude. If it does not exist, the constants will give the location of the singularity. We carry out a study of the influence of compressibility on the existence of supercritical equilibrium. The resutls show that supercritical equilibrium exists at relatively low Mach numbers, while singularity is more likely at high Mach numbers. The stability characteristics of compressible mixing layers are essential to efficient mixing bet,ween the fuel and OXidant in scramjet engine combustion. Stabi1it.y results also pro\-ide appropriate initial conditions for DYS. Comprehensive studies of this problem using linear theory were carried out by Jackson”2.3, h1ichalke4, Ragabs, Day6: Shin’, et.c. These st,udies contribute to the understanding of the phenomena and allow correct prediction of the first stage of the mixing process. In order to gain more knowledge of the evolution of the disturbance in mixing layers: we must rely on * Graduate student, Research .4ssistant t Associate Professor, Senior Member, AIA.4 f Professor 5 .41.4A member non-linear theory. Early non-linear studies were carried out by Landaus, Stuartg, Watsonl’,Reynoldsl’, Eagles12, et,c. Landau’s consideration lends to the well-known and most commonly used expansion for non-linear analysis which is referred to as the Landau’s series. This is also the basis for the present study. Using the Landau’s expansion, Stuart9 and Watsonl’ investigated the non-linear st,ability behavior of parallel flows. The procedure lends itself to the analysis of circular Couette flow and plane Poiseuille flow. Eagles12 studied the stability of Taylor vortices using the fifth-order amplitude expansion. STABILITY FORMULATIONS We define z, y and z as streamwise, normal and spanwise directions. The dependent variables of interest are p, U, ZI, w, T, and p, which are density, velocities in the z, y, z directions, temperature and pressure, respectively. For the convenience, we use variables, such as u, u, without any decorations to represent the total quantities. Paramet.ers for the base flow are denoted by small letters with subscript ‘00’ (e.g. uaa). Variables with subscript ‘10’ (e.g. 2~10) denot,e eigenfunctions of the linear part, which are equivalent to those with ‘ ’ on top (e.g. U) in most of the linear stability literature. Subscripts ‘Ol’, ‘20’, ‘11’ are used to denot.e the eigenfunctions for different non-linear parts. The reference quantities for nondimenionalization are those of the fast free stream, wherein the vorticity thickness 6, = Uc’n(+m)-uoo(-m) I* InlaX serves as the length scale. To simplify the stabilit.y analysis, we assume the following:
Cited by 0
Guillaume Alain, Yoshua Bengio · arXiv (Cornell University) · 2016
Neural network models have a reputation for being black boxes. We propose to monitor the features at every layer of a model and measure how suitable they are for classification. We use linear classifiers, which we refer to as "probes", trained entirely independently of the model itself. This helps us better understand the roles and dynamics of the intermediate layers. We demonstrate how this can be used to develop a better intuition about models and to diagnose potential problems. We apply this technique to the popular models Inception v3 and Resnet-50. Among other things, we observe experimentally that the linear separability of features increase monotonically along the depth of the model.
Cited by 117Open access
С. А. Гапонов, Б. В. Смородский · Journal of Applied Mechanics and Technical Physics · 2008
Cited by 16