C. R. Cahn, G. K. Huth, Christopher I. Moore · IEEE Transactions on Communications · 1973
Convolutional encoding with biphase modulation is an attractive combination of modulation/coding for the Gaussian channel, but a means of tracking carrier phase must be provided. The performance of combined sequential decoding and suppressed carrier (Costas loop) tracking in the receiver is difficult to analyze but can be obtained by computer simulations. Results of such simulations are presented in the form of a Pareto fit to the computation distributions for various tracking loop bandwidths and demodulator quantization (soft decisions). These distributions are applied to predict probability of error performance for typical decoder parameters (buffer size, speed factor). The anticipated advantage of quantized demodulation is observed. However, narrow tracking bandwidths are required, with sufficient interleaving to overcome effects of correlated phase perturbations due to noise.
Cited by 4
Bin Xiao, Chunan Shi, Xiaonan Nie et al. · arXiv (Cornell University) · 2024
Large language models (LLMs) suffer from low efficiency as the mismatch between the requirement of auto-regressive decoding and the design of most contemporary GPUs. Specifically, billions to trillions of parameters must be loaded to the GPU cache through its limited memory bandwidth for computation, but only a small batch of tokens is actually computed. Consequently, the GPU spends most of its time on memory transfer instead of computation. Recently, parallel decoding, a type of speculative decoding algorithms, is becoming more popular and has demonstrated impressive efficiency improvement in generation. It introduces extra decoding heads to large models, enabling them to predict multiple subsequent tokens simultaneously and verify these candidate continuations in a single decoding step. However, this approach deviates from the training objective of next token prediction used during pre-training, resulting in a low hit rate for candidate tokens. In this paper, we propose a new speculative decoding algorithm, Clover, which integrates sequential knowledge into the parallel decoding process. This enhancement improves the hit rate of speculators and thus boosts the overall efficiency. Clover transmits the sequential knowledge from pre-speculated tokens via the Regressive Connection, then employs an Attention Decoder to integrate these speculated tokens. Additionally, Clover incorporates an Augmenting Block that modifies the hidden states to better align with the purpose of speculative generation rather than next token prediction. The experiment results demonstrate that Clover outperforms the baseline by up to 91% on Baichuan-Small and 146% on Baichuan-Large, respectively, and exceeds the performance of the previously top-performing method, Medusa, by up to 37% on Baichuan-Small and 57% on Baichuan-Large, respectively.
Cited by 0Open access
Savage,J. E. · Defense Technical Information Center (DTIC) · 1964
A model for a binary channel with additive Markovian noise is presented. The behavior of the Sequential Decoding cutoff rate, R(comp), for this channel is studied under various degrees of memory. It is shown that R(comp) decreases from its memoryless value whereas channel capacity increases when errors begin to cluster. This apparent paradox is explained and the smallmemory deterioation in R(comp) is shown to be correctable by elimination of the channel memory through scrambling. This study should serve primarily as an instrument for understanding the effects of memory on Sequential Decoding. (Author)
Cited by 0
Dmitry Osipov, D. V. Titov · Cybernetics and Information Technologies · 2018
Abstract The following paper adapts the classical Zigangirov-Jelinek algorithm to the decoding of nonbinary block codes under severe mixed jamming. To ensure reliable communications in this scenario we combine reception techniques based on distribution free statistical tests with sequential decoding on syndrome trellises. It will be shown that the proposed approach can ensure relatively high transmission rate with reasonable complexity.
Cited by 4Open access
Lenny Grokop · 2005
Sequential decoding algorithms decode convolutional codes by guessing their way through the expanding tree of possible transmitted sequences. In this way computational complexity is reduced. Generally it comes at the cost of having to communicate at rates distinctly below capacity. The computational cutoff rate Rcomp delineates the border between those rates for which the average decoding time is bounded and those for which it is infinite. A series of papers establishes Rcomp = E0(1), where E0(ρ) is the Gallager function. 1 Convolutional Codes Convolutional codes use memory to spread the message digits over time in order to average out the channel noise. An example of a convolutional encoder is shown in figure (1). The encoder is defined by three parameters λ-the number of digits inputed at each time point, L-the constraint length and ν-the number of digits outputed at each time point. The rate of the code is defined as R � λ/ν log(2) in nats. The constraint length plays the same role as the length of a block code. If the sequence of message digits is denoted u (1) 1, u (2)
Cited by 0
Guo, Li · eScholarship (California Digital Library) · 2023
Complex behaviors are assembled from simpler movements executed in sequence. However, how each component is controlled and how their combination is coordinated is not well understood. Here, we used fly grooming as an advantageous system to investigate two fundamental questions of motor control in sequential behavior.1) How are different actions coordinated within a sequence? Descending neurons, which connect the brain to the nerve cord, are a key bottleneck in the sensorimotor circuits and are therefore idea to address this question. For anterior grooming behavior in fly, we find that its subroutines, head sweeps and leg rubs, can be initiated separately, or as a set, by different descending neurons. Moreover, through genetic manipulation, behavioral assays, and connectomic analysis, we show that both the circuits in the brain and ventral nerve cord participate in coordinating the alternation of these two movements. We propose that the existence of redundant regulation strategies can provide backup plan for an important behavior. It ensures the behavior can be executed the best way under different scenarios and ensure the sequence to be executed without confusion.\n 2) How is each action coordinated? Flies rub their legs in response to the light touch which is location-dependent: the stimulated leg becomes the target. Depending on the segment, leg rubbing involves either contralateral legs or the neighboring ipsilateral, intersegmental legs. Here, we are interested in how sensory inputs from different body regions are processed to generate proper motor outputs. We identified a class of segmentally repeated command-like interneurons whose activation exhibits segment-specific leg rubbing responses. We uncovered a local sensorimotor circuit present in all segments that we propose reinforces the selection of the target leg. We also found segmental differences in the pre-motor wring that explains which leg are recruited for leg rubbings. These anatomical and behavioral results demonstrate how neuron wiring coordinates the segmentally different leg rubbing in grooming.\nIn conclusion, our studies on fly grooming shed light on the neural circuits and coordination rules underlying this complex behavior, offering valuable insights into broader principles of motor control. \n
Cited by 0
C.Y. Chang, K. Yao · International Journal of High Speed Computing · 1989
A systolic array processing technique is applied to implementing the stack algorithm form of the sequential decoding algorithm. It is shown that sorting, a key function in the stack algorithm, can be efficiently realized by a special type of systolic arrays known as systolic priority queues. Compared to the stack-bucket algorithm, this approach is shown to have the advantages that the decoding always moves along the optimal path, that it has a fast and constant decoding speed and that its simple and regular hardware architecture is suitable for VLSI implementation. Three types of systolic priority queues are discussed: random access scheme, shift register scheme and ripple register scheme. The property of the entries stored in the systolic priority queue is also investigated. The results are applicable to many other basic sorting type problems.
Cited by 2
V.N. Koshelev · IEEE Transactions on Information Theory · 1973
The method of sequential encoding and decoding is generalized to the case of a source with redundancy. A computational entropy of the source analogous to the computational cutoff rate of the channel is introduced. A range of transmission rates is found for which the average number of decoding computations is finite.
Cited by 20
Tsui, Steven · Library and Archives Canada (Government of Canada) · 1998
Cited by 0
M. Cedervall, Rolf Johannesson, Kamil Sh. Zigangirov · Lund University Publications (Lund University) · 1984
Cited by 1
Mikhail Kamenev · arXiv (Cornell University) · 2021
A soft-input sequential decoder for Reed-Muller (RM) codes of length $2^m$ and order $m - 3$ is proposed. The considered algorithm sequentially processes different permuted versions of the received vector using a decoder of an extended Hamming code, with permutations being selected on-the-fly from the RM codes' automorphism group based on soft information from a channel. It is shown that the proposed algorithm outperforms the recursive list decoder with similar computational complexity and achieves near maximum-likelihood decoding performance with reasonable computational complexity for RM codes of length 512 and 1024.
Cited by 0Open access
Gunilla Bratt, Rolf Johannesson, Kamil Sh. Zigangirov · Lund University Publications (Lund University) · 1987
Cited by 0
N. Stolte, Ulrich Sorger, G. M. Sessler · 2000
In this paper we propose a sequential stack algorithm to decode binary Reed-Muller codes which is based on the GC structure of these codes. A key problem of sequential decoding is the estimation of errors in the unexplored part of the code tree which in case of convolutional codes is traditionally done using the Fano-metric. We show that with proper defined soft values it is possible to calculate for each element in the stack a lower bound on the squared Euclidean distance between the received sequence and all codewords containing the element. This value can be used as a metric to evaluate the elements in the stack. It is further shown how this metric can be improved by lower bounding the metric increase in future decoding steps. The influence of this improvement on the necessary stack size is demonstrated by simulation results which underline the performance of sequential decoding of binary ReedMuller codes.
Cited by 6
Anisha Banerjee, Andreas Lenz, Antonia Wachter-Zeh · 2022 IEEE Information Theory Workshop (ITW) · 2022
In this work, a sequential decoder for convolutional codes over channels that are vulnerable to insertion, deletion, and substitution errors, is described and analyzed. The decoder expands the code trellis by introducing a new channel state variable, called drift state, as proposed by Davey-MacKay. A suitable decoding metric on that trellis for sequential decoding is derived, in a manner that generalizes the original Fano metric. Under low-noise environments, this approach reduces the decoding complexity by a couple orders of magnitude in comparison to Viterbi’s algorithm. An analytical method to determine the computational cutoff rate is also suggested. This analysis is supported with numerical evaluations of bit error rates and computational complexity, which are compared with respect to optimal Viterbi decoding.
Cited by 3
Si Wu, Danmei Chen, Mahesan Niranjan et al. · Neural Computation · 2003
Population coding is a simplified model of distributed information processing in the brain. This study investigates the performance and implementation of a sequential Bayesian decoding (SBD) paradigm in the framework of population coding. In the first step of decoding, when no prior knowledge is available, maximum likelihood inference is used; the result forms the prior knowledge of stimulus for the second step of decoding. Estimates are propagated sequentially to apply maximum a posteriori (MAP) decoding in which prior knowledge for any step is taken from estimates from the previous step. Not only do we analyze the performance of SBD, obtaining the optimal form of prior knowledge that achieves the best estimation result, but we also investigate its possible biological realization, in the sense that all operations are performed by the dynamics of a recurrent network. In order to achieve MAP, a crucial point is to identify a mechanism that propagates prior knowledge. We find that this could be achieved by short-term adaptation of network weights according to the Hebbian learning rule. Simulation results on both constant and time-varying stimulus support the analysis.
Cited by 20
Horner, Daniel A.; Rescigno, Thomas N.; McCurdy, C. William · University of North Texas Digital Library (University of North Texas) · 2008
Above 54.4 eV, two-photon double ionization of helium isdominated by a sequential absorption process, producing characteristicbehavior in the single and triple differential cross sections. We showthat the signature of this process is visible in the nuclear recoil crosssection, integrated over all energy sharings of the ejected electrons,even below the threshold for the sequential process. Since nuclear recoilmomentum imaging does not require coincident photoelectron measurement,the predicted images present a viable target for future experiments withnew short-pulse VUV and soft X-ray sources.
Cited by 0Open access
Ajay Dholakia, H. Koorapaty, Donald L. Bitzer et al. · 1995
Cited by 0
Ron Dabora, Sergio D. Servetto · 2006
discrete output alphabets Y1 and Y and a probability density function p(y,y1|x1,x2) giving the probability distribution on Y×Y1 for each (x1,x2) ∈ X1×X2. The relay channel is called memoryless if the probability of a block of n transmissions is given by p(y,y1|x1,x2) = ∏ n i=1 p (yi,y1,i|x1,i,x2,i). Definition 2 A (2 nR,n) code for the relay channel consists of a source message set W = { 1,2,...,2 nR}, a mapping function f at the encoder, f: W ↦ → X n 1, a set of n relay functions x2,i = ti (y1,1,y1,2,...,y1,i−1), where the i’th relay function ti maps the first i − 1 channel inputs at the relay into a transmitted relay symbol at time i. Lastly we have a decoder g: Y n ↦ → W. Definition 3 The average probability of error of a code with length n for the relay channel is defined as = Pr(g(Y n) = W), where W is selected uniformly over W.
Cited by 0
Chen‐Yi Lee · IEE Proceedings - Communications · 1994
The authors present a new sequential decoding algorithm based on dynamic searching strategy to improve decoding efficiency. The searching strategy is to exploit both 'sorting' and 'path recording' techniques. By means of sorting, it is possible to identify the correct path in a very fast way and then, by path recording, the bit sequence can be recovered without degrading decoding performance. The authors also develop a conditional resetting scheme to overcome the buffer overflow problem encountered in conventional sequential decoding algorithms. Simulation results show that, for a given code, the decoding efficiency remains the same as that obtained from maximum likelihood function by appropriately selecting sorting length and decoding depth. In addition, this algorithm can easily be mapped onto an area-efficient VLSI architecture to implement long constraint length convolutional decoders for high-speed digital communications.
Cited by 1
Ravi Sivasankaran · UPT. Syiah Kuala University Library (Syiah Kuala University) · 2000
Cited by 0Open access