Multilingual Computing, Sas Unicode Server · 2013
In today’s business world, information comes in many languages and you may have customers and employees in various countries all over the globe. It is very possible that your mission-critical data will be created and stored in more than one language. SAS offers several features that allow you to store and process multilingual data. With SAS 9.1, it is now possible to write a SAS application that processes data from many languages all in the same SAS session. This paper introduces the Unicode support that is provided in SAS 9.1 and discusses several scenarios for how you might use this support to deliver multilingual data to users around the world. CONCEPTS You should become familiar with the following basic concepts in order to understand this paper.
Cited by 0
Richard L. Apodaca · Front Matter · 2022
The CTfile family includes the widely-used serialization formats Molfile and SDfile. Users rarely need to consider the many details of these formats because they're handled by software. But data corruption can result when one utility misreads the output of another. The problem is especially hard to diagnose when the root cause lies close to the base layer.
Cited by 0Open access
L.R. Henderson, A.M. Mumford · Elsevier eBooks · 1990
Cited by 0
Richard L. Apodaca · Front Matter · 2022
The CTfile family includes the widely-used serialization formats Molfile and SDfile. Users rarely need to consider the many details of these formats because they're handled by software. But data corruption can result when one utility misreads the output of another. The problem is especially hard to diagnose when the root cause lies close to the base layer.
Cited by 0Open access
McEnery, A. M., Xiao, R. Z. · Lancaster EPrints (Lancaster University) · 2005
This chapter first briefly reviews the history of character encoding. Following from this is a discussion of standard and non-standard native encoding systems, and an evaluation of the efforts to unify these character codes. Then we move on to discuss Unicode as well as various Unicode Transformation Formats (UTFs). As a conclusion, we recommend that Unicode (UTF-8, to be precise) be used in corpus construction.
Cited by 2Open access
Christian Beck, Martín Lotze, Silke Anders · 2014
Cited by 0
Tony McEnery, R. Xiao · 2005
This chapter first briefly reviews the history of character encoding. Following from this is a discussion of standard and non-standard native encoding systems, and an evaluation of the efforts to unify these character codes. Then we move on to discuss Unicode as well as various Unicode Transformation Formats (UTFs). As a conclusion, we recommend that Unicode (UTF-8, to be precise) be used in corpus construction.
Cited by 16
Bossard Antoine, Keiichi Kaneko · Frontiers in artificial intelligence and applications · 2019
The glyphs of the Japanese writing system mainly consist of Chinese characters, and there are tens of thousands of such characters. Because of the amount of characters involved, glyph database creation and character representation in general on computer systems has been the focus of numerous researches and various software systems. Character information is usually represented in a computer system by an encoding. Some encodings target specifically Chinese characters: this is the case for instance of Big-5 and Shift-JIS. Tere are also encodings that aim at covering several, possibly all, writing systems: this the case for instance of Unicode. However, whichever the solution adopted, a significant part of Chinese characters remain uncovered by the current encoding methods. Thanks to the properties and relations featured by Chinese characters, they can be classified into a database with respect to various attributes. First, the formal structure of such a database is described in this paper as a character encoding, thus addressing the character representation issue. Importantly, we show that the proposed logical structure overcome the limitations of existing encodings, most notably the glyph number restriction and the lack of coherency in the code. This theoretical proposal will then be followed by the practical realisation of the proposed database and the visualisation of the corresponding code structure. Finally, an additional experiment is conducted to measure the memory size overhead that is induced by the proposed encoding, comparing with the memory size required by an implementation of Unicode. Once the files are compressed, the memory size overhead is significantly reduced.
Cited by 4
Andysah Putera Utama Siahaan · 2017
Security is crucial to maintaining the confidentiality of the information. Secure information is the information should not be known to the unreliable person, especially information concerning the state and the government. This information is often transmitted using a public network. If the data is not secured in advance, would be easily intercepted and the contents of the information known by the people who stole it. The method used to secure data is to use a cryptographic system by changing plaintext into ciphertext. The Base64 algorithm is one of the encryption processes that is ideal for use in data transmission. Ciphertext obtained is the arrangement of the characters that have been tabulated. These tables have been designed to facilitate the delivery of data during transmission. By applying this algorithm, errors would be avoided, and security would also be ensured.
Cited by 23
George M. White · 1972
An isolated word recognition system that uses character string encoding is described that has achieved 98% correct recognition scores on limited vocabularies (20-54 words). Speaker normalization, word segmentation, and learning paradigms have been incorporated. Audio input passes through a 6-channel octave band pass filter bank. The output of each channel is time integrated for 10 ms, and log mapped. An utterance is represented by a succession of points (a new point is generated every 10 ms) in the 6- dimensional space defined by the 6 octave bands. Reference points are scattered throughout the space. Each time interval is assigned the label of the nearest reference point. We call the resulting string of labels a "character string". Encoding an utterance into a character string may proceed with an arbitrary degree of precision, greater resolution resulting from the use of more reference points. Only 24 reference points are needed to achieve 98% correct recognition scores for 54 words in near real time. String generation techniques are explored. Several learning schemes based on character strings are described. Finally, experiments with a software classifier that uses "deformable templates" based on character strings are presented.
Cited by 6
NTUR (臺灣機構典藏) · 1985
Cited by 0
顾俊娟 · Institutional Repository of Institute of Psychology, Chinese Academy of Sciences (Institute of Psychology, Chinese Academy of Sciences) · 2014
大量研究证明：在拼音文字中，字母位置编码是非常灵活的。字母转置非词（单词中相邻两个字母调换位置）比字母替换非词（调换位置的两个字母被替换成另外两个字母）更像其基词，即转置非词比替换非词更容易激活其基词，这就是字母转置效应。在汉语书面或印刷文本中，词与词之间没有空格等明显的边界标记。本研究利用中文文本的特性，考察了中文阅读中的汉字位置编码问题。 研究一探讨了在中文双字词识别中，词素边界是否影响汉字位置编码。我们选取了两种词素结构不同的双字词：单纯词和合成词。在实验一，我们使用了掩蔽启动词汇决定任务。启动刺激或与靶子词相同（相同条件），或是靶子词的汉字转置非词（转置条件），或是靶子词的汉字替换非词（替换条件）。研究结果显示，替换条件的正确率低于转置条件；替换条件的反应时长于转置条件。无论是正确率还是反应时指标，词类型和启动条件的交互作用不显著。因此，中文词汇识别过程中存在汉字转置效应。但是，词素边界不能调节汉字位置编码。在实验二，我们使用边界范式，把靶子词嵌入到句子中。我们同样操纵了双字词的类型和预视刺激的类型。预视刺激类型也是相同条件、转置条件和替换条件。结果发现，靶子词上的注视时间在替换条件下比转置条件下的长得多。词类型和预视条件的交互作用不显著。因此，在副中央凹视野中存在汉字转置效应。词素边界不影响中文阅读中的汉字位置编码。在实验三，我们考察在一般的句子阅读场景中词素内转置和词素间转置对词汇识别的模式是否有差异。结果发现注视时间在转置条件显著长于相同条件。可见，在中央凹视野，词素边界信息不影响汉字位置编码。而且，汉字位置编码发生在词汇加工的早期阶段。 研究二探讨了在中文阅读中词边界是否影响汉字位置编码。我们选取了四字成语和两个双字词作为不同类型的靶子刺激。与研究一相同，在实验四和实验五，我们分别使用了掩蔽启动词汇决定任务和边界范式。启动和预视刺激或与靶子刺激相同（相同条件），或是靶子刺激中间两个字转置后的非词（转置条件），或是靶子刺激中间两个字被其它字替换后的非词（替换条件）。结果显示：在两个双字词条件，转置条件的反应时和注视时间显著长于相同条件；而在一个词条件，反应时和注视时间在转置条件和相同条件没有差异，而反应时和正确率在替换条件比转置条件更长，即存在很强的汉字转置效应。实验六在更为自然的句子阅读场景考察词边界效应。结果发现，靶子词的首次注视时间在跨词边界汉字转置条件显著长于相同条件。首次注视时间在词内汉字转置和相同条件没有差异。 总之，在中文阅读中汉字位置得到编码，存在汉字转置效应。词边界信息影响汉字位置编码。词的切分发生在词汇识别的早期阶段，可能与汉字位置编码同时发生，或者早于汉字位置编码。
Cited by 0
Hf. Zhu, Tsinghua U, Dy. Hu et al. · 1922
Cited by 0
Wei Liu, Xu Tongge, Qinghua Xu et al. · 2019
Wei Liu, Tongge Xu, Qinghua Xu, Jiayu Song, Yueran Zu. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Cited by 86Open access
Y Wang · 2014
According to the present situation of various character encoded modes,it has become a problem,which can't be ignored,that how softwares judge which character set fi le the target fi le belongs to.In this paper, aiming at the fact that Notepad can't display the Unicom correctly,the Windows Bug,using software Winhex analysis the fi le to obtain 16 binary codes,according to the codes to guess the cause of misjudgment,exegesis function IsTextUTF8 of Notepad to prove that,fi nally it properly showed the root cause of that Unicom can't display correctly,and found more Chinese characters which Windows Notepad can't display correctly.
Cited by 0
Sabri A. Mahmoud · Pattern Recognition · 1994
Cited by 91
Constable, Peter G. · The COCOON platform (University of Paris) · 2001
https://www.sil.org/resources/archives/5753
Cited by 0
Bahit, Eugenia · Figshare · 2019
Alnum is theoretical model of an alphanumeric character encoding system. This model starts from the premise that the alphanumeric treatment of characters, at a high level, could suppose a reduction to zero of the risk of injections of malicious code.
Cited by 0Open access
Jun Murai, Kazuhiko Yamamoto, Eiiti Wada · 1999
Cited by 0
Bahit, Eugenia · Figshare · 2019
Alnum is theoretical model of an alphanumeric character encoding system. This model starts from the premise that the alphanumeric treatment of characters, at a high level, could suppose a reduction to zero of the risk of injections of malicious code.
Cited by 0Open access