1 2TYPE ROWCOL 3NAME UCS/DEVANAGA 4SRC_ZONE 0x0000-0x2212 5OOB_MODE INVALID 6DST_INVALID 0x100 7DST_UNIT_BITS 16 8#======================================================================= 9# File name: DEVANAGA.TXT 10# 11# Contents: Map (external version) from Mac OS Devanagari 12# encoding to Unicode 2.1 and later. 13# 14# Copyright: (c) 1995-2002, 2005 by Apple Computer, Inc., all rights 15# reserved. 16# 17# Contact: charsets@apple.com 18# 19# Changes: 20# 21# c02 2005-Apr-05 Update header comments; add section on 22# roundtrip considerations. Matches internal 23# xml <c1.1> and Text Encoding Converter 2.0. 24# b3,c1 2002-Dec-19 Update URLs. Matches internal utom<b1>. 25# b02 1999-Sep-22 Update contact e-mail address. Matches 26# internal utom<b1>, ufrm<b1>, and Text 27# Encoding Converter version 1.5. 28# n04 1998-Feb-05 First version; matches internal utom<n9>, 29# ufrm<n15>. 30# 31# Standard header: 32# ---------------- 33# 34# Apple, the Apple logo, and Macintosh are trademarks of Apple 35# Computer, Inc., registered in the United States and other countries. 36# Unicode is a trademark of Unicode Inc. For the sake of brevity, 37# throughout this document, "Macintosh" can be used to refer to 38# Macintosh computers and "Unicode" can be used to refer to the 39# Unicode standard. 40# 41# Apple Computer, Inc. ("Apple") makes no warranty or representation, 42# either express or implied, with respect to this document and the 43# included data, its quality, accuracy, or fitness for a particular 44# purpose. In no event will Apple be liable for direct, indirect, 45# special, incidental, or consequential damages resulting from any 46# defect or inaccuracy in this document or the included data. 47# 48# These mapping tables and character lists are subject to change. 49# The latest tables should be available from the following: 50# 51# <http://www.unicode.org/Public/MAPPINGS/VENDORS/APPLE/> 52# 53# For general information about Mac OS encodings and these mapping 54# tables, see the file "README.TXT". 55# 56# Format: 57# ------- 58# 59# Three tab-separated columns; 60# '#' begins a comment which continues to the end of the line. 61# Column #1 is the Mac OS Devanagari code or code sequence 62# (in hex as 0xNN or 0xNN+0xNN) 63# Column #2 is the corresponding Unicode or Unicode sequence 64# (in hex as 0xNNNN or 0xNNNN+0xNNNN). 65# Column #3 is a comment containing the Unicode name or sequence 66# of names. In some cases an additional comment follows the 67# Unicode name(s). 68# 69# The entries are in two sections. The first section is for pairs of 70# Mac OS Devanagari code points that must be mapped in a special way. 71# The second section maps individual code points. 72# 73# Within each section, the entries are in Mac OS Devanagari code order. 74# 75# Control character mappings are not shown in this table, following 76# the conventions of the standard UTC mapping tables. However, the 77# Mac OS Devanagari character set uses the standard control characters 78# at 0x00-0x1F and 0x7F. 79# 80# Notes on Mac OS Devanagari: 81# --------------------------- 82# 83# This is a legacy Mac OS encoding; in the Mac OS X Carbon and Cocoa 84# environments, it is only supported via transcoding to and from 85# Unicode. 86# 87# Mac OS Devanagari is based on IS 13194:1991 (ISCII-91), with the 88# addition of several punctuation and symbol characters. However, 89# Mac OS Devanagari does not support the ATR (attribute) mechanism of 90# ISCII-91. 91# 92# 1. ISCII-91 features in Mac OS Devanagari include: 93# 94# a) Overloading of nukta 95# 96# In addition to using the nukta (0xE9) like a combining dot below, 97# nukta is overloaded to function as a general character modifier. 98# In this role, certain code points followed by 0xE9 are treated as 99# a two-byte code point representing a character which may be 100# rather different than the characters represented by either of 101# the code points alone. For example, the character DEVANAGARI OM 102# (U+0950) is represented in ISCII-91 as candrabindu + nukta. 103# 104# b) Explicit halant and soft halant 105# 106# A double halant (0xE8 + 0xE8) constitutes an "explicit halant", 107# which will always appear as a halant instead of causing formation 108# of a ligature or half-form consonant. 109# 110# Halant followed by nukta (0xE8 + 0xE9) constitutes a "soft 111# halant", which prevents formation of a ligature and instead 112# retains the half-form of the first consonant. 113# 114# c) Invisible consonant 115# 116# The byte 0xD9 (called INV in ISCII-91) is an invisible consonant: 117# It behaves like a consonant but has no visible appearance. It is 118# intended to be used (often in combination with halant) to display 119# dependent forms in isolation, such as the RA forms or consonant 120# half-forms. 121# 122# d) Extensions for Vedic, etc. 123# 124# The byte 0xF0 (called EXT in ISCII-91) followed by any byte in 125# the range 0xA1-0xEE constitutes a two-byte code point which can 126# be used to represent additional characters for Vedic (or other 127# extensions); 0xF0 followed by any other byte value constitutes 128# malformed text. Mac OS Devanagari supports this mechanism, but 129# does not currently map any of these two-byte code points to 130# anything. 131# 132# 2. Mac OS Devanagari additions 133# 134# Mac OS Devanagari adds characters using the code points 135# 0x80-0x8A and 0x90-0x91 (the latter are some Devanagari additions 136# from Unicode). 137# 138# 3. Unused code points 139# 140# The following code points are currently unused, and are not shown 141# here: 0x8B-0x8F, 0x92-0xA0, 0xEB-0xEF, 0xFB-0xFF. In addition, 142# 0xF0 is not shown here, but it has a special function as described 143# above. 144# 145# Unicode mapping issues and notes: 146# --------------------------------- 147# 148# 1. Mapping the byte pairs 149# 150# If one of the following byte values is encountered when mapping 151# Mac OS Devanagari text - 0xA1, 0xA6, 0xA7, 0xAA, 0xDB, 0xDC, 0xDF, 152# 0xE8, or 0xEA - then the next byte (if there is one) should be 153# examined. If the next byte is 0xE9 - or also 0xE8, if the first 154# byte was 0xE8 - then the byte pair should be mapped using the 155# first section of the mapping table below. Otherwise, each byte 156# should be mapped using the second section of the mapping table 157# below. 158# 159# - The Unicode Standard, Version 2.0, specifies how explicit 160# halant and soft halant should be represented in Unicode; 161# these mappings are used below. 162# 163# If the byte value 0xF0 is encountered when mapping Mac OS 164# Devanagari text, then the next byte should be examined. If there 165# is no next byte (e.g. 0xF0 at end of buffer), the mapping 166# process should indicate incomplete character. If there is a next 167# byte but it is not in the range 0xA1-0xEE, the mapping process 168# should indicate malformed text. Otherwise, the mapping process 169# should treat the byte pair as a valid two-byte code point with no 170# mapping (e.g. map it to QUESTION MARK, REPLACEMENT CHARACTER, 171# etc.). 172# 173# 2. Mapping the invisible consonant 174# 175# It has been suggested that INV in ISCII-91 should map to ZERO 176# WIDTH NON-JOINER in Unicode. However, this causes problems with 177# roundtrip fidelity: The ISCII-91 sequences 0xE8+0xE8 and 0xE8+0xD9 178# would map to the same sequence of Unicode characters. We have 179# instead mapped INV to LEFT-TO-RIGHT MARK, which avoids these 180# problems. 181# 182# 3. Additional loose mappings from Unicode 183# 184# These are not preserved in roundtrip mappings. 185# 186# U+0958 0xB3+0xE9 # DEVANAGARI LETTER QA 187# U+0959 0xB4+0xE9 # DEVANAGARI LETTER KHHA 188# U+095A 0xB5+0xE9 # DEVANAGARI LETTER GHHA 189# U+095B 0xBA+0xE9 # DEVANAGARI LETTER ZA 190# U+095C 0xBF+0xE9 # DEVANAGARI LETTER DDDHA 191# U+095D 0xC0+0xE9 # DEVANAGARI LETTER RHA 192# U+095E 0xC9+0xE9 # DEVANAGARI LETTER FA 193# 194# 4. Roundtrip considerations when mapping to decomposed Unicode 195# 196# Both ISCII-91 (hence Mac OS Devanagari) and Unicode provide multiple 197# ways of representing certain Devanagari consonants. For example, 198# DEVANAGARI LETTER NNNA can be represented in Unicode as the single 199# character 0x0929 or as the sequence 0x0928 0x093C; similarly, this 200# consonant can be represented in Mac OS Devanagari as 0xC7 or as the 201# sequence 0xC6 0xE9. This leads to some roundtrip problems. First 202# note that we have the following mappings without such problems: 203# 204# ISCII/ standard decomposition of reverse mapping 205# Mac OS Unicode mapping standard mapping of decomposition 206# ------ ----------------------- ---------------- ---------------- 207# 0xC6 0x0928 ... LETTER NA 0x0928 (same) 0xC6 208# 0xCD 0x092F ... LETTER YA 0x092F (same) 0xCD 209# 0xCF 0x0930 ... LETTER RA 0x0930 (same) 0xCF 210# 0xD2 0x0933 ... LETTER LLA 0x0933 (same) 0xD2 211# 0xE9 0x093C ... SIGN NUKTA 0x093C (same) 0xE9 212# 213# However, those mappings above cause roundtrip problems for the 214# the following mappings if they are decomposed: 215# 216# ISCII/ standard decomposition of reverse mapping 217# Mac OS Unicode mapping standard mapping of decomposition 218# ------ ----------------------- ---------------- ---------------- 219# 0xC7 0x0929 ... LETTER NNNA 0x0928 0x093C 0xC6 0xE9 220# 0xCE 0x095F ... LETTER YYA 0x092F 0x093C 0xCD 0xE9 221# 0xD0 0x0931 ... LETTER RRA 0x0930 0x093C 0xCF 0xE9 222# 0xD3 0x0934 ... LETTER LLLA 0x0933 0x093C 0xD2 0xE9 223# 224# One solution is to use a grouping transcoding hint with the four 225# decompositions above to mark the decomposed sequence for special 226# treatment in transcoding. This yields the following mappings to 227# decomposed Unicode: 228# 229# ISCII/ decomposed 230# Mac OS Unicode mapping 231# ------ ---------------- 232# 0xC7 0xF860 0x0928 0x093C 233# 0xCE 0xF860 0x092F 0x093C 234# 0xD0 0xF860 0x0930 0x093C 235# 0xD3 0xF860 0x0933 0x093C 236# 237# Details of mapping changes in each version: 238# ------------------------------------------- 239# 240################## 241# Section 1: Map the following byte pairs as indicated: 242# (ZWNJ means ZERO WIDTH NON-JOINER, ZWJ means ZERO WIDTH JOINER) 243# (Also see note about 0xF0 in comments above) 244# Section 2: Map the remaining bytes as follows: 245# 246# 247# 248# 249BEGIN_MAP 2500x0000 - 0x007F = 0x00 - 2510x00A9 = 0x88 2520x00AE = 0x89 2530x00D7 = 0x80 2540x0901 = 0xA1 2550x0902 = 0xA2 2560x0903 = 0xA3 2570x0905 = 0xA4 2580x0906 = 0xA5 2590x0907 = 0xA6 2600x0908 = 0xA7 2610x0909 = 0xA8 2620x090A = 0xA9 2630x090B = 0xAA 264#0x090C = 0xA6+0xE9 2650x090D = 0xAE 2660x090E = 0xAB 2670x090F = 0xAC 2680x0910 = 0xAD 2690x0911 = 0xB2 2700x0912 = 0xAF 2710x0913 = 0xB0 2720x0914 = 0xB1 2730x0915 = 0xB3 2740x0916 = 0xB4 2750x0917 = 0xB5 2760x0918 = 0xB6 2770x0919 = 0xB7 2780x091A = 0xB8 2790x091B = 0xB9 2800x091C = 0xBA 2810x091D = 0xBB 2820x091E = 0xBC 2830x091F = 0xBD 2840x0920 = 0xBE 2850x0921 = 0xBF 2860x0922 = 0xC0 2870x0923 = 0xC1 2880x0924 = 0xC2 2890x0925 = 0xC3 2900x0926 = 0xC4 2910x0927 = 0xC5 2920x0928 = 0xC6 2930x0929 = 0xC7 2940x092A = 0xC8 2950x092B = 0xC9 2960x092C = 0xCA 2970x092D = 0xCB 2980x092E = 0xCC 2990x092F = 0xCD 3000x0930 = 0xCF 3010x0931 = 0xD0 3020x0932 = 0xD1 3030x0933 = 0xD2 3040x0934 = 0xD3 3050x0935 = 0xD4 3060x0936 = 0xD5 3070x0937 = 0xD6 3080x0938 = 0xD7 3090x0939 = 0xD8 3100x093C = 0xE9 311#0x093D = 0xEA+0xE9 3120x093E = 0xDA 3130x093F = 0xDB 3140x0940 = 0xDC 3150x0941 = 0xDD 3160x0942 = 0xDE 3170x0943 = 0xDF 318#0x0944 = 0xDF+0xE9 3190x0945 = 0xE3 3200x0946 = 0xE0 3210x0947 = 0xE1 3220x0948 = 0xE2 3230x0949 = 0xE7 3240x094A = 0xE4 3250x094B = 0xE5 3260x094C = 0xE6 3270x094D = 0xE8 328#0x094D+0x200C = 0xE8+0xE8 329#0x094D+0x200D = 0xE8+0xE9 330#0x0950 = 0xA1+0xE9 3310x095F = 0xCE 332#0x0960 = 0xAA+0xE9 333#0x0961 = 0xA7+0xE9 334#0x0962 = 0xDB+0xE9 335#0x0963 = 0xDC+0xE9 3360x0964 = 0xEA 3370x0965 = 0x90 3380x0966 = 0xF1 3390x0967 = 0xF2 3400x0968 = 0xF3 3410x0969 = 0xF4 3420x096A = 0xF5 3430x096B = 0xF6 3440x096C = 0xF7 3450x096D = 0xF8 3460x096E = 0xF9 3470x096F = 0xFA 3480x0970 = 0x91 3490x200E = 0xD9 3500x2013 = 0x82 3510x2014 = 0x83 3520x2018 = 0x84 3530x2019 = 0x85 3540x2022 = 0x87 3550x2026 = 0x86 3560x2122 = 0x8A 3570x2212 = 0x81 358END_MAP 359