CDAC-IDN-Critique

Malayalam IDN Policy Draft by CDAC - Critique by SMC

The Draft policy document by CDAC is available here

File:Response to C-DAC-IDN-Critique.pdf

Criticism on the policy

General Comments

1. The variant table is defined based on random glyphs taken from a list of 900+ possible glyphs for Malayalam. No explanation is given on how two entries in variant table become homo morphs. One entry in variant table is just because of the fact that one is mirror image of other. Since b, d are not excluded from English, there's no need to exclude mirror imaged glyphs in variant table.

   CDAC's Response: The IDN system devised for Malayalam is based only on the modern script. It doesn't address the old script or the fonts based on old script. Also, a detailed study was done before proposing homographs in each of the languages. The study included observing the visual form of the conjunct in the point size of the Address bars of major browsers. The mirror imaged nature of the glyphs was not the criterion for the two glyphs to be qualified as variants.

2. Visually identical glyphs are not the only entries to be considered for the variant table. Unicode chart itself has ambiguous dual representations for the same code point without canonical equivalence. An example for this is au signs in Tamil and Malayalam. ௗ- ௌ and ൗ - ൌ . The document does not consider these special cases.

   CDAC's Response: The IDN policy does not permit the entry of syllables having structure CMM or MCM, where M stands for Matra or vowel sign. The ABNF rules takes care of this.

   ൗ - ൌ are neither CMM nor MCM case. It is single code pointed Mathra(vowel signs), appearing with consonants in CM format alone. സന്തോഷ് 09:03, 2 December 2010 (PST)

3. There are different orthographic forms for many glyphs in Malayalam. The variant table does not address different scenarios arising while considering the visual similarity. For example in traditional orthography TTA is written in stacked form (റ്റ). While in modern orthography it can be written in non-stacked form and this non-stacked form is visually identical to two RA sequence (ററ).

   CDAC's Response : Only the stacked form is considered to be the conjunct TTA in modern orthography.

ABNF rules

1. Section 2 says ക് as pure consonant of ക. Chillu of ക്‍ is considered as pure consonant of ka.

   CDAC's Response: The policy document doesn't address the obsolete characters in the script, although those characters might have been included in Unicode code chart.

2. Section 2.a says CM can be followed by only D (anuswara) or X (visarga). This excludes the Samvruthokarams of Malayalam. All consonant can have cons + u vowel sign + virama and forming samvruthokaram form of that consonant. Examples: തു് , കു് , പു് , രു് .

   CDAC's Response: The use of samvruthokarams is considered to be the part of traditional orthography which the policy doesn't permit.

3. Section 3.a restrict the count of consonant in syllable as 4. But ഗ്ദ്ധ്ര്യ has 5 consonants

   CDAC's Response : Complex conjuncts like ഗ്ദ്ധ്ര്യ have been simplified in modern orthography.

4. Section 3.b excludes syllables with samvruthokram like ക്കു് .
5. Section 4 states a chillu can be followed by a vowel sign. Since chillu is dead consonant, there is no possibility of having virama after chillu.

   CDAC's Response : The document doesn't state that a chillu can be followed by a vowel sign. The observation that a virama can appear after a chillu is based on the recommendation of Unicode

6. The example used for LHC - ന്‍്റ does not exist in printing or digital format. None of the input methods or Malayalam writers ന്റ in this way. The sequence for nta is ന + ് + റ . ie there is no LHC sequence in Malayalam.

   CDAC's Response: The ന്‍്റ happened because none of the rendering engines available today does support the rendering of 'nTa' in the Unicode 5.1 way. as displayed in the document is the wrongly rendered form of the conjunct 'nTa'. This

5.1 official document on rendering the conjunct 'nTa'.

1. Since LHC is invalid for Malayalam, including L = ന്‍ , section 5 of the document cancels itself.
2. Because of argument #6, section 6 also cancels itself.
3. Because of arguments #1 to #8 the IDN rule "Consonant Sequence → *3(CH) C [H|D|X|M[D|X]] | L[HC[D|H|M[D]]]" is completely wrong and need to be reformulated.

Restriction Rules

1. Section 2 says "H is not permitted after V, D, X, M, digit and dash" This is wrong since samvruthokaram requires H after V

   CDAC's Response : See the explanation for section 2 under ABNF Rules

2. Section 7 says H can follow L if it is followed by റ , This is wrong as explained above. L can never followed by H. It can only followed by C

   CDAC's Response : See the explanation for section 5 under ABNF Rules

nta criticism

This document does not address the case of stacked and non stacked forms of nta, which are interchangeably used. For example എന്റെ can be spoofed with എന്‍റെ. Severity of this issue is increased by having one more sequence to represent the same conjunct (ന്‍ + ് + റ) is introduced in Unicode 5.1

CDAC's Response : Modern orthography treats ന്‍റ as stacked and ന്‍റ as non-stacked forms for the conjunct 'nTa' is wrong by convention. as 'nRa' and ന as 'nTa'. The interchangeable usage of stacked and non-stacked forms for the conjunct 'nTa' is wrong by convention.

Chart of allowed characters

1. Malayalam chillus - the 5.1 version ക്‍ is removed from the tables. which is having same characteristics and use cases of other chillus. So excluding it from the allowed code points does not make any sense. Moreover the existing chillu representation - non-atomic - is not mentioned in the document at all.
2. Malayalam au sign - ൌ is not allowed. Instead the au length mark ൗ is provided. The inscript standard does not allow one to type ൗ and allows only ൌ. Other input methods allows to type both. But the document does not say anything on the equivalence of both. Allowing both vowel signs is also a spoofing issue. And hence this should be handled in variant table.

   CDAC's Response : The inscript standard being revised. The new standard allows both the characters to be inputted. For restricting spoofing and phishing, only one form i.e. ൗ by IDN policy (used in modern text) has been allowed

Variant Table and Visual Spoofing

Variant table is not logical. Only ളള and ള്ള makes sense. None of the other entries should be considered as spoofing. ന്ത and ന്ന is not even close. Mirror images are already used in Latin, eg. b and d. Hence സ്സ and ഡ്ഡ cannot be blocked. Moreover it is not clear why the same logic does not apply for സ and ഡ. It did not consider the case of ററ and non stacked form of റ്റ common in new lipi.

CDAC's Response : The variant table is based on the observations how Malayalam characters and conjuncts are rendered in the address bars of standard browsers like IE, Mozilla and Safari. While ന്ത and ന്ന are perfectly rendered in Mozilla and Safari, they are not legibly rendered in various versions of IE. The mirror imaged nature of the glyphs was not the criterion for the two glyphs to be qualified as variants. Also note that the variant table is not a full-proof mechanism which can prevent spoofing. Even though similarity is considered, dual encoding is not mentioned. In case of dual encoding of chillus, both forms (atomic chillu and consonant chandrakkala ZWJ) of chillus will look SAME.

Even though similarity is considered, dual encoding is not mentioned. In case of dual encoding of chillus, both forms (atomic chillu and consonant chandrakkala ZWJ) of chillus will look SAME.

CDAC's Response : IDN policy doesn't allow control characters such as ZWJ and ZWNJ to be part of domain names.

Conclusion

The CDAC Policy document on Malayalam IDN is not acceptable without correcting the above explained errors. In its current format, the document was prepared with lot of false assumptions and contains many technical mistakes as pointed out above. Consultation is must with all stake holders of Malayalam Computing before preparing such an important document. This was not happened and we expect such an initiative from the authorities.

Discussion

1. Discussion about the CDACs response: http://lists.smc.org.in/pipermail/discuss-smc.org.in/2010-December/011985.html