1 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
3 * create-mscompat-collation-table.cs : fixed a few diacritical weight
4 on Cyrillic characters. Fixed ParseTailoringSource() to handle
5 non-heading escape sequence (\uXXXX) as expected.
7 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
9 * create-mscompat-collation-table.cs,
10 MSCompatUnicodeTableUtil.cs,
11 MSCompatUnicodeTable.cs : added more aggressive index limits for
12 table optimization at data size, in cost of speed.
14 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
16 * create-mscompat-collation-table.cs : fixed Arabic thirtial weight.
18 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
20 * create-mscompat-collation-table.cs : Mapping for hyphens and
21 punctuation are kinda finished. Rewrote batch mapping method to
22 collect all NFKD. Required modification on mapping is done.
24 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
26 * create-mscompat-collation-table.cs : minor mapping fixes on accent
27 marks and punctuations.
29 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
31 * create-mscompat-collation-table.cs : Fixed some MathSymbol mapping
32 and Box drawing mapping.
34 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
36 * create-mscompat-collation-table.cs : Fixed almost all numbers.
38 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
40 * create-mscompat-collation-table.cs : Symbol mappings are almost done.
41 Removed hack that gave dummy mappings to blank symbols.
43 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
45 * create-mscompat-collation-table.cs : more fix on arrows. Fix on box
46 drawings. Some code refactoring to eliminate hack.
48 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
50 * create-mscompat-collation-table.cs : Fixed some secondary weight
51 in Devanagari and arrows.
53 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
55 * create-mscompat-collation-table.cs : a set of tiny mapping fixes.
57 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
59 * create-mscompat-collation-table.cs : some diacritical fixes for
60 Latin. Added batch mapping method that considers computed
61 diacritical weight (for numbers).
63 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
65 * managed-collation.patch : forgot to add System.String patch.
67 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
69 * MSCompatUnicodeTable.cs : added resource existence check (required
70 for mscorlib transient time from the one without resources to the
73 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
75 * create-mscompat-collation-table.cs : fixed punctuations and hyphen
76 (shift) primary weight.
78 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
80 * create-mscompat-collation-table.cs : more nonspacing mark fixes.
81 Some non-basic Cyrillic diacritical weight fixes.
83 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
85 * create-mscompat-collation-table.cs : some Gurmukhi fixes on level 1
86 and level 3. Tiny Hangul weight fixes.
87 * MSCompatUnicodeTable.cs : U+30F5 and U+30F6 are small Japanese.
89 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
91 * create-mscompat-collation-table.cs : some normal characters who have
92 "narrow" NFKD mapping are regarded as "wide" and thus level 3 weight
93 values were different. Handle U+30FB as category A.
94 * MSCompatUnicodeTable.cs : U+30FB does not have special weight.
96 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
98 * create-mscompat-collation-table.cs : more diacritical weight fixes.
99 Removed some unused code.
101 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
103 * create-mscompat-collation-table.cs : Fixed some Thai and Arabic
106 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
108 * create-mscompat-collation-table.cs : Fixed Syriac nonspacing marks.
110 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
112 * create-mscompat-collation-table.cs : Fixed nonspacing marks in
113 Malayalam, Thai and Lao. Removed extraneous hack.
115 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
117 * SimpleCollator.cs : rewrote LastIndexOf() to handle source extenders.
118 Some refactoring on IndexOf() code. Removed unused Matches().
119 * Collation-notes.txt : some methods needed to be reimplemented, so
120 rewrote the description.
122 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
124 * SimpleCollator.cs : rewrote IsSuffix() to use CompareInternal().
125 Thus supported extenders in IsSuffix().
127 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
129 * SimpleCollator.cs : more IsSuffix() simplification, but it will be
130 stopped here since it cannot handle extenders (implementing new
133 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
135 * SimpleCollator.cs : simplified IsSuffix() code.
137 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
139 * SimpleCollator.cs : Fixed IndexOf() and LasIndexOf() to search the
140 entire replacement string if char target was an expansion.
141 IsSuffix() was using a method for IsPrefix() which was incorrect.
142 Removed old IsPrefix() code.
144 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
146 * SimpleCollator.cs : IndexOf() was incorrectly sharing the same
147 byte[] field in different areas of code. Now extenders in both
148 source and target really work in IndexOf().
150 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
152 * create-mscompat-collation-table.cs : fixed U+FF9F diacritical weight.
153 * SimpleCollator.cs : handle U+FF9E and U+FF9F as extenders.
155 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
157 * SimpleCollator.cs : Now FilterExtender() handles all extender
158 support. IndexOf() and LastIndexOf() now supports extenders.
159 IndexOf() and LastIndexOf() did not proceed contraction source
160 length as expected. Tiny refactoring on private IsPrefix() to take
163 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
165 * SimpleCollator.cs : when restoring from expansion, go back to the
166 top of the loop (to avoid index out of range).
167 Now IsPrefix() is implemented to reuse Compare() and thus it now
168 supports extender as well.
169 * Collation-notes.txt : status update. Deleted optimization part in
170 status section (it is duplicate).
172 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
174 * SimpleCollator.cs : some code reordering.
175 * create-mscompat-collation-table.cs : it was still missing U+3094.
177 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
179 * SimpleCollator.cs : Compare() now supports extender (e.g. U+39FC).
181 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
183 * SimpleCollator.cs : In GetSortKey(), don't update previousChar when
184 it is not primary (e.g. don't "extend" diacritical mark).
186 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
188 * managed-collation.patch : CompareInfo.Compare() should consider
189 the possibilities that non-empty string might be actually empty
190 in culture-sensitive context.
192 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
194 * SimpleCollator.cs : IndexOf() and LastIndexOf() returns start when
195 target is "empty" (in culture-sensitive context).
197 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
199 * SimpleCollator.cs : In IndexOf() and LastIndexOf(), skip ignorable
200 characters in target string.
202 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
204 * SimpleCollator.cs : When IgnoreWidth is specified, all Kana
205 characters are regarded as half-width.
206 Even though IgnoreWidth is specified, it should not ignore case.
207 For special weight comparison, the default values (E4) are bigger
208 than non-default values.
209 * SortKeyBuffer.cs : It should save LCID and original string.
210 * create-mscompat-collation-table.cs : For Japanese half-width kana,
211 it should not be counted in widthCompat map since IgnoreWidth does
212 not really ignore those differences.
214 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
216 * create-mscompat-collation-table.cs : Fixed missing Japanese bits.
218 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
220 * create-mscompat-collation-table.cs :
221 tiny diacritical weight fix for U+20D0-U+20E1.
223 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
225 * create-mscompat-collation-table.cs : ja CJK ideograph got completed.
227 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
229 * create-mscompat-collation-table.cs : Fixed CJK custom Japanese
230 mapping. It (maybe as well as other CJK tables) mixes NFKD. For
231 Japanese, modified NFKD table (because of Windows lame design).
233 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
235 * Makefile : added MONO_USE_MANAGED_COLLATION=no almost everywhere.
236 * MSCompatUnicodeTable.cs : FillCJK() was not invoked. Now it is
237 invoked at any time it is required.
238 * SimpleCollator.cs : call FillCJK() above in .ctor().
239 * MSCompatUnicodeTableUtil.cs : CJK range was wider.
240 * create-mscompat-collation-table.cs : CJK binary was missing the
241 length. CJK remapping is being moved to ModifyUnidata().
242 For cjk-ja mapping, we have to consider compat characters to be
243 added to the map, besides the raw UCA table.
245 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
247 * SortKeyBuffer.cs : Fixed shift level computation to match w/ Windows.
249 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
251 * SimpleCollator.cs : fixed LastIndexOf() to handle _target's_
252 contraction as expected. Fixed Compare() to save s2's contraction
254 * TestDriver.cs :added LastIndexOf() tester w/ indexes.
256 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
258 * managed-collation.patch : Fixed IsPrefix() and IsSuffix(). They
259 incorrectly use Compare().
260 * TestDriver.cs : more moved to nunit tests.
262 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
264 * SimpleCollator.cs : several fixes on Compare().
265 - Ignorable characters are skippted at the top of the loop.
266 - IgnoreNonSpace is checked to avoid extraneous level 2 comparison.
267 - In such case that s1 index is increased while s2 contraction is
268 replaced, s1 is inconsistently proceeded (bug).
269 - IsIgnorable() now also checks IgnoreNonSpace.
270 - Fixed FilterOptions() that does not work for IgnoreWidth at all.
271 * TestDriver.cs : now some are moved to nunit tests.
272 * Collation-notes.txt : minor todo update.
274 2005-07-11 Atsushi Enomoto <atsushi@ximian.com>
276 * SimpleCollator.cs : Compare() was ignoring such case that both
277 entire strings have '-' to be compared.
278 * Collation-notes.txt : more status updates.
279 * TestDriver.cs : added '-' use cases.
281 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
283 * SimpleCollator.cs : to be same as other buggy part, it now handles
284 U+3005, U+3031 and U+3032 as buggy as Windows. It just repeats
286 Fixed GetSortKey(): if the repeater is U+3005, second weight is 5.
287 * create-mscompat-collation-table.cs : dummy values for extenders.
289 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
291 * SimpleCollator.cs : Special weight fixes on GetSortKey(). Dash type
292 should be computed from ExtenderType, and voice mark weight should
294 * MSCompatUnicodeTable.cs : added tiny comment.
296 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
298 * SortKey.cs : It borked when MONO_USE_MANAGED_COLLATION is not yes.
299 * SimpleCollator.cs : support for extender (U+309D etc.).
301 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
303 * create-mscompat-collation-table.cs : some punct/symbols fix.
304 * managed-collation.patch : new (and temporary) file to support
305 managed collation in mscorlib.
306 * README : described how to use managed collation.
308 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
310 * create-mscompat-collation-table.cs : Further Cyrillic fixes. Handle
311 U+482-4C8 (though needs diacritical fixes).
312 * MSCompatUnicodeTable.cs : tiny comment for alternative impl.
314 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
316 * create-mscompat-collation-table.cs : Reimplemented Cyrillic weight
317 computation code, since it looks like the same way as Latin letters
318 have. Thus removed all other approach (UCA, by letter name).
320 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
322 * create-mscompat-collation-table.cs : diacritical fix for "double-
323 struck". Syriac nonspacing fixes.
325 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
327 * create-mscompat-collation-table.cs : more math symbol weight fixes.
329 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
331 * create-mscompat-collation-table.cs : fixed Hebrew character sortkeys.
333 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
335 * create-mscompat-collation-table.cs : math symbols U+25A0-U+2600 are
336 implemented (no stub). Some other fixes on category 8-A.
338 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
340 * create-mscompat-collation-table.cs : some minor fixes on Arabic,
341 Korean and Japanese sortkey weights.
343 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
345 * create-mscompat-collation-table.cs : More diacritical fixes.
346 Georgian characters do not have level 2 weights but level 3.
348 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
350 * create-mscompat-collation-table.cs : Roman numeral characters
351 have diacritical weight. quick hack for control signs (U+2400..)
354 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
356 * create-mscompat-collation-table.cs : improving Latin mappings.
357 Setting non-ASCII Latin characters' primary weight between those
358 ASCII characters, and setting diacritical weight (hacky).
359 * MSCompatUnicodeTable.cs :
360 Kanatype check: fixed (voice marks) and improved (comparison order).
362 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
364 * create-mscompat-collation-table.cs : more diacritical fixes.
365 primary weight fixes on punctuations in category 07.
367 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
369 * create-mscompat-collation-table.cs : several diacritical fixes.
370 * TestDriver.cs : sortkey dumper should use StringSort.
372 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
374 * SimpleCollator.cs : fixed incorrect indexer setup. Optimized
375 GetContraction() call a bit.
377 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
379 * create-mscompat-collation-table.cs : fixed incorrect level 2
381 * MSCompatUnicodeTable.cs : remove debug line.
383 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
385 * MSCompatUnicodeTableUtil.cs,
386 MSCompatUnicodeTable.cs,
388 create-mscompat-collation-table.cs : made some members internal and
389 accessible from other classes. Many indexes could be 0 by default.
390 * SimpleCollator.cs : optimizations. avoid method call.
392 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
394 * Collation-notes.txt : more updates.
395 * SimpleCollator.cs : Added quick check for Ordinal comparison.
396 Fixed special weight comparison. It cannot be customizable in the
397 implementation (and it won't be harmful).
398 * mono-tailoring-source.txt : thus updated comment.
400 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
402 * SimpleCollator.cs : Compare() was missing French sort support.
403 * TestDriver.cs : added example case.
405 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
407 * Collation-notes.txt : updated status. Eliminated descriptions on
408 "iterator" (I avoided it for performance concern). Fixed misc.
409 incorrect descriptions.
411 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
413 * Collator.cs : Now that SimpleCollator became feature complete, it is
416 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
418 * SimpleCollator.cs : implemented decent Compare() that immediately
419 stops at first primary difference.
421 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
423 * SimpleCollator.cs : indexers might return -1.
425 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
427 * SimpleCollator.cs : IsPrefix() and IsSuffix() optimization code was
428 buggy (length check for source was missing).
430 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
432 * create-mscompat-collation-table.cs : Fixed tailoring table output
433 to be in correct and countable order. Now if tailoring alias was not
434 found, just stop the build.
435 * MSCompatUnicodeTable.cs : several build fixes. Now it works to read
437 * mono-tailoring-source.txt : commented out CJK aliases that miss
439 * Makefile : needed further filename fixes.
441 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
443 * MSCompatUnicodeTable.cs : renamed from MSCompatUnicodeTable.template
444 (now it is working as a standalone file).
445 * Makefile : renamed generated file as MSCompatUnicodeTableGenerated.cs
446 (the generator now creates both binary resources and C# source).
448 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
450 * create-mscompat-collation-table.cs : Now it generates binary
451 resources (to parent directory).
452 * MSCompatUnicodeTable.template : added conditional code that fills
453 collation tables from manifest resources.
454 * Makefile : remove collation table binaries as well on "make clean".
455 Removed extraneous dependency.
457 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
459 * MSCompatUnicodeTable.template,
460 SimpleCollator.cs : removed extraneous GetExpansion().
462 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
464 * SimpleCollator.cs : IsSuffix() also supports contractions.
465 * TestDriver.cs : IsSuffix() example contraction cases.
467 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
469 * SimpleCollator.cs : reverted IsSuffix() to return bool (to match w/
470 what current IsPrefix() does). For expansion of target, IsPrefix()
471 should check the no-match case that expansion is longer than input.
472 Some refactory on IsPrefix().
473 Added GetContractionTal() for IsSuffix() (not used yet).
475 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
477 * TestDriver.cs : added IsPrefix() expansion cases.
478 * SimpleCollator.cs : IsPrefix() now supports contractions (with much
479 of complexity), and it now returns bool again.
480 IndexOf() for replacement should make use of IndexOfPrimitiveChar()
481 since expansions won't be expanded recursively.
483 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
485 * SimpleCollator.cs : commonized character comparison in IsPrefix()
486 and IsSuffix(). csc compile fix.
487 * CompareInfoImpl.cs : deleted.
489 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
491 * TestDriver.cs : added SimpleCollator.ctor() sanity check.
492 Added replacement contraction example.
493 * SimpleCollator.cs : Now IndexOf() and LastIndexOf() support
494 contraction in source string. Extracted matching code to Matches().
495 Replacement contraction was including extraneous '\x0'.
497 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
499 * Collation-notes.txt : updated status.
500 * CollationDataStructures.txt : tiny fixes.
501 * SimpleCollator.cs :
502 Renamed alias Util to UUtil (MS sys.enterprisesvc has sucky global
503 namespace Util and csc borked).
504 GetContraction was incorrectly returning first item.
505 Private IsPrefix() now returns int (but it might not be in real use).
506 Extracted simple char comparison to CompareCharSimple().
507 IndexOf() and LastIndexOf() now fully handle contractions (both
508 binary key and string replacement) in "target" (for "s" not yet).
509 * TestDriver.cs : be more verbose.
510 * mono-tailoring-source.txt : added comment.
511 * MSCompatUnicodeTable.template :
512 Renamed alias Util to UUtil (MS sys.enterprisesvc has sucky global
514 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
516 * create-mscompat-collation-table.cs : compute COMBINING blah marks as
517 well as those characters WITH blah.
518 * TestDriver.cs : added combining sortkey cases.
520 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
522 * mono-tailoring-source.txt : fixed description on '*' in sortkeys.
523 * SimpleCollator.cs : Now it fully uses tailoring info. Fixed
524 contraction search that worked only when string is contraction.
525 Removed commented code. Minor refactoring.
526 * TestDriver.cs : added example that uses "ZS" in Hungarian sorting.
528 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
530 * create-mscompat-collation-table.cs,
531 * mono-tailoring-source.txt : removed extraneous level 4 sortkey
532 which cannot be supported.
533 * SimpleCollator.cs : added GetContraction() and used in some places.
534 Now CompareOptions is set only once. Reordered some code (e.g.
535 ignorable check -> get compat char -> compare).
537 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
539 * SimpleCollator.cs : sort tailoring tables before actual usage.
540 Support diacritical remappings (it is customized collation rule
541 which does not exist in UCA).
543 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
545 * SimpleCollator.cs : build culture specific tailoring table from
546 TailoringInfo and unified data array.
547 * create-mscompat-collation-table.cs : Added null termination to
548 sortkey map tailorings (mostly to save my eyes).
549 * MSCompatUnicodeTable.template : added public TailoringValues.
551 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
553 * SortKeyBuffer.cs : handle special weight (category 06) characters.
554 * Collation-notes.txt : Updated description on special weight (it was
556 * TestDriver.cs : added special weight cases.
558 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
560 * MSCompatUnicodeTable.template : added GetTailoringInfo().
561 * SimpleCollator.cs : Now tailoring information is acquired and used.
562 (FrenchSort is supported but Compare() won't work expectedly since
563 the table is still incomplete for those diacritical marks).
564 * SortKeyBuffer.cs : On reversing diacritical weights, it should
565 ignore zeros. Reset() should reset frenchSorted flag.
567 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
569 * create-mscompat-collation-table.cs : Further fixes on Jamo,
570 diacritical weights by character name, and *Numbers primary weights.
572 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
574 * create-mscompat-collation-table.cs : More fix on Devanagari,
575 Gujarati, Oliya, Tamil and Lao sortkeys.
577 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
579 * create-mscompat-collation-table.cs : Fixed Georgian, Thai, Gurmukhi
582 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
584 * create-mscompat-collation-table.cs : Fixed Thai character primary
585 and secondary values. Fixed Thaana letters. Added more LAMESPEC
586 CJK compat. Fixed some circled CJK secondary weight.
587 Hacked some nonspacing mark sortkey value adjustment.
589 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
591 * create-mscompat-collation-table.cs : CP932.TXT was not parsed as
592 expected. JIS ordering was incorrect. OtherNumbers that represents
593 10 or more values were incorrectly computed the offset. Some Hangul
594 compat characters has different offset.
596 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
598 * create-mscompat-collation-table.cs : Fixed 0x8 category characters.
599 Added hack for need-to-be-fixed characters to fall into 0xA category.
600 * create-collation-element-table.cs : previous checkin seem failed :(
601 * README: updated a bit.
603 2005-06-24 Atsushi Enomoto <atsushi@ximian.com>
605 * CodePointIndexer.cs :
606 removed extraneous switch (I could use empty array for that need).
607 * CollationElementTableUtil.cs : primary weight type became ushort.
608 * create-collation-element-table.cs : several bugfixes.
609 collElem should be int. It was skipping most of entries because of
610 incorrect string tokenization.
612 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
614 * create-mscompat-collation-table.cs : handle some Jamo NKFD.
616 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
618 * SimpleCollator.cs : forgot to commit in the last checkin.
619 * create-mscompat-collation-table.cs : fixed arabic shift weight chars.
620 * TestDriver.cs : switch table dumper and collator testing.
621 * SortKey.cs : for now comment out internal indexes (not in use).
623 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
625 * MSCompatUnicodeTable.template,
626 SimpleCollator.cs : support for culture dependent CJK table.
628 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
630 * create-mscompat-collation-table.cs,
631 MSCompatUnicodeTableUtil.cs : make CJK table more compact.
633 2005-06-22 Atsushi Enomoto <atsushi@ximian.com>
635 * SimpleCollator.cs : Fixed stupid index search when start != 0.
637 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
639 * SimpleCollator.cs : fixed my misunderstanding on LastIndexOf(). It
640 now starts from "start" and proceeds backward by "length".
641 * TestDriver.cs : fix warning.
643 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
645 * TestDriver.cs : more tests.
646 * SimpleCollator.cs : LastIndexOf() is not setting search length
647 on iteration. Quick workaround fro String.LastIndexOf() bug (maybe).
649 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
651 * create-normalization-source.cs : output propValue as uint.
653 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
655 * SortKey.cs : Now it is System.Globalization.SortKey.
656 To replace existing implementation, it now requires lcid and
657 CompareOptions. Added required members.
658 * SortKeyBuffer.cs : thus .ctor() requires LCID.
659 * SimpleCollator.cs : made required changes above.
661 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
663 * CodePointIndexer.cs : added CompressArray(). Now it requires two more
664 parameters for default index and codepoint.
665 * CollationElementTableUtil.cs,
666 NormalizationTableUtil.cs : required changes wrt above change.
667 * MSCompatUnicodeTableUtil.cs : added for several codepoint indexers.
668 * MSCompatUnicodeTable.template : Now it uses codepoint indexer.
669 * create-mscompat-collation-table.cs : Now it outputs compressed array.
670 * Makefile : now collation requires MSCompatUnicodeTableUtil.cs
672 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
674 * SimpleCollator.cs :
675 Implemented IsSuffix() and LastIndexOf().
676 Several fixes on index > 0 cases.
677 * TestDriver.cs : sample IsSuffix() and LastIndexOf() usage and more.
679 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
681 * Collation-notes.txt : updated (status, impl. classes).
682 * MSCompatUnicodeTable.cs : Korean Jamo are not really expansions.
684 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
686 * SimpleCollator.cs : implemented IndexOf(string,string,CompareOptions)
687 and IsPrefix(). Tiny code refactory.
688 * TestDriver.cs : sample IsPrefix() and IndexOf() usage.
689 * MSCompatUnicodeTable.cs : tiny refactory for CodePointIndexer use.
691 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
693 * SimpleCollator.cs :
694 IndexOf(string, char, CompareOptions) implementation.
695 * TestDriver.cs : sample IndexOf() usage.
697 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
699 * create-mscompat-collation-table.cs : was missing most important
700 kind of blocks - equivalent expansions (e.g. invariant mappings).
701 More readable mappings.
703 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
705 * mono-tailoring-source.txt : new file. It describes tailoring
706 information. Basically examined under .NET 1.x.
707 * create-mscompat-collation-table.cs : consume the file above.
708 * MSCompatUnicodeTable.template : now tailorings is not a stub.
709 * CollationDataStructures.txt : minor fixes.
711 SimpleCollator.cs : added FrenchSort support.
712 * Collation-notes.txt : added description on Latin primary weights.
713 * ldml-limited.rng : added note.
714 * create-tailorings.cs : added note. more serialization (but won't be
717 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
719 * SortKeyBuffer.cs : non-primary character is added to previous
721 * TestDriver.cs : added example case of above.
723 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
725 * SimpleCollator.cs : IgnoreSymbols support.
726 * TestDriver.cs : compilation fix. IgnoreSymbols example.
727 * create-mscompat-collation-table.cs : more Hangul fixes.
729 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
731 * create-mscompat-collation-table.cs : more Hangul fixes.
732 * SortKey.cs : it will replace sys.globalization.SortKey. It has
733 some internal members.
734 * SortKeyBuffer.cs : now it uses SortKey instead of byte[].
735 * SimpleCollator.cs : CompareOptions support. However I don't think
736 it will be developed anymore since SortKey never enables IndexOf().
737 * TestDriver.cs : a few CompareOptions cases.
739 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
741 * SimpleCollator.cs : simple collator implementation that just will
742 use GetSortKey() for all its basis.
743 * TestDriver.cs : sample code that uses this collator set.
744 * MSCompatUnicodeTable.template : removed test driver from here.
746 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
748 * create-mscompat-collation-table.cs : Hangul fixes.
749 Now less than 300 characters that does not have sortkey weights.
750 * MSCompatUnicodeTable.template : added FIXME info for Hangul Jamo.
752 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
754 * create-mscompat-collation-table.cs : Added control picture mappings.
755 Minor primary weight fixes.
757 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
759 * create-mscompat-collation-table.cs : Added mappings for box
762 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
764 * create-mscompat-collation-table.cs : Added mappings for arrows.
766 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
768 * create-mscompat-collation-table.cs : added support for letterlike
769 characters and squared CJK compatibility characters, ordered by
770 character names (0x0E category).
771 * Collation-notes.txt : added description on that.
773 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
775 * MSCompatUnicodeTable.template : Now expansions are simulated.
776 * create-mscompat-collation-table.cs : filled Korean number level2.
777 Reordered some code blocks to fill correct diacritical differences.
778 * Collation-notes.txt : some corrections and minor additions.
780 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
782 * MSCompatUnicodeTable.template :
783 Now dumper test driver uses SortKeyBuffer for dogfooding.
784 * create-mscompat-collation-table.cs : some diacritical level fixes
785 (with non-working extra latin check).
786 * SortKeyBuffer.cs : several fixes to get working as a practical code.
787 * Collator.cs : make it compilable, leaving things as NotImplemented.
789 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
791 * create-mscompat-collation-table.cs : some fixes on primary category
792 07 (miscellaneous symbols and punctuations).
794 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
796 * create-mscompat-collation-table.cs : more mapping fix on numbers,
797 letters, variable weight characters, circled Japanese and CJK.
798 * MSCompatUnicodeTable.template : fixed HasSpecialWeight() to be more
799 inclusive. Simplified dumper code.
801 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
803 * create-mscompat-collation-table.cs : finished Hangul (both Jamo
804 and Syllables). sortkey dumper diff lines became 8000 from 30000.
806 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
808 * create-mscompat-collation-table.cs : added some nonspacing marks in
809 either correct or hacky way.
811 2005-06-13 Atsushi Enomoto <atsushi@ximian.com>
813 * create-mscompat-collation-table.cs : several improvements. Japanese
814 Kana support, Hebrew accents, Bengali nonspacing marks, sorting of
815 numeric characters, diacritically decorated latin alphabets. Fixed
816 some diacritical weights detection.
817 * MSCompatUnicodeTable.cs : tiny Japanese fix. Handle nonspacing
818 marks' primary weight as empty.
819 * Collation-notes.txt : some updates.
821 2005-06-13 Atsushi Enomoto <atsushi@ximian.com>
823 * create-mscompat-collation-table.cs : don't process nonexact NFKD
824 mapping as equivalent, however store CJK extensions into NFKD map
825 even if one does not strictly match.
826 Now am going to fill Hangul into tables (unlike UCA it does not look
827 possible to calculate sortkey value).
828 Fixed Cyrillic and Georgian UCA based orderings.
829 * MSCompatUnicodeTable.template : added CJK extension sortkey
832 2005-06-10 Atsushi Enomoto <atsushi@ximian.com>
834 * create-mscompat-collation-table.cs : Fixed latin alphabet support.
835 Added latin with diacritical and CJK extension.
836 * MSCompatUnicodeTable.cs : modified dumper code a bit (for my purpose).
838 2005-06-10 Atsushi Enomoto <atsushi@ximian.com>
840 * create-mscompat-collation-table.cs : now parses DerivedAge.txt (right
841 now not used thouth). Filled CJK ideograph, still not perfect.
842 Fixed number primary keys. NFKD numbers and CJK ideographs are now
843 considered, including brackets elimination.
844 * Makefile : now it downloads DerivedAge.txt.
845 * MSCompatUnicodeTable.template : added dummy code dumper. It computes
846 PrivateUse, Surrogate and Hangul Syllables.
847 * Collation-notes.txt : Noted that Hangul Syllables need more love.
849 2005-06-09 Atsushi Enomoto <atsushi@ximian.com>
851 * create-tailorings.cs : added configuration support. sort them.
852 I wonder if it is really usable. Having own format might be better.
853 * create-mscompat-collation-table.cs : fixing some sortkey numbers,
854 making closer to windows. Now it handles NFKD in some places.
855 * MSCompatUnicodeTable.template : Added dummy sortkey dumper driver.
856 * CollationDataStructures.txt : added description on tailoring
857 fields, though they are subject to change.
859 2005-06-07 Atsushi Enomoto <atsushi@ximian.com>
861 * create-tailorings.cs, ldml-limited.rng : new file.
862 * LdmlReader.cs : removed old file.
864 2005-06-07 Atsushi Enomoto <atsushi@ximian.com>
866 * SortKeyBuffer.cs : split from Collator.cs. Now it considers
867 practical use, reflecting updated sortkey constant design.
868 Especially level 4 weight is split to 4 arrays that are merged in
869 the last stage of GetSortKey().
870 * Collator.cs : thus SortKeyBuffer is removed from here.
871 Additionally, removed some extraneous bits in other classes.
872 * Collation-notes.txt : Some editorial fixes. Added information on
873 Korean matter (how to compute Hangle Syllables / Hangul Jamo cannot
874 be stored in simple byte arrays).
875 * CodePointIndexer.cs,
876 create-collation-element-table.cs,
877 CollationElementTable.template,
878 NormalizationTableUtil.cs : short CodePointIndexer method names.
879 * create-mscompat-collation-table.cs : Additional info on why some
880 meaningful characters are ignored in Windows (Unicode version
881 difference). Removed U+070F from special check (was extraneous).
883 2005-06-06 Atsushi Enomoto <atsushi@ximian.com>
885 * MSCompatUnicodeTable.template:
886 Moved body implementation to table creator and put those bool
887 results into an array.
888 * create-mscompat-collation-table.cs :
889 So imported those methods. Modified array output to emit "0x"
890 only for more than 9.
891 * create-normalization-source.cs : ditto on "0x" output matter.
892 * CollationDataStructures.txt : so now it holds ignorableFlags.
894 2005-06-03 Atsushi Enomoto <atsushi@ximian.com>
896 * Collation-notes.txt, CollationDataStructures.txt :
897 separate document for data structure design.
899 2005-06-03 Atsushi Enomoto <atsushi@ximian.com>
901 * create-mscompat-collation-table.cs : added culture-dependent CJK
902 table creation. It uses CLDR as its basis. (Culture independent CJK
904 * Makefile : added CLDR archive downloading support.
905 * MSCompatUnicodeTable.template : tiny renamings.
906 * Collation-notes.txt : additional CJK info.
908 2005-06-02 Atsushi Enomoto <atsushi@ximian.com>
910 * Collation-notes.txt, create-mscompat-collation-table.cs :
911 added secondary weight support for BlahNumber characters.
913 2005-06-01 Atsushi Enomoto <atsushi@ximian.com>
915 * downloaded : added directory. All downloaded files are stored here.
916 * Makefile : use "downloaded" directory.
917 Added more auto-download stuff.
918 * create-mscompat-collation-table.cs :
919 Added Japanese square kana support.
921 2005-06-01 Atsushi Enomoto <atsushi@ximian.com>
923 * Collation-notes.txt : added Estrangela (ancient Syriac) and Thaana.
924 * create-mscompat-collation-table.cs : added support for Arabic abjad,
925 Estrangela and Thaana.
926 * MSCompatUnicodeTable.template : removed BOM.
928 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
930 * Collation-notes.txt : wrong comment cleanup and spelling fixes.
931 * create-mscompat-collation-table.cs : added diacritic support for
932 Latin letters (as long as covered in primary weight).
934 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
936 * Makefile : minor fixes. Added warning lines to generated sources.
938 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
940 * create-char-mapping-source.cs :
941 Removed ToWidthInsensitive() generation.
943 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
945 * create-mscompat-collation-table.cs : Now it dumps level1 to 3 values.
946 ToWidthInsensitive() is implemented here, using an array (which is
947 to be optimized using CodePointIndexer).
948 * MSCompatUnicodeTable.cs : renamed as MSCompatUnicodeTable.template
949 * MSCompatUnicodeTable.template : now it is used to generate
950 MSCompatUnicodeTable.cs which got ready to be used.
951 * Makefile : added MSCompatUnicodeTable.cs build support. Now it
952 supports "make normalization" and "make collation".
954 2005-05-30 Atsushi Enomoto <atsushi@ximian.com>
956 * Collation-notes.txt : Description on ICU is very incorrect. Now it
957 became more rational and sane.
958 * create-mscompat-collation-table.cs : fixed some indexes.
959 * Makefile : added "mstablegen" target.
960 * MSCompatUnicodeTable.cs : removed GetPrimaryWeight(). Minor fix.
962 2005-05-26 Atsushi Enomoto <atsushi@ximian.com>
964 * Collation-notes.txt : more analysis on "letters".
965 * create-mscompat-collation-table.cs : more proof of concepts.
967 2005-05-25 Atsushi Enomoto <atsushi@ximian.com>
969 * Collation-notes.txt : more info. Started letter sortkey analysis
970 (some of other stuff are really non-understandable right now.)
971 * create-mscompat-collation-table.cs : table generator proof-of-
972 concept source (not compilable).
973 * MSCompatUnicodeTable.cs : moved some code to the new source.
976 2005-05-20 Atsushi Enomoto <atsushi@ximian.com>
978 * Collation-notes.txt : started level 2 weight analysis.
980 2005-05-19 Atsushi Enomoto <atsushi@ximian.com>
982 * Collation-notes.txt : Additional information on how to create
984 * MSCompatUnicodeTable.cs : implemented part of GetLevel3Weight().
986 2005-05-19 Atsushi Enomoto <atsushi@ximian.com>
988 * Collation-notes.txt : More case weight (level 3) analysis. I'm
989 likely to just write table generator.
991 2005-05-18 Atsushi Enomoto <atsushi@ximian.com>
993 * MSCompatUnicodeTable.cs : part of level 4 weight implementation.
995 2005-05-18 Atsushi Enomoto <atsushi@ximian.com>
997 * Collation-notes.txt :
999 Revised comparison methods; backward iteration is possible.
1000 More on char-by-char comparison.
1001 Level 4 comparison is actually a bit more complex.
1003 * Collator.cs : some conceptual updates wrt above.
1005 2005-05-17 Atsushi Enomoto <atsushi@ximian.com>
1007 * Collation-notes.txt : Japanese voice mark is level 2, and Hangul
1008 properties are level 3.
1010 2005-05-17 Atsushi Enomoto <atsushi@ximian.com>
1012 * Collation-notes.txt : Make it more readable. More analysis on
1013 level 3 and 4 sortkey structures.
1014 * Collator.cs : some compilation fixes (not compilable yet).
1016 2005-05-16 Atsushi Enomoto <atsushi@ximian.com>
1018 * Collation-notes.txt : Analysis on variable-weighting (level 5)
1020 * Collator.cs : updated corresponding part of level 5, and more.
1022 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1024 * Collation-notes.txt : more updates.
1025 * Collator.cs : rewrote from scratch. Some rough sketch for sortkey
1026 buffer, character iterator and collator methods. Not compiling.
1028 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1030 * Collator.cs : Am going to replace it with new one. No need for
1031 CompareOptions-dependent Comparer.
1033 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1035 * Collation-notes.txt : There seems a bit more complexity.
1037 2005-05-10 Atsushi Enomoto <atsushi@ximian.com>
1039 * Collation-notes.txt : more updates, being close to write sortkey
1042 2005-05-09 Atsushi Enomoto <atsushi@ximian.com>
1044 * CompareInfoImpl.cs, Collator.cs : conceptual update
1045 * Collation-notes.txt : some corrections and additions.
1046 * Makefile : added LDML input (but it won't be used at all).
1048 2005-04-28 Atsushi Enomoto <atsushi@ximian.com>
1050 * Collation-notes.txt : more updates.
1052 2005-04-26 Atsushi Enomoto <atsushi@ximian.com>
1054 * Collation-notes.txt : more updates.
1056 2005-04-26 Atsushi Enomoto <atsushi@ximian.com>
1058 * Collation-notes.txt : some updates.
1059 * create-mapping-char-source.cs : superscripts and subscripts are also
1060 ignored in IgnoreWidth comparison.
1061 * Makefile : tiny touch fix.
1063 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1065 * CompareInfoImpl.cs, Collator.cs : conceptual stuff (not working).
1067 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1069 * create-char-mapping-source.cs : Now it generates
1070 ToWidthInsensitive() from combining category <wide> and <narrow>.
1071 * MSCompatUnicodeTable.cs : added ToKanaTypeInsensitive() and
1072 ToWidthInsensitive() for IgnoreKanaType and IgnoreWidth.
1074 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1076 * README, LdmlReader.cs, DataStructures.txt : new files.
1078 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1080 * CodePointIndexer.cs,
1081 Collation-notes.txt,
1082 CollationElementTable.template,
1083 CollationElementTableUtil.cs,
1084 create-char-mapping-source.cs,
1085 create-collation-element-table.cs,
1086 create-combining-class-source.cs,
1087 create-normalization-source.cs,
1089 MSCompatUnicodeTable.cs,
1090 Normalization.template,
1091 NormalizationTableUtil.cs : initial checkin (to private branch).