1 2008-06-19 Atsushi Enomoto <atsushi@ximian.com>
4 - reverted the previous index calculation change. It was correctly
5 implemented and I rather broke it.
6 - fix index calculation on combining.
7 - NFKD was incorrectly directed to combining path. It should not.
8 - Simplify quick check.
10 2008-06-15 Atsushi Enomoto <atsushi@ximian.com>
12 * Normalization.cs : For NFC and NFKC, IsNormalized() was not working
13 enough to check composed characters. It's not possible without
14 the actual composition, so just call Normalize() and compare them.
15 In Normalize() mapping helper didn't pick correct map index since
16 the table for index stores index for "uncompressed" numbers.
17 * NormalizationTableUtil.cs : updated to the latest UCD.
18 * Makefile : to build test, source file must be downloaded too.
20 2008-11-05 Atsushi Enomoto <atsushi@ximian.com>
22 * ucd.cs : Write type for *_count. Add notice to not edit
23 unicode-data.h directly.
25 2008-11-04 Atsushi Enomoto <atsushi@ximian.com>
27 * ucd.cs : new code to generate unicode table for eglib.
29 2008-07-04 Andreas Nahr <ClassDevelopment@A-SoftTech.com>
31 * SortKey: Fix parameter names, add attribute, small formatting
33 2008-06-27 Rodrigo Kumpera <rkumpera@novell.com>
35 * CodePointIndexer.cs : Make TableRange a struct instead
36 of a class so we save 2 memory ops per ToIndex loop.
38 2008-04-02 Atsushi Enomoto <atsushi@ximian.com>
40 * SortKey.cs : check null arguments. Fixed bug #376171.
42 2007-07-20 Atsushi Enomoto <atsushi@ximian.com>
44 * create-mscompat-collation-table.cs : I wonder how long its build
47 2007-03-06 Atsushi Enomoto <atsushi@ximian.com>
49 * SimpleCollator.cs : disable QuickCheckPossible(), which is
50 inaccurate and inefficient. Fixed bug #79714.
52 2007-02-15 Atsushi Enomoto <atsushi@ximian.com>
54 * SimpleCollator.cs : character filtering is needed for
55 OrdinalIgnoreCase in 2.0 profile. Fixed bug #80865.
57 2007-01-25 Atsushi Enomoto <atsushi@ximian.com>
59 * SimpleCollator.cs : GetTailContraction() was broken to pick correct
60 contraction/special sortkey out and thus LastIndexOf() failed when
61 it is involved. Fixed bug #80612.
63 2007-01-22 Atsushi Enomoto <atsushi@ximian.com>
65 * SimpleCollator.cs : for non-StringSort comparison, level5 (- and ')
66 should be still skipped after initial level5 check is done (while
67 they were simply treated as a normal character). Fixed bug #78748.
68 * SortKeyBuffer.cs : Fixed NRE in french sort.
70 2006-12-25 Atsushi Enomoto <atsushi@ximian.com>
72 * SimpleCollator.cs : added IndexOf() implementation for Ordinal
73 and OrdinalIgnoreCase, though Ordinal version is not used (since
74 it is slower than icall).
76 2006-05-30 Miguel de Icaza <miguel@novell.com>
78 * MSCompatUnicodeTable.cs: Remove the fixed loading and compute it
79 just when we actually consume it. This only fixes the
82 2006-04-14 Atsushi Enomoto <atsushi@ximian.com>
84 * README: removed obsolete info.
85 * Normalization.cs : canonical reordering should participate in the
86 decomposition step. In reordering, string append was incomplete.
87 Combining class check is required in NFD check. Icall is written
90 2005-12-07 Zoltan Varga <vargaz@gmail.com>
92 * SimpleCollator.cs: Fix a warning.
94 2005-11-30 Sebastien Pouliot <sebastien@ximian.com>
96 * SimpleCollator.cs: Fix CAS support. The static ctor/var try to get
97 the environment variable MUCH too soon (i.e. the security manager
100 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
102 * SimpleCollator.cs : direct fast-path optimization for IndexOf().
104 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
106 * SimpleCollator.cs :
107 - CompareQuick(): added immediateBreakup to avoid extraneous sortkey
109 - QuickCheckPossible(): index used for s1 was incorrect.
111 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
113 * SimpleCollator.cs : added another quick check for CompareInternal()
114 that does almost ordinal comparison for quick-checkable strings.
115 (It affects on Compare(), IndexOf(), IsSuffix() etc. as well.)
117 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
119 * MSCompatUnicodeTable.cs : (IsIgnorable) \0 is not ignorable.
122 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
124 * SimpleCollator.cs :
125 Created another struct to reduce method arguments. Created another
126 flags that keeps "once-matched" state (counterpart of
127 checkedFlags, now neverMatchFlags).
129 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
131 * SimpleCollator.cs :
132 - Added CompareOrdinalIgnoreCase() for NET_2_0 RTM.
133 - Reduced extra parameter from LastIndexOfSortKey().
134 - LastIndexOf() should use GetTailContraction for the source string.
135 And then, target could match in the middle of the possible
136 "replacement contraction" of the source string, so use
137 LastIndexOfSortKey() to catch them.
138 - Fixed GetTailContraction() that caused index out of range.
140 2005-11-11 Atsushi Enomoto <atsushi@ximian.com>
142 * Makefile : Now use MONO_DISABLE_MANAGED_COLLATION.
143 * SortKey.cs : some members are virtual.
145 2005-10-14 Atsushi Enomoto <atsushi@ximian.com>
147 * SimpleCollator.cs : modified to use stackalloc for byte array.
149 2005-09-27 Atsushi Enomoto <atsushi@ximian.com>
151 * SimpleCollator.cs : in CompareInternal(), there was a possibility of
152 infinite loop. Fixed bug #76243.
154 2005-09-20 Atsushi Enomoto <atsushi@ximian.com>
156 * SimpleCollator.cs : In IsPrefix/IsSuffix, if target is an empty string,
157 immediately return true.
159 2005-09-09 Atsushi Enomoto <atsushi@ximian.com>
161 * SimpleCollator.cs : IsSuffix() optimization logic was buggy, so just
162 use pretty simple way with LastIndexOf() (no significant perf.
165 2005-09-01 Atsushi Enomoto <atsushi@ximian.com>
167 * README, Collation-notes.txt, CollationDataStructures.txt :
168 removing obsolete info and some added some notes.
170 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
172 * Normalization.cs : remove warned code.
173 * managed-collation.patch : now it's not required anymore.
175 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
177 * MSCompatUnicodeTable.cs : added IsSortable(string).
179 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
181 * SimpleCollator.cs : Now all collator methods are thread safe.
183 All instance non-readonly fields turned into arguments of every
184 methods that use those fields.
185 (Sadly it is the end of no-memory-cost collator era. mcs bootstrap
186 now needs +100KB memory consumption.)
188 2005-08-09 Atsushi Enomoto <atsushi@ximian.com>
190 * SimpleCollator.cs : made "checkedFlags" as nullable and made it as
191 an argument of every index methods (to make it thread safe).
193 2005-08-09 Atsushi Enomoto <atsushi@ximian.com>
196 MSCompatUnicodeTable.cs :
197 - Now IsIgnorable() is aggregated to be one invokation to check
198 completely ignorable, nonspacing and symbols.
199 - Introduced "already checked" flags for IndexOf() and LastIndexOf()
200 to skip sortkey binary check on the same characters. Significant
201 perf. improvement for such case as IndexOf("AABCBABC...Z",'Z').
203 2005-08-08 Gert Driesen <drieseng@users.sourceforge.net>
205 * SortKey.cs: Marked Serializable to match MS.NET.
207 2005-08-08 Atsushi Enomoto <atsushi@ximian.com>
209 * create-mscompat-collation-table.cs,
210 Makefile : changed resources output directory.
212 2005-08-04 Atsushi Enomoto <atsushi@ximian.com>
214 * create-normalization-tests.cs,
215 StringNormalizationTestSource.cs : new files for Unicode
216 Normalization test generator.
217 * Makefile : added support for above.
219 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
221 * NormalizationTableUtil.cs : oops, it does not compile.
222 * managed-collation.patch : I guess having managed resource would be
223 better for collation. At least current code has such #define so
224 Makefile should be in sync with it.
226 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
228 * create-normalization-source.cs : Fixed CharMapComparer which
229 incorrectly returned 0 when the second arg is shorter. Reduced
230 extraneous helperIndex map. Other minor fixes and code removal.
231 * Normalization.cs : several fixes to support blocked combine handling.
232 * NormalizationTableUtil.cs : tiny member renaming.
234 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
236 * create-normalization-source.cs,
237 NormalizationTableUtil.cs,
238 Normalization.cs : several bugfixes on index miscomputation.
239 Renamed using aliases (csc will bork). Primary combine safety is now
240 computed during UnicodeData.txt parse.
241 Maximum NFKD length was 18, not 4 (U+FDFA).
243 2005-08-02 Atsushi Enomoto <atsushi@ximian.com>
245 * managed-collation.patch : added Normalization support.
246 * managed-collation-icall.patch : added, including normalization stuff.
248 BTW when will collation code checked in?
250 2005-08-02 Atsushi Enomoto <atsushi@ximian.com>
252 * create-normalization-source.cs : Unified three normalization source
253 generators, to compute IsUnsafe flag. Fixed helperIndex array type
255 * create-char-mapping-source.cs,
256 create-combining-class-source.cs : thus removed.
257 * Makefile : thus modified for the above integration.
258 * NormalizationTableUtil.cs : Extended to contain IsUnsafe flag.
259 * Normalization.cs : Several fixes to make Normalize() actually work.
261 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
263 * create-normalization-source.cs,
265 create-char-mapping-source.cs,
266 create-combining-class-source.cs,
267 Makefile : converted managed array to pointers (like collation stuff).
269 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
271 * NormalizationTableUtil.cs : further table range optimization.
272 * create-normalization-source.cs,
273 create-char-mapping-source.cs,
274 create-combining-class-source.cs : added C header output support.
276 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
278 * create-normalization-source.cs, Normalization.cs :
279 Now property size is < 256, so directly embed value in "props" array.
280 Add QuickCheck(c,checkType) and remove IsNFD/C/KD/KC and delegates.
282 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
284 * create-combining-class-source.cs,
285 create-char-mapping-source.cs,
286 create-normalization-source.cs,
287 NormalizationTableUtil.cs,
288 Normalization.cs : String.Normalize() does not handle surrogate
289 characters. mapping information in DerivedNormalizationProps.txt
290 are not used in the code (those from UnicodeData.txt is used).
291 Hangul syllables are computed instead of embedded in the tables.
292 * managed-collation.patch : removed IntPtrStream and Makefile patches.
294 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
296 * MSCompatUnicodeTable.cs : IsSortable() was broken.
298 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
300 * MSCompatUnicodeTable.cs : added helper for CompareInfo.IsSortable().
302 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
304 * create-tailoring.cfg : added for convenience of contraction check.
306 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
308 * create-normalization-source.cs,
311 create-mscompat-collation-table.cs,
312 MSCompatUnicodeTableUtil.cs,
314 create-collation-element-table.cs,
315 MSCompatUnicodeTable.cs,
317 create-combining-class-source.cs : added copyright lines.
319 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
321 MSCompatUnicodeTable.cs : removed extraneous definition.
323 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
325 * create-mscompat-collation-table.cs
326 MSCompatUnicodeTable.cs : full C header support, finally.
328 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
331 NormalizationTableUtil.cs,
332 create-char-mapping-source.cs : more aggressive data compression.
333 It now ignores characters that are >= U+10000.
335 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
338 Normalization.template,
339 Normalization.cs : renamed existing file.
341 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
343 * NormalizationTableUtil.cs,
344 Normalization.template,
345 create-combining-class-source.cs : GetCombiningClass is now
346 implemented as indexer based array.
347 * Makefile : renamed output filename.
348 * create-mscompat-collation-table.cs : removed comments that does not
350 * create-tailoring.cs : use utf-8 output (and fixed filename).
352 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
354 * create-mscompat-collation-table.cs : hacked safer IPA extensions.
355 * Collation-notes.txt : status of sortkey table.
357 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
359 * create-mscompat-collation-table.cs : some Greek mapping fix.
361 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
363 * create-mscompat-collation-table.cs : diacritical weight is not
364 treated correctly when they are picked from letter names, as flags.
366 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
368 * create-mscompat-collation-table.cs : fixed culture-dependent
369 nonspacing mark weight.
371 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
373 * create-mscompat-collation-table.cs : some Hebrew case letter fixes.
374 Some diacritical fixes on symbols.
376 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
378 * create-mscompat-collation-table.cs : Fixed level 3 weight of
379 Arabic presentation forms.
381 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
383 * create-mscompat-collation-table.cs : Fixed some diacritical weight
384 of Arabic presentation forms.
386 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
388 * SimpleCollator.cs : more status updates. It's almost complete,
389 except for sortkey values.
391 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
393 * SimpleCollator.cs : similar optimization also for LastIndexOf().
395 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
397 * SimpleCollator.cs : the previous patch was missing IgnoreNonSpace
400 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
402 * SimpleCollator.cs : reduced extra sortkey value computation in
403 MatchesForward(). It makes IndexOf() roughly 30% faster.
405 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
407 * SortKey.cs : GetHashCode() returns a value based on its byte data.
410 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
412 * SimpleCollator.cs : consider extractions in invariant culture.
414 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
416 * SimpleCollator.cs : (unsafeFlags) be compact ;-)
418 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
420 * SimpleCollator.cs : When the tail of the target does not match more
421 than 3 times, then IsSuffix() will never be true (3 is the max
422 length of an expansion; \uFB03 -> ffi). It brings significant
423 performance boost when "source" string is very long.
424 * MSCompatUnicodeTable.cs : added MaxExpansionLength constant.
425 Reordered code lines.
427 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
429 * Collation-notes.txt : updated implementation status.
431 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
433 * SimpleCollator.cs : Implemented quick codepoint comparison in
434 Compare(). Comparison became 125x faster.
435 * mono-tailoring-source.txt : added tiny comment.
437 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
439 * mono-tailoring-source.txt : Added all single sortkey remapping to
440 all cultures (still need to fill contractions and annotate possible
441 buggy mapping referencing to CLDR).
442 * SimpleCollator.cs : removed unused code.
443 * MSCompatUnicodeTable.cs : tiny cast removal.
445 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
448 create-mscompat-collation-table.cs
449 MSCompatUnicodeTableUtil.cs
450 MSCompatUnicodeTable.cs : Now CJK mapping data is stored as byte
451 arrays. Thus SimpleCollator does not need to use bitwise and shift
452 operations to get sortkey value and they could be managed resources.
454 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
456 * create-mscompat-collation-table.cs,
457 MSCompatUnicodeTable.cs,
458 MSCompatUnicodeTableUtil.cs : From the result of sortkey comparison
459 between None and IgnoreWidth, width compat table could be computed
460 in somewhat simple way. So removed that table and all related code.
461 Increased the collation resource version.
463 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
465 * create-mscompat-collation-table.cs : Added C header output support.
467 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
469 * create-mscompat-collation-table.cs : FillLetterNFKD() could also be
470 applied to Cyrillic letters. Saved some of them.
472 2005-07-24 Atsushi Enomoto <atsushi@ximian.com>
474 * MSCompatUnicodeTable.cs : oh, ok, so we already have
475 GetManifestResourceInternal() ;-)
476 * managed-collation.patch : in Assembly.cs made that method internal.
478 2005-07-24 Atsushi Enomoto <atsushi@ximian.com>
480 * MSCompatUnicodeTable.cs : the pointer based icall code could be
481 also applicable for USE_MANAGED_RESOURCE mode.
483 2005-07-23 Atsushi Enomoto <atsushi@ximian.com>
485 * MSCompatUnicodeTable.cs : added icall support code (not enabled
486 unless the first line is commented out).
488 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
490 * create-mscompat-collation-table.cs,
491 MSCompatUnicodeTableUtil.cs,
492 MSCompatUnicodeTable.cs : Added resource version output (and ignore
493 in case of version mismatch). Removed obsolete, commented out code.
495 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
498 MSCompatUnicodeTable.cs,
499 create-mscompat-collation-table.cs : Now they use unmanaged pointers
500 instead of managed arrays.
501 * managed-collation.patch : Now it contains patch for IntPtrStream.cs
502 and Assembly.cs as well.
504 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
506 * MSCompatUnicodeTable.cs,
507 SimpleCollator.cs : Moved tailoring support classes to
508 MSCompatUnicodeTable.cs and drawn out from SimpleCollator.
509 Now that cjk and tailoring support are filled inside
510 MSCompatUnicodeTable, no managed array is exposed.
512 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
514 * create-mscompat-collation-table.cs,
516 MSCompatUnicodeTable.cs : Now it's not exposing collation table
517 internals as managed arrays (to switch to unmanaged pointers).
519 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
521 * create-mscompat-collation-table.cs : tiny nonspacing mark fix.
523 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
525 * create-mscompat-collation-table.cs : Fixed most of Greek mappings.
526 * MSCompatUnicodeTable.cs : don't lock string.
528 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
530 * create-mscompat-collation-table.cs : More Cyrillic diacritical fixes.
532 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
534 * create-mscompat-collation-table.cs : More Latin diacritical fixes.
536 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
538 * create-mscompat-collation-table.cs : There were still missing
539 math symbol mappings. Added several hacky diacritical weight for
542 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
544 * create-mscompat-collation-table.cs : fixed a few diacritical weight
545 on Cyrillic characters. Fixed ParseTailoringSource() to handle
546 non-heading escape sequence (\uXXXX) as expected.
548 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
550 * create-mscompat-collation-table.cs,
551 MSCompatUnicodeTableUtil.cs,
552 MSCompatUnicodeTable.cs : added more aggressive index limits for
553 table optimization at data size, in cost of speed.
555 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
557 * create-mscompat-collation-table.cs : fixed Arabic thirtial weight.
559 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
561 * create-mscompat-collation-table.cs : Mapping for hyphens and
562 punctuation are kinda finished. Rewrote batch mapping method to
563 collect all NFKD. Required modification on mapping is done.
565 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
567 * create-mscompat-collation-table.cs : minor mapping fixes on accent
568 marks and punctuations.
570 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
572 * create-mscompat-collation-table.cs : Fixed some MathSymbol mapping
573 and Box drawing mapping.
575 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
577 * create-mscompat-collation-table.cs : Fixed almost all numbers.
579 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
581 * create-mscompat-collation-table.cs : Symbol mappings are almost done.
582 Removed hack that gave dummy mappings to blank symbols.
584 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
586 * create-mscompat-collation-table.cs : more fix on arrows. Fix on box
587 drawings. Some code refactoring to eliminate hack.
589 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
591 * create-mscompat-collation-table.cs : Fixed some secondary weight
592 in Devanagari and arrows.
594 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
596 * create-mscompat-collation-table.cs : a set of tiny mapping fixes.
598 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
600 * create-mscompat-collation-table.cs : some diacritical fixes for
601 Latin. Added batch mapping method that considers computed
602 diacritical weight (for numbers).
604 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
606 * managed-collation.patch : forgot to add System.String patch.
608 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
610 * MSCompatUnicodeTable.cs : added resource existence check (required
611 for mscorlib transient time from the one without resources to the
614 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
616 * create-mscompat-collation-table.cs : fixed punctuations and hyphen
617 (shift) primary weight.
619 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
621 * create-mscompat-collation-table.cs : more nonspacing mark fixes.
622 Some non-basic Cyrillic diacritical weight fixes.
624 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
626 * create-mscompat-collation-table.cs : some Gurmukhi fixes on level 1
627 and level 3. Tiny Hangul weight fixes.
628 * MSCompatUnicodeTable.cs : U+30F5 and U+30F6 are small Japanese.
630 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
632 * create-mscompat-collation-table.cs : some normal characters who have
633 "narrow" NFKD mapping are regarded as "wide" and thus level 3 weight
634 values were different. Handle U+30FB as category A.
635 * MSCompatUnicodeTable.cs : U+30FB does not have special weight.
637 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
639 * create-mscompat-collation-table.cs : more diacritical weight fixes.
640 Removed some unused code.
642 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
644 * create-mscompat-collation-table.cs : Fixed some Thai and Arabic
647 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
649 * create-mscompat-collation-table.cs : Fixed Syriac nonspacing marks.
651 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
653 * create-mscompat-collation-table.cs : Fixed nonspacing marks in
654 Malayalam, Thai and Lao. Removed extraneous hack.
656 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
658 * SimpleCollator.cs : rewrote LastIndexOf() to handle source extenders.
659 Some refactoring on IndexOf() code. Removed unused Matches().
660 * Collation-notes.txt : some methods needed to be reimplemented, so
661 rewrote the description.
663 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
665 * SimpleCollator.cs : rewrote IsSuffix() to use CompareInternal().
666 Thus supported extenders in IsSuffix().
668 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
670 * SimpleCollator.cs : more IsSuffix() simplification, but it will be
671 stopped here since it cannot handle extenders (implementing new
674 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
676 * SimpleCollator.cs : simplified IsSuffix() code.
678 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
680 * SimpleCollator.cs : Fixed IndexOf() and LasIndexOf() to search the
681 entire replacement string if char target was an expansion.
682 IsSuffix() was using a method for IsPrefix() which was incorrect.
683 Removed old IsPrefix() code.
685 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
687 * SimpleCollator.cs : IndexOf() was incorrectly sharing the same
688 byte[] field in different areas of code. Now extenders in both
689 source and target really work in IndexOf().
691 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
693 * create-mscompat-collation-table.cs : fixed U+FF9F diacritical weight.
694 * SimpleCollator.cs : handle U+FF9E and U+FF9F as extenders.
696 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
698 * SimpleCollator.cs : Now FilterExtender() handles all extender
699 support. IndexOf() and LastIndexOf() now supports extenders.
700 IndexOf() and LastIndexOf() did not proceed contraction source
701 length as expected. Tiny refactoring on private IsPrefix() to take
704 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
706 * SimpleCollator.cs : when restoring from expansion, go back to the
707 top of the loop (to avoid index out of range).
708 Now IsPrefix() is implemented to reuse Compare() and thus it now
709 supports extender as well.
710 * Collation-notes.txt : status update. Deleted optimization part in
711 status section (it is duplicate).
713 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
715 * SimpleCollator.cs : some code reordering.
716 * create-mscompat-collation-table.cs : it was still missing U+3094.
718 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
720 * SimpleCollator.cs : Compare() now supports extender (e.g. U+39FC).
722 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
724 * SimpleCollator.cs : In GetSortKey(), don't update previousChar when
725 it is not primary (e.g. don't "extend" diacritical mark).
727 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
729 * managed-collation.patch : CompareInfo.Compare() should consider
730 the possibilities that non-empty string might be actually empty
731 in culture-sensitive context.
733 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
735 * SimpleCollator.cs : IndexOf() and LastIndexOf() returns start when
736 target is "empty" (in culture-sensitive context).
738 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
740 * SimpleCollator.cs : In IndexOf() and LastIndexOf(), skip ignorable
741 characters in target string.
743 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
745 * SimpleCollator.cs : When IgnoreWidth is specified, all Kana
746 characters are regarded as half-width.
747 Even though IgnoreWidth is specified, it should not ignore case.
748 For special weight comparison, the default values (E4) are bigger
749 than non-default values.
750 * SortKeyBuffer.cs : It should save LCID and original string.
751 * create-mscompat-collation-table.cs : For Japanese half-width kana,
752 it should not be counted in widthCompat map since IgnoreWidth does
753 not really ignore those differences.
755 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
757 * create-mscompat-collation-table.cs : Fixed missing Japanese bits.
759 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
761 * create-mscompat-collation-table.cs :
762 tiny diacritical weight fix for U+20D0-U+20E1.
764 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
766 * create-mscompat-collation-table.cs : ja CJK ideograph got completed.
768 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
770 * create-mscompat-collation-table.cs : Fixed CJK custom Japanese
771 mapping. It (maybe as well as other CJK tables) mixes NFKD. For
772 Japanese, modified NFKD table (because of Windows lame design).
774 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
776 * Makefile : added MONO_USE_MANAGED_COLLATION=no almost everywhere.
777 * MSCompatUnicodeTable.cs : FillCJK() was not invoked. Now it is
778 invoked at any time it is required.
779 * SimpleCollator.cs : call FillCJK() above in .ctor().
780 * MSCompatUnicodeTableUtil.cs : CJK range was wider.
781 * create-mscompat-collation-table.cs : CJK binary was missing the
782 length. CJK remapping is being moved to ModifyUnidata().
783 For cjk-ja mapping, we have to consider compat characters to be
784 added to the map, besides the raw UCA table.
786 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
788 * SortKeyBuffer.cs : Fixed shift level computation to match w/ Windows.
790 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
792 * SimpleCollator.cs : fixed LastIndexOf() to handle _target's_
793 contraction as expected. Fixed Compare() to save s2's contraction
795 * TestDriver.cs :added LastIndexOf() tester w/ indexes.
797 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
799 * managed-collation.patch : Fixed IsPrefix() and IsSuffix(). They
800 incorrectly use Compare().
801 * TestDriver.cs : more moved to nunit tests.
803 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
805 * SimpleCollator.cs : several fixes on Compare().
806 - Ignorable characters are skippted at the top of the loop.
807 - IgnoreNonSpace is checked to avoid extraneous level 2 comparison.
808 - In such case that s1 index is increased while s2 contraction is
809 replaced, s1 is inconsistently proceeded (bug).
810 - IsIgnorable() now also checks IgnoreNonSpace.
811 - Fixed FilterOptions() that does not work for IgnoreWidth at all.
812 * TestDriver.cs : now some are moved to nunit tests.
813 * Collation-notes.txt : minor todo update.
815 2005-07-11 Atsushi Enomoto <atsushi@ximian.com>
817 * SimpleCollator.cs : Compare() was ignoring such case that both
818 entire strings have '-' to be compared.
819 * Collation-notes.txt : more status updates.
820 * TestDriver.cs : added '-' use cases.
822 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
824 * SimpleCollator.cs : to be same as other buggy part, it now handles
825 U+3005, U+3031 and U+3032 as buggy as Windows. It just repeats
827 Fixed GetSortKey(): if the repeater is U+3005, second weight is 5.
828 * create-mscompat-collation-table.cs : dummy values for extenders.
830 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
832 * SimpleCollator.cs : Special weight fixes on GetSortKey(). Dash type
833 should be computed from ExtenderType, and voice mark weight should
835 * MSCompatUnicodeTable.cs : added tiny comment.
837 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
839 * SortKey.cs : It borked when MONO_USE_MANAGED_COLLATION is not yes.
840 * SimpleCollator.cs : support for extender (U+309D etc.).
842 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
844 * create-mscompat-collation-table.cs : some punct/symbols fix.
845 * managed-collation.patch : new (and temporary) file to support
846 managed collation in mscorlib.
847 * README : described how to use managed collation.
849 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
851 * create-mscompat-collation-table.cs : Further Cyrillic fixes. Handle
852 U+482-4C8 (though needs diacritical fixes).
853 * MSCompatUnicodeTable.cs : tiny comment for alternative impl.
855 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
857 * create-mscompat-collation-table.cs : Reimplemented Cyrillic weight
858 computation code, since it looks like the same way as Latin letters
859 have. Thus removed all other approach (UCA, by letter name).
861 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
863 * create-mscompat-collation-table.cs : diacritical fix for "double-
864 struck". Syriac nonspacing fixes.
866 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
868 * create-mscompat-collation-table.cs : more math symbol weight fixes.
870 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
872 * create-mscompat-collation-table.cs : fixed Hebrew character sortkeys.
874 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
876 * create-mscompat-collation-table.cs : math symbols U+25A0-U+2600 are
877 implemented (no stub). Some other fixes on category 8-A.
879 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
881 * create-mscompat-collation-table.cs : some minor fixes on Arabic,
882 Korean and Japanese sortkey weights.
884 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
886 * create-mscompat-collation-table.cs : More diacritical fixes.
887 Georgian characters do not have level 2 weights but level 3.
889 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
891 * create-mscompat-collation-table.cs : Roman numeral characters
892 have diacritical weight. quick hack for control signs (U+2400..)
895 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
897 * create-mscompat-collation-table.cs : improving Latin mappings.
898 Setting non-ASCII Latin characters' primary weight between those
899 ASCII characters, and setting diacritical weight (hacky).
900 * MSCompatUnicodeTable.cs :
901 Kanatype check: fixed (voice marks) and improved (comparison order).
903 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
905 * create-mscompat-collation-table.cs : more diacritical fixes.
906 primary weight fixes on punctuations in category 07.
908 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
910 * create-mscompat-collation-table.cs : several diacritical fixes.
911 * TestDriver.cs : sortkey dumper should use StringSort.
913 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
915 * SimpleCollator.cs : fixed incorrect indexer setup. Optimized
916 GetContraction() call a bit.
918 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
920 * create-mscompat-collation-table.cs : fixed incorrect level 2
922 * MSCompatUnicodeTable.cs : remove debug line.
924 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
926 * MSCompatUnicodeTableUtil.cs,
927 MSCompatUnicodeTable.cs,
929 create-mscompat-collation-table.cs : made some members internal and
930 accessible from other classes. Many indexes could be 0 by default.
931 * SimpleCollator.cs : optimizations. avoid method call.
933 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
935 * Collation-notes.txt : more updates.
936 * SimpleCollator.cs : Added quick check for Ordinal comparison.
937 Fixed special weight comparison. It cannot be customizable in the
938 implementation (and it won't be harmful).
939 * mono-tailoring-source.txt : thus updated comment.
941 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
943 * SimpleCollator.cs : Compare() was missing French sort support.
944 * TestDriver.cs : added example case.
946 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
948 * Collation-notes.txt : updated status. Eliminated descriptions on
949 "iterator" (I avoided it for performance concern). Fixed misc.
950 incorrect descriptions.
952 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
954 * Collator.cs : Now that SimpleCollator became feature complete, it is
957 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
959 * SimpleCollator.cs : implemented decent Compare() that immediately
960 stops at first primary difference.
962 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
964 * SimpleCollator.cs : indexers might return -1.
966 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
968 * SimpleCollator.cs : IsPrefix() and IsSuffix() optimization code was
969 buggy (length check for source was missing).
971 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
973 * create-mscompat-collation-table.cs : Fixed tailoring table output
974 to be in correct and countable order. Now if tailoring alias was not
975 found, just stop the build.
976 * MSCompatUnicodeTable.cs : several build fixes. Now it works to read
978 * mono-tailoring-source.txt : commented out CJK aliases that miss
980 * Makefile : needed further filename fixes.
982 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
984 * MSCompatUnicodeTable.cs : renamed from MSCompatUnicodeTable.template
985 (now it is working as a standalone file).
986 * Makefile : renamed generated file as MSCompatUnicodeTableGenerated.cs
987 (the generator now creates both binary resources and C# source).
989 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
991 * create-mscompat-collation-table.cs : Now it generates binary
992 resources (to parent directory).
993 * MSCompatUnicodeTable.template : added conditional code that fills
994 collation tables from manifest resources.
995 * Makefile : remove collation table binaries as well on "make clean".
996 Removed extraneous dependency.
998 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1000 * MSCompatUnicodeTable.template,
1001 SimpleCollator.cs : removed extraneous GetExpansion().
1003 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1005 * SimpleCollator.cs : IsSuffix() also supports contractions.
1006 * TestDriver.cs : IsSuffix() example contraction cases.
1008 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1010 * SimpleCollator.cs : reverted IsSuffix() to return bool (to match w/
1011 what current IsPrefix() does). For expansion of target, IsPrefix()
1012 should check the no-match case that expansion is longer than input.
1013 Some refactory on IsPrefix().
1014 Added GetContractionTal() for IsSuffix() (not used yet).
1016 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1018 * TestDriver.cs : added IsPrefix() expansion cases.
1019 * SimpleCollator.cs : IsPrefix() now supports contractions (with much
1020 of complexity), and it now returns bool again.
1021 IndexOf() for replacement should make use of IndexOfPrimitiveChar()
1022 since expansions won't be expanded recursively.
1024 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1026 * SimpleCollator.cs : commonized character comparison in IsPrefix()
1027 and IsSuffix(). csc compile fix.
1028 * CompareInfoImpl.cs : deleted.
1030 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
1032 * TestDriver.cs : added SimpleCollator.ctor() sanity check.
1033 Added replacement contraction example.
1034 * SimpleCollator.cs : Now IndexOf() and LastIndexOf() support
1035 contraction in source string. Extracted matching code to Matches().
1036 Replacement contraction was including extraneous '\x0'.
1038 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
1040 * Collation-notes.txt : updated status.
1041 * CollationDataStructures.txt : tiny fixes.
1042 * SimpleCollator.cs :
1043 Renamed alias Util to UUtil (MS sys.enterprisesvc has sucky global
1044 namespace Util and csc borked).
1045 GetContraction was incorrectly returning first item.
1046 Private IsPrefix() now returns int (but it might not be in real use).
1047 Extracted simple char comparison to CompareCharSimple().
1048 IndexOf() and LastIndexOf() now fully handle contractions (both
1049 binary key and string replacement) in "target" (for "s" not yet).
1050 * TestDriver.cs : be more verbose.
1051 * mono-tailoring-source.txt : added comment.
1052 * MSCompatUnicodeTable.template :
1053 Renamed alias Util to UUtil (MS sys.enterprisesvc has sucky global
1055 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
1057 * create-mscompat-collation-table.cs : compute COMBINING blah marks as
1058 well as those characters WITH blah.
1059 * TestDriver.cs : added combining sortkey cases.
1061 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
1063 * mono-tailoring-source.txt : fixed description on '*' in sortkeys.
1064 * SimpleCollator.cs : Now it fully uses tailoring info. Fixed
1065 contraction search that worked only when string is contraction.
1066 Removed commented code. Minor refactoring.
1067 * TestDriver.cs : added example that uses "ZS" in Hungarian sorting.
1069 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1071 * create-mscompat-collation-table.cs,
1072 * mono-tailoring-source.txt : removed extraneous level 4 sortkey
1073 which cannot be supported.
1074 * SimpleCollator.cs : added GetContraction() and used in some places.
1075 Now CompareOptions is set only once. Reordered some code (e.g.
1076 ignorable check -> get compat char -> compare).
1078 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1080 * SimpleCollator.cs : sort tailoring tables before actual usage.
1081 Support diacritical remappings (it is customized collation rule
1082 which does not exist in UCA).
1084 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1086 * SimpleCollator.cs : build culture specific tailoring table from
1087 TailoringInfo and unified data array.
1088 * create-mscompat-collation-table.cs : Added null termination to
1089 sortkey map tailorings (mostly to save my eyes).
1090 * MSCompatUnicodeTable.template : added public TailoringValues.
1092 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1094 * SortKeyBuffer.cs : handle special weight (category 06) characters.
1095 * Collation-notes.txt : Updated description on special weight (it was
1097 * TestDriver.cs : added special weight cases.
1099 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1101 * MSCompatUnicodeTable.template : added GetTailoringInfo().
1102 * SimpleCollator.cs : Now tailoring information is acquired and used.
1103 (FrenchSort is supported but Compare() won't work expectedly since
1104 the table is still incomplete for those diacritical marks).
1105 * SortKeyBuffer.cs : On reversing diacritical weights, it should
1106 ignore zeros. Reset() should reset frenchSorted flag.
1108 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1110 * create-mscompat-collation-table.cs : Further fixes on Jamo,
1111 diacritical weights by character name, and *Numbers primary weights.
1113 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1115 * create-mscompat-collation-table.cs : More fix on Devanagari,
1116 Gujarati, Oliya, Tamil and Lao sortkeys.
1118 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1120 * create-mscompat-collation-table.cs : Fixed Georgian, Thai, Gurmukhi
1123 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1125 * create-mscompat-collation-table.cs : Fixed Thai character primary
1126 and secondary values. Fixed Thaana letters. Added more LAMESPEC
1127 CJK compat. Fixed some circled CJK secondary weight.
1128 Hacked some nonspacing mark sortkey value adjustment.
1130 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1132 * create-mscompat-collation-table.cs : CP932.TXT was not parsed as
1133 expected. JIS ordering was incorrect. OtherNumbers that represents
1134 10 or more values were incorrectly computed the offset. Some Hangul
1135 compat characters has different offset.
1137 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1139 * create-mscompat-collation-table.cs : Fixed 0x8 category characters.
1140 Added hack for need-to-be-fixed characters to fall into 0xA category.
1141 * create-collation-element-table.cs : previous checkin seem failed :(
1142 * README: updated a bit.
1144 2005-06-24 Atsushi Enomoto <atsushi@ximian.com>
1146 * CodePointIndexer.cs :
1147 removed extraneous switch (I could use empty array for that need).
1148 * CollationElementTableUtil.cs : primary weight type became ushort.
1149 * create-collation-element-table.cs : several bugfixes.
1150 collElem should be int. It was skipping most of entries because of
1151 incorrect string tokenization.
1153 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1155 * create-mscompat-collation-table.cs : handle some Jamo NKFD.
1157 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1159 * SimpleCollator.cs : forgot to commit in the last checkin.
1160 * create-mscompat-collation-table.cs : fixed arabic shift weight chars.
1161 * TestDriver.cs : switch table dumper and collator testing.
1162 * SortKey.cs : for now comment out internal indexes (not in use).
1164 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1166 * MSCompatUnicodeTable.template,
1167 SimpleCollator.cs : support for culture dependent CJK table.
1169 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1171 * create-mscompat-collation-table.cs,
1172 MSCompatUnicodeTableUtil.cs : make CJK table more compact.
1174 2005-06-22 Atsushi Enomoto <atsushi@ximian.com>
1176 * SimpleCollator.cs : Fixed stupid index search when start != 0.
1178 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1180 * SimpleCollator.cs : fixed my misunderstanding on LastIndexOf(). It
1181 now starts from "start" and proceeds backward by "length".
1182 * TestDriver.cs : fix warning.
1184 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1186 * TestDriver.cs : more tests.
1187 * SimpleCollator.cs : LastIndexOf() is not setting search length
1188 on iteration. Quick workaround fro String.LastIndexOf() bug (maybe).
1190 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1192 * create-normalization-source.cs : output propValue as uint.
1194 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1196 * SortKey.cs : Now it is System.Globalization.SortKey.
1197 To replace existing implementation, it now requires lcid and
1198 CompareOptions. Added required members.
1199 * SortKeyBuffer.cs : thus .ctor() requires LCID.
1200 * SimpleCollator.cs : made required changes above.
1202 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1204 * CodePointIndexer.cs : added CompressArray(). Now it requires two more
1205 parameters for default index and codepoint.
1206 * CollationElementTableUtil.cs,
1207 NormalizationTableUtil.cs : required changes wrt above change.
1208 * MSCompatUnicodeTableUtil.cs : added for several codepoint indexers.
1209 * MSCompatUnicodeTable.template : Now it uses codepoint indexer.
1210 * create-mscompat-collation-table.cs : Now it outputs compressed array.
1211 * Makefile : now collation requires MSCompatUnicodeTableUtil.cs
1213 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1215 * SimpleCollator.cs :
1216 Implemented IsSuffix() and LastIndexOf().
1217 Several fixes on index > 0 cases.
1218 * TestDriver.cs : sample IsSuffix() and LastIndexOf() usage and more.
1220 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1222 * Collation-notes.txt : updated (status, impl. classes).
1223 * MSCompatUnicodeTable.cs : Korean Jamo are not really expansions.
1225 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1227 * SimpleCollator.cs : implemented IndexOf(string,string,CompareOptions)
1228 and IsPrefix(). Tiny code refactory.
1229 * TestDriver.cs : sample IsPrefix() and IndexOf() usage.
1230 * MSCompatUnicodeTable.cs : tiny refactory for CodePointIndexer use.
1232 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1234 * SimpleCollator.cs :
1235 IndexOf(string, char, CompareOptions) implementation.
1236 * TestDriver.cs : sample IndexOf() usage.
1238 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1240 * create-mscompat-collation-table.cs : was missing most important
1241 kind of blocks - equivalent expansions (e.g. invariant mappings).
1242 More readable mappings.
1244 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1246 * mono-tailoring-source.txt : new file. It describes tailoring
1247 information. Basically examined under .NET 1.x.
1248 * create-mscompat-collation-table.cs : consume the file above.
1249 * MSCompatUnicodeTable.template : now tailorings is not a stub.
1250 * CollationDataStructures.txt : minor fixes.
1252 SimpleCollator.cs : added FrenchSort support.
1253 * Collation-notes.txt : added description on Latin primary weights.
1254 * ldml-limited.rng : added note.
1255 * create-tailorings.cs : added note. more serialization (but won't be
1258 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1260 * SortKeyBuffer.cs : non-primary character is added to previous
1262 * TestDriver.cs : added example case of above.
1264 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1266 * SimpleCollator.cs : IgnoreSymbols support.
1267 * TestDriver.cs : compilation fix. IgnoreSymbols example.
1268 * create-mscompat-collation-table.cs : more Hangul fixes.
1270 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1272 * create-mscompat-collation-table.cs : more Hangul fixes.
1273 * SortKey.cs : it will replace sys.globalization.SortKey. It has
1274 some internal members.
1275 * SortKeyBuffer.cs : now it uses SortKey instead of byte[].
1276 * SimpleCollator.cs : CompareOptions support. However I don't think
1277 it will be developed anymore since SortKey never enables IndexOf().
1278 * TestDriver.cs : a few CompareOptions cases.
1280 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1282 * SimpleCollator.cs : simple collator implementation that just will
1283 use GetSortKey() for all its basis.
1284 * TestDriver.cs : sample code that uses this collator set.
1285 * MSCompatUnicodeTable.template : removed test driver from here.
1287 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1289 * create-mscompat-collation-table.cs : Hangul fixes.
1290 Now less than 300 characters that does not have sortkey weights.
1291 * MSCompatUnicodeTable.template : added FIXME info for Hangul Jamo.
1293 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1295 * create-mscompat-collation-table.cs : Added control picture mappings.
1296 Minor primary weight fixes.
1298 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1300 * create-mscompat-collation-table.cs : Added mappings for box
1301 drawings and blocks.
1303 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1305 * create-mscompat-collation-table.cs : Added mappings for arrows.
1307 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1309 * create-mscompat-collation-table.cs : added support for letterlike
1310 characters and squared CJK compatibility characters, ordered by
1311 character names (0x0E category).
1312 * Collation-notes.txt : added description on that.
1314 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1316 * MSCompatUnicodeTable.template : Now expansions are simulated.
1317 * create-mscompat-collation-table.cs : filled Korean number level2.
1318 Reordered some code blocks to fill correct diacritical differences.
1319 * Collation-notes.txt : some corrections and minor additions.
1321 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1323 * MSCompatUnicodeTable.template :
1324 Now dumper test driver uses SortKeyBuffer for dogfooding.
1325 * create-mscompat-collation-table.cs : some diacritical level fixes
1326 (with non-working extra latin check).
1327 * SortKeyBuffer.cs : several fixes to get working as a practical code.
1328 * Collator.cs : make it compilable, leaving things as NotImplemented.
1330 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1332 * create-mscompat-collation-table.cs : some fixes on primary category
1333 07 (miscellaneous symbols and punctuations).
1335 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1337 * create-mscompat-collation-table.cs : more mapping fix on numbers,
1338 letters, variable weight characters, circled Japanese and CJK.
1339 * MSCompatUnicodeTable.template : fixed HasSpecialWeight() to be more
1340 inclusive. Simplified dumper code.
1342 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1344 * create-mscompat-collation-table.cs : finished Hangul (both Jamo
1345 and Syllables). sortkey dumper diff lines became 8000 from 30000.
1347 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1349 * create-mscompat-collation-table.cs : added some nonspacing marks in
1350 either correct or hacky way.
1352 2005-06-13 Atsushi Enomoto <atsushi@ximian.com>
1354 * create-mscompat-collation-table.cs : several improvements. Japanese
1355 Kana support, Hebrew accents, Bengali nonspacing marks, sorting of
1356 numeric characters, diacritically decorated latin alphabets. Fixed
1357 some diacritical weights detection.
1358 * MSCompatUnicodeTable.cs : tiny Japanese fix. Handle nonspacing
1359 marks' primary weight as empty.
1360 * Collation-notes.txt : some updates.
1362 2005-06-13 Atsushi Enomoto <atsushi@ximian.com>
1364 * create-mscompat-collation-table.cs : don't process nonexact NFKD
1365 mapping as equivalent, however store CJK extensions into NFKD map
1366 even if one does not strictly match.
1367 Now am going to fill Hangul into tables (unlike UCA it does not look
1368 possible to calculate sortkey value).
1369 Fixed Cyrillic and Georgian UCA based orderings.
1370 * MSCompatUnicodeTable.template : added CJK extension sortkey
1373 2005-06-10 Atsushi Enomoto <atsushi@ximian.com>
1375 * create-mscompat-collation-table.cs : Fixed latin alphabet support.
1376 Added latin with diacritical and CJK extension.
1377 * MSCompatUnicodeTable.cs : modified dumper code a bit (for my purpose).
1379 2005-06-10 Atsushi Enomoto <atsushi@ximian.com>
1381 * create-mscompat-collation-table.cs : now parses DerivedAge.txt (right
1382 now not used thouth). Filled CJK ideograph, still not perfect.
1383 Fixed number primary keys. NFKD numbers and CJK ideographs are now
1384 considered, including brackets elimination.
1385 * Makefile : now it downloads DerivedAge.txt.
1386 * MSCompatUnicodeTable.template : added dummy code dumper. It computes
1387 PrivateUse, Surrogate and Hangul Syllables.
1388 * Collation-notes.txt : Noted that Hangul Syllables need more love.
1390 2005-06-09 Atsushi Enomoto <atsushi@ximian.com>
1392 * create-tailorings.cs : added configuration support. sort them.
1393 I wonder if it is really usable. Having own format might be better.
1394 * create-mscompat-collation-table.cs : fixing some sortkey numbers,
1395 making closer to windows. Now it handles NFKD in some places.
1396 * MSCompatUnicodeTable.template : Added dummy sortkey dumper driver.
1397 * CollationDataStructures.txt : added description on tailoring
1398 fields, though they are subject to change.
1400 2005-06-07 Atsushi Enomoto <atsushi@ximian.com>
1402 * create-tailorings.cs, ldml-limited.rng : new file.
1403 * LdmlReader.cs : removed old file.
1405 2005-06-07 Atsushi Enomoto <atsushi@ximian.com>
1407 * SortKeyBuffer.cs : split from Collator.cs. Now it considers
1408 practical use, reflecting updated sortkey constant design.
1409 Especially level 4 weight is split to 4 arrays that are merged in
1410 the last stage of GetSortKey().
1411 * Collator.cs : thus SortKeyBuffer is removed from here.
1412 Additionally, removed some extraneous bits in other classes.
1413 * Collation-notes.txt : Some editorial fixes. Added information on
1414 Korean matter (how to compute Hangle Syllables / Hangul Jamo cannot
1415 be stored in simple byte arrays).
1416 * CodePointIndexer.cs,
1417 create-collation-element-table.cs,
1418 CollationElementTable.template,
1419 NormalizationTableUtil.cs : short CodePointIndexer method names.
1420 * create-mscompat-collation-table.cs : Additional info on why some
1421 meaningful characters are ignored in Windows (Unicode version
1422 difference). Removed U+070F from special check (was extraneous).
1424 2005-06-06 Atsushi Enomoto <atsushi@ximian.com>
1426 * MSCompatUnicodeTable.template:
1427 Moved body implementation to table creator and put those bool
1428 results into an array.
1429 * create-mscompat-collation-table.cs :
1430 So imported those methods. Modified array output to emit "0x"
1431 only for more than 9.
1432 * create-normalization-source.cs : ditto on "0x" output matter.
1433 * CollationDataStructures.txt : so now it holds ignorableFlags.
1435 2005-06-03 Atsushi Enomoto <atsushi@ximian.com>
1437 * Collation-notes.txt, CollationDataStructures.txt :
1438 separate document for data structure design.
1440 2005-06-03 Atsushi Enomoto <atsushi@ximian.com>
1442 * create-mscompat-collation-table.cs : added culture-dependent CJK
1443 table creation. It uses CLDR as its basis. (Culture independent CJK
1445 * Makefile : added CLDR archive downloading support.
1446 * MSCompatUnicodeTable.template : tiny renamings.
1447 * Collation-notes.txt : additional CJK info.
1449 2005-06-02 Atsushi Enomoto <atsushi@ximian.com>
1451 * Collation-notes.txt, create-mscompat-collation-table.cs :
1452 added secondary weight support for BlahNumber characters.
1454 2005-06-01 Atsushi Enomoto <atsushi@ximian.com>
1456 * downloaded : added directory. All downloaded files are stored here.
1457 * Makefile : use "downloaded" directory.
1458 Added more auto-download stuff.
1459 * create-mscompat-collation-table.cs :
1460 Added Japanese square kana support.
1462 2005-06-01 Atsushi Enomoto <atsushi@ximian.com>
1464 * Collation-notes.txt : added Estrangela (ancient Syriac) and Thaana.
1465 * create-mscompat-collation-table.cs : added support for Arabic abjad,
1466 Estrangela and Thaana.
1467 * MSCompatUnicodeTable.template : removed BOM.
1469 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1471 * Collation-notes.txt : wrong comment cleanup and spelling fixes.
1472 * create-mscompat-collation-table.cs : added diacritic support for
1473 Latin letters (as long as covered in primary weight).
1475 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1477 * Makefile : minor fixes. Added warning lines to generated sources.
1479 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1481 * create-char-mapping-source.cs :
1482 Removed ToWidthInsensitive() generation.
1484 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1486 * create-mscompat-collation-table.cs : Now it dumps level1 to 3 values.
1487 ToWidthInsensitive() is implemented here, using an array (which is
1488 to be optimized using CodePointIndexer).
1489 * MSCompatUnicodeTable.cs : renamed as MSCompatUnicodeTable.template
1490 * MSCompatUnicodeTable.template : now it is used to generate
1491 MSCompatUnicodeTable.cs which got ready to be used.
1492 * Makefile : added MSCompatUnicodeTable.cs build support. Now it
1493 supports "make normalization" and "make collation".
1495 2005-05-30 Atsushi Enomoto <atsushi@ximian.com>
1497 * Collation-notes.txt : Description on ICU is very incorrect. Now it
1498 became more rational and sane.
1499 * create-mscompat-collation-table.cs : fixed some indexes.
1500 * Makefile : added "mstablegen" target.
1501 * MSCompatUnicodeTable.cs : removed GetPrimaryWeight(). Minor fix.
1503 2005-05-26 Atsushi Enomoto <atsushi@ximian.com>
1505 * Collation-notes.txt : more analysis on "letters".
1506 * create-mscompat-collation-table.cs : more proof of concepts.
1508 2005-05-25 Atsushi Enomoto <atsushi@ximian.com>
1510 * Collation-notes.txt : more info. Started letter sortkey analysis
1511 (some of other stuff are really non-understandable right now.)
1512 * create-mscompat-collation-table.cs : table generator proof-of-
1513 concept source (not compilable).
1514 * MSCompatUnicodeTable.cs : moved some code to the new source.
1517 2005-05-20 Atsushi Enomoto <atsushi@ximian.com>
1519 * Collation-notes.txt : started level 2 weight analysis.
1521 2005-05-19 Atsushi Enomoto <atsushi@ximian.com>
1523 * Collation-notes.txt : Additional information on how to create
1525 * MSCompatUnicodeTable.cs : implemented part of GetLevel3Weight().
1527 2005-05-19 Atsushi Enomoto <atsushi@ximian.com>
1529 * Collation-notes.txt : More case weight (level 3) analysis. I'm
1530 likely to just write table generator.
1532 2005-05-18 Atsushi Enomoto <atsushi@ximian.com>
1534 * MSCompatUnicodeTable.cs : part of level 4 weight implementation.
1536 2005-05-18 Atsushi Enomoto <atsushi@ximian.com>
1538 * Collation-notes.txt :
1540 Revised comparison methods; backward iteration is possible.
1541 More on char-by-char comparison.
1542 Level 4 comparison is actually a bit more complex.
1544 * Collator.cs : some conceptual updates wrt above.
1546 2005-05-17 Atsushi Enomoto <atsushi@ximian.com>
1548 * Collation-notes.txt : Japanese voice mark is level 2, and Hangul
1549 properties are level 3.
1551 2005-05-17 Atsushi Enomoto <atsushi@ximian.com>
1553 * Collation-notes.txt : Make it more readable. More analysis on
1554 level 3 and 4 sortkey structures.
1555 * Collator.cs : some compilation fixes (not compilable yet).
1557 2005-05-16 Atsushi Enomoto <atsushi@ximian.com>
1559 * Collation-notes.txt : Analysis on variable-weighting (level 5)
1561 * Collator.cs : updated corresponding part of level 5, and more.
1563 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1565 * Collation-notes.txt : more updates.
1566 * Collator.cs : rewrote from scratch. Some rough sketch for sortkey
1567 buffer, character iterator and collator methods. Not compiling.
1569 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1571 * Collator.cs : Am going to replace it with new one. No need for
1572 CompareOptions-dependent Comparer.
1574 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1576 * Collation-notes.txt : There seems a bit more complexity.
1578 2005-05-10 Atsushi Enomoto <atsushi@ximian.com>
1580 * Collation-notes.txt : more updates, being close to write sortkey
1583 2005-05-09 Atsushi Enomoto <atsushi@ximian.com>
1585 * CompareInfoImpl.cs, Collator.cs : conceptual update
1586 * Collation-notes.txt : some corrections and additions.
1587 * Makefile : added LDML input (but it won't be used at all).
1589 2005-04-28 Atsushi Enomoto <atsushi@ximian.com>
1591 * Collation-notes.txt : more updates.
1593 2005-04-26 Atsushi Enomoto <atsushi@ximian.com>
1595 * Collation-notes.txt : more updates.
1597 2005-04-26 Atsushi Enomoto <atsushi@ximian.com>
1599 * Collation-notes.txt : some updates.
1600 * create-mapping-char-source.cs : superscripts and subscripts are also
1601 ignored in IgnoreWidth comparison.
1602 * Makefile : tiny touch fix.
1604 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1606 * CompareInfoImpl.cs, Collator.cs : conceptual stuff (not working).
1608 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1610 * create-char-mapping-source.cs : Now it generates
1611 ToWidthInsensitive() from combining category <wide> and <narrow>.
1612 * MSCompatUnicodeTable.cs : added ToKanaTypeInsensitive() and
1613 ToWidthInsensitive() for IgnoreKanaType and IgnoreWidth.
1615 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1617 * README, LdmlReader.cs, DataStructures.txt : new files.
1619 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1621 * CodePointIndexer.cs,
1622 Collation-notes.txt,
1623 CollationElementTable.template,
1624 CollationElementTableUtil.cs,
1625 create-char-mapping-source.cs,
1626 create-collation-element-table.cs,
1627 create-combining-class-source.cs,
1628 create-normalization-source.cs,
1630 MSCompatUnicodeTable.cs,
1631 Normalization.template,
1632 NormalizationTableUtil.cs : initial checkin (to private branch).