1 2009-09-17 Atsushi Enomoto <atsushi@ximian.com>
3 * Normalization.cs : some renaming for disambiguation.
4 * NormalizationTableUtil.cs : fix some wrong ranges in
5 mapIdxToComposite. This fixes some Arabic normalization (and more).
6 * normalization-notes.txt : added some notes on the implementation.
8 2008-06-19 Atsushi Enomoto <atsushi@ximian.com>
11 - reverted the previous index calculation change. It was correctly
12 implemented and I rather broke it.
13 - fix index calculation on combining.
14 - NFKD was incorrectly directed to combining path. It should not.
15 - Simplify quick check.
17 2008-06-15 Atsushi Enomoto <atsushi@ximian.com>
19 * Normalization.cs : For NFC and NFKC, IsNormalized() was not working
20 enough to check composed characters. It's not possible without
21 the actual composition, so just call Normalize() and compare them.
22 In Normalize() mapping helper didn't pick correct map index since
23 the table for index stores index for "uncompressed" numbers.
24 * NormalizationTableUtil.cs : updated to the latest UCD.
25 * Makefile : to build test, source file must be downloaded too.
27 2008-11-05 Atsushi Enomoto <atsushi@ximian.com>
29 * ucd.cs : Write type for *_count. Add notice to not edit
30 unicode-data.h directly.
32 2008-11-04 Atsushi Enomoto <atsushi@ximian.com>
34 * ucd.cs : new code to generate unicode table for eglib.
36 2008-07-04 Andreas Nahr <ClassDevelopment@A-SoftTech.com>
38 * SortKey: Fix parameter names, add attribute, small formatting
40 2008-06-27 Rodrigo Kumpera <rkumpera@novell.com>
42 * CodePointIndexer.cs : Make TableRange a struct instead
43 of a class so we save 2 memory ops per ToIndex loop.
45 2008-04-02 Atsushi Enomoto <atsushi@ximian.com>
47 * SortKey.cs : check null arguments. Fixed bug #376171.
49 2007-07-20 Atsushi Enomoto <atsushi@ximian.com>
51 * create-mscompat-collation-table.cs : I wonder how long its build
54 2007-03-06 Atsushi Enomoto <atsushi@ximian.com>
56 * SimpleCollator.cs : disable QuickCheckPossible(), which is
57 inaccurate and inefficient. Fixed bug #79714.
59 2007-02-15 Atsushi Enomoto <atsushi@ximian.com>
61 * SimpleCollator.cs : character filtering is needed for
62 OrdinalIgnoreCase in 2.0 profile. Fixed bug #80865.
64 2007-01-25 Atsushi Enomoto <atsushi@ximian.com>
66 * SimpleCollator.cs : GetTailContraction() was broken to pick correct
67 contraction/special sortkey out and thus LastIndexOf() failed when
68 it is involved. Fixed bug #80612.
70 2007-01-22 Atsushi Enomoto <atsushi@ximian.com>
72 * SimpleCollator.cs : for non-StringSort comparison, level5 (- and ')
73 should be still skipped after initial level5 check is done (while
74 they were simply treated as a normal character). Fixed bug #78748.
75 * SortKeyBuffer.cs : Fixed NRE in french sort.
77 2006-12-25 Atsushi Enomoto <atsushi@ximian.com>
79 * SimpleCollator.cs : added IndexOf() implementation for Ordinal
80 and OrdinalIgnoreCase, though Ordinal version is not used (since
81 it is slower than icall).
83 2006-05-30 Miguel de Icaza <miguel@novell.com>
85 * MSCompatUnicodeTable.cs: Remove the fixed loading and compute it
86 just when we actually consume it. This only fixes the
89 2006-04-14 Atsushi Enomoto <atsushi@ximian.com>
91 * README: removed obsolete info.
92 * Normalization.cs : canonical reordering should participate in the
93 decomposition step. In reordering, string append was incomplete.
94 Combining class check is required in NFD check. Icall is written
97 2005-12-07 Zoltan Varga <vargaz@gmail.com>
99 * SimpleCollator.cs: Fix a warning.
101 2005-11-30 Sebastien Pouliot <sebastien@ximian.com>
103 * SimpleCollator.cs: Fix CAS support. The static ctor/var try to get
104 the environment variable MUCH too soon (i.e. the security manager
107 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
109 * SimpleCollator.cs : direct fast-path optimization for IndexOf().
111 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
113 * SimpleCollator.cs :
114 - CompareQuick(): added immediateBreakup to avoid extraneous sortkey
116 - QuickCheckPossible(): index used for s1 was incorrect.
118 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
120 * SimpleCollator.cs : added another quick check for CompareInternal()
121 that does almost ordinal comparison for quick-checkable strings.
122 (It affects on Compare(), IndexOf(), IsSuffix() etc. as well.)
124 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
126 * MSCompatUnicodeTable.cs : (IsIgnorable) \0 is not ignorable.
129 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
131 * SimpleCollator.cs :
132 Created another struct to reduce method arguments. Created another
133 flags that keeps "once-matched" state (counterpart of
134 checkedFlags, now neverMatchFlags).
136 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
138 * SimpleCollator.cs :
139 - Added CompareOrdinalIgnoreCase() for NET_2_0 RTM.
140 - Reduced extra parameter from LastIndexOfSortKey().
141 - LastIndexOf() should use GetTailContraction for the source string.
142 And then, target could match in the middle of the possible
143 "replacement contraction" of the source string, so use
144 LastIndexOfSortKey() to catch them.
145 - Fixed GetTailContraction() that caused index out of range.
147 2005-11-11 Atsushi Enomoto <atsushi@ximian.com>
149 * Makefile : Now use MONO_DISABLE_MANAGED_COLLATION.
150 * SortKey.cs : some members are virtual.
152 2005-10-14 Atsushi Enomoto <atsushi@ximian.com>
154 * SimpleCollator.cs : modified to use stackalloc for byte array.
156 2005-09-27 Atsushi Enomoto <atsushi@ximian.com>
158 * SimpleCollator.cs : in CompareInternal(), there was a possibility of
159 infinite loop. Fixed bug #76243.
161 2005-09-20 Atsushi Enomoto <atsushi@ximian.com>
163 * SimpleCollator.cs : In IsPrefix/IsSuffix, if target is an empty string,
164 immediately return true.
166 2005-09-09 Atsushi Enomoto <atsushi@ximian.com>
168 * SimpleCollator.cs : IsSuffix() optimization logic was buggy, so just
169 use pretty simple way with LastIndexOf() (no significant perf.
172 2005-09-01 Atsushi Enomoto <atsushi@ximian.com>
174 * README, Collation-notes.txt, CollationDataStructures.txt :
175 removing obsolete info and some added some notes.
177 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
179 * Normalization.cs : remove warned code.
180 * managed-collation.patch : now it's not required anymore.
182 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
184 * MSCompatUnicodeTable.cs : added IsSortable(string).
186 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
188 * SimpleCollator.cs : Now all collator methods are thread safe.
190 All instance non-readonly fields turned into arguments of every
191 methods that use those fields.
192 (Sadly it is the end of no-memory-cost collator era. mcs bootstrap
193 now needs +100KB memory consumption.)
195 2005-08-09 Atsushi Enomoto <atsushi@ximian.com>
197 * SimpleCollator.cs : made "checkedFlags" as nullable and made it as
198 an argument of every index methods (to make it thread safe).
200 2005-08-09 Atsushi Enomoto <atsushi@ximian.com>
203 MSCompatUnicodeTable.cs :
204 - Now IsIgnorable() is aggregated to be one invokation to check
205 completely ignorable, nonspacing and symbols.
206 - Introduced "already checked" flags for IndexOf() and LastIndexOf()
207 to skip sortkey binary check on the same characters. Significant
208 perf. improvement for such case as IndexOf("AABCBABC...Z",'Z').
210 2005-08-08 Gert Driesen <drieseng@users.sourceforge.net>
212 * SortKey.cs: Marked Serializable to match MS.NET.
214 2005-08-08 Atsushi Enomoto <atsushi@ximian.com>
216 * create-mscompat-collation-table.cs,
217 Makefile : changed resources output directory.
219 2005-08-04 Atsushi Enomoto <atsushi@ximian.com>
221 * create-normalization-tests.cs,
222 StringNormalizationTestSource.cs : new files for Unicode
223 Normalization test generator.
224 * Makefile : added support for above.
226 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
228 * NormalizationTableUtil.cs : oops, it does not compile.
229 * managed-collation.patch : I guess having managed resource would be
230 better for collation. At least current code has such #define so
231 Makefile should be in sync with it.
233 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
235 * create-normalization-source.cs : Fixed CharMapComparer which
236 incorrectly returned 0 when the second arg is shorter. Reduced
237 extraneous helperIndex map. Other minor fixes and code removal.
238 * Normalization.cs : several fixes to support blocked combine handling.
239 * NormalizationTableUtil.cs : tiny member renaming.
241 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
243 * create-normalization-source.cs,
244 NormalizationTableUtil.cs,
245 Normalization.cs : several bugfixes on index miscomputation.
246 Renamed using aliases (csc will bork). Primary combine safety is now
247 computed during UnicodeData.txt parse.
248 Maximum NFKD length was 18, not 4 (U+FDFA).
250 2005-08-02 Atsushi Enomoto <atsushi@ximian.com>
252 * managed-collation.patch : added Normalization support.
253 * managed-collation-icall.patch : added, including normalization stuff.
255 BTW when will collation code checked in?
257 2005-08-02 Atsushi Enomoto <atsushi@ximian.com>
259 * create-normalization-source.cs : Unified three normalization source
260 generators, to compute IsUnsafe flag. Fixed helperIndex array type
262 * create-char-mapping-source.cs,
263 create-combining-class-source.cs : thus removed.
264 * Makefile : thus modified for the above integration.
265 * NormalizationTableUtil.cs : Extended to contain IsUnsafe flag.
266 * Normalization.cs : Several fixes to make Normalize() actually work.
268 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
270 * create-normalization-source.cs,
272 create-char-mapping-source.cs,
273 create-combining-class-source.cs,
274 Makefile : converted managed array to pointers (like collation stuff).
276 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
278 * NormalizationTableUtil.cs : further table range optimization.
279 * create-normalization-source.cs,
280 create-char-mapping-source.cs,
281 create-combining-class-source.cs : added C header output support.
283 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
285 * create-normalization-source.cs, Normalization.cs :
286 Now property size is < 256, so directly embed value in "props" array.
287 Add QuickCheck(c,checkType) and remove IsNFD/C/KD/KC and delegates.
289 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
291 * create-combining-class-source.cs,
292 create-char-mapping-source.cs,
293 create-normalization-source.cs,
294 NormalizationTableUtil.cs,
295 Normalization.cs : String.Normalize() does not handle surrogate
296 characters. mapping information in DerivedNormalizationProps.txt
297 are not used in the code (those from UnicodeData.txt is used).
298 Hangul syllables are computed instead of embedded in the tables.
299 * managed-collation.patch : removed IntPtrStream and Makefile patches.
301 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
303 * MSCompatUnicodeTable.cs : IsSortable() was broken.
305 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
307 * MSCompatUnicodeTable.cs : added helper for CompareInfo.IsSortable().
309 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
311 * create-tailoring.cfg : added for convenience of contraction check.
313 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
315 * create-normalization-source.cs,
318 create-mscompat-collation-table.cs,
319 MSCompatUnicodeTableUtil.cs,
321 create-collation-element-table.cs,
322 MSCompatUnicodeTable.cs,
324 create-combining-class-source.cs : added copyright lines.
326 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
328 MSCompatUnicodeTable.cs : removed extraneous definition.
330 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
332 * create-mscompat-collation-table.cs
333 MSCompatUnicodeTable.cs : full C header support, finally.
335 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
338 NormalizationTableUtil.cs,
339 create-char-mapping-source.cs : more aggressive data compression.
340 It now ignores characters that are >= U+10000.
342 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
345 Normalization.template,
346 Normalization.cs : renamed existing file.
348 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
350 * NormalizationTableUtil.cs,
351 Normalization.template,
352 create-combining-class-source.cs : GetCombiningClass is now
353 implemented as indexer based array.
354 * Makefile : renamed output filename.
355 * create-mscompat-collation-table.cs : removed comments that does not
357 * create-tailoring.cs : use utf-8 output (and fixed filename).
359 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
361 * create-mscompat-collation-table.cs : hacked safer IPA extensions.
362 * Collation-notes.txt : status of sortkey table.
364 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
366 * create-mscompat-collation-table.cs : some Greek mapping fix.
368 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
370 * create-mscompat-collation-table.cs : diacritical weight is not
371 treated correctly when they are picked from letter names, as flags.
373 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
375 * create-mscompat-collation-table.cs : fixed culture-dependent
376 nonspacing mark weight.
378 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
380 * create-mscompat-collation-table.cs : some Hebrew case letter fixes.
381 Some diacritical fixes on symbols.
383 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
385 * create-mscompat-collation-table.cs : Fixed level 3 weight of
386 Arabic presentation forms.
388 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
390 * create-mscompat-collation-table.cs : Fixed some diacritical weight
391 of Arabic presentation forms.
393 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
395 * SimpleCollator.cs : more status updates. It's almost complete,
396 except for sortkey values.
398 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
400 * SimpleCollator.cs : similar optimization also for LastIndexOf().
402 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
404 * SimpleCollator.cs : the previous patch was missing IgnoreNonSpace
407 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
409 * SimpleCollator.cs : reduced extra sortkey value computation in
410 MatchesForward(). It makes IndexOf() roughly 30% faster.
412 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
414 * SortKey.cs : GetHashCode() returns a value based on its byte data.
417 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
419 * SimpleCollator.cs : consider extractions in invariant culture.
421 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
423 * SimpleCollator.cs : (unsafeFlags) be compact ;-)
425 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
427 * SimpleCollator.cs : When the tail of the target does not match more
428 than 3 times, then IsSuffix() will never be true (3 is the max
429 length of an expansion; \uFB03 -> ffi). It brings significant
430 performance boost when "source" string is very long.
431 * MSCompatUnicodeTable.cs : added MaxExpansionLength constant.
432 Reordered code lines.
434 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
436 * Collation-notes.txt : updated implementation status.
438 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
440 * SimpleCollator.cs : Implemented quick codepoint comparison in
441 Compare(). Comparison became 125x faster.
442 * mono-tailoring-source.txt : added tiny comment.
444 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
446 * mono-tailoring-source.txt : Added all single sortkey remapping to
447 all cultures (still need to fill contractions and annotate possible
448 buggy mapping referencing to CLDR).
449 * SimpleCollator.cs : removed unused code.
450 * MSCompatUnicodeTable.cs : tiny cast removal.
452 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
455 create-mscompat-collation-table.cs
456 MSCompatUnicodeTableUtil.cs
457 MSCompatUnicodeTable.cs : Now CJK mapping data is stored as byte
458 arrays. Thus SimpleCollator does not need to use bitwise and shift
459 operations to get sortkey value and they could be managed resources.
461 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
463 * create-mscompat-collation-table.cs,
464 MSCompatUnicodeTable.cs,
465 MSCompatUnicodeTableUtil.cs : From the result of sortkey comparison
466 between None and IgnoreWidth, width compat table could be computed
467 in somewhat simple way. So removed that table and all related code.
468 Increased the collation resource version.
470 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
472 * create-mscompat-collation-table.cs : Added C header output support.
474 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
476 * create-mscompat-collation-table.cs : FillLetterNFKD() could also be
477 applied to Cyrillic letters. Saved some of them.
479 2005-07-24 Atsushi Enomoto <atsushi@ximian.com>
481 * MSCompatUnicodeTable.cs : oh, ok, so we already have
482 GetManifestResourceInternal() ;-)
483 * managed-collation.patch : in Assembly.cs made that method internal.
485 2005-07-24 Atsushi Enomoto <atsushi@ximian.com>
487 * MSCompatUnicodeTable.cs : the pointer based icall code could be
488 also applicable for USE_MANAGED_RESOURCE mode.
490 2005-07-23 Atsushi Enomoto <atsushi@ximian.com>
492 * MSCompatUnicodeTable.cs : added icall support code (not enabled
493 unless the first line is commented out).
495 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
497 * create-mscompat-collation-table.cs,
498 MSCompatUnicodeTableUtil.cs,
499 MSCompatUnicodeTable.cs : Added resource version output (and ignore
500 in case of version mismatch). Removed obsolete, commented out code.
502 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
505 MSCompatUnicodeTable.cs,
506 create-mscompat-collation-table.cs : Now they use unmanaged pointers
507 instead of managed arrays.
508 * managed-collation.patch : Now it contains patch for IntPtrStream.cs
509 and Assembly.cs as well.
511 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
513 * MSCompatUnicodeTable.cs,
514 SimpleCollator.cs : Moved tailoring support classes to
515 MSCompatUnicodeTable.cs and drawn out from SimpleCollator.
516 Now that cjk and tailoring support are filled inside
517 MSCompatUnicodeTable, no managed array is exposed.
519 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
521 * create-mscompat-collation-table.cs,
523 MSCompatUnicodeTable.cs : Now it's not exposing collation table
524 internals as managed arrays (to switch to unmanaged pointers).
526 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
528 * create-mscompat-collation-table.cs : tiny nonspacing mark fix.
530 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
532 * create-mscompat-collation-table.cs : Fixed most of Greek mappings.
533 * MSCompatUnicodeTable.cs : don't lock string.
535 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
537 * create-mscompat-collation-table.cs : More Cyrillic diacritical fixes.
539 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
541 * create-mscompat-collation-table.cs : More Latin diacritical fixes.
543 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
545 * create-mscompat-collation-table.cs : There were still missing
546 math symbol mappings. Added several hacky diacritical weight for
549 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
551 * create-mscompat-collation-table.cs : fixed a few diacritical weight
552 on Cyrillic characters. Fixed ParseTailoringSource() to handle
553 non-heading escape sequence (\uXXXX) as expected.
555 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
557 * create-mscompat-collation-table.cs,
558 MSCompatUnicodeTableUtil.cs,
559 MSCompatUnicodeTable.cs : added more aggressive index limits for
560 table optimization at data size, in cost of speed.
562 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
564 * create-mscompat-collation-table.cs : fixed Arabic thirtial weight.
566 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
568 * create-mscompat-collation-table.cs : Mapping for hyphens and
569 punctuation are kinda finished. Rewrote batch mapping method to
570 collect all NFKD. Required modification on mapping is done.
572 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
574 * create-mscompat-collation-table.cs : minor mapping fixes on accent
575 marks and punctuations.
577 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
579 * create-mscompat-collation-table.cs : Fixed some MathSymbol mapping
580 and Box drawing mapping.
582 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
584 * create-mscompat-collation-table.cs : Fixed almost all numbers.
586 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
588 * create-mscompat-collation-table.cs : Symbol mappings are almost done.
589 Removed hack that gave dummy mappings to blank symbols.
591 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
593 * create-mscompat-collation-table.cs : more fix on arrows. Fix on box
594 drawings. Some code refactoring to eliminate hack.
596 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
598 * create-mscompat-collation-table.cs : Fixed some secondary weight
599 in Devanagari and arrows.
601 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
603 * create-mscompat-collation-table.cs : a set of tiny mapping fixes.
605 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
607 * create-mscompat-collation-table.cs : some diacritical fixes for
608 Latin. Added batch mapping method that considers computed
609 diacritical weight (for numbers).
611 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
613 * managed-collation.patch : forgot to add System.String patch.
615 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
617 * MSCompatUnicodeTable.cs : added resource existence check (required
618 for mscorlib transient time from the one without resources to the
621 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
623 * create-mscompat-collation-table.cs : fixed punctuations and hyphen
624 (shift) primary weight.
626 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
628 * create-mscompat-collation-table.cs : more nonspacing mark fixes.
629 Some non-basic Cyrillic diacritical weight fixes.
631 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
633 * create-mscompat-collation-table.cs : some Gurmukhi fixes on level 1
634 and level 3. Tiny Hangul weight fixes.
635 * MSCompatUnicodeTable.cs : U+30F5 and U+30F6 are small Japanese.
637 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
639 * create-mscompat-collation-table.cs : some normal characters who have
640 "narrow" NFKD mapping are regarded as "wide" and thus level 3 weight
641 values were different. Handle U+30FB as category A.
642 * MSCompatUnicodeTable.cs : U+30FB does not have special weight.
644 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
646 * create-mscompat-collation-table.cs : more diacritical weight fixes.
647 Removed some unused code.
649 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
651 * create-mscompat-collation-table.cs : Fixed some Thai and Arabic
654 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
656 * create-mscompat-collation-table.cs : Fixed Syriac nonspacing marks.
658 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
660 * create-mscompat-collation-table.cs : Fixed nonspacing marks in
661 Malayalam, Thai and Lao. Removed extraneous hack.
663 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
665 * SimpleCollator.cs : rewrote LastIndexOf() to handle source extenders.
666 Some refactoring on IndexOf() code. Removed unused Matches().
667 * Collation-notes.txt : some methods needed to be reimplemented, so
668 rewrote the description.
670 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
672 * SimpleCollator.cs : rewrote IsSuffix() to use CompareInternal().
673 Thus supported extenders in IsSuffix().
675 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
677 * SimpleCollator.cs : more IsSuffix() simplification, but it will be
678 stopped here since it cannot handle extenders (implementing new
681 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
683 * SimpleCollator.cs : simplified IsSuffix() code.
685 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
687 * SimpleCollator.cs : Fixed IndexOf() and LasIndexOf() to search the
688 entire replacement string if char target was an expansion.
689 IsSuffix() was using a method for IsPrefix() which was incorrect.
690 Removed old IsPrefix() code.
692 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
694 * SimpleCollator.cs : IndexOf() was incorrectly sharing the same
695 byte[] field in different areas of code. Now extenders in both
696 source and target really work in IndexOf().
698 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
700 * create-mscompat-collation-table.cs : fixed U+FF9F diacritical weight.
701 * SimpleCollator.cs : handle U+FF9E and U+FF9F as extenders.
703 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
705 * SimpleCollator.cs : Now FilterExtender() handles all extender
706 support. IndexOf() and LastIndexOf() now supports extenders.
707 IndexOf() and LastIndexOf() did not proceed contraction source
708 length as expected. Tiny refactoring on private IsPrefix() to take
711 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
713 * SimpleCollator.cs : when restoring from expansion, go back to the
714 top of the loop (to avoid index out of range).
715 Now IsPrefix() is implemented to reuse Compare() and thus it now
716 supports extender as well.
717 * Collation-notes.txt : status update. Deleted optimization part in
718 status section (it is duplicate).
720 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
722 * SimpleCollator.cs : some code reordering.
723 * create-mscompat-collation-table.cs : it was still missing U+3094.
725 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
727 * SimpleCollator.cs : Compare() now supports extender (e.g. U+39FC).
729 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
731 * SimpleCollator.cs : In GetSortKey(), don't update previousChar when
732 it is not primary (e.g. don't "extend" diacritical mark).
734 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
736 * managed-collation.patch : CompareInfo.Compare() should consider
737 the possibilities that non-empty string might be actually empty
738 in culture-sensitive context.
740 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
742 * SimpleCollator.cs : IndexOf() and LastIndexOf() returns start when
743 target is "empty" (in culture-sensitive context).
745 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
747 * SimpleCollator.cs : In IndexOf() and LastIndexOf(), skip ignorable
748 characters in target string.
750 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
752 * SimpleCollator.cs : When IgnoreWidth is specified, all Kana
753 characters are regarded as half-width.
754 Even though IgnoreWidth is specified, it should not ignore case.
755 For special weight comparison, the default values (E4) are bigger
756 than non-default values.
757 * SortKeyBuffer.cs : It should save LCID and original string.
758 * create-mscompat-collation-table.cs : For Japanese half-width kana,
759 it should not be counted in widthCompat map since IgnoreWidth does
760 not really ignore those differences.
762 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
764 * create-mscompat-collation-table.cs : Fixed missing Japanese bits.
766 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
768 * create-mscompat-collation-table.cs :
769 tiny diacritical weight fix for U+20D0-U+20E1.
771 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
773 * create-mscompat-collation-table.cs : ja CJK ideograph got completed.
775 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
777 * create-mscompat-collation-table.cs : Fixed CJK custom Japanese
778 mapping. It (maybe as well as other CJK tables) mixes NFKD. For
779 Japanese, modified NFKD table (because of Windows lame design).
781 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
783 * Makefile : added MONO_USE_MANAGED_COLLATION=no almost everywhere.
784 * MSCompatUnicodeTable.cs : FillCJK() was not invoked. Now it is
785 invoked at any time it is required.
786 * SimpleCollator.cs : call FillCJK() above in .ctor().
787 * MSCompatUnicodeTableUtil.cs : CJK range was wider.
788 * create-mscompat-collation-table.cs : CJK binary was missing the
789 length. CJK remapping is being moved to ModifyUnidata().
790 For cjk-ja mapping, we have to consider compat characters to be
791 added to the map, besides the raw UCA table.
793 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
795 * SortKeyBuffer.cs : Fixed shift level computation to match w/ Windows.
797 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
799 * SimpleCollator.cs : fixed LastIndexOf() to handle _target's_
800 contraction as expected. Fixed Compare() to save s2's contraction
802 * TestDriver.cs :added LastIndexOf() tester w/ indexes.
804 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
806 * managed-collation.patch : Fixed IsPrefix() and IsSuffix(). They
807 incorrectly use Compare().
808 * TestDriver.cs : more moved to nunit tests.
810 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
812 * SimpleCollator.cs : several fixes on Compare().
813 - Ignorable characters are skippted at the top of the loop.
814 - IgnoreNonSpace is checked to avoid extraneous level 2 comparison.
815 - In such case that s1 index is increased while s2 contraction is
816 replaced, s1 is inconsistently proceeded (bug).
817 - IsIgnorable() now also checks IgnoreNonSpace.
818 - Fixed FilterOptions() that does not work for IgnoreWidth at all.
819 * TestDriver.cs : now some are moved to nunit tests.
820 * Collation-notes.txt : minor todo update.
822 2005-07-11 Atsushi Enomoto <atsushi@ximian.com>
824 * SimpleCollator.cs : Compare() was ignoring such case that both
825 entire strings have '-' to be compared.
826 * Collation-notes.txt : more status updates.
827 * TestDriver.cs : added '-' use cases.
829 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
831 * SimpleCollator.cs : to be same as other buggy part, it now handles
832 U+3005, U+3031 and U+3032 as buggy as Windows. It just repeats
834 Fixed GetSortKey(): if the repeater is U+3005, second weight is 5.
835 * create-mscompat-collation-table.cs : dummy values for extenders.
837 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
839 * SimpleCollator.cs : Special weight fixes on GetSortKey(). Dash type
840 should be computed from ExtenderType, and voice mark weight should
842 * MSCompatUnicodeTable.cs : added tiny comment.
844 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
846 * SortKey.cs : It borked when MONO_USE_MANAGED_COLLATION is not yes.
847 * SimpleCollator.cs : support for extender (U+309D etc.).
849 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
851 * create-mscompat-collation-table.cs : some punct/symbols fix.
852 * managed-collation.patch : new (and temporary) file to support
853 managed collation in mscorlib.
854 * README : described how to use managed collation.
856 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
858 * create-mscompat-collation-table.cs : Further Cyrillic fixes. Handle
859 U+482-4C8 (though needs diacritical fixes).
860 * MSCompatUnicodeTable.cs : tiny comment for alternative impl.
862 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
864 * create-mscompat-collation-table.cs : Reimplemented Cyrillic weight
865 computation code, since it looks like the same way as Latin letters
866 have. Thus removed all other approach (UCA, by letter name).
868 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
870 * create-mscompat-collation-table.cs : diacritical fix for "double-
871 struck". Syriac nonspacing fixes.
873 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
875 * create-mscompat-collation-table.cs : more math symbol weight fixes.
877 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
879 * create-mscompat-collation-table.cs : fixed Hebrew character sortkeys.
881 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
883 * create-mscompat-collation-table.cs : math symbols U+25A0-U+2600 are
884 implemented (no stub). Some other fixes on category 8-A.
886 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
888 * create-mscompat-collation-table.cs : some minor fixes on Arabic,
889 Korean and Japanese sortkey weights.
891 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
893 * create-mscompat-collation-table.cs : More diacritical fixes.
894 Georgian characters do not have level 2 weights but level 3.
896 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
898 * create-mscompat-collation-table.cs : Roman numeral characters
899 have diacritical weight. quick hack for control signs (U+2400..)
902 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
904 * create-mscompat-collation-table.cs : improving Latin mappings.
905 Setting non-ASCII Latin characters' primary weight between those
906 ASCII characters, and setting diacritical weight (hacky).
907 * MSCompatUnicodeTable.cs :
908 Kanatype check: fixed (voice marks) and improved (comparison order).
910 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
912 * create-mscompat-collation-table.cs : more diacritical fixes.
913 primary weight fixes on punctuations in category 07.
915 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
917 * create-mscompat-collation-table.cs : several diacritical fixes.
918 * TestDriver.cs : sortkey dumper should use StringSort.
920 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
922 * SimpleCollator.cs : fixed incorrect indexer setup. Optimized
923 GetContraction() call a bit.
925 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
927 * create-mscompat-collation-table.cs : fixed incorrect level 2
929 * MSCompatUnicodeTable.cs : remove debug line.
931 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
933 * MSCompatUnicodeTableUtil.cs,
934 MSCompatUnicodeTable.cs,
936 create-mscompat-collation-table.cs : made some members internal and
937 accessible from other classes. Many indexes could be 0 by default.
938 * SimpleCollator.cs : optimizations. avoid method call.
940 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
942 * Collation-notes.txt : more updates.
943 * SimpleCollator.cs : Added quick check for Ordinal comparison.
944 Fixed special weight comparison. It cannot be customizable in the
945 implementation (and it won't be harmful).
946 * mono-tailoring-source.txt : thus updated comment.
948 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
950 * SimpleCollator.cs : Compare() was missing French sort support.
951 * TestDriver.cs : added example case.
953 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
955 * Collation-notes.txt : updated status. Eliminated descriptions on
956 "iterator" (I avoided it for performance concern). Fixed misc.
957 incorrect descriptions.
959 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
961 * Collator.cs : Now that SimpleCollator became feature complete, it is
964 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
966 * SimpleCollator.cs : implemented decent Compare() that immediately
967 stops at first primary difference.
969 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
971 * SimpleCollator.cs : indexers might return -1.
973 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
975 * SimpleCollator.cs : IsPrefix() and IsSuffix() optimization code was
976 buggy (length check for source was missing).
978 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
980 * create-mscompat-collation-table.cs : Fixed tailoring table output
981 to be in correct and countable order. Now if tailoring alias was not
982 found, just stop the build.
983 * MSCompatUnicodeTable.cs : several build fixes. Now it works to read
985 * mono-tailoring-source.txt : commented out CJK aliases that miss
987 * Makefile : needed further filename fixes.
989 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
991 * MSCompatUnicodeTable.cs : renamed from MSCompatUnicodeTable.template
992 (now it is working as a standalone file).
993 * Makefile : renamed generated file as MSCompatUnicodeTableGenerated.cs
994 (the generator now creates both binary resources and C# source).
996 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
998 * create-mscompat-collation-table.cs : Now it generates binary
999 resources (to parent directory).
1000 * MSCompatUnicodeTable.template : added conditional code that fills
1001 collation tables from manifest resources.
1002 * Makefile : remove collation table binaries as well on "make clean".
1003 Removed extraneous dependency.
1005 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1007 * MSCompatUnicodeTable.template,
1008 SimpleCollator.cs : removed extraneous GetExpansion().
1010 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1012 * SimpleCollator.cs : IsSuffix() also supports contractions.
1013 * TestDriver.cs : IsSuffix() example contraction cases.
1015 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1017 * SimpleCollator.cs : reverted IsSuffix() to return bool (to match w/
1018 what current IsPrefix() does). For expansion of target, IsPrefix()
1019 should check the no-match case that expansion is longer than input.
1020 Some refactory on IsPrefix().
1021 Added GetContractionTal() for IsSuffix() (not used yet).
1023 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1025 * TestDriver.cs : added IsPrefix() expansion cases.
1026 * SimpleCollator.cs : IsPrefix() now supports contractions (with much
1027 of complexity), and it now returns bool again.
1028 IndexOf() for replacement should make use of IndexOfPrimitiveChar()
1029 since expansions won't be expanded recursively.
1031 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
1033 * SimpleCollator.cs : commonized character comparison in IsPrefix()
1034 and IsSuffix(). csc compile fix.
1035 * CompareInfoImpl.cs : deleted.
1037 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
1039 * TestDriver.cs : added SimpleCollator.ctor() sanity check.
1040 Added replacement contraction example.
1041 * SimpleCollator.cs : Now IndexOf() and LastIndexOf() support
1042 contraction in source string. Extracted matching code to Matches().
1043 Replacement contraction was including extraneous '\x0'.
1045 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
1047 * Collation-notes.txt : updated status.
1048 * CollationDataStructures.txt : tiny fixes.
1049 * SimpleCollator.cs :
1050 Renamed alias Util to UUtil (MS sys.enterprisesvc has sucky global
1051 namespace Util and csc borked).
1052 GetContraction was incorrectly returning first item.
1053 Private IsPrefix() now returns int (but it might not be in real use).
1054 Extracted simple char comparison to CompareCharSimple().
1055 IndexOf() and LastIndexOf() now fully handle contractions (both
1056 binary key and string replacement) in "target" (for "s" not yet).
1057 * TestDriver.cs : be more verbose.
1058 * mono-tailoring-source.txt : added comment.
1059 * MSCompatUnicodeTable.template :
1060 Renamed alias Util to UUtil (MS sys.enterprisesvc has sucky global
1062 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
1064 * create-mscompat-collation-table.cs : compute COMBINING blah marks as
1065 well as those characters WITH blah.
1066 * TestDriver.cs : added combining sortkey cases.
1068 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
1070 * mono-tailoring-source.txt : fixed description on '*' in sortkeys.
1071 * SimpleCollator.cs : Now it fully uses tailoring info. Fixed
1072 contraction search that worked only when string is contraction.
1073 Removed commented code. Minor refactoring.
1074 * TestDriver.cs : added example that uses "ZS" in Hungarian sorting.
1076 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1078 * create-mscompat-collation-table.cs,
1079 * mono-tailoring-source.txt : removed extraneous level 4 sortkey
1080 which cannot be supported.
1081 * SimpleCollator.cs : added GetContraction() and used in some places.
1082 Now CompareOptions is set only once. Reordered some code (e.g.
1083 ignorable check -> get compat char -> compare).
1085 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1087 * SimpleCollator.cs : sort tailoring tables before actual usage.
1088 Support diacritical remappings (it is customized collation rule
1089 which does not exist in UCA).
1091 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1093 * SimpleCollator.cs : build culture specific tailoring table from
1094 TailoringInfo and unified data array.
1095 * create-mscompat-collation-table.cs : Added null termination to
1096 sortkey map tailorings (mostly to save my eyes).
1097 * MSCompatUnicodeTable.template : added public TailoringValues.
1099 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1101 * SortKeyBuffer.cs : handle special weight (category 06) characters.
1102 * Collation-notes.txt : Updated description on special weight (it was
1104 * TestDriver.cs : added special weight cases.
1106 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1108 * MSCompatUnicodeTable.template : added GetTailoringInfo().
1109 * SimpleCollator.cs : Now tailoring information is acquired and used.
1110 (FrenchSort is supported but Compare() won't work expectedly since
1111 the table is still incomplete for those diacritical marks).
1112 * SortKeyBuffer.cs : On reversing diacritical weights, it should
1113 ignore zeros. Reset() should reset frenchSorted flag.
1115 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1117 * create-mscompat-collation-table.cs : Further fixes on Jamo,
1118 diacritical weights by character name, and *Numbers primary weights.
1120 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1122 * create-mscompat-collation-table.cs : More fix on Devanagari,
1123 Gujarati, Oliya, Tamil and Lao sortkeys.
1125 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1127 * create-mscompat-collation-table.cs : Fixed Georgian, Thai, Gurmukhi
1130 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1132 * create-mscompat-collation-table.cs : Fixed Thai character primary
1133 and secondary values. Fixed Thaana letters. Added more LAMESPEC
1134 CJK compat. Fixed some circled CJK secondary weight.
1135 Hacked some nonspacing mark sortkey value adjustment.
1137 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1139 * create-mscompat-collation-table.cs : CP932.TXT was not parsed as
1140 expected. JIS ordering was incorrect. OtherNumbers that represents
1141 10 or more values were incorrectly computed the offset. Some Hangul
1142 compat characters has different offset.
1144 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1146 * create-mscompat-collation-table.cs : Fixed 0x8 category characters.
1147 Added hack for need-to-be-fixed characters to fall into 0xA category.
1148 * create-collation-element-table.cs : previous checkin seem failed :(
1149 * README: updated a bit.
1151 2005-06-24 Atsushi Enomoto <atsushi@ximian.com>
1153 * CodePointIndexer.cs :
1154 removed extraneous switch (I could use empty array for that need).
1155 * CollationElementTableUtil.cs : primary weight type became ushort.
1156 * create-collation-element-table.cs : several bugfixes.
1157 collElem should be int. It was skipping most of entries because of
1158 incorrect string tokenization.
1160 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1162 * create-mscompat-collation-table.cs : handle some Jamo NKFD.
1164 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1166 * SimpleCollator.cs : forgot to commit in the last checkin.
1167 * create-mscompat-collation-table.cs : fixed arabic shift weight chars.
1168 * TestDriver.cs : switch table dumper and collator testing.
1169 * SortKey.cs : for now comment out internal indexes (not in use).
1171 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1173 * MSCompatUnicodeTable.template,
1174 SimpleCollator.cs : support for culture dependent CJK table.
1176 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1178 * create-mscompat-collation-table.cs,
1179 MSCompatUnicodeTableUtil.cs : make CJK table more compact.
1181 2005-06-22 Atsushi Enomoto <atsushi@ximian.com>
1183 * SimpleCollator.cs : Fixed stupid index search when start != 0.
1185 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1187 * SimpleCollator.cs : fixed my misunderstanding on LastIndexOf(). It
1188 now starts from "start" and proceeds backward by "length".
1189 * TestDriver.cs : fix warning.
1191 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1193 * TestDriver.cs : more tests.
1194 * SimpleCollator.cs : LastIndexOf() is not setting search length
1195 on iteration. Quick workaround fro String.LastIndexOf() bug (maybe).
1197 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1199 * create-normalization-source.cs : output propValue as uint.
1201 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1203 * SortKey.cs : Now it is System.Globalization.SortKey.
1204 To replace existing implementation, it now requires lcid and
1205 CompareOptions. Added required members.
1206 * SortKeyBuffer.cs : thus .ctor() requires LCID.
1207 * SimpleCollator.cs : made required changes above.
1209 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1211 * CodePointIndexer.cs : added CompressArray(). Now it requires two more
1212 parameters for default index and codepoint.
1213 * CollationElementTableUtil.cs,
1214 NormalizationTableUtil.cs : required changes wrt above change.
1215 * MSCompatUnicodeTableUtil.cs : added for several codepoint indexers.
1216 * MSCompatUnicodeTable.template : Now it uses codepoint indexer.
1217 * create-mscompat-collation-table.cs : Now it outputs compressed array.
1218 * Makefile : now collation requires MSCompatUnicodeTableUtil.cs
1220 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1222 * SimpleCollator.cs :
1223 Implemented IsSuffix() and LastIndexOf().
1224 Several fixes on index > 0 cases.
1225 * TestDriver.cs : sample IsSuffix() and LastIndexOf() usage and more.
1227 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1229 * Collation-notes.txt : updated (status, impl. classes).
1230 * MSCompatUnicodeTable.cs : Korean Jamo are not really expansions.
1232 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1234 * SimpleCollator.cs : implemented IndexOf(string,string,CompareOptions)
1235 and IsPrefix(). Tiny code refactory.
1236 * TestDriver.cs : sample IsPrefix() and IndexOf() usage.
1237 * MSCompatUnicodeTable.cs : tiny refactory for CodePointIndexer use.
1239 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1241 * SimpleCollator.cs :
1242 IndexOf(string, char, CompareOptions) implementation.
1243 * TestDriver.cs : sample IndexOf() usage.
1245 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1247 * create-mscompat-collation-table.cs : was missing most important
1248 kind of blocks - equivalent expansions (e.g. invariant mappings).
1249 More readable mappings.
1251 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1253 * mono-tailoring-source.txt : new file. It describes tailoring
1254 information. Basically examined under .NET 1.x.
1255 * create-mscompat-collation-table.cs : consume the file above.
1256 * MSCompatUnicodeTable.template : now tailorings is not a stub.
1257 * CollationDataStructures.txt : minor fixes.
1259 SimpleCollator.cs : added FrenchSort support.
1260 * Collation-notes.txt : added description on Latin primary weights.
1261 * ldml-limited.rng : added note.
1262 * create-tailorings.cs : added note. more serialization (but won't be
1265 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1267 * SortKeyBuffer.cs : non-primary character is added to previous
1269 * TestDriver.cs : added example case of above.
1271 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1273 * SimpleCollator.cs : IgnoreSymbols support.
1274 * TestDriver.cs : compilation fix. IgnoreSymbols example.
1275 * create-mscompat-collation-table.cs : more Hangul fixes.
1277 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1279 * create-mscompat-collation-table.cs : more Hangul fixes.
1280 * SortKey.cs : it will replace sys.globalization.SortKey. It has
1281 some internal members.
1282 * SortKeyBuffer.cs : now it uses SortKey instead of byte[].
1283 * SimpleCollator.cs : CompareOptions support. However I don't think
1284 it will be developed anymore since SortKey never enables IndexOf().
1285 * TestDriver.cs : a few CompareOptions cases.
1287 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1289 * SimpleCollator.cs : simple collator implementation that just will
1290 use GetSortKey() for all its basis.
1291 * TestDriver.cs : sample code that uses this collator set.
1292 * MSCompatUnicodeTable.template : removed test driver from here.
1294 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1296 * create-mscompat-collation-table.cs : Hangul fixes.
1297 Now less than 300 characters that does not have sortkey weights.
1298 * MSCompatUnicodeTable.template : added FIXME info for Hangul Jamo.
1300 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1302 * create-mscompat-collation-table.cs : Added control picture mappings.
1303 Minor primary weight fixes.
1305 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1307 * create-mscompat-collation-table.cs : Added mappings for box
1308 drawings and blocks.
1310 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1312 * create-mscompat-collation-table.cs : Added mappings for arrows.
1314 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1316 * create-mscompat-collation-table.cs : added support for letterlike
1317 characters and squared CJK compatibility characters, ordered by
1318 character names (0x0E category).
1319 * Collation-notes.txt : added description on that.
1321 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1323 * MSCompatUnicodeTable.template : Now expansions are simulated.
1324 * create-mscompat-collation-table.cs : filled Korean number level2.
1325 Reordered some code blocks to fill correct diacritical differences.
1326 * Collation-notes.txt : some corrections and minor additions.
1328 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1330 * MSCompatUnicodeTable.template :
1331 Now dumper test driver uses SortKeyBuffer for dogfooding.
1332 * create-mscompat-collation-table.cs : some diacritical level fixes
1333 (with non-working extra latin check).
1334 * SortKeyBuffer.cs : several fixes to get working as a practical code.
1335 * Collator.cs : make it compilable, leaving things as NotImplemented.
1337 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1339 * create-mscompat-collation-table.cs : some fixes on primary category
1340 07 (miscellaneous symbols and punctuations).
1342 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1344 * create-mscompat-collation-table.cs : more mapping fix on numbers,
1345 letters, variable weight characters, circled Japanese and CJK.
1346 * MSCompatUnicodeTable.template : fixed HasSpecialWeight() to be more
1347 inclusive. Simplified dumper code.
1349 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1351 * create-mscompat-collation-table.cs : finished Hangul (both Jamo
1352 and Syllables). sortkey dumper diff lines became 8000 from 30000.
1354 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1356 * create-mscompat-collation-table.cs : added some nonspacing marks in
1357 either correct or hacky way.
1359 2005-06-13 Atsushi Enomoto <atsushi@ximian.com>
1361 * create-mscompat-collation-table.cs : several improvements. Japanese
1362 Kana support, Hebrew accents, Bengali nonspacing marks, sorting of
1363 numeric characters, diacritically decorated latin alphabets. Fixed
1364 some diacritical weights detection.
1365 * MSCompatUnicodeTable.cs : tiny Japanese fix. Handle nonspacing
1366 marks' primary weight as empty.
1367 * Collation-notes.txt : some updates.
1369 2005-06-13 Atsushi Enomoto <atsushi@ximian.com>
1371 * create-mscompat-collation-table.cs : don't process nonexact NFKD
1372 mapping as equivalent, however store CJK extensions into NFKD map
1373 even if one does not strictly match.
1374 Now am going to fill Hangul into tables (unlike UCA it does not look
1375 possible to calculate sortkey value).
1376 Fixed Cyrillic and Georgian UCA based orderings.
1377 * MSCompatUnicodeTable.template : added CJK extension sortkey
1380 2005-06-10 Atsushi Enomoto <atsushi@ximian.com>
1382 * create-mscompat-collation-table.cs : Fixed latin alphabet support.
1383 Added latin with diacritical and CJK extension.
1384 * MSCompatUnicodeTable.cs : modified dumper code a bit (for my purpose).
1386 2005-06-10 Atsushi Enomoto <atsushi@ximian.com>
1388 * create-mscompat-collation-table.cs : now parses DerivedAge.txt (right
1389 now not used thouth). Filled CJK ideograph, still not perfect.
1390 Fixed number primary keys. NFKD numbers and CJK ideographs are now
1391 considered, including brackets elimination.
1392 * Makefile : now it downloads DerivedAge.txt.
1393 * MSCompatUnicodeTable.template : added dummy code dumper. It computes
1394 PrivateUse, Surrogate and Hangul Syllables.
1395 * Collation-notes.txt : Noted that Hangul Syllables need more love.
1397 2005-06-09 Atsushi Enomoto <atsushi@ximian.com>
1399 * create-tailorings.cs : added configuration support. sort them.
1400 I wonder if it is really usable. Having own format might be better.
1401 * create-mscompat-collation-table.cs : fixing some sortkey numbers,
1402 making closer to windows. Now it handles NFKD in some places.
1403 * MSCompatUnicodeTable.template : Added dummy sortkey dumper driver.
1404 * CollationDataStructures.txt : added description on tailoring
1405 fields, though they are subject to change.
1407 2005-06-07 Atsushi Enomoto <atsushi@ximian.com>
1409 * create-tailorings.cs, ldml-limited.rng : new file.
1410 * LdmlReader.cs : removed old file.
1412 2005-06-07 Atsushi Enomoto <atsushi@ximian.com>
1414 * SortKeyBuffer.cs : split from Collator.cs. Now it considers
1415 practical use, reflecting updated sortkey constant design.
1416 Especially level 4 weight is split to 4 arrays that are merged in
1417 the last stage of GetSortKey().
1418 * Collator.cs : thus SortKeyBuffer is removed from here.
1419 Additionally, removed some extraneous bits in other classes.
1420 * Collation-notes.txt : Some editorial fixes. Added information on
1421 Korean matter (how to compute Hangle Syllables / Hangul Jamo cannot
1422 be stored in simple byte arrays).
1423 * CodePointIndexer.cs,
1424 create-collation-element-table.cs,
1425 CollationElementTable.template,
1426 NormalizationTableUtil.cs : short CodePointIndexer method names.
1427 * create-mscompat-collation-table.cs : Additional info on why some
1428 meaningful characters are ignored in Windows (Unicode version
1429 difference). Removed U+070F from special check (was extraneous).
1431 2005-06-06 Atsushi Enomoto <atsushi@ximian.com>
1433 * MSCompatUnicodeTable.template:
1434 Moved body implementation to table creator and put those bool
1435 results into an array.
1436 * create-mscompat-collation-table.cs :
1437 So imported those methods. Modified array output to emit "0x"
1438 only for more than 9.
1439 * create-normalization-source.cs : ditto on "0x" output matter.
1440 * CollationDataStructures.txt : so now it holds ignorableFlags.
1442 2005-06-03 Atsushi Enomoto <atsushi@ximian.com>
1444 * Collation-notes.txt, CollationDataStructures.txt :
1445 separate document for data structure design.
1447 2005-06-03 Atsushi Enomoto <atsushi@ximian.com>
1449 * create-mscompat-collation-table.cs : added culture-dependent CJK
1450 table creation. It uses CLDR as its basis. (Culture independent CJK
1452 * Makefile : added CLDR archive downloading support.
1453 * MSCompatUnicodeTable.template : tiny renamings.
1454 * Collation-notes.txt : additional CJK info.
1456 2005-06-02 Atsushi Enomoto <atsushi@ximian.com>
1458 * Collation-notes.txt, create-mscompat-collation-table.cs :
1459 added secondary weight support for BlahNumber characters.
1461 2005-06-01 Atsushi Enomoto <atsushi@ximian.com>
1463 * downloaded : added directory. All downloaded files are stored here.
1464 * Makefile : use "downloaded" directory.
1465 Added more auto-download stuff.
1466 * create-mscompat-collation-table.cs :
1467 Added Japanese square kana support.
1469 2005-06-01 Atsushi Enomoto <atsushi@ximian.com>
1471 * Collation-notes.txt : added Estrangela (ancient Syriac) and Thaana.
1472 * create-mscompat-collation-table.cs : added support for Arabic abjad,
1473 Estrangela and Thaana.
1474 * MSCompatUnicodeTable.template : removed BOM.
1476 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1478 * Collation-notes.txt : wrong comment cleanup and spelling fixes.
1479 * create-mscompat-collation-table.cs : added diacritic support for
1480 Latin letters (as long as covered in primary weight).
1482 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1484 * Makefile : minor fixes. Added warning lines to generated sources.
1486 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1488 * create-char-mapping-source.cs :
1489 Removed ToWidthInsensitive() generation.
1491 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1493 * create-mscompat-collation-table.cs : Now it dumps level1 to 3 values.
1494 ToWidthInsensitive() is implemented here, using an array (which is
1495 to be optimized using CodePointIndexer).
1496 * MSCompatUnicodeTable.cs : renamed as MSCompatUnicodeTable.template
1497 * MSCompatUnicodeTable.template : now it is used to generate
1498 MSCompatUnicodeTable.cs which got ready to be used.
1499 * Makefile : added MSCompatUnicodeTable.cs build support. Now it
1500 supports "make normalization" and "make collation".
1502 2005-05-30 Atsushi Enomoto <atsushi@ximian.com>
1504 * Collation-notes.txt : Description on ICU is very incorrect. Now it
1505 became more rational and sane.
1506 * create-mscompat-collation-table.cs : fixed some indexes.
1507 * Makefile : added "mstablegen" target.
1508 * MSCompatUnicodeTable.cs : removed GetPrimaryWeight(). Minor fix.
1510 2005-05-26 Atsushi Enomoto <atsushi@ximian.com>
1512 * Collation-notes.txt : more analysis on "letters".
1513 * create-mscompat-collation-table.cs : more proof of concepts.
1515 2005-05-25 Atsushi Enomoto <atsushi@ximian.com>
1517 * Collation-notes.txt : more info. Started letter sortkey analysis
1518 (some of other stuff are really non-understandable right now.)
1519 * create-mscompat-collation-table.cs : table generator proof-of-
1520 concept source (not compilable).
1521 * MSCompatUnicodeTable.cs : moved some code to the new source.
1524 2005-05-20 Atsushi Enomoto <atsushi@ximian.com>
1526 * Collation-notes.txt : started level 2 weight analysis.
1528 2005-05-19 Atsushi Enomoto <atsushi@ximian.com>
1530 * Collation-notes.txt : Additional information on how to create
1532 * MSCompatUnicodeTable.cs : implemented part of GetLevel3Weight().
1534 2005-05-19 Atsushi Enomoto <atsushi@ximian.com>
1536 * Collation-notes.txt : More case weight (level 3) analysis. I'm
1537 likely to just write table generator.
1539 2005-05-18 Atsushi Enomoto <atsushi@ximian.com>
1541 * MSCompatUnicodeTable.cs : part of level 4 weight implementation.
1543 2005-05-18 Atsushi Enomoto <atsushi@ximian.com>
1545 * Collation-notes.txt :
1547 Revised comparison methods; backward iteration is possible.
1548 More on char-by-char comparison.
1549 Level 4 comparison is actually a bit more complex.
1551 * Collator.cs : some conceptual updates wrt above.
1553 2005-05-17 Atsushi Enomoto <atsushi@ximian.com>
1555 * Collation-notes.txt : Japanese voice mark is level 2, and Hangul
1556 properties are level 3.
1558 2005-05-17 Atsushi Enomoto <atsushi@ximian.com>
1560 * Collation-notes.txt : Make it more readable. More analysis on
1561 level 3 and 4 sortkey structures.
1562 * Collator.cs : some compilation fixes (not compilable yet).
1564 2005-05-16 Atsushi Enomoto <atsushi@ximian.com>
1566 * Collation-notes.txt : Analysis on variable-weighting (level 5)
1568 * Collator.cs : updated corresponding part of level 5, and more.
1570 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1572 * Collation-notes.txt : more updates.
1573 * Collator.cs : rewrote from scratch. Some rough sketch for sortkey
1574 buffer, character iterator and collator methods. Not compiling.
1576 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1578 * Collator.cs : Am going to replace it with new one. No need for
1579 CompareOptions-dependent Comparer.
1581 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1583 * Collation-notes.txt : There seems a bit more complexity.
1585 2005-05-10 Atsushi Enomoto <atsushi@ximian.com>
1587 * Collation-notes.txt : more updates, being close to write sortkey
1590 2005-05-09 Atsushi Enomoto <atsushi@ximian.com>
1592 * CompareInfoImpl.cs, Collator.cs : conceptual update
1593 * Collation-notes.txt : some corrections and additions.
1594 * Makefile : added LDML input (but it won't be used at all).
1596 2005-04-28 Atsushi Enomoto <atsushi@ximian.com>
1598 * Collation-notes.txt : more updates.
1600 2005-04-26 Atsushi Enomoto <atsushi@ximian.com>
1602 * Collation-notes.txt : more updates.
1604 2005-04-26 Atsushi Enomoto <atsushi@ximian.com>
1606 * Collation-notes.txt : some updates.
1607 * create-mapping-char-source.cs : superscripts and subscripts are also
1608 ignored in IgnoreWidth comparison.
1609 * Makefile : tiny touch fix.
1611 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1613 * CompareInfoImpl.cs, Collator.cs : conceptual stuff (not working).
1615 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1617 * create-char-mapping-source.cs : Now it generates
1618 ToWidthInsensitive() from combining category <wide> and <narrow>.
1619 * MSCompatUnicodeTable.cs : added ToKanaTypeInsensitive() and
1620 ToWidthInsensitive() for IgnoreKanaType and IgnoreWidth.
1622 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1624 * README, LdmlReader.cs, DataStructures.txt : new files.
1626 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1628 * CodePointIndexer.cs,
1629 Collation-notes.txt,
1630 CollationElementTable.template,
1631 CollationElementTableUtil.cs,
1632 create-char-mapping-source.cs,
1633 create-collation-element-table.cs,
1634 create-combining-class-source.cs,
1635 create-normalization-source.cs,
1637 MSCompatUnicodeTable.cs,
1638 Normalization.template,
1639 NormalizationTableUtil.cs : initial checkin (to private branch).