1 2006-05-30 Miguel de Icaza <miguel@novell.com>
3 * MSCompatUnicodeTable.cs: Remove the fixed loading and compute it
4 just when we actually consume it. This only fixes the
7 2006-04-14 Atsushi Enomoto <atsushi@ximian.com>
9 * README: removed obsolete info.
10 * Normalization.cs : canonical reordering should participate in the
11 decomposition step. In reordering, string append was incomplete.
12 Combining class check is required in NFD check. Icall is written
15 2005-12-07 Zoltan Varga <vargaz@gmail.com>
17 * SimpleCollator.cs: Fix a warning.
19 2005-11-30 Sebastien Pouliot <sebastien@ximian.com>
21 * SimpleCollator.cs: Fix CAS support. The static ctor/var try to get
22 the environment variable MUCH too soon (i.e. the security manager
25 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
27 * SimpleCollator.cs : direct fast-path optimization for IndexOf().
29 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
32 - CompareQuick(): added immediateBreakup to avoid extraneous sortkey
34 - QuickCheckPossible(): index used for s1 was incorrect.
36 2005-11-29 Atsushi Enomoto <atsushi@ximian.com>
38 * SimpleCollator.cs : added another quick check for CompareInternal()
39 that does almost ordinal comparison for quick-checkable strings.
40 (It affects on Compare(), IndexOf(), IsSuffix() etc. as well.)
42 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
44 * MSCompatUnicodeTable.cs : (IsIgnorable) \0 is not ignorable.
47 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
50 Created another struct to reduce method arguments. Created another
51 flags that keeps "once-matched" state (counterpart of
52 checkedFlags, now neverMatchFlags).
54 2005-11-14 Atsushi Enomoto <atsushi@ximian.com>
57 - Added CompareOrdinalIgnoreCase() for NET_2_0 RTM.
58 - Reduced extra parameter from LastIndexOfSortKey().
59 - LastIndexOf() should use GetTailContraction for the source string.
60 And then, target could match in the middle of the possible
61 "replacement contraction" of the source string, so use
62 LastIndexOfSortKey() to catch them.
63 - Fixed GetTailContraction() that caused index out of range.
65 2005-11-11 Atsushi Enomoto <atsushi@ximian.com>
67 * Makefile : Now use MONO_DISABLE_MANAGED_COLLATION.
68 * SortKey.cs : some members are virtual.
70 2005-10-14 Atsushi Enomoto <atsushi@ximian.com>
72 * SimpleCollator.cs : modified to use stackalloc for byte array.
74 2005-09-27 Atsushi Enomoto <atsushi@ximian.com>
76 * SimpleCollator.cs : in CompareInternal(), there was a possibility of
77 infinite loop. Fixed bug #76243.
79 2005-09-20 Atsushi Enomoto <atsushi@ximian.com>
81 * SimpleCollator.cs : In IsPrefix/IsSuffix, if target is an empty string,
82 immediately return true.
84 2005-09-09 Atsushi Enomoto <atsushi@ximian.com>
86 * SimpleCollator.cs : IsSuffix() optimization logic was buggy, so just
87 use pretty simple way with LastIndexOf() (no significant perf.
90 2005-09-01 Atsushi Enomoto <atsushi@ximian.com>
92 * README, Collation-notes.txt, CollationDataStructures.txt :
93 removing obsolete info and some added some notes.
95 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
97 * Normalization.cs : remove warned code.
98 * managed-collation.patch : now it's not required anymore.
100 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
102 * MSCompatUnicodeTable.cs : added IsSortable(string).
104 2005-08-10 Atsushi Enomoto <atsushi@ximian.com>
106 * SimpleCollator.cs : Now all collator methods are thread safe.
108 All instance non-readonly fields turned into arguments of every
109 methods that use those fields.
110 (Sadly it is the end of no-memory-cost collator era. mcs bootstrap
111 now needs +100KB memory consumption.)
113 2005-08-09 Atsushi Enomoto <atsushi@ximian.com>
115 * SimpleCollator.cs : made "checkedFlags" as nullable and made it as
116 an argument of every index methods (to make it thread safe).
118 2005-08-09 Atsushi Enomoto <atsushi@ximian.com>
121 MSCompatUnicodeTable.cs :
122 - Now IsIgnorable() is aggregated to be one invokation to check
123 completely ignorable, nonspacing and symbols.
124 - Introduced "already checked" flags for IndexOf() and LastIndexOf()
125 to skip sortkey binary check on the same characters. Significant
126 perf. improvement for such case as IndexOf("AABCBABC...Z",'Z').
128 2005-08-08 Gert Driesen <drieseng@users.sourceforge.net>
130 * SortKey.cs: Marked Serializable to match MS.NET.
132 2005-08-08 Atsushi Enomoto <atsushi@ximian.com>
134 * create-mscompat-collation-table.cs,
135 Makefile : changed resources output directory.
137 2005-08-04 Atsushi Enomoto <atsushi@ximian.com>
139 * create-normalization-tests.cs,
140 StringNormalizationTestSource.cs : new files for Unicode
141 Normalization test generator.
142 * Makefile : added support for above.
144 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
146 * NormalizationTableUtil.cs : oops, it does not compile.
147 * managed-collation.patch : I guess having managed resource would be
148 better for collation. At least current code has such #define so
149 Makefile should be in sync with it.
151 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
153 * create-normalization-source.cs : Fixed CharMapComparer which
154 incorrectly returned 0 when the second arg is shorter. Reduced
155 extraneous helperIndex map. Other minor fixes and code removal.
156 * Normalization.cs : several fixes to support blocked combine handling.
157 * NormalizationTableUtil.cs : tiny member renaming.
159 2005-08-03 Atsushi Enomoto <atsushi@ximian.com>
161 * create-normalization-source.cs,
162 NormalizationTableUtil.cs,
163 Normalization.cs : several bugfixes on index miscomputation.
164 Renamed using aliases (csc will bork). Primary combine safety is now
165 computed during UnicodeData.txt parse.
166 Maximum NFKD length was 18, not 4 (U+FDFA).
168 2005-08-02 Atsushi Enomoto <atsushi@ximian.com>
170 * managed-collation.patch : added Normalization support.
171 * managed-collation-icall.patch : added, including normalization stuff.
173 BTW when will collation code checked in?
175 2005-08-02 Atsushi Enomoto <atsushi@ximian.com>
177 * create-normalization-source.cs : Unified three normalization source
178 generators, to compute IsUnsafe flag. Fixed helperIndex array type
180 * create-char-mapping-source.cs,
181 create-combining-class-source.cs : thus removed.
182 * Makefile : thus modified for the above integration.
183 * NormalizationTableUtil.cs : Extended to contain IsUnsafe flag.
184 * Normalization.cs : Several fixes to make Normalize() actually work.
186 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
188 * create-normalization-source.cs,
190 create-char-mapping-source.cs,
191 create-combining-class-source.cs,
192 Makefile : converted managed array to pointers (like collation stuff).
194 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
196 * NormalizationTableUtil.cs : further table range optimization.
197 * create-normalization-source.cs,
198 create-char-mapping-source.cs,
199 create-combining-class-source.cs : added C header output support.
201 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
203 * create-normalization-source.cs, Normalization.cs :
204 Now property size is < 256, so directly embed value in "props" array.
205 Add QuickCheck(c,checkType) and remove IsNFD/C/KD/KC and delegates.
207 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
209 * create-combining-class-source.cs,
210 create-char-mapping-source.cs,
211 create-normalization-source.cs,
212 NormalizationTableUtil.cs,
213 Normalization.cs : String.Normalize() does not handle surrogate
214 characters. mapping information in DerivedNormalizationProps.txt
215 are not used in the code (those from UnicodeData.txt is used).
216 Hangul syllables are computed instead of embedded in the tables.
217 * managed-collation.patch : removed IntPtrStream and Makefile patches.
219 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
221 * MSCompatUnicodeTable.cs : IsSortable() was broken.
223 2005-07-29 Atsushi Enomoto <atsushi@ximian.com>
225 * MSCompatUnicodeTable.cs : added helper for CompareInfo.IsSortable().
227 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
229 * create-tailoring.cfg : added for convenience of contraction check.
231 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
233 * create-normalization-source.cs,
236 create-mscompat-collation-table.cs,
237 MSCompatUnicodeTableUtil.cs,
239 create-collation-element-table.cs,
240 MSCompatUnicodeTable.cs,
242 create-combining-class-source.cs : added copyright lines.
244 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
246 MSCompatUnicodeTable.cs : removed extraneous definition.
248 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
250 * create-mscompat-collation-table.cs
251 MSCompatUnicodeTable.cs : full C header support, finally.
253 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
256 NormalizationTableUtil.cs,
257 create-char-mapping-source.cs : more aggressive data compression.
258 It now ignores characters that are >= U+10000.
260 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
263 Normalization.template,
264 Normalization.cs : renamed existing file.
266 2005-07-28 Atsushi Enomoto <atsushi@ximian.com>
268 * NormalizationTableUtil.cs,
269 Normalization.template,
270 create-combining-class-source.cs : GetCombiningClass is now
271 implemented as indexer based array.
272 * Makefile : renamed output filename.
273 * create-mscompat-collation-table.cs : removed comments that does not
275 * create-tailoring.cs : use utf-8 output (and fixed filename).
277 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
279 * create-mscompat-collation-table.cs : hacked safer IPA extensions.
280 * Collation-notes.txt : status of sortkey table.
282 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
284 * create-mscompat-collation-table.cs : some Greek mapping fix.
286 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
288 * create-mscompat-collation-table.cs : diacritical weight is not
289 treated correctly when they are picked from letter names, as flags.
291 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
293 * create-mscompat-collation-table.cs : fixed culture-dependent
294 nonspacing mark weight.
296 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
298 * create-mscompat-collation-table.cs : some Hebrew case letter fixes.
299 Some diacritical fixes on symbols.
301 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
303 * create-mscompat-collation-table.cs : Fixed level 3 weight of
304 Arabic presentation forms.
306 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
308 * create-mscompat-collation-table.cs : Fixed some diacritical weight
309 of Arabic presentation forms.
311 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
313 * SimpleCollator.cs : more status updates. It's almost complete,
314 except for sortkey values.
316 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
318 * SimpleCollator.cs : similar optimization also for LastIndexOf().
320 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
322 * SimpleCollator.cs : the previous patch was missing IgnoreNonSpace
325 2005-07-27 Atsushi Enomoto <atsushi@ximian.com>
327 * SimpleCollator.cs : reduced extra sortkey value computation in
328 MatchesForward(). It makes IndexOf() roughly 30% faster.
330 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
332 * SortKey.cs : GetHashCode() returns a value based on its byte data.
335 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
337 * SimpleCollator.cs : consider extractions in invariant culture.
339 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
341 * SimpleCollator.cs : (unsafeFlags) be compact ;-)
343 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
345 * SimpleCollator.cs : When the tail of the target does not match more
346 than 3 times, then IsSuffix() will never be true (3 is the max
347 length of an expansion; \uFB03 -> ffi). It brings significant
348 performance boost when "source" string is very long.
349 * MSCompatUnicodeTable.cs : added MaxExpansionLength constant.
350 Reordered code lines.
352 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
354 * Collation-notes.txt : updated implementation status.
356 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
358 * SimpleCollator.cs : Implemented quick codepoint comparison in
359 Compare(). Comparison became 125x faster.
360 * mono-tailoring-source.txt : added tiny comment.
362 2005-07-26 Atsushi Enomoto <atsushi@ximian.com>
364 * mono-tailoring-source.txt : Added all single sortkey remapping to
365 all cultures (still need to fill contractions and annotate possible
366 buggy mapping referencing to CLDR).
367 * SimpleCollator.cs : removed unused code.
368 * MSCompatUnicodeTable.cs : tiny cast removal.
370 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
373 create-mscompat-collation-table.cs
374 MSCompatUnicodeTableUtil.cs
375 MSCompatUnicodeTable.cs : Now CJK mapping data is stored as byte
376 arrays. Thus SimpleCollator does not need to use bitwise and shift
377 operations to get sortkey value and they could be managed resources.
379 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
381 * create-mscompat-collation-table.cs,
382 MSCompatUnicodeTable.cs,
383 MSCompatUnicodeTableUtil.cs : From the result of sortkey comparison
384 between None and IgnoreWidth, width compat table could be computed
385 in somewhat simple way. So removed that table and all related code.
386 Increased the collation resource version.
388 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
390 * create-mscompat-collation-table.cs : Added C header output support.
392 2005-07-25 Atsushi Enomoto <atsushi@ximian.com>
394 * create-mscompat-collation-table.cs : FillLetterNFKD() could also be
395 applied to Cyrillic letters. Saved some of them.
397 2005-07-24 Atsushi Enomoto <atsushi@ximian.com>
399 * MSCompatUnicodeTable.cs : oh, ok, so we already have
400 GetManifestResourceInternal() ;-)
401 * managed-collation.patch : in Assembly.cs made that method internal.
403 2005-07-24 Atsushi Enomoto <atsushi@ximian.com>
405 * MSCompatUnicodeTable.cs : the pointer based icall code could be
406 also applicable for USE_MANAGED_RESOURCE mode.
408 2005-07-23 Atsushi Enomoto <atsushi@ximian.com>
410 * MSCompatUnicodeTable.cs : added icall support code (not enabled
411 unless the first line is commented out).
413 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
415 * create-mscompat-collation-table.cs,
416 MSCompatUnicodeTableUtil.cs,
417 MSCompatUnicodeTable.cs : Added resource version output (and ignore
418 in case of version mismatch). Removed obsolete, commented out code.
420 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
423 MSCompatUnicodeTable.cs,
424 create-mscompat-collation-table.cs : Now they use unmanaged pointers
425 instead of managed arrays.
426 * managed-collation.patch : Now it contains patch for IntPtrStream.cs
427 and Assembly.cs as well.
429 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
431 * MSCompatUnicodeTable.cs,
432 SimpleCollator.cs : Moved tailoring support classes to
433 MSCompatUnicodeTable.cs and drawn out from SimpleCollator.
434 Now that cjk and tailoring support are filled inside
435 MSCompatUnicodeTable, no managed array is exposed.
437 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
439 * create-mscompat-collation-table.cs,
441 MSCompatUnicodeTable.cs : Now it's not exposing collation table
442 internals as managed arrays (to switch to unmanaged pointers).
444 2005-07-22 Atsushi Enomoto <atsushi@ximian.com>
446 * create-mscompat-collation-table.cs : tiny nonspacing mark fix.
448 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
450 * create-mscompat-collation-table.cs : Fixed most of Greek mappings.
451 * MSCompatUnicodeTable.cs : don't lock string.
453 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
455 * create-mscompat-collation-table.cs : More Cyrillic diacritical fixes.
457 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
459 * create-mscompat-collation-table.cs : More Latin diacritical fixes.
461 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
463 * create-mscompat-collation-table.cs : There were still missing
464 math symbol mappings. Added several hacky diacritical weight for
467 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
469 * create-mscompat-collation-table.cs : fixed a few diacritical weight
470 on Cyrillic characters. Fixed ParseTailoringSource() to handle
471 non-heading escape sequence (\uXXXX) as expected.
473 2005-07-21 Atsushi Enomoto <atsushi@ximian.com>
475 * create-mscompat-collation-table.cs,
476 MSCompatUnicodeTableUtil.cs,
477 MSCompatUnicodeTable.cs : added more aggressive index limits for
478 table optimization at data size, in cost of speed.
480 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
482 * create-mscompat-collation-table.cs : fixed Arabic thirtial weight.
484 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
486 * create-mscompat-collation-table.cs : Mapping for hyphens and
487 punctuation are kinda finished. Rewrote batch mapping method to
488 collect all NFKD. Required modification on mapping is done.
490 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
492 * create-mscompat-collation-table.cs : minor mapping fixes on accent
493 marks and punctuations.
495 2005-07-20 Atsushi Enomoto <atsushi@ximian.com>
497 * create-mscompat-collation-table.cs : Fixed some MathSymbol mapping
498 and Box drawing mapping.
500 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
502 * create-mscompat-collation-table.cs : Fixed almost all numbers.
504 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
506 * create-mscompat-collation-table.cs : Symbol mappings are almost done.
507 Removed hack that gave dummy mappings to blank symbols.
509 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
511 * create-mscompat-collation-table.cs : more fix on arrows. Fix on box
512 drawings. Some code refactoring to eliminate hack.
514 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
516 * create-mscompat-collation-table.cs : Fixed some secondary weight
517 in Devanagari and arrows.
519 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
521 * create-mscompat-collation-table.cs : a set of tiny mapping fixes.
523 2005-07-19 Atsushi Enomoto <atsushi@ximian.com>
525 * create-mscompat-collation-table.cs : some diacritical fixes for
526 Latin. Added batch mapping method that considers computed
527 diacritical weight (for numbers).
529 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
531 * managed-collation.patch : forgot to add System.String patch.
533 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
535 * MSCompatUnicodeTable.cs : added resource existence check (required
536 for mscorlib transient time from the one without resources to the
539 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
541 * create-mscompat-collation-table.cs : fixed punctuations and hyphen
542 (shift) primary weight.
544 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
546 * create-mscompat-collation-table.cs : more nonspacing mark fixes.
547 Some non-basic Cyrillic diacritical weight fixes.
549 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
551 * create-mscompat-collation-table.cs : some Gurmukhi fixes on level 1
552 and level 3. Tiny Hangul weight fixes.
553 * MSCompatUnicodeTable.cs : U+30F5 and U+30F6 are small Japanese.
555 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
557 * create-mscompat-collation-table.cs : some normal characters who have
558 "narrow" NFKD mapping are regarded as "wide" and thus level 3 weight
559 values were different. Handle U+30FB as category A.
560 * MSCompatUnicodeTable.cs : U+30FB does not have special weight.
562 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
564 * create-mscompat-collation-table.cs : more diacritical weight fixes.
565 Removed some unused code.
567 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
569 * create-mscompat-collation-table.cs : Fixed some Thai and Arabic
572 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
574 * create-mscompat-collation-table.cs : Fixed Syriac nonspacing marks.
576 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
578 * create-mscompat-collation-table.cs : Fixed nonspacing marks in
579 Malayalam, Thai and Lao. Removed extraneous hack.
581 2005-07-15 Atsushi Enomoto <atsushi@ximian.com>
583 * SimpleCollator.cs : rewrote LastIndexOf() to handle source extenders.
584 Some refactoring on IndexOf() code. Removed unused Matches().
585 * Collation-notes.txt : some methods needed to be reimplemented, so
586 rewrote the description.
588 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
590 * SimpleCollator.cs : rewrote IsSuffix() to use CompareInternal().
591 Thus supported extenders in IsSuffix().
593 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
595 * SimpleCollator.cs : more IsSuffix() simplification, but it will be
596 stopped here since it cannot handle extenders (implementing new
599 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
601 * SimpleCollator.cs : simplified IsSuffix() code.
603 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
605 * SimpleCollator.cs : Fixed IndexOf() and LasIndexOf() to search the
606 entire replacement string if char target was an expansion.
607 IsSuffix() was using a method for IsPrefix() which was incorrect.
608 Removed old IsPrefix() code.
610 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
612 * SimpleCollator.cs : IndexOf() was incorrectly sharing the same
613 byte[] field in different areas of code. Now extenders in both
614 source and target really work in IndexOf().
616 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
618 * create-mscompat-collation-table.cs : fixed U+FF9F diacritical weight.
619 * SimpleCollator.cs : handle U+FF9E and U+FF9F as extenders.
621 2005-07-14 Atsushi Enomoto <atsushi@ximian.com>
623 * SimpleCollator.cs : Now FilterExtender() handles all extender
624 support. IndexOf() and LastIndexOf() now supports extenders.
625 IndexOf() and LastIndexOf() did not proceed contraction source
626 length as expected. Tiny refactoring on private IsPrefix() to take
629 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
631 * SimpleCollator.cs : when restoring from expansion, go back to the
632 top of the loop (to avoid index out of range).
633 Now IsPrefix() is implemented to reuse Compare() and thus it now
634 supports extender as well.
635 * Collation-notes.txt : status update. Deleted optimization part in
636 status section (it is duplicate).
638 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
640 * SimpleCollator.cs : some code reordering.
641 * create-mscompat-collation-table.cs : it was still missing U+3094.
643 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
645 * SimpleCollator.cs : Compare() now supports extender (e.g. U+39FC).
647 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
649 * SimpleCollator.cs : In GetSortKey(), don't update previousChar when
650 it is not primary (e.g. don't "extend" diacritical mark).
652 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
654 * managed-collation.patch : CompareInfo.Compare() should consider
655 the possibilities that non-empty string might be actually empty
656 in culture-sensitive context.
658 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
660 * SimpleCollator.cs : IndexOf() and LastIndexOf() returns start when
661 target is "empty" (in culture-sensitive context).
663 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
665 * SimpleCollator.cs : In IndexOf() and LastIndexOf(), skip ignorable
666 characters in target string.
668 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
670 * SimpleCollator.cs : When IgnoreWidth is specified, all Kana
671 characters are regarded as half-width.
672 Even though IgnoreWidth is specified, it should not ignore case.
673 For special weight comparison, the default values (E4) are bigger
674 than non-default values.
675 * SortKeyBuffer.cs : It should save LCID and original string.
676 * create-mscompat-collation-table.cs : For Japanese half-width kana,
677 it should not be counted in widthCompat map since IgnoreWidth does
678 not really ignore those differences.
680 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
682 * create-mscompat-collation-table.cs : Fixed missing Japanese bits.
684 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
686 * create-mscompat-collation-table.cs :
687 tiny diacritical weight fix for U+20D0-U+20E1.
689 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
691 * create-mscompat-collation-table.cs : ja CJK ideograph got completed.
693 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
695 * create-mscompat-collation-table.cs : Fixed CJK custom Japanese
696 mapping. It (maybe as well as other CJK tables) mixes NFKD. For
697 Japanese, modified NFKD table (because of Windows lame design).
699 2005-07-13 Atsushi Enomoto <atsushi@ximian.com>
701 * Makefile : added MONO_USE_MANAGED_COLLATION=no almost everywhere.
702 * MSCompatUnicodeTable.cs : FillCJK() was not invoked. Now it is
703 invoked at any time it is required.
704 * SimpleCollator.cs : call FillCJK() above in .ctor().
705 * MSCompatUnicodeTableUtil.cs : CJK range was wider.
706 * create-mscompat-collation-table.cs : CJK binary was missing the
707 length. CJK remapping is being moved to ModifyUnidata().
708 For cjk-ja mapping, we have to consider compat characters to be
709 added to the map, besides the raw UCA table.
711 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
713 * SortKeyBuffer.cs : Fixed shift level computation to match w/ Windows.
715 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
717 * SimpleCollator.cs : fixed LastIndexOf() to handle _target's_
718 contraction as expected. Fixed Compare() to save s2's contraction
720 * TestDriver.cs :added LastIndexOf() tester w/ indexes.
722 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
724 * managed-collation.patch : Fixed IsPrefix() and IsSuffix(). They
725 incorrectly use Compare().
726 * TestDriver.cs : more moved to nunit tests.
728 2005-07-12 Atsushi Enomoto <atsushi@ximian.com>
730 * SimpleCollator.cs : several fixes on Compare().
731 - Ignorable characters are skippted at the top of the loop.
732 - IgnoreNonSpace is checked to avoid extraneous level 2 comparison.
733 - In such case that s1 index is increased while s2 contraction is
734 replaced, s1 is inconsistently proceeded (bug).
735 - IsIgnorable() now also checks IgnoreNonSpace.
736 - Fixed FilterOptions() that does not work for IgnoreWidth at all.
737 * TestDriver.cs : now some are moved to nunit tests.
738 * Collation-notes.txt : minor todo update.
740 2005-07-11 Atsushi Enomoto <atsushi@ximian.com>
742 * SimpleCollator.cs : Compare() was ignoring such case that both
743 entire strings have '-' to be compared.
744 * Collation-notes.txt : more status updates.
745 * TestDriver.cs : added '-' use cases.
747 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
749 * SimpleCollator.cs : to be same as other buggy part, it now handles
750 U+3005, U+3031 and U+3032 as buggy as Windows. It just repeats
752 Fixed GetSortKey(): if the repeater is U+3005, second weight is 5.
753 * create-mscompat-collation-table.cs : dummy values for extenders.
755 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
757 * SimpleCollator.cs : Special weight fixes on GetSortKey(). Dash type
758 should be computed from ExtenderType, and voice mark weight should
760 * MSCompatUnicodeTable.cs : added tiny comment.
762 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
764 * SortKey.cs : It borked when MONO_USE_MANAGED_COLLATION is not yes.
765 * SimpleCollator.cs : support for extender (U+309D etc.).
767 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
769 * create-mscompat-collation-table.cs : some punct/symbols fix.
770 * managed-collation.patch : new (and temporary) file to support
771 managed collation in mscorlib.
772 * README : described how to use managed collation.
774 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
776 * create-mscompat-collation-table.cs : Further Cyrillic fixes. Handle
777 U+482-4C8 (though needs diacritical fixes).
778 * MSCompatUnicodeTable.cs : tiny comment for alternative impl.
780 2005-07-08 Atsushi Enomoto <atsushi@ximian.com>
782 * create-mscompat-collation-table.cs : Reimplemented Cyrillic weight
783 computation code, since it looks like the same way as Latin letters
784 have. Thus removed all other approach (UCA, by letter name).
786 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
788 * create-mscompat-collation-table.cs : diacritical fix for "double-
789 struck". Syriac nonspacing fixes.
791 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
793 * create-mscompat-collation-table.cs : more math symbol weight fixes.
795 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
797 * create-mscompat-collation-table.cs : fixed Hebrew character sortkeys.
799 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
801 * create-mscompat-collation-table.cs : math symbols U+25A0-U+2600 are
802 implemented (no stub). Some other fixes on category 8-A.
804 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
806 * create-mscompat-collation-table.cs : some minor fixes on Arabic,
807 Korean and Japanese sortkey weights.
809 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
811 * create-mscompat-collation-table.cs : More diacritical fixes.
812 Georgian characters do not have level 2 weights but level 3.
814 2005-07-07 Atsushi Enomoto <atsushi@ximian.com>
816 * create-mscompat-collation-table.cs : Roman numeral characters
817 have diacritical weight. quick hack for control signs (U+2400..)
820 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
822 * create-mscompat-collation-table.cs : improving Latin mappings.
823 Setting non-ASCII Latin characters' primary weight between those
824 ASCII characters, and setting diacritical weight (hacky).
825 * MSCompatUnicodeTable.cs :
826 Kanatype check: fixed (voice marks) and improved (comparison order).
828 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
830 * create-mscompat-collation-table.cs : more diacritical fixes.
831 primary weight fixes on punctuations in category 07.
833 2005-07-06 Atsushi Enomoto <atsushi@ximian.com>
835 * create-mscompat-collation-table.cs : several diacritical fixes.
836 * TestDriver.cs : sortkey dumper should use StringSort.
838 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
840 * SimpleCollator.cs : fixed incorrect indexer setup. Optimized
841 GetContraction() call a bit.
843 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
845 * create-mscompat-collation-table.cs : fixed incorrect level 2
847 * MSCompatUnicodeTable.cs : remove debug line.
849 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
851 * MSCompatUnicodeTableUtil.cs,
852 MSCompatUnicodeTable.cs,
854 create-mscompat-collation-table.cs : made some members internal and
855 accessible from other classes. Many indexes could be 0 by default.
856 * SimpleCollator.cs : optimizations. avoid method call.
858 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
860 * Collation-notes.txt : more updates.
861 * SimpleCollator.cs : Added quick check for Ordinal comparison.
862 Fixed special weight comparison. It cannot be customizable in the
863 implementation (and it won't be harmful).
864 * mono-tailoring-source.txt : thus updated comment.
866 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
868 * SimpleCollator.cs : Compare() was missing French sort support.
869 * TestDriver.cs : added example case.
871 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
873 * Collation-notes.txt : updated status. Eliminated descriptions on
874 "iterator" (I avoided it for performance concern). Fixed misc.
875 incorrect descriptions.
877 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
879 * Collator.cs : Now that SimpleCollator became feature complete, it is
882 2005-07-05 Atsushi Enomoto <atsushi@ximian.com>
884 * SimpleCollator.cs : implemented decent Compare() that immediately
885 stops at first primary difference.
887 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
889 * SimpleCollator.cs : indexers might return -1.
891 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
893 * SimpleCollator.cs : IsPrefix() and IsSuffix() optimization code was
894 buggy (length check for source was missing).
896 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
898 * create-mscompat-collation-table.cs : Fixed tailoring table output
899 to be in correct and countable order. Now if tailoring alias was not
900 found, just stop the build.
901 * MSCompatUnicodeTable.cs : several build fixes. Now it works to read
903 * mono-tailoring-source.txt : commented out CJK aliases that miss
905 * Makefile : needed further filename fixes.
907 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
909 * MSCompatUnicodeTable.cs : renamed from MSCompatUnicodeTable.template
910 (now it is working as a standalone file).
911 * Makefile : renamed generated file as MSCompatUnicodeTableGenerated.cs
912 (the generator now creates both binary resources and C# source).
914 2005-07-04 Atsushi Enomoto <atsushi@ximian.com>
916 * create-mscompat-collation-table.cs : Now it generates binary
917 resources (to parent directory).
918 * MSCompatUnicodeTable.template : added conditional code that fills
919 collation tables from manifest resources.
920 * Makefile : remove collation table binaries as well on "make clean".
921 Removed extraneous dependency.
923 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
925 * MSCompatUnicodeTable.template,
926 SimpleCollator.cs : removed extraneous GetExpansion().
928 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
930 * SimpleCollator.cs : IsSuffix() also supports contractions.
931 * TestDriver.cs : IsSuffix() example contraction cases.
933 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
935 * SimpleCollator.cs : reverted IsSuffix() to return bool (to match w/
936 what current IsPrefix() does). For expansion of target, IsPrefix()
937 should check the no-match case that expansion is longer than input.
938 Some refactory on IsPrefix().
939 Added GetContractionTal() for IsSuffix() (not used yet).
941 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
943 * TestDriver.cs : added IsPrefix() expansion cases.
944 * SimpleCollator.cs : IsPrefix() now supports contractions (with much
945 of complexity), and it now returns bool again.
946 IndexOf() for replacement should make use of IndexOfPrimitiveChar()
947 since expansions won't be expanded recursively.
949 2005-07-01 Atsushi Enomoto <atsushi@ximian.com>
951 * SimpleCollator.cs : commonized character comparison in IsPrefix()
952 and IsSuffix(). csc compile fix.
953 * CompareInfoImpl.cs : deleted.
955 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
957 * TestDriver.cs : added SimpleCollator.ctor() sanity check.
958 Added replacement contraction example.
959 * SimpleCollator.cs : Now IndexOf() and LastIndexOf() support
960 contraction in source string. Extracted matching code to Matches().
961 Replacement contraction was including extraneous '\x0'.
963 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
965 * Collation-notes.txt : updated status.
966 * CollationDataStructures.txt : tiny fixes.
967 * SimpleCollator.cs :
968 Renamed alias Util to UUtil (MS sys.enterprisesvc has sucky global
969 namespace Util and csc borked).
970 GetContraction was incorrectly returning first item.
971 Private IsPrefix() now returns int (but it might not be in real use).
972 Extracted simple char comparison to CompareCharSimple().
973 IndexOf() and LastIndexOf() now fully handle contractions (both
974 binary key and string replacement) in "target" (for "s" not yet).
975 * TestDriver.cs : be more verbose.
976 * mono-tailoring-source.txt : added comment.
977 * MSCompatUnicodeTable.template :
978 Renamed alias Util to UUtil (MS sys.enterprisesvc has sucky global
980 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
982 * create-mscompat-collation-table.cs : compute COMBINING blah marks as
983 well as those characters WITH blah.
984 * TestDriver.cs : added combining sortkey cases.
986 2005-06-30 Atsushi Enomoto <atsushi@ximian.com>
988 * mono-tailoring-source.txt : fixed description on '*' in sortkeys.
989 * SimpleCollator.cs : Now it fully uses tailoring info. Fixed
990 contraction search that worked only when string is contraction.
991 Removed commented code. Minor refactoring.
992 * TestDriver.cs : added example that uses "ZS" in Hungarian sorting.
994 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
996 * create-mscompat-collation-table.cs,
997 * mono-tailoring-source.txt : removed extraneous level 4 sortkey
998 which cannot be supported.
999 * SimpleCollator.cs : added GetContraction() and used in some places.
1000 Now CompareOptions is set only once. Reordered some code (e.g.
1001 ignorable check -> get compat char -> compare).
1003 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1005 * SimpleCollator.cs : sort tailoring tables before actual usage.
1006 Support diacritical remappings (it is customized collation rule
1007 which does not exist in UCA).
1009 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1011 * SimpleCollator.cs : build culture specific tailoring table from
1012 TailoringInfo and unified data array.
1013 * create-mscompat-collation-table.cs : Added null termination to
1014 sortkey map tailorings (mostly to save my eyes).
1015 * MSCompatUnicodeTable.template : added public TailoringValues.
1017 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1019 * SortKeyBuffer.cs : handle special weight (category 06) characters.
1020 * Collation-notes.txt : Updated description on special weight (it was
1022 * TestDriver.cs : added special weight cases.
1024 2005-06-29 Atsushi Enomoto <atsushi@ximian.com>
1026 * MSCompatUnicodeTable.template : added GetTailoringInfo().
1027 * SimpleCollator.cs : Now tailoring information is acquired and used.
1028 (FrenchSort is supported but Compare() won't work expectedly since
1029 the table is still incomplete for those diacritical marks).
1030 * SortKeyBuffer.cs : On reversing diacritical weights, it should
1031 ignore zeros. Reset() should reset frenchSorted flag.
1033 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1035 * create-mscompat-collation-table.cs : Further fixes on Jamo,
1036 diacritical weights by character name, and *Numbers primary weights.
1038 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1040 * create-mscompat-collation-table.cs : More fix on Devanagari,
1041 Gujarati, Oliya, Tamil and Lao sortkeys.
1043 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1045 * create-mscompat-collation-table.cs : Fixed Georgian, Thai, Gurmukhi
1048 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1050 * create-mscompat-collation-table.cs : Fixed Thai character primary
1051 and secondary values. Fixed Thaana letters. Added more LAMESPEC
1052 CJK compat. Fixed some circled CJK secondary weight.
1053 Hacked some nonspacing mark sortkey value adjustment.
1055 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1057 * create-mscompat-collation-table.cs : CP932.TXT was not parsed as
1058 expected. JIS ordering was incorrect. OtherNumbers that represents
1059 10 or more values were incorrectly computed the offset. Some Hangul
1060 compat characters has different offset.
1062 2005-06-28 Atsushi Enomoto <atsushi@ximian.com>
1064 * create-mscompat-collation-table.cs : Fixed 0x8 category characters.
1065 Added hack for need-to-be-fixed characters to fall into 0xA category.
1066 * create-collation-element-table.cs : previous checkin seem failed :(
1067 * README: updated a bit.
1069 2005-06-24 Atsushi Enomoto <atsushi@ximian.com>
1071 * CodePointIndexer.cs :
1072 removed extraneous switch (I could use empty array for that need).
1073 * CollationElementTableUtil.cs : primary weight type became ushort.
1074 * create-collation-element-table.cs : several bugfixes.
1075 collElem should be int. It was skipping most of entries because of
1076 incorrect string tokenization.
1078 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1080 * create-mscompat-collation-table.cs : handle some Jamo NKFD.
1082 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1084 * SimpleCollator.cs : forgot to commit in the last checkin.
1085 * create-mscompat-collation-table.cs : fixed arabic shift weight chars.
1086 * TestDriver.cs : switch table dumper and collator testing.
1087 * SortKey.cs : for now comment out internal indexes (not in use).
1089 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1091 * MSCompatUnicodeTable.template,
1092 SimpleCollator.cs : support for culture dependent CJK table.
1094 2005-06-23 Atsushi Enomoto <atsushi@ximian.com>
1096 * create-mscompat-collation-table.cs,
1097 MSCompatUnicodeTableUtil.cs : make CJK table more compact.
1099 2005-06-22 Atsushi Enomoto <atsushi@ximian.com>
1101 * SimpleCollator.cs : Fixed stupid index search when start != 0.
1103 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1105 * SimpleCollator.cs : fixed my misunderstanding on LastIndexOf(). It
1106 now starts from "start" and proceeds backward by "length".
1107 * TestDriver.cs : fix warning.
1109 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1111 * TestDriver.cs : more tests.
1112 * SimpleCollator.cs : LastIndexOf() is not setting search length
1113 on iteration. Quick workaround fro String.LastIndexOf() bug (maybe).
1115 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1117 * create-normalization-source.cs : output propValue as uint.
1119 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1121 * SortKey.cs : Now it is System.Globalization.SortKey.
1122 To replace existing implementation, it now requires lcid and
1123 CompareOptions. Added required members.
1124 * SortKeyBuffer.cs : thus .ctor() requires LCID.
1125 * SimpleCollator.cs : made required changes above.
1127 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1129 * CodePointIndexer.cs : added CompressArray(). Now it requires two more
1130 parameters for default index and codepoint.
1131 * CollationElementTableUtil.cs,
1132 NormalizationTableUtil.cs : required changes wrt above change.
1133 * MSCompatUnicodeTableUtil.cs : added for several codepoint indexers.
1134 * MSCompatUnicodeTable.template : Now it uses codepoint indexer.
1135 * create-mscompat-collation-table.cs : Now it outputs compressed array.
1136 * Makefile : now collation requires MSCompatUnicodeTableUtil.cs
1138 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1140 * SimpleCollator.cs :
1141 Implemented IsSuffix() and LastIndexOf().
1142 Several fixes on index > 0 cases.
1143 * TestDriver.cs : sample IsSuffix() and LastIndexOf() usage and more.
1145 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1147 * Collation-notes.txt : updated (status, impl. classes).
1148 * MSCompatUnicodeTable.cs : Korean Jamo are not really expansions.
1150 2005-06-21 Atsushi Enomoto <atsushi@ximian.com>
1152 * SimpleCollator.cs : implemented IndexOf(string,string,CompareOptions)
1153 and IsPrefix(). Tiny code refactory.
1154 * TestDriver.cs : sample IsPrefix() and IndexOf() usage.
1155 * MSCompatUnicodeTable.cs : tiny refactory for CodePointIndexer use.
1157 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1159 * SimpleCollator.cs :
1160 IndexOf(string, char, CompareOptions) implementation.
1161 * TestDriver.cs : sample IndexOf() usage.
1163 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1165 * create-mscompat-collation-table.cs : was missing most important
1166 kind of blocks - equivalent expansions (e.g. invariant mappings).
1167 More readable mappings.
1169 2005-06-20 Atsushi Enomoto <atsushi@ximian.com>
1171 * mono-tailoring-source.txt : new file. It describes tailoring
1172 information. Basically examined under .NET 1.x.
1173 * create-mscompat-collation-table.cs : consume the file above.
1174 * MSCompatUnicodeTable.template : now tailorings is not a stub.
1175 * CollationDataStructures.txt : minor fixes.
1177 SimpleCollator.cs : added FrenchSort support.
1178 * Collation-notes.txt : added description on Latin primary weights.
1179 * ldml-limited.rng : added note.
1180 * create-tailorings.cs : added note. more serialization (but won't be
1183 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1185 * SortKeyBuffer.cs : non-primary character is added to previous
1187 * TestDriver.cs : added example case of above.
1189 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1191 * SimpleCollator.cs : IgnoreSymbols support.
1192 * TestDriver.cs : compilation fix. IgnoreSymbols example.
1193 * create-mscompat-collation-table.cs : more Hangul fixes.
1195 2005-06-17 Atsushi Enomoto <atsushi@ximian.com>
1197 * create-mscompat-collation-table.cs : more Hangul fixes.
1198 * SortKey.cs : it will replace sys.globalization.SortKey. It has
1199 some internal members.
1200 * SortKeyBuffer.cs : now it uses SortKey instead of byte[].
1201 * SimpleCollator.cs : CompareOptions support. However I don't think
1202 it will be developed anymore since SortKey never enables IndexOf().
1203 * TestDriver.cs : a few CompareOptions cases.
1205 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1207 * SimpleCollator.cs : simple collator implementation that just will
1208 use GetSortKey() for all its basis.
1209 * TestDriver.cs : sample code that uses this collator set.
1210 * MSCompatUnicodeTable.template : removed test driver from here.
1212 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1214 * create-mscompat-collation-table.cs : Hangul fixes.
1215 Now less than 300 characters that does not have sortkey weights.
1216 * MSCompatUnicodeTable.template : added FIXME info for Hangul Jamo.
1218 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1220 * create-mscompat-collation-table.cs : Added control picture mappings.
1221 Minor primary weight fixes.
1223 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1225 * create-mscompat-collation-table.cs : Added mappings for box
1226 drawings and blocks.
1228 2005-06-16 Atsushi Enomoto <atsushi@ximian.com>
1230 * create-mscompat-collation-table.cs : Added mappings for arrows.
1232 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1234 * create-mscompat-collation-table.cs : added support for letterlike
1235 characters and squared CJK compatibility characters, ordered by
1236 character names (0x0E category).
1237 * Collation-notes.txt : added description on that.
1239 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1241 * MSCompatUnicodeTable.template : Now expansions are simulated.
1242 * create-mscompat-collation-table.cs : filled Korean number level2.
1243 Reordered some code blocks to fill correct diacritical differences.
1244 * Collation-notes.txt : some corrections and minor additions.
1246 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1248 * MSCompatUnicodeTable.template :
1249 Now dumper test driver uses SortKeyBuffer for dogfooding.
1250 * create-mscompat-collation-table.cs : some diacritical level fixes
1251 (with non-working extra latin check).
1252 * SortKeyBuffer.cs : several fixes to get working as a practical code.
1253 * Collator.cs : make it compilable, leaving things as NotImplemented.
1255 2005-06-15 Atsushi Enomoto <atsushi@ximian.com>
1257 * create-mscompat-collation-table.cs : some fixes on primary category
1258 07 (miscellaneous symbols and punctuations).
1260 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1262 * create-mscompat-collation-table.cs : more mapping fix on numbers,
1263 letters, variable weight characters, circled Japanese and CJK.
1264 * MSCompatUnicodeTable.template : fixed HasSpecialWeight() to be more
1265 inclusive. Simplified dumper code.
1267 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1269 * create-mscompat-collation-table.cs : finished Hangul (both Jamo
1270 and Syllables). sortkey dumper diff lines became 8000 from 30000.
1272 2005-06-14 Atsushi Enomoto <atsushi@ximian.com>
1274 * create-mscompat-collation-table.cs : added some nonspacing marks in
1275 either correct or hacky way.
1277 2005-06-13 Atsushi Enomoto <atsushi@ximian.com>
1279 * create-mscompat-collation-table.cs : several improvements. Japanese
1280 Kana support, Hebrew accents, Bengali nonspacing marks, sorting of
1281 numeric characters, diacritically decorated latin alphabets. Fixed
1282 some diacritical weights detection.
1283 * MSCompatUnicodeTable.cs : tiny Japanese fix. Handle nonspacing
1284 marks' primary weight as empty.
1285 * Collation-notes.txt : some updates.
1287 2005-06-13 Atsushi Enomoto <atsushi@ximian.com>
1289 * create-mscompat-collation-table.cs : don't process nonexact NFKD
1290 mapping as equivalent, however store CJK extensions into NFKD map
1291 even if one does not strictly match.
1292 Now am going to fill Hangul into tables (unlike UCA it does not look
1293 possible to calculate sortkey value).
1294 Fixed Cyrillic and Georgian UCA based orderings.
1295 * MSCompatUnicodeTable.template : added CJK extension sortkey
1298 2005-06-10 Atsushi Enomoto <atsushi@ximian.com>
1300 * create-mscompat-collation-table.cs : Fixed latin alphabet support.
1301 Added latin with diacritical and CJK extension.
1302 * MSCompatUnicodeTable.cs : modified dumper code a bit (for my purpose).
1304 2005-06-10 Atsushi Enomoto <atsushi@ximian.com>
1306 * create-mscompat-collation-table.cs : now parses DerivedAge.txt (right
1307 now not used thouth). Filled CJK ideograph, still not perfect.
1308 Fixed number primary keys. NFKD numbers and CJK ideographs are now
1309 considered, including brackets elimination.
1310 * Makefile : now it downloads DerivedAge.txt.
1311 * MSCompatUnicodeTable.template : added dummy code dumper. It computes
1312 PrivateUse, Surrogate and Hangul Syllables.
1313 * Collation-notes.txt : Noted that Hangul Syllables need more love.
1315 2005-06-09 Atsushi Enomoto <atsushi@ximian.com>
1317 * create-tailorings.cs : added configuration support. sort them.
1318 I wonder if it is really usable. Having own format might be better.
1319 * create-mscompat-collation-table.cs : fixing some sortkey numbers,
1320 making closer to windows. Now it handles NFKD in some places.
1321 * MSCompatUnicodeTable.template : Added dummy sortkey dumper driver.
1322 * CollationDataStructures.txt : added description on tailoring
1323 fields, though they are subject to change.
1325 2005-06-07 Atsushi Enomoto <atsushi@ximian.com>
1327 * create-tailorings.cs, ldml-limited.rng : new file.
1328 * LdmlReader.cs : removed old file.
1330 2005-06-07 Atsushi Enomoto <atsushi@ximian.com>
1332 * SortKeyBuffer.cs : split from Collator.cs. Now it considers
1333 practical use, reflecting updated sortkey constant design.
1334 Especially level 4 weight is split to 4 arrays that are merged in
1335 the last stage of GetSortKey().
1336 * Collator.cs : thus SortKeyBuffer is removed from here.
1337 Additionally, removed some extraneous bits in other classes.
1338 * Collation-notes.txt : Some editorial fixes. Added information on
1339 Korean matter (how to compute Hangle Syllables / Hangul Jamo cannot
1340 be stored in simple byte arrays).
1341 * CodePointIndexer.cs,
1342 create-collation-element-table.cs,
1343 CollationElementTable.template,
1344 NormalizationTableUtil.cs : short CodePointIndexer method names.
1345 * create-mscompat-collation-table.cs : Additional info on why some
1346 meaningful characters are ignored in Windows (Unicode version
1347 difference). Removed U+070F from special check (was extraneous).
1349 2005-06-06 Atsushi Enomoto <atsushi@ximian.com>
1351 * MSCompatUnicodeTable.template:
1352 Moved body implementation to table creator and put those bool
1353 results into an array.
1354 * create-mscompat-collation-table.cs :
1355 So imported those methods. Modified array output to emit "0x"
1356 only for more than 9.
1357 * create-normalization-source.cs : ditto on "0x" output matter.
1358 * CollationDataStructures.txt : so now it holds ignorableFlags.
1360 2005-06-03 Atsushi Enomoto <atsushi@ximian.com>
1362 * Collation-notes.txt, CollationDataStructures.txt :
1363 separate document for data structure design.
1365 2005-06-03 Atsushi Enomoto <atsushi@ximian.com>
1367 * create-mscompat-collation-table.cs : added culture-dependent CJK
1368 table creation. It uses CLDR as its basis. (Culture independent CJK
1370 * Makefile : added CLDR archive downloading support.
1371 * MSCompatUnicodeTable.template : tiny renamings.
1372 * Collation-notes.txt : additional CJK info.
1374 2005-06-02 Atsushi Enomoto <atsushi@ximian.com>
1376 * Collation-notes.txt, create-mscompat-collation-table.cs :
1377 added secondary weight support for BlahNumber characters.
1379 2005-06-01 Atsushi Enomoto <atsushi@ximian.com>
1381 * downloaded : added directory. All downloaded files are stored here.
1382 * Makefile : use "downloaded" directory.
1383 Added more auto-download stuff.
1384 * create-mscompat-collation-table.cs :
1385 Added Japanese square kana support.
1387 2005-06-01 Atsushi Enomoto <atsushi@ximian.com>
1389 * Collation-notes.txt : added Estrangela (ancient Syriac) and Thaana.
1390 * create-mscompat-collation-table.cs : added support for Arabic abjad,
1391 Estrangela and Thaana.
1392 * MSCompatUnicodeTable.template : removed BOM.
1394 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1396 * Collation-notes.txt : wrong comment cleanup and spelling fixes.
1397 * create-mscompat-collation-table.cs : added diacritic support for
1398 Latin letters (as long as covered in primary weight).
1400 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1402 * Makefile : minor fixes. Added warning lines to generated sources.
1404 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1406 * create-char-mapping-source.cs :
1407 Removed ToWidthInsensitive() generation.
1409 2005-05-31 Atsushi Enomoto <atsushi@ximian.com>
1411 * create-mscompat-collation-table.cs : Now it dumps level1 to 3 values.
1412 ToWidthInsensitive() is implemented here, using an array (which is
1413 to be optimized using CodePointIndexer).
1414 * MSCompatUnicodeTable.cs : renamed as MSCompatUnicodeTable.template
1415 * MSCompatUnicodeTable.template : now it is used to generate
1416 MSCompatUnicodeTable.cs which got ready to be used.
1417 * Makefile : added MSCompatUnicodeTable.cs build support. Now it
1418 supports "make normalization" and "make collation".
1420 2005-05-30 Atsushi Enomoto <atsushi@ximian.com>
1422 * Collation-notes.txt : Description on ICU is very incorrect. Now it
1423 became more rational and sane.
1424 * create-mscompat-collation-table.cs : fixed some indexes.
1425 * Makefile : added "mstablegen" target.
1426 * MSCompatUnicodeTable.cs : removed GetPrimaryWeight(). Minor fix.
1428 2005-05-26 Atsushi Enomoto <atsushi@ximian.com>
1430 * Collation-notes.txt : more analysis on "letters".
1431 * create-mscompat-collation-table.cs : more proof of concepts.
1433 2005-05-25 Atsushi Enomoto <atsushi@ximian.com>
1435 * Collation-notes.txt : more info. Started letter sortkey analysis
1436 (some of other stuff are really non-understandable right now.)
1437 * create-mscompat-collation-table.cs : table generator proof-of-
1438 concept source (not compilable).
1439 * MSCompatUnicodeTable.cs : moved some code to the new source.
1442 2005-05-20 Atsushi Enomoto <atsushi@ximian.com>
1444 * Collation-notes.txt : started level 2 weight analysis.
1446 2005-05-19 Atsushi Enomoto <atsushi@ximian.com>
1448 * Collation-notes.txt : Additional information on how to create
1450 * MSCompatUnicodeTable.cs : implemented part of GetLevel3Weight().
1452 2005-05-19 Atsushi Enomoto <atsushi@ximian.com>
1454 * Collation-notes.txt : More case weight (level 3) analysis. I'm
1455 likely to just write table generator.
1457 2005-05-18 Atsushi Enomoto <atsushi@ximian.com>
1459 * MSCompatUnicodeTable.cs : part of level 4 weight implementation.
1461 2005-05-18 Atsushi Enomoto <atsushi@ximian.com>
1463 * Collation-notes.txt :
1465 Revised comparison methods; backward iteration is possible.
1466 More on char-by-char comparison.
1467 Level 4 comparison is actually a bit more complex.
1469 * Collator.cs : some conceptual updates wrt above.
1471 2005-05-17 Atsushi Enomoto <atsushi@ximian.com>
1473 * Collation-notes.txt : Japanese voice mark is level 2, and Hangul
1474 properties are level 3.
1476 2005-05-17 Atsushi Enomoto <atsushi@ximian.com>
1478 * Collation-notes.txt : Make it more readable. More analysis on
1479 level 3 and 4 sortkey structures.
1480 * Collator.cs : some compilation fixes (not compilable yet).
1482 2005-05-16 Atsushi Enomoto <atsushi@ximian.com>
1484 * Collation-notes.txt : Analysis on variable-weighting (level 5)
1486 * Collator.cs : updated corresponding part of level 5, and more.
1488 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1490 * Collation-notes.txt : more updates.
1491 * Collator.cs : rewrote from scratch. Some rough sketch for sortkey
1492 buffer, character iterator and collator methods. Not compiling.
1494 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1496 * Collator.cs : Am going to replace it with new one. No need for
1497 CompareOptions-dependent Comparer.
1499 2005-05-13 Atsushi Enomoto <atsushi@ximian.com>
1501 * Collation-notes.txt : There seems a bit more complexity.
1503 2005-05-10 Atsushi Enomoto <atsushi@ximian.com>
1505 * Collation-notes.txt : more updates, being close to write sortkey
1508 2005-05-09 Atsushi Enomoto <atsushi@ximian.com>
1510 * CompareInfoImpl.cs, Collator.cs : conceptual update
1511 * Collation-notes.txt : some corrections and additions.
1512 * Makefile : added LDML input (but it won't be used at all).
1514 2005-04-28 Atsushi Enomoto <atsushi@ximian.com>
1516 * Collation-notes.txt : more updates.
1518 2005-04-26 Atsushi Enomoto <atsushi@ximian.com>
1520 * Collation-notes.txt : more updates.
1522 2005-04-26 Atsushi Enomoto <atsushi@ximian.com>
1524 * Collation-notes.txt : some updates.
1525 * create-mapping-char-source.cs : superscripts and subscripts are also
1526 ignored in IgnoreWidth comparison.
1527 * Makefile : tiny touch fix.
1529 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1531 * CompareInfoImpl.cs, Collator.cs : conceptual stuff (not working).
1533 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1535 * create-char-mapping-source.cs : Now it generates
1536 ToWidthInsensitive() from combining category <wide> and <narrow>.
1537 * MSCompatUnicodeTable.cs : added ToKanaTypeInsensitive() and
1538 ToWidthInsensitive() for IgnoreKanaType and IgnoreWidth.
1540 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1542 * README, LdmlReader.cs, DataStructures.txt : new files.
1544 2005-04-25 Atsushi Enomoto <atsushi@ximian.com>
1546 * CodePointIndexer.cs,
1547 Collation-notes.txt,
1548 CollationElementTable.template,
1549 CollationElementTableUtil.cs,
1550 create-char-mapping-source.cs,
1551 create-collation-element-table.cs,
1552 create-combining-class-source.cs,
1553 create-normalization-source.cs,
1555 MSCompatUnicodeTable.cs,
1556 Normalization.template,
1557 NormalizationTableUtil.cs : initial checkin (to private branch).