* Collation Data structure and code

** This document is not up to date

	Currently the table generator generates all three kind of files
	(C header, C# source and binary resources) but only binary resources
	are used in fact. C# source is useful for debugging. C header is
	historical.


** Manual tasks required to maintain the source.

	We should always pay attention to the contants for CodePointIndexer
	that is likely not be in sync with the constant arrays.


** collation-tables.h

	Note: the structures are similar in MSCompatUnicodeTable.cs but now
	it is managed code.

	typedef struct {
		ushort lcid;
		ushort tailoringIndex;
		ushort tailoringCount;
		short reverseAccentOrder; /* 1:French sort. 0:Normal */
	} TailoringInfo;

	Those [*] characters will be compressed using CodePointIndexer
	whose max value is char.MaxValue+1.

	// Holds sortkey basis.
	guint8 [*] category;
	guint8 [*] level1;
	guint8 [*] level2;
	guint8 [*] level3;
	guint8 [*] ignorableFlags; // 1:complete, 2:symbol, 3:nonspace
	gunichar [*] widthCompat;

	// Holds special arrays for CJK order which is culture dependent.
	guint16 [*] cjkCHS;
	guint16 [*] cjkCHT;
	guint16 [*] cjkJA;
	guint16 [*] cjkKO;
	guint8 [*] cjkKOlv2;
	gunichar [whole_tailoring_count] tailorings;
	CollationSource [culture_count] collationSources;

	"tailorings" table holds the entire contract mappings and expansion
	mappings for all cultures.
	Actually it is not only "culture dependent tailorings" but also
	contains expansions for Invariant culture.
	CollationSource.tailoringIndex holds the index of "tailorings" array
	to point where its tailorings begin.
	Tailorings for the culture is counted by tailoringCount.

	If tailoringIndex is 0 then there is no tailorings for the specific
	culture (it should still handle invariant tailorings).

** CollationSourceUtil.cs
	static CodePointIndexer Category;
	static CodePointIndexer Level1;
	static CodePointIndexer Level2;
	static CodePointIndexer Level3;
	static CodePointIndexer Ignorable;
	static CodePointIndexer WidthCompat;
	static CodePointIndexer CjkCHS;
	static CodePointIndexer Cjk;

** CollatorSource.cs

	static byte [*] category;
	static byte [*] level1;
	static byte [*] level2;
	static byte [*] level3;
	static byte [*] ignorableFlags; // 1:complete, 2:symbol, 3:nonspace
	static char [*] widthCompat;
	static char [] tailorings;
	static ushort [] cjkCHS;
	static ushort [] cjkCHT;
	static ushort [] cjkJA;
	static ushort [] cjkKO;
	static byte [] cjkKOlv2;

	class TailoringInfo // instantiated for each CultureInfo
	{
		// Primary constants
		int tailoringIndex;
		int tailoringCount;
		bool frenchSort;
	}