1 The Internals of the Mono C# Compiler
9 The Mono C# compiler is a C# compiler written in C# itself.
10 Its goals are to provide a free and alternate implementation
11 of the C# language. The Mono C# compiler generates ECMA CIL
12 images through the use of the System.Reflection.Emit API which
13 enable the compiler to be platform independent.
15 * Overview: How the compiler fits together
17 The compilation process is managed by the compiler driver (it
20 The compiler reads a set of C# source code files, and parses
21 them. Any assemblies or modules that the user might want to
22 use with his project are loaded after parsing is done.
24 Once all the files have been parsed, the type hierarchy is
25 resolved. First interfaces are resolved, then types and
28 Once the type hierarchy is resolved, every type is populated:
29 fields, methods, indexers, properties, events and delegates
30 are entered into the type system.
32 At this point the program skeleton has been completed. The
33 next process is to actually emit the code for each of the
34 executable methods. The compiler drives this from
37 Each type then has to populate its methods: populating a
38 method requires creating a structure that is used as the state
39 of the block being emitted (this is the EmitContext class) and
40 then generating code for the topmost statement (the Block).
42 Code generation has two steps: the first step is the semantic
43 analysis (Resolve method) that resolves any pending tasks, and
44 guarantees that the code is correct. The second phase is the
45 actual code emission. All errors are flagged during in the
48 After all code has been emitted, then the compiler closes all
49 the types (this basically tells the Reflection.Emit library to
50 finish up the types), resources, and definition of the entry
51 point are done at this point, and the output is saved to
56 All the input files that make up a program need to be read in
57 advance, because C# allows declarations to happen after an
58 entity is used, for example, the following is a valid program:
63 a = "hello"; b = "world";
72 At the time the assignment expression `a = "hello"' is parsed,
73 it is not know whether a is a class field from this class, or
74 its parents, or whether it is a property access or a variable
75 reference. The actual meaning of `a' will not be discvored
76 until the semantic analysis phase.
78 ** The Tokenizer and the pre-processor
80 The tokenizer is contained in the file `cs-tokenizer.cs', and
81 the main entry point is the `token ()' method. The tokenizer
82 implements the `yyParser.yyInput' interface, which is what the
83 Yacc/Jay parser will use when fetching tokens.
85 Token definitions are generated by jay during the compilation
86 process, and those can be references from the tokenizer class
87 with the `Token.' prefix.
89 Each time a token is returned, the location for the token is
90 recorded into the `Location' property, that can be accessed by
91 the parser. The parser retrieves the Location properties as
92 it builds its internal representation to allow the semantic
93 analysis phase to produce error messages that can pin point
94 the location of the problem.
96 Some tokens have values associated with it, for example when
97 the tokenizer encounters a string, it will return a
98 LITERAL_STRING token, and the actual string parsed will be
99 available in the `Value' property of the tokenizer. The same
100 mechanism is used to return integers and floating point
103 C# has a limited pre-processor that allows conditional
104 compilation, but it is not as fully featured as the C
105 pre-processor, and most notably, macros are missing. This
106 makes it simple to implement in very few lines and mesh it
109 The `handle_preprocessing_directive' method in the tokenizer
110 handles all the pre-processing, and it is invoked when the '#'
111 symbol is found as the first token in a line.
113 The state of the pre-processor is contained in a Stack called
114 `ifstack', this state is used to track the if/elif/else/endif
115 nesting and the current state. The state is encoded in the
116 top of the stack as a number of values `TAKING',
117 `TAKEN_BEFORE', `ELSE_SEEN', `PARENT_TAKING'.
121 Locations are encoded as a 32-bit number (the Location
122 struct) that map each input source line to a linear number.
123 As new files are parsed, the Location manager is informed of
124 the new file, to allow it to map back from an int constant to
125 a file + line number.
127 The tokenizer also tracks the column number for a token, but
128 this is currently not being used or encoded. It could
129 probably be encoded in the low 9 bits, allowing for columns
130 from 1 to 512 to be encoded.
134 The parser is written using Jay, which is a port of Berkeley
135 Yacc to Java, that I later ported to C#.
137 Many people ask why the grammar of the parser does not match
138 exactly the definition in the C# specification. The reason is
139 simple: the grammar in the C# specification is designed to be
140 consumed by humans, and not by a computer program. Before
141 you can feed this grammar to a tool, it needs to be simplified
142 to allow the tool to generate a correct parser for it.
144 In the Mono C# compiler, we use a class for each of the
145 statements and expressions in the C# language. For example,
146 there is a `While' class for the the `while' statement, a
147 `Cast' class to represent a cast expression and so on.
149 There is a Statement class, and an Expression class which are
150 the base classes for statements and expressions.
156 * Internal Representation
160 Expressions in the Mono C# compiler are represented by the
161 `Expression' class. This is an abstract class that particular
162 kinds of expressions have to inherit from and override a few
165 The base Expression class contains two fields: `eclass' which
166 represents the "expression classification" (from the C#
167 specs) and the type of the expression.
169 Expressions have to be resolved before they are can be used.
170 The resolution process is implemented by overriding the
171 `DoResolve' method. The DoResolve method has to set the
172 `eclass' field and the `type', perform all error checking and
173 computations that will be required for code generation at this
176 The return value from DoResolve is an expression. Most of the
177 time an Expression derived class will return itself (return
178 this) when it will handle the emission of the code itself, or
179 it can return a new Expression.
181 For example, the parser will create an "ElementAccess" class
186 During the resolution process, the compiler will know whether
187 this is an array access, or an indexer access. And will
188 return either an ArrayAccess expression or an IndexerAccess
189 expression from DoResolve.
193 *** The Expression Class
195 The utility functions that can be called by all children of
200 Constants in the Mono C# compiler are reprensented by the
201 abstract class `Constant'. Constant is in turn derived from
202 Expression. The base constructor for `Constant' just sets the
203 expression class to be an `ExprClass.Value', Constants are
204 born in a fully resolved state, so the `DoResolve' method
205 only returns a reference to itself.
207 Each Constant should implement the `GetValue' method which
208 returns an object with the actual contents of this constant, a
209 utility virtual method called `AsString' is used to render a
210 diagnostic message. The output of AsString is shown to the
211 developer when an error or a warning is triggered.
213 Constant classes also participate in the constant folding
214 process. Constant folding is invoked by those expressions
215 that can be constant folded invoking the functionality
216 provided by the ConstantFold class (cfold.cs).
218 Each Constant has to implement a number of methods to convert
219 itself into a Constant of a different type. These methods are
220 called `ConvertToXXXX' and they are invoked by the wrapper
221 functions `ToXXXX'. These methods only perform implicit
222 numeric conversions. Explicit conversions are handled by the
223 `Cast' expression class.
225 The `ToXXXX' methods are the entry point, and provide error
226 reporting in case a conversion can not be performed.
230 The C# language requires constant folding to be implemented.
231 Constant folding is hooked up in the Binary.Resolve method.
232 If both sides of a binary expression are constants, then the
233 ConstantFold.BinaryFold routine is invoked.
235 This routine implements all the binary operator rules, it
236 is a mirror of the code that generates code for binary
237 operators, but that has to be evaluated at runtime.
239 If the constants can be folded, then a new constant expression
240 is returned, if not, then the null value is returned (for
241 example, the concatenation of a string constant and a numeric
242 constant is deferred to the runtime).
251 * The semantic analysis
253 Hence, the compiler driver has to parse all the input files.
254 Once all the input files have been parsed, and an internal
255 representation of the input program exists, the following
258 * The interface hierarchy is resolved first.
259 As the interface hierarchy is constructed,
260 TypeBuilder objects are created for each one of
263 * Classes and structure hierarchy is resolved next,
264 TypeBuilder objects are created for them.
266 * Constants and enumerations are resolved.
268 * Method, indexer, properties, delegates and event
269 definitions are now entered into the TypeBuilders.
271 * Elements that contain code are now invoked to
272 perform semantic analysis and code generation.
278 The EmitContext class is created any time that IL code is to
279 be generated (methods, properties, indexers and attributes all
280 create EmitContexts).
282 The EmitContext keeps track of the current namespace and type
283 container. This is used during name resolution.
285 An EmitContext is used by the underlying code generation
286 facilities to track the state of code generation:
288 * The ILGenerator used to generate code for this
291 * The TypeContainer where the code lives, this is used
292 to access the TypeBuilder.
294 * The DeclSpace, this is used to resolve names through
295 RootContext.LookupType in the various statements and
298 Code generation state is also tracked here:
302 This variable tracks the `checked' state of the
303 compilation, it controls whether we should generate
304 code that does overflow checking, or if we generate
305 code that ignores overflows.
307 The default setting comes from the command line
308 option to generate checked or unchecked code plus
309 any source code changes using the checked/unchecked
310 statements or expressions. Contrast this with the
311 ConstantCheckState flag.
315 The constant check state is always set to `true' and
316 cant be changed from the command line. The source
317 code can change this setting with the `checked' and
318 `unchecked' statements and expressions.
322 Whether we are emitting code inside a static or
327 The value that is allowed to be returned or NULL if
328 there is no return type.
333 Points to the Type (extracted from the
334 TypeContainer) that declares this body of code
340 Whether this is generating code for a constructor
344 Tracks the current block being generated.
348 The location where return has to jump to return the
351 A few variables are used to track the state for checking in
352 for loops, or in try/catch statements:
356 Whether we are in a Finally block
360 Whether we are in a Try block
364 Whether we are in a Catch block
367 Whether we are inside an unsafe block
373 Errors are reported during the various stages of the
374 compilation process. The compiler stops its processing if
375 there are errors between the various phases. This simplifies
376 the code, because it is safe to assume always that the data
377 structures that the compiler is operating on are always
380 The error codes in the Mono C# compiler are the same as those
381 found in the Microsoft C# compiler, with a few exceptions
382 (where we report a few more errors, those are documented in
383 mcs/errors/errors.txt). The goal is to reduce confussion to
384 the users, and also to help us track the progress of the
385 compiler in terms of the errors we report.
387 The Report class provides error and warning display functions,
388 and also keeps an error count which is used to stop the
389 compiler between the phases.
391 A couple of debugging tools are available here, and are useful
392 when extending or fixing bugs in the compiler. If the
393 `--fatal' flag is passed to the compiler, the Report.Error
394 routine will throw an exception. This can be used to pinpoint
395 the location of the bug and examine the variables around the
398 Warnings can be turned into errors by using the `--werror'
399 flag to the compiler.
401 The report class also ignores warnings that have been
402 specified on the command line with the `--nowarn' flag.
404 Finally, code in the compiler uses the global variable
405 RootContext.WarningLevel in a few places to decide whether a
406 warning is worth reporting to the user or not.