Application domains are used to isolate multiple applications on a single Mono virtual machine. They are conceptually similiar to processes, the difference is that processes are managed by the operating system, while application domains are managed by the Mono virtual machine.
For more information on applications domains see the AppDomain FAQ.
The MonoDomain is the unmanaged representation of the System.AppDomain managed type, while the MonoAppDomain type represents the managed version (MonoAppDomain has a pointer to a MonoDomain).