From: Niklas Therning <niklas@therning.org>
Date: Mon, 28 Nov 2016 14:25:01 +0000 (+0100)
Subject: Use __cdecl rather than __stdcall for icalls on Windows 32-bit
X-Git-Url: http://wien.tomnetworks.com/gitweb/?a=commitdiff_plain;h=61c239fb7f55bf802cb0bfc71a8680e21e324be9;p=mono.git

Use __cdecl rather than __stdcall for icalls on Windows 32-bit

Mono assumes the same calling convention for icalls as the default P/Invoke
calling convention which is __stdcall on Windows. But none of the icalls are
marked with the required __stdcall attribute on Windows so there will be a
mismatch between the caller and callee.

This is the reason why the System.Web.HttpUtilityTest.JavaScriptStringEncode()
test crashes on Windows 32-bit. It generates a call to an icall
(char.ToString() inlines to new string(char,int)) which is assumed to be
__stdcall while it's actually __cdecl. The caller will decrement ESP as if the
callee was __stdcall resulting in ESP decreasing on each iteration. After a
couple of thousand iterations in the loop in that test the stack guard page is
hit and a stack overflow occurs.
---

diff --git a/mono/metadata/loader.c b/mono/metadata/loader.c
index 9e09748c344..0309ce8404e 100644
--- a/mono/metadata/loader.c
+++ b/mono/metadata/loader.c
@@ -2434,9 +2434,16 @@ mono_method_signature_checked (MonoMethod *m, MonoError *error)
 		mono_error_set_method_load (error, m->klass, m->name, "generic_params table claims method has generic parameters, but signature says it doesn't for method 0x%08x from image %s", idx, img->name);
 		return NULL;
 	}
-	if (m->iflags & METHOD_IMPL_ATTRIBUTE_INTERNAL_CALL)
+	if (m->iflags & METHOD_IMPL_ATTRIBUTE_INTERNAL_CALL) {
 		signature->pinvoke = 1;
-	else if (m->flags & METHOD_ATTRIBUTE_PINVOKE_IMPL) {
+#ifdef HOST_WIN32
+		/*
+		 * On Windows the default pinvoke calling convention is STDCALL but
+		 * we need CDECL since this is actually an icall.
+		 */
+		signature->call_convention = MONO_CALL_C;
+#endif
+	} else if (m->flags & METHOD_ATTRIBUTE_PINVOKE_IMPL) {
 		MonoCallConvention conv = (MonoCallConvention)0;
 		MonoMethodPInvoke *piinfo = (MonoMethodPInvoke *)m;
 		signature->pinvoke = 1;