Fix #6349: Do not use KQueue on Darwin, as their kernel behaves differently than BSD

Introduce a new define that determines whether we can use kqueue for the threadpool,
and keep the old check for kqueue, in case we need to use it in the future for something
else.

From Gonzalo's notes:

I could not reproduce the problem at all on FreeBSD, but it happened
immediately on the Mac. And when it happened, I could never kill the
process (tried kill from outside and another thread with
Environment.Exit() but no luck). This led me to believe that it was an
OS problem. Both asynchronous operations work fine when they are the
only asynchronous operation on the socket (I turned the other into
synch). When read and writes are asychronous kqueue() changes the
event we are waiting for to be read/write. That is usually not a
problem, but in this particular case, it looked like 2 separate
threads were trying to do the same thing (one for read and one for
write). I tried a simple spin lock per file-descriptor to no avail. I
also tried a fat lock and again no luck. Then I found this:
http://lists.apple.com/archives/darwin-dev/2011/Jun/msg00016.html and
then remembered that back in 2006, when I first tried a patch to
support kqueue(), Geoff found a similar problem to this one (yeah,
what the fuck, Steve!). The problem has not surfaced earlier because
most people that use sockets know how to do it. Setting up read and
write asynch. calls at the same time is not something that pros do,
but...

Solutions?

1. Provide a way to disable kqueue at runtime for cases like this (can
we set the MONO_DISABLE_AIO environment variable in an iOS mono
program? Just Environment.SetEnvironmentVariable before the first use
of sockets/threadpool should work.

2. Disable kqueue by default. The benefits of things like epoll and
kqueue are negligible, really, unless people are dealing with several
hundreds of sockets (~500 and above) and the default poll based
implementation is ok for most applications (xsp would suffer a lot of
contention under load, but xsp uses asynch ops the right way :).

diff --git a/configure.in b/configure.in
index 221eee8cd2f210b7789906cd4070f40c02372b1c..3540c558a02e01cbc8a7e818bf98f8bbef977b08 100644 (file)
--- a/configure.in
+++ b/configure.in
@@ -89,6 +89,7 @@ esac
 host_win32=no
 target_win32=no
 platform_android=no
+platform_darwin=no
 case "$host" in
        *-mingw*|*-*-cygwin*)
                AC_DEFINE(HOST_WIN32,1,[Host Platform is Win32])
@@ -1661,9 +1662,22 @@ if test x$target_win32 = xno; then
        havekqueue=no
         AC_CHECK_FUNCS(kqueue, , AC_MSG_CHECKING(for kqueue in sys/event.h)
                 AC_TRY_LINK([#include <sys/event.h>], 
-                [ kqueue(); ], 
-                AC_DEFINE(HAVE_KQUEUE, 1, [Have kqueue]) AC_MSG_RESULT(yes),
-                AC_MSG_RESULT(no)))
+                [ kqueue(); ],[havekqueue=yes],[]))
+       AC_MSG_RESULT($havekqueue);
+
+       dnl **************************************
+       dnl * Darwin has a race that prevents us from using reliably:
+       dnl * http://lists.apple.com/archives/darwin-dev/2011/Jun/msg00016.html
+       dnl * Since kqueue is mostly used for scaling large web servers, 
+       dnl * and very few folks run Mono on large web servers on OSX, falling
+       dnl * back 
+       dnl **************************************
+       if test x$havekqueue = xyes; then
+               if x$platform_darwin = xno; then
+                       AC_DEFINE(USE_KQUEUE_FOR_THREADPOOL, 1, [Use kqueue for the threadpool])
+               fi
+       fi
+
        dnl ******************************
        dnl *** Checks for SIOCGIFCONF ***
        dnl ******************************

diff --git a/mono/metadata/threadpool.c b/mono/metadata/threadpool.c
index ad2b3de06a0e155fe9558b6f2cd67bcba1f5f273..35cf47b56b6de0a960093330e1e7f0c360587e20 100644 (file)
--- a/mono/metadata/threadpool.c
+++ b/mono/metadata/threadpool.c
@@ -189,7 +189,7 @@ enum {
 #include <mono/metadata/tpool-poll.c>
 #ifdef HAVE_EPOLL
 #include <mono/metadata/tpool-epoll.c>
-#elif defined(HAVE_KQUEUE)
+#elif defined(USE_KQUEUE_FOR_THREADPOOL)
 #include <mono/metadata/tpool-kqueue.c>
 #endif
 /*
@@ -496,7 +496,7 @@ init_event_system (SocketIOData *data)
                        data->event_system = POLL_BACKEND;
                }
        }
-#elif defined(HAVE_KQUEUE)
+#elif defined(USE_KQUEUE_FOR_THREADPOOL)
        if (data->event_system == KQUEUE_BACKEND)
                data->event_data = tp_kqueue_init (data);
 #endif
@@ -525,7 +525,7 @@ socket_io_init (SocketIOData *data)
        data->sock_to_state = mono_g_hash_table_new_type (g_direct_hash, g_direct_equal, MONO_HASH_VALUE_GC);
 #ifdef HAVE_EPOLL
        data->event_system = EPOLL_BACKEND;
-#elif defined(HAVE_KQUEUE)
+#elif defined(USE_KQUEUE_FOR_THREADPOOL)
        data->event_system = KQUEUE_BACKEND;
 #else
        data->event_system = POLL_BACKEND;