mono.git
7 years agoMerge pull request #3499 from kumpera/fix_logging
monojenkins [Thu, 1 Sep 2016 02:40:12 +0000 (03:40 +0100)]
Merge pull request #3499 from kumpera/fix_logging

[logging] Ensure glib logging redirection and remove unwanted new pub api functions.

7 years ago[logging] Ensure that glib logging goes through the new logging mechanics and that...
Rodrigo Kumpera [Thu, 1 Sep 2016 00:37:35 +0000 (17:37 -0700)]
[logging] Ensure that glib logging goes through the new logging mechanics and that the domain prefix is preserved.

Call g_log_set_default_handler in mono_trace_set_log_handler and mono_trace_set_log_handler_internal to ensure that g_ logging is properly routed.

Additionally, change the internal logging callback to take the log message as a single string to simplify things.

7 years ago[logging] Move internals that were landed in public headers by mistake.
Rodrigo Kumpera [Wed, 31 Aug 2016 23:23:21 +0000 (16:23 -0700)]
[logging] Move internals that were landed in public headers by mistake.

7 years ago[runtime] Fix a warning.
Zoltan Varga [Wed, 31 Aug 2016 23:15:34 +0000 (01:15 +0200)]
[runtime] Fix a warning.

7 years ago[profiler] Fix some off-by-ones in header size calculation.
Alex Rønne Petersen [Wed, 31 Aug 2016 23:03:55 +0000 (01:03 +0200)]
[profiler] Fix some off-by-ones in header size calculation.

7 years agoMerge pull request #3493 from alexrp/master
Alex Rønne Petersen [Wed, 31 Aug 2016 23:03:41 +0000 (01:03 +0200)]
Merge pull request #3493 from alexrp/master

Follow-up PR for #3481

7 years ago[corlib] Disable CS0169 for RuntimeStructs.cs.
Zoltan Varga [Wed, 31 Aug 2016 22:59:30 +0000 (00:59 +0200)]
[corlib] Disable CS0169 for RuntimeStructs.cs.

7 years ago[llvm] Error out when using --aot=llvmonly in a runtime compiled without llvm.
Zoltan Varga [Wed, 31 Aug 2016 22:56:52 +0000 (00:56 +0200)]
[llvm] Error out when using --aot=llvmonly in a runtime compiled without llvm.

7 years ago[arm] Make mono_arch_get_call_target () work on direct branches too, previously it...
Zoltan Varga [Wed, 31 Aug 2016 22:41:12 +0000 (00:41 +0200)]
[arm] Make mono_arch_get_call_target () work on direct branches too, previously it returned NULL, which broke the code in handle_thunk () for tail calls.

7 years ago[profiler] Include arguments, architecture, and OS in format header.
Alex Rønne Petersen [Wed, 31 Aug 2016 19:12:54 +0000 (21:12 +0200)]
[profiler] Include arguments, architecture, and OS in format header.

Bumped the major/minor version to 1.0 for this change.

7 years ago[profiler] Fix dump_header () to do fflush () in all non-gzip code paths.
Alex Rønne Petersen [Wed, 31 Aug 2016 19:12:16 +0000 (21:12 +0200)]
[profiler] Fix dump_header () to do fflush () in all non-gzip code paths.

7 years ago[runtime] Export API functions to get arch/os/wordsize strings.
Alex Rønne Petersen [Wed, 31 Aug 2016 19:09:56 +0000 (21:09 +0200)]
[runtime] Export API functions to get arch/os/wordsize strings.

7 years ago[profiler] Remove duplicate #include.
Alex Rønne Petersen [Wed, 31 Aug 2016 17:39:00 +0000 (19:39 +0200)]
[profiler] Remove duplicate #include.

7 years ago[profiler] Update documentation for time fields.
Alex Rønne Petersen [Wed, 31 Aug 2016 11:06:33 +0000 (13:06 +0200)]
[profiler] Update documentation for time fields.

7 years ago[profiler] Document the TYPE_EXCEPTION format.
Alex Rønne Petersen [Wed, 31 Aug 2016 08:26:34 +0000 (10:26 +0200)]
[profiler] Document the TYPE_EXCEPTION format.

7 years ago[profiler] Clean up some enums a bit. No format change.
Alex Rønne Petersen [Wed, 31 Aug 2016 08:11:09 +0000 (10:11 +0200)]
[profiler] Clean up some enums a bit. No format change.

7 years ago[profiler] Update documentation on the backtrace format.
Alex Rønne Petersen [Wed, 31 Aug 2016 07:49:19 +0000 (09:49 +0200)]
[profiler] Update documentation on the backtrace format.

7 years ago[profiler] Pump the hazard free queue when shutting down.
Alex Rønne Petersen [Wed, 31 Aug 2016 07:39:42 +0000 (09:39 +0200)]
[profiler] Pump the hazard free queue when shutting down.

7 years ago[utils/hp] Export mono_thread_hazardous_try_free_all () for use in the profiler.
Alex Rønne Petersen [Wed, 31 Aug 2016 07:35:45 +0000 (09:35 +0200)]
[utils/hp] Export mono_thread_hazardous_try_free_all () for use in the profiler.

7 years ago[utils/hp] Keep pumping the free queue even when we encounter hazardous items.
Alex Rønne Petersen [Wed, 31 Aug 2016 07:34:04 +0000 (09:34 +0200)]
[utils/hp] Keep pumping the free queue even when we encounter hazardous items.

Previously, a single item being hazardous for a while could completely block
pumping of the queue. This fixes that.

7 years ago[runtime] Fix the method stored in the WrapperInfo for delegate-invoke wrappers for...
Zoltan Varga [Wed, 31 Aug 2016 22:25:21 +0000 (00:25 +0200)]
[runtime] Fix the method stored in the WrapperInfo for delegate-invoke wrappers for shared methods. Fixes #43696.

7 years agoMerge pull request #3473 from ntherning/fix-thread-double-close-handle-on-appdomain...
Niklas Therning [Wed, 31 Aug 2016 20:08:29 +0000 (22:08 +0200)]
Merge pull request #3473 from ntherning/fix-thread-double-close-handle-on-appdomain-unload

Fix for double CloseHandle() on AppDomain unload

7 years agoMerge pull request #3496 from ntherning/fix-heap-corruption-on-windows
Niklas Therning [Wed, 31 Aug 2016 18:22:13 +0000 (20:22 +0200)]
Merge pull request #3496 from ntherning/fix-heap-corruption-on-windows

Fixes two heap corruptions in the corlib tests on Windows

7 years agoMerge pull request #3395 from lambdageek/dev/handles-strings
Aleksey Kliger (λgeek) [Wed, 31 Aug 2016 17:23:52 +0000 (13:23 -0400)]
Merge pull request #3395 from lambdageek/dev/handles-strings

[runtime] Use coop handles in some String-returning icalls

7 years agoFixes two heap corruptions in the corlib tests on Windows
Niklas Therning [Wed, 31 Aug 2016 16:40:41 +0000 (18:40 +0200)]
Fixes two heap corruptions in the corlib tests on Windows

The patch of mono/metadata/marshal.c makes sure that
Marshal.StringToHGlobalAnsi() allocates a memory area for the returned C
string which is at least as large as the length of the string object. This is
the behavior on non-Windows platforms. Without this patch the
MarshalTest.StringToHGlobalAnsiWithNullValues() test wrote outside of a memory
area and corrupted the heap which later on caused a heap corruption crash when
running the corlib test suite on Windows.

The patch of mono/metadata/mono-security.c fixes the freeing of the memory
returned by GetNamedSecurityInfoW(). When GetNamedSecurityInfoW() is used to
request a DACL one must also pass an argument for the ppSecurityDescriptor
parameter and this is the one that should be freed later on, not the pointer
to the DACL as the original code used to free. Before this patch
DSACryptoServiceProviderTest.UseMachineKeyStore() crashed because of a heap
corruption.

7 years agoBump cecil
Marek Safar [Wed, 31 Aug 2016 14:10:56 +0000 (16:10 +0200)]
Bump cecil

7 years agoMerge pull request #3494 from akoeplinger/netstd-fixes
Alexander Köplinger [Wed, 31 Aug 2016 11:52:49 +0000 (13:52 +0200)]
Merge pull request #3494 from akoeplinger/netstd-fixes

Expose a few more types/methods for netstandard

7 years ago[corlib] Add Environment APIs that take EnvironmentVariableTarget on mobile
Alexander Köplinger [Wed, 31 Aug 2016 11:33:53 +0000 (13:33 +0200)]
[corlib] Add Environment APIs that take EnvironmentVariableTarget on mobile

We only do something when target is "Process", this matches CoreFX behavior.

7 years ago[System.Core] Add a couple of System.IO.Pipes classes to the mobile profiles
Alexander Köplinger [Wed, 31 Aug 2016 11:15:14 +0000 (13:15 +0200)]
[System.Core] Add a couple of System.IO.Pipes classes to the mobile profiles

These are required for netstandard parity.

7 years ago[System] HttpWebRequest::Headers setter was only setting local variable. Fixes #42408
Marek Safar [Wed, 31 Aug 2016 10:11:30 +0000 (12:11 +0200)]
[System] HttpWebRequest::Headers setter was only setting local variable. Fixes #42408

7 years agoMerge pull request #3470 from lewurm/fix-aot-compiler-toolchain-ld-invoke
Bernhard Urban [Wed, 31 Aug 2016 09:45:58 +0000 (11:45 +0200)]
Merge pull request #3470 from lewurm/fix-aot-compiler-toolchain-ld-invoke

[aot] fix linker invocation

7 years agoMerge pull request #3487 from marek-safar/rs-Threading
Marek Safar [Wed, 31 Aug 2016 05:57:02 +0000 (07:57 +0200)]
Merge pull request #3487 from marek-safar/rs-Threading

[corlib] Flow CultureInfo from parent thread. Fixes #43193

7 years agoMerge pull request #3489 from akoeplinger/fix-latin1-surrogatepair
Marek Safar [Wed, 31 Aug 2016 05:52:38 +0000 (07:52 +0200)]
Merge pull request #3489 from akoeplinger/fix-latin1-surrogatepair

[corlib] Fix replacement of UTF-16 surrogate pairs in Latin1Encoding

7 years agoOnly set the attached flag when a client attaches (#3490)
Zoltan Varga [Wed, 31 Aug 2016 00:56:54 +0000 (02:56 +0200)]
Only set the attached flag when a client attaches (#3490)

In defer mode, the debugger thread is started as soon as possible before
any clients are attached, but the debugger is_attached flag was being set
immediately. This would cause the runtime (and the System.Diagnostics.Debugger
class) to wrongly report a client as being attached.

Originally fixed in the Unity fork here: https://github.com/Unity-Technologies/mono/commit/797a210e0f2416d779c448bb9e03bb16933e261f

7 years agoOnly set the attached flag when a client attaches
Josh Peterson [Tue, 30 Aug 2016 15:00:32 +0000 (11:00 -0400)]
Only set the attached flag when a client attaches

In defer mode, the debugger thread is started as soon as possible before
any clients are attached, but the debugger is_attached flag was being set
immediately. This would cause the runtime (and the System.Diagnostics.Debugger
class) to wrongly report a client as being attached.

Originally fixed in the Unity fork here: https://github.com/Unity-Technologies/mono/commit/797a210e0f2416d779c448bb9e03bb16933e261f

7 years agoMerge pull request #3488 from Unity-Technologies/threadpool-unhandled-exception-polic...
Rodrigo Kumpera [Tue, 30 Aug 2016 22:44:20 +0000 (15:44 -0700)]
Merge pull request #3488 from Unity-Technologies/threadpool-unhandled-exception-policy-legacy

Always clear thread abort exception for internal/threadpool threads

7 years agoMerge pull request #3481 from alexrp/master
Alex Rønne Petersen [Tue, 30 Aug 2016 22:23:43 +0000 (00:23 +0200)]
Merge pull request #3481 from alexrp/master

Even more profiler fixes/improvements

7 years agoMerge pull request #3464 from alexrp/arm64-native-atomics
Alex Rønne Petersen [Tue, 30 Aug 2016 19:47:14 +0000 (21:47 +0200)]
Merge pull request #3464 from alexrp/arm64-native-atomics

[runtime] Fix C and LLVM atomics on ARM64

7 years ago[utils] Remove ia64-specific atomics/membar code.
Alex Rønne Petersen [Sun, 28 Aug 2016 15:12:19 +0000 (17:12 +0200)]
[utils] Remove ia64-specific atomics/membar code.

All ia64 compilers have intrinsics that we use. Also, this ia64 code was very
out of date (missing many functions we use now) so it wouldn't compile anyway.

7 years ago[llvm] Fix atomic opcodes on ARM64 to emit correct fences.
Alex Rønne Petersen [Sun, 28 Aug 2016 14:52:21 +0000 (16:52 +0200)]
[llvm] Fix atomic opcodes on ARM64 to emit correct fences.

This is a workaround. It's needed for the same reason that we need the fix for
Clang's atomic intrinsics in mono/utils/atomic.h.

7 years ago[utils/atomic] Fix native atomic intrinsics on buggy ARM64 compilers.
Alex Rønne Petersen [Sun, 28 Aug 2016 12:50:38 +0000 (14:50 +0200)]
[utils/atomic] Fix native atomic intrinsics on buggy ARM64 compilers.

See the comment in atomic.h for details.

7 years ago[utils] Add a MONO_GNUC_VERSION macro.
Alex Rønne Petersen [Sun, 28 Aug 2016 12:50:09 +0000 (14:50 +0200)]
[utils] Add a MONO_GNUC_VERSION macro.

7 years ago[corlib] Flow CultureInfo from parent thread. Fixes #43193
Marek Safar [Tue, 30 Aug 2016 15:00:35 +0000 (17:00 +0200)]
[corlib] Flow CultureInfo from parent thread. Fixes #43193

7 years ago[corlib] Fix replacement of UTF-16 surrogate pairs in Latin1Encoding
Alexander Köplinger [Tue, 30 Aug 2016 18:25:37 +0000 (20:25 +0200)]
[corlib] Fix replacement of UTF-16 surrogate pairs in Latin1Encoding

When encoding a UTF-16 C# string into bytes using "latin-1"/"iso-8859-1" encoding we were
not properly taking surrogate pairs into account. MSDN[1] provides a good introduction about the topic.

For example, this string would fail to encode and throw an IndexOutOfRangeException: "\ud83d\ude0a "
However, this string would not: "\ud83d\ude0a" (note missing space at the end). More on that later.

In https://bugzilla.xamarin.com/show_bug.cgi?id=42887 it was noted that encoding the string
didn't cause the exception on Mono 2.10.8.1 and 3.2.8. Indeed, it doesn't happen until Mono 4.0
where we switched most of the string code to referencesources.

While looking at the output of `Encoding.GetEncoding("latin1").GetBytes("\ud83d\ude0a ")` on the
"working" Mono verions I noticed that they essentially returned wrong results, e.g.:

```
csharp> Encoding.GetEncoding("latin1").GetBytes("\ud83d\ude0a ")
{ 63, 32, 0 }
```

This is wrong, because a) there's only one 63 (i.e. the '?' question mark char which is used as
a replacement char whenever the target encoding doesn't support a specific char,
and b) there is a trailing '\0' which should not be there.

Checking on .NET confirms the expected result: { 63, 63, 32 } i.e. '?? '

Back to the IndexOutOfRangeException in Mono 4.0+: digging into Latin1Encoding.InternalGetBytes() it
turns out that when we detect a surrogate pair we were incrementing charIndex, but we didn't decrement count.
This means that after the fallback characters (the two '?') were added we would still try to encode one more
char in the loop even though we're already at the end of the string, resulting in the exception.
For the "\ud83d\ude0a" example we would not hit the `count > 1` code path which made it work in this case.

Adding the decrement fixes the bug and makes the code consistent with MonoEncoding.cs[2].

The reason why it "worked" in earlier Mono versions was that we had a very simple EncoderReplacementFallbackBuffer
implementation[3] which only output a single '?' for the surrogate pair:

```
csharp> var buf = Encoding.GetEncoding("latin1").EncoderFallback.CreateFallbackBuffer()
csharp> buf.Fallback((char)0xD800, (char)0xDC00, 0)
true
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'\x0

csharp> buf.Fallback((char)0xD800, (char)0xDC00, 0)
true
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'?'
csharp> buf.GetNextChar()
'\x0'
```

This means that for the "\ud83d\ude0a " example from above we would only add one '?' character
for the surrogate pair to the result buffer, then the space and then we'd be done, leaving the
uninitialized array value 0 in the buffer on index 2.

I added tests for Latin1Encoding, mostly copied from ASCIIEncoding with additions for these
specific cases.

Fixes https://bugzilla.xamarin.com/show_bug.cgi?id=42887

[1] https://msdn.microsoft.com/en-us/library/ms404377(v=vs.110).aspx
[2] https://github.com/mono/mono/blob/5b69e839e12b3d3512f310aa5a0e873346d7e50e/mcs/class/I18N/Common/MonoEncoding.cs#L62-L66
[3] https://github.com/mono/mono/blob/mono-3.12.0-branch/mcs/class/corlib/System.Text/EncoderReplacementFallbackBuffer.cs

7 years agoMerge pull request #3477 from Unity-Technologies/create-delegate-exception
Marek Safar [Tue, 30 Aug 2016 17:05:29 +0000 (19:05 +0200)]
Merge pull request #3477 from Unity-Technologies/create-delegate-exception

Throw an ArgumentException when a delegate is bound to an open generi…

7 years agoReset thread abort exception for internal threads regardless of unhandled exception...
Jonathan Chambers [Tue, 30 Aug 2016 15:32:53 +0000 (11:32 -0400)]
Reset thread abort exception for internal threads regardless of unhandled exception policy.

7 years agoMake improvments after code review:
Josh Peterson [Tue, 30 Aug 2016 14:46:02 +0000 (10:46 -0400)]
Make improvments after code review:

* Use mono_set_pending_exception instead of mono_raise_exception
* Make the test an nunit test instead of a runtime test.

7 years ago[mono-threads] Use create_suspended_sem on windows (#3482)
Ludovic Henry [Tue, 30 Aug 2016 10:45:53 +0000 (12:45 +0200)]
[mono-threads] Use create_suspended_sem on windows (#3482)

7 years agoMerge pull request #3483 from alexavator/bugfix-keepalive
Marek Safar [Tue, 30 Aug 2016 09:02:43 +0000 (11:02 +0200)]
Merge pull request #3483 from alexavator/bugfix-keepalive

[System] Set KeepAlive flag for HttpWebRequest

7 years ago[System] Set KeepAlive flag for HttpWebRequest
Pozdeev [Tue, 30 Aug 2016 08:36:21 +0000 (13:36 +0500)]
[System] Set KeepAlive flag for HttpWebRequest

7 years ago[pedump] Fix assertion error
Ludovic Henry [Tue, 30 Aug 2016 08:24:12 +0000 (10:24 +0200)]
[pedump] Fix assertion error

We would previously observe the following crash:

```
ludovic-laptop :: ~/Xamarin/mono ‹16c07b5› » (cd tools/pedump && MONO_PATH=../../mcs/class/lib/net_4_x lldb -- ./pedump --verify metadata,code ../.././mono/tests/xdomain-threads.exe)
(lldb) target create "./pedump"
Current executable set to './pedump' (x86_64).
(lldb) settings set -- target.run-args  "--verify" "metadata,code" "../.././mono/tests/xdomain-threads.exe"
(lldb) r
Process 14407 launched: './pedump' (x86_64)
mono_os_mutex_lock: pthread_mutex_lock failed with "Invalid argument" (22)
Process 14407 stopped
* thread #1: tid = 0x27c9715, 0x00007fff93cfcf06 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
    frame #0: 0x00007fff93cfcf06 libsystem_kernel.dylib`__pthread_kill + 10
libsystem_kernel.dylib`__pthread_kill:
->  0x7fff93cfcf06 <+10>: jae    0x7fff93cfcf10            ; <+20>
    0x7fff93cfcf08 <+12>: movq   %rax, %rdi
        0x7fff93cfcf0b <+15>: jmp    0x7fff93cf77cd            ; cerror_nocancel
    0x7fff93cfcf10 <+20>: retq
(lldb) bt
* thread #1: tid = 0x27c9715, 0x00007fff93cfcf06 libsystem_kernel.dylib`__pthread_kill + 10, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00007fff93cfcf06 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff8b9b74ec libsystem_pthread.dylib`pthread_kill + 90
frame #2: 0x00007fff908d56e7 libsystem_c.dylib`abort + 129
frame #3: 0x000000010022da6a pedump`monoeg_log_default_handler(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, message="mono_os_mutex_lock: pthread_mutex_lock failed with \"Invalid argument\" (22)", unused_data=0x0000000000000000) + 202 at goutput.c:231
frame #4: 0x000000010022d98d pedump`monoeg_g_logv(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, format="%s: pthread_mutex_lock failed with \"%s\" (%d)", args=0x00007fff5fbff4a0) + 109 at goutput.c:111
frame #5: 0x000000010022dbd9 pedump`monoeg_g_log(log_domain=0x0000000000000000, log_level=G_LOG_LEVEL_ERROR, format="%s: pthread_mutex_lock failed with \"%s\" (%d)") + 361 at goutput.c:121
frame #6: 0x000000010022301e pedump`mono_os_mutex_lock(mutex=0x00000001002ec630) + 110 at mono-os-mutex.h:98
frame #7: 0x0000000100223a66 pedump`mono_w32handle_new(type=MONO_W32HANDLE_PROCESS, handle_specific=0x00007fff5fbff540) + 246 at w32handle.c:440
frame #8: 0x00000001001ef8bb pedump`_wapi_processes_init + 123 at processes.c:1149
frame #9: 0x00000001001fbcc3 pedump`wapi_init + 19 at wapi.c:23
frame #10: 0x0000000100116486 pedump`mono_init_internal(filename="pedump", exe_filename=0x0000000000000000, runtime_version="v4.0.30319") + 102 at domain.c:528
frame #11: 0x0000000100117124 pedump`mono_init_version(domain_name="pedump", version="v4.0.30319") + 36 at domain.c:868
frame #12: 0x0000000100001496 pedump`verify_image_file(fname="../.././mono/tests/xdomain-threads.exe") + 678 at pedump.c:466
frame #13: 0x0000000100000f87 pedump`main(argc=4, argv=0x00007fff5fbff9a0) + 1095 at pedump.c:708
frame #14: 0x00007fff93e655ad libdyld.dylib`start + 1
```

That would be because the w32handle wouldn't be initialized before calling into `wapi_init` which allocates w32handle.

7 years agoMerge pull request #3476 from lambdageek/fix-conc-ht
Ludovic Henry [Tue, 30 Aug 2016 08:04:32 +0000 (10:04 +0200)]
Merge pull request #3476 from lambdageek/fix-conc-ht

[test-conc-hashtable] Init w32 handles. Fixes OSX+coop.

7 years ago[profiler] Reject heapshot requests if we're not in on-demand mode.
Alex Rønne Petersen [Sun, 28 Aug 2016 18:25:10 +0000 (20:25 +0200)]
[profiler] Reject heapshot requests if we're not in on-demand mode.

Previously, we would try to satisfy the heapshot request every single time
process_requests () was invoked, but since heap_walk () would never unset
heapshot_requested, we'd end up doing GCs over and over, causing massive
overhead.

7 years ago[profiler] Process on-demand heapshot requests on the finalizer thread.
Alex Rønne Petersen [Tue, 30 Aug 2016 07:59:17 +0000 (09:59 +0200)]
[profiler] Process on-demand heapshot requests on the finalizer thread.

This way, we avoid attempting to fully attached the helper thread when it's
already a tools thread, which can lead to all sorts of problems.

7 years ago[profiler] Add GC finalization events to the format.
Alex Rønne Petersen [Tue, 30 Aug 2016 07:57:14 +0000 (09:57 +0200)]
[profiler] Add GC finalization events to the format.

7 years ago[gc] Expose mono_gc_pending_finalizers () and mono_gc_finalize_notify () as public...
Alex Rønne Petersen [Sun, 28 Aug 2016 17:41:18 +0000 (19:41 +0200)]
[gc] Expose mono_gc_pending_finalizers () and mono_gc_finalize_notify () as public API.

We need these in the profiler, but they are also generally useful functions
that embedders may very well want to use.

7 years ago[profiler] Add GC finalization callbacks to the profiler API.
Alex Rønne Petersen [Sun, 28 Aug 2016 17:22:22 +0000 (19:22 +0200)]
[profiler] Add GC finalization callbacks to the profiler API.

7 years ago[profiler] Initialize counters in the runtime_initialized () callback.
Alex Rønne Petersen [Sun, 28 Aug 2016 18:38:51 +0000 (20:38 +0200)]
[profiler] Initialize counters in the runtime_initialized () callback.

This way, we don't have to depend on the counters subsystem having been
initialized when the profiler library is loaded (very early).

7 years ago[profiler] Normalize time fields in the format.
Alex Rønne Petersen [Tue, 30 Aug 2016 06:32:50 +0000 (08:32 +0200)]
[profiler] Normalize time fields in the format.

All events now come with a time field right after the event byte, and it is
encoded in the same way for all events (based on LogBuffer.time_base).

This allows readers to queue up events until a sync point event, then sort all
of those events by time, and process them in the order they were actually
written in.

7 years ago[profiler] Make the comment in proflog.c only represent the latest version of the...
Alex Rønne Petersen [Sun, 28 Aug 2016 10:36:40 +0000 (12:36 +0200)]
[profiler] Make the comment in proflog.c only represent the latest version of the format.

Maintaining it for every format version is getting unwieldy and most readers
likely won't care about ancient format versions. Further, we haven't really
stabilized the format yet, so documenting every iteration of it doesn't make
a whole lot of sense. Finally, if somebody does care about older versions,
they can simply look in decode.c to see how mprof-report deals with them.

7 years ago[profiler] Fix a few format bloat and encoding issues since we're bumping the format...
Alex Rønne Petersen [Sun, 28 Aug 2016 09:13:08 +0000 (11:13 +0200)]
[profiler] Fix a few format bloat and encoding issues since we're bumping the format version.

* Methods in backtraces are now properly encoded based on the method_base
  rather than the ptr_base.
* The unused IL and native offset fields in TYPE_SAMPLE_HIT's backtrace frame
  format have been removed.
* The unused flags fields in the backtrace format and in metadata events have
  been removed.
* Changed a number of fields to be encoded as a single byte instead of LEB128.

7 years ago[profiler] Emit sync point events when flushing all buffers.
Alex Rønne Petersen [Wed, 17 Aug 2016 05:50:24 +0000 (07:50 +0200)]
[profiler] Emit sync point events when flushing all buffers.

These events basically establish points in the event stream where all previous
events can be processed and a complete view of the profiling state can be
formed. For instance, if an allocation event for object A of type T was seen
before a sync point, then it's guaranteed that the class load event for T also
came before that sync point (although it may have arrived after the allocation
event for A).

7 years ago[profiler] Acquire the exclusive buffer lock after the suspend lock.
Alex Rønne Petersen [Wed, 17 Aug 2016 05:01:25 +0000 (07:01 +0200)]
[profiler] Acquire the exclusive buffer lock after the suspend lock.

This prevents possible STW deadlocks.

7 years ago[profiler] Only flush buffers when they exceed the base capacity.
Alex Rønne Petersen [Wed, 3 Aug 2016 12:28:43 +0000 (14:28 +0200)]
[profiler] Only flush buffers when they exceed the base capacity.

Other than timely arrival, there was never really a need to force buffers to be
flushed at these arbitrary points. Now that we flush all buffers periodically,
that reason is no longer relevant.

7 years ago[profiler] Process requests in enter/leave callbacks regardless of call depth.
Alex Rønne Petersen [Wed, 3 Aug 2016 11:58:03 +0000 (13:58 +0200)]
[profiler] Process requests in enter/leave callbacks regardless of call depth.

Heapshots have nothing to do with call depth, so they should be processed in
any case.

Also flush the local buffer in all enter/leave callbacks if needed.

7 years ago[profiler] Periodically flush all thread-local buffers.
Alex Rønne Petersen [Thu, 21 Jul 2016 09:24:05 +0000 (11:24 +0200)]
[profiler] Periodically flush all thread-local buffers.

This ensures that even for low-activity threads, it won't take ages before the
events from it arrive. We do this flush every 1sec (as with sampling of perf
counters).

7 years ago[profiler] Implement global flush points for thread-local buffers.
Alex Rønne Petersen [Wed, 20 Jul 2016 16:21:53 +0000 (18:21 +0200)]
[profiler] Implement global flush points for thread-local buffers.

The problem is this: Given an allocation of object O on address A1 in thread T1
and a subsequent GC move of O from A1 to A2 in T2, if T2's event buffer is
flushed before T1's buffer, the reader of the log file will a seemingly
nonexistent object being moved, and then see the actual allocation of that
object later. This makes it extremely difficult to keep track of the managed
heap.

The way we solve this is by flushing all thread-local buffers when the GC stops
the world. This way, we know that all prior allocations in the program have
been committed to the log file before any moves that are about to follow as
part of the collection. Similarly, we flush the GC thread's buffer before
restarting the world to ensure that all moves are committed before allocations
that may happen later (which could use some of the addresses freed up for use
due to the moves).

This is reasonably easy now that we have a global LLS of threads which are
attached to the profiler. However, we have to use some kind of lock to ensure
that buffers aren't being written to while we're trying to flush them, or that
a suspended thread was in the middle of modifying its buffer. For this, we use
a reader/writer spin lock of sorts, somewhat similar to the lock used by the
SGen binary protocol. When threads are modifying their own buffer (whether it
be writing to it or flushing it), they take the reader lock. This means that
many different threads can be operating on their own buffers without any lock
contention (which would kinda defeat the purpose of the thread-local buffers).
Now, when a thread -- such as the thread that initiated a GC -- wants to access
the buffers of other threads, it takes the writer lock. In doing so, all other
threads are prevented from modifying their own buffers. This gives the thread
holding the writer lock an opportunity to flush the buffers of other threads.

7 years ago[utils/threads] Export mono_thread_info_yield () for use in the profiler.
Alex Rønne Petersen [Wed, 20 Jul 2016 13:48:01 +0000 (15:48 +0200)]
[utils/threads] Export mono_thread_info_yield () for use in the profiler.

7 years ago[profiler] Remove LogBuffer.locked and related debugging code.
Alex Rønne Petersen [Wed, 20 Jul 2016 01:39:47 +0000 (03:39 +0200)]
[profiler] Remove LogBuffer.locked and related debugging code.

This hasn't been needed for a long time.

7 years ago[profiler] Rewrite TLS code.
Alex Rønne Petersen [Tue, 19 Jul 2016 10:35:01 +0000 (12:35 +0200)]
[profiler] Rewrite TLS code.

* Actual thread-local state is now stored in a MonoProfilerThread struct which
  is stored in a single TLS slot per thread. This avoids storing things like
  call_depth in LogBuffer. The current LogBuffer of a thread is also stored in
  the MonoProfilerThread.
* All MonoProfilerThread structs are stored in a global linked list set. This
  LLS is used to manually clean up the thread-local state of threads that, for
  whatever reason, don't have the thread_end () callback invoked.
* Lifetime management of thread-local buffers is now much more straightforward.
* The thread-local list of referenced methods is now only allocated if needed.
* Fixed a MethodInfo memory leak in handle_writer_queue_entry ().
* Fixed some class load events that could be lost in the writer thread when
  fetching the full name of a method.
* TLS slots are now actually freed on shutdown.

7 years ago[profiler] Implement process_id () for Windows.
Alex Rønne Petersen [Tue, 19 Jul 2016 04:13:59 +0000 (06:13 +0200)]
[profiler] Implement process_id () for Windows.

7 years ago[utils/lls] Export the linked list set API for use in the profiler.
Alex Rønne Petersen [Tue, 19 Jul 2016 02:57:57 +0000 (04:57 +0200)]
[utils/lls] Export the linked list set API for use in the profiler.

Also exported a couple of hazard pointer functions used by the LLS macros.

7 years ago[utils/lls] Use mono_thread_hazardous_queue_free () instead of mono_thread_hazardous_...
Alex Rønne Petersen [Tue, 19 Jul 2016 02:52:55 +0000 (04:52 +0200)]
[utils/lls] Use mono_thread_hazardous_queue_free () instead of mono_thread_hazardous_try_free ().

We cannot use the latter as we do not know whether the free function is safe to call.

7 years ago[utils/lls] Rename get_hazardous_pointer_with_mask () to mono_lls_get_hazardous_point...
Alex Rønne Petersen [Tue, 19 Jul 2016 02:51:03 +0000 (04:51 +0200)]
[utils/lls] Rename get_hazardous_pointer_with_mask () to mono_lls_get_hazardous_pointer_with_mask ().

7 years ago[utils/hp] Rename get_hazardous_pointer () to mono_get_hazardous_pointer ().
Alex Rønne Petersen [Tue, 19 Jul 2016 02:50:44 +0000 (04:50 +0200)]
[utils/hp] Rename get_hazardous_pointer () to mono_get_hazardous_pointer ().

7 years ago[utils/hp] Remove HazardFreeLocking and HazardFreeContext enums.
Alex Rønne Petersen [Tue, 19 Jul 2016 02:43:21 +0000 (04:43 +0200)]
[utils/hp] Remove HazardFreeLocking and HazardFreeContext enums.

These are no longer useful in the new hazard free API.

7 years ago[profiler] Only forward declare structs that need it.
Alex Rønne Petersen [Tue, 19 Jul 2016 02:18:50 +0000 (04:18 +0200)]
[profiler] Only forward declare structs that need it.

7 years agoRevert "[runtime] Fix pedump by using the normal embedding APIs instead of in…" ...
Zoltan Varga [Mon, 29 Aug 2016 22:01:18 +0000 (00:01 +0200)]
Revert "[runtime] Fix pedump by using the normal embedding APIs instead of in…" (#3479)

7 years agoMerge pull request #3478 from vargaz/fix-pedump
Rodrigo Kumpera [Mon, 29 Aug 2016 21:51:30 +0000 (17:51 -0400)]
Merge pull request #3478 from vargaz/fix-pedump

[runtime] Fix pedump by using the normal embedding APIs instead of in…

7 years ago[runtime] Fix pedump by using the normal embedding APIs instead of initializing parts...
Zoltan Varga [Mon, 29 Aug 2016 20:37:22 +0000 (22:37 +0200)]
[runtime] Fix pedump by using the normal embedding APIs instead of initializing parts of the runtime. Fixes #43786.

7 years agoMerge pull request #3422 from xmcclure/tarjan-doublefan
monojenkins [Mon, 29 Aug 2016 20:30:06 +0000 (22:30 +0200)]
Merge pull request #3422 from xmcclure/tarjan-doublefan

Make GC bridge perform well in the "double fan" scenario

The GC bridge currently performs VERY poorly when the object graph has a "double fan" shape. If M Java objects point to 1 C# object which points to N Java objects, the graph exported to Monodroid will currently drop the C# object node and replace it with (M\*N) edges. It is easy to write code where (M\*N) grows ludicrously big.

In testing on device, this patch solves that problem while leaving other cases unaffected. In testing on device, I found:
* All performance tests except double fan: No discernible performance difference after patch
* Double fan test with M=N=1000: Before the patch this took between three and seven seconds. After the patch this took between 0.2 and 0.3 seconds.
* Double fan test with M=N=4000: Before the patch this took anywhere from one to two minutes. After the patch, this took about 1.2 seconds.
* Double fan test with M=N=6000: Before the patch, this took so long I was not able to get it to ever complete. After the patch, this took about 1.3 seconds.
* Double fan test with M=N=20000: I did not attempt this test pre-patch. After the patch, it took about 5.6 seconds.

The commit messages describe the changes in some detail but the short version is:
* Add a mechanism for exporting non-bridged SCCs to the bridge client
* Turn that mechanism on whenever the product fanin*fanout grows too large
* Make the merge cache more aggressive

To function, this patch requires a corresponding change to monodroid. See https://github.com/xamarin/xamarin-android/pull/154

7 years ago[Facades] Remove private facade
Marek Safar [Mon, 29 Aug 2016 19:40:50 +0000 (21:40 +0200)]
[Facades] Remove private facade

7 years agoMerge pull request #3463 from david-mitchell/al-platform-switch
Marek Safar [Mon, 29 Aug 2016 19:31:33 +0000 (21:31 +0200)]
Merge pull request #3463 from david-mitchell/al-platform-switch

Add missing switch and functionality to al.

7 years agoGC bridge: Fix non-buildable assert ()s added in previous commits
Andi McClure [Mon, 29 Aug 2016 18:34:22 +0000 (14:34 -0400)]
GC bridge: Fix non-buildable assert ()s added in previous commits

7 years agoMerge pull request #3468 from alexrp/hwcap-improvements
Alex Rønne Petersen [Mon, 29 Aug 2016 18:26:46 +0000 (20:26 +0200)]
Merge pull request #3468 from alexrp/hwcap-improvements

[utils/hwcap] Clean up and improve the hwcap code.

7 years agoGC bridge: Move bridge-require-precise-merge to MONO_GC_PARAMS
Andi McClure [Mon, 29 Aug 2016 17:48:48 +0000 (13:48 -0400)]
GC bridge: Move bridge-require-precise-merge to MONO_GC_PARAMS

This was previously in MONO_GC_DEBUG, but we do not use this for
production-viable flags.

7 years ago[corlib] Improve CanceledContinuationExecuteSynchronouslyTest test
Marek Safar [Mon, 29 Aug 2016 16:39:14 +0000 (18:39 +0200)]
[corlib] Improve CanceledContinuationExecuteSynchronouslyTest test

7 years agoThrow an ArgumentException when a delegate is bound to an open generic method.
Josh Peterson [Fri, 26 Aug 2016 18:54:27 +0000 (14:54 -0400)]
Throw an ArgumentException when a delegate is bound to an open generic method.

* This change was made in Unity's old version of Mono at: https://github.com/Unity-Technologies/mono/commit/9e27edaec9a91a75d6ebf0d21654dedcab74653d

7 years agoAdd missing switch and functionality to al.
David Mitchell [Fri, 26 Aug 2016 23:24:32 +0000 (16:24 -0700)]
Add missing switch and functionality to al.

When running msbuild on mono against an architecture-specific
project that also defines satellite assemblies, the /platform
switch will be passed to al.exe. Mono's implementation of al.exe
does not support this switch, which results in a failed build.

This commit adds the /platform switch, which also required
changing the implementation from System.Reflection to
IKVM.Reflection because Mono's implementation of AssemblyBuilder
does not support the necessary options.

While testing this change, I also noticed a few attributes that
the MS version of al.exe will copy from "template" assemblies,
so I added support for those, as well.

7 years ago[runtime] Use coop handles in some String-returning icalls
Aleksey Kliger [Thu, 11 Aug 2016 20:54:53 +0000 (16:54 -0400)]
[runtime] Use coop handles in some String-returning icalls

Also begin adding handle definitions for some System.Reflection classes
when needed.

7 years ago[mcs] Add reference to parent storey when current value type async storey needs fabri...
Marek Safar [Mon, 29 Aug 2016 16:01:39 +0000 (18:01 +0200)]
[mcs] Add reference to parent storey when current value type async storey needs fabricated this access. Fixes #40860

7 years ago[test-conc-hashtable] Init w32 handles. Fixes OSX+coop.
Aleksey Kliger [Mon, 29 Aug 2016 15:57:31 +0000 (11:57 -0400)]
[test-conc-hashtable] Init w32 handles. Fixes OSX+coop.

7 years ago[llvm] Fix the arm cross compiler build when using llvm 3.9.
Zoltan Varga [Mon, 29 Aug 2016 14:49:05 +0000 (16:49 +0200)]
[llvm] Fix the arm cross compiler build when using llvm 3.9.

7 years agoMerge pull request #3453 from ntherning/cleanup-AssemblyTest-temp-file-handling
Marek Safar [Mon, 29 Aug 2016 14:46:04 +0000 (16:46 +0200)]
Merge pull request #3453 from ntherning/cleanup-AssemblyTest-temp-file-handling

Cleanup temp file handling in AssemblyTest

7 years agoCleanup temp file handling in AssemblyTest
Niklas Therning [Thu, 25 Aug 2016 15:33:14 +0000 (17:33 +0200)]
Cleanup temp file handling in AssemblyTest

SetUp() already sets up a temp folder for the test methods and TearDown()
handles cleaning up and taking into account that on Windows, loaded assemblies
cannot be deleted. However, some of the tests created temp files outside of
this folder and had their own cleanup code which didn't expect temp files to
be undeletable on Windows.

This patch removes any custom temp file creation/cleanup from individual test
methods. It also slightly changes the way SetUp() created the temp folder for
each test, now the paths will be shallower.

7 years agoFix for double CloseHandle() on AppDomain unload
Niklas Therning [Mon, 29 Aug 2016 12:00:48 +0000 (14:00 +0200)]
Fix for double CloseHandle() on AppDomain unload

When AppDomain.Unload() is called a new thread is created and attached. When
this thread is done the thread handle is immediately closed using
CloseHandle() but the same handle is also passed to CloseHandle() later on by
the InternalThread destructor. On the second CloseHandle() call Windows will
abort the process with an "Invalid handle" error.

This patch removes the CloseHandle() calls in mono_domain_try_unload() which
is called by AppDomain.Unload(). The patch also wires up some more functions
in mono/utils/mono-threads-windows.c to set the MonoThreadInfo.handle field to
NULL when a thread has exited.

7 years ago[aot] fix linker invocation
Bernhard Urban [Mon, 29 Aug 2016 10:06:30 +0000 (12:06 +0200)]
[aot] fix linker invocation

fixes regression introduced by 7d238ddd5edeeb42cec5e07b6d6527d16332f921

7 years ago[utils/hwcap] Clean up and improve the hwcap code.
Alex Rønne Petersen [Thu, 28 Jul 2016 15:26:49 +0000 (17:26 +0200)]
[utils/hwcap] Clean up and improve the hwcap code.

* Get rid of the arch-specific headers.
* Do verbose printing in one place.
* Simplify the cross compilation case to avoid repetition.
* Define all variables with a macro in a single header.
* Print detected hwcap info via g_print () so it's visible on mobile.
* Expose plain bool variables in the s390x implementation.

7 years ago[runtime] Bundle support, implement missing stream processing (#3423)
Miguel de Icaza [Sun, 28 Aug 2016 01:51:53 +0000 (21:51 -0400)]
[runtime] Bundle support, implement missing stream processing (#3423)

7 years ago[corlib] Fix Marshal test to not rely on Assert.Less
Alexander Köplinger [Sat, 27 Aug 2016 19:24:10 +0000 (21:24 +0200)]
[corlib] Fix Marshal test to not rely on Assert.Less

It doesn't work on nunitlite e.g. on XA:

```
src/corlib/Test/System.Runtime.InteropServices/MarshalTest.cs(268,11): error CS0117: `NUnit.Framework.Assert' does not contain a definition for `Less'
```