tdm compiler as a plugin to pacman

[revelator]

As you all probably know i been using an offshot of the compilers here based on TDM mingw, lately with gcc-10.2.0 it turned out the old patchset was no longer viable and i had to set for a more standard build which no longer defaulted to linking to the static gcc runtimes. I finally managed to get the patchset updated so they work with gcc-10.2.0 but as i discovered there are some things that need to be handled differently when building the mingw packages with this. For one if you have libunwind from clang in the compiler it will conflict pretty badly with gcc's libgcc.a because the TDM based compiler links to that by default the unwinders end up
linked into every library that supports exceptions and you get multiple definition errors as a result (glog is one example).
I created a modified pull of the packages from here which i modified for use with this compiler instead and i modified pacman's build chain to accept the TDM based compiler as a plugin (like the clang compiler). If there is interrest in it i could create packages for msys2 so that people who use it could. TDM uses shared memory to support casting exceptions across dll boundaries without having to link in the gcc runtime but it is not compatible with ASLR and probably newer will be so keep that in mind.

[revelator]

Well in fact i rebuilt allmost all packages for the TDM compiler plugin now and things work quite fine :) so i could setup a repository for it if wanted with a clone of the currrent mingw packages modified towards this.

Had to build everything from scratch which was a major pain in the neck because we have so many circular dependencies now 🤕 but its upto speed with the current code and the only downside is that aslr is a nogo and probably allways will be as TDM's code for handling exceptions rely on a very odd way of doing shared memory access via atomic tables which do not like aslr one bit. The original code for this stems back from the days of mingw.org's first gcc port which was not able to do dll builds of the runtimes so they had to hack around this by allowing things that had been linked to the static runtimes to throw exceptions across dll boundaries.

I replaced all integers in the code with code more appropriate for 64 bit compilers size_t uintptr_t etc. to try and get around the aslr problem but with little success so far. If someone can fix it completely i can upload the current patch for this feature for them to have a look at.

[revelator]

currently only 3 packages seem to have any problem linking to the static gcc runtimes -> ogre (though it seems this is a problem in ogre itself as the normal compiler also exhibits the same failures if i force it to link to the static runtimes) -> scite and this one has me a bit perplexed it newer opens but strangely there is no error gdb does not report anything either and even the evenlog says all is honky dory, if i replace libscilexer.dll with the one from the normal mingw gcc it works so wtf ?!?. -> lastly openshadinglanguage again no error is generated and im beginning to suspect the inclusion of clangs libunwind is making a mess of things because gcc's unwinder is linked directly into every dll and library (static linked runtimes remember...) this in turn seems to cause some rather annoying things like multiple definition errors when the source in question tries to link in libunwind.

[revelator]

new patch has been up for some time now, and aslr sorta works actually Oo only problem has been a few packages who seem to get hissy when built with introspection on (mostly glib2 based ones). Might actually be because i need to recompile a lot of dependencies with aslr on where previously they where non aslr non nxcompat.

Also been doing some testing with gcc-11.2 which like the upstream one from here had broken exceptions :S turned out of all things it was grep which broke the camels back urgh. after recompiling gcc-11.2 with the reverted grep package things have been more stable and the sjlj build can actually bootstrap itself unlike the dwarf exception based one.

Tbh i kinda dont get the hate for sjlj ? sure its a bit slower but way more stable and in difference to dwarf it will not crash the compiler if something throws an exception involving a windows dll since sjlj is pretty close to what seh on 32 bit is. You can actually get so unlucky with the right call (or rather a very wrong one that dwarf throwing an exception will crash your PC, sjlj might crash the compiler in that case but i newer seen it crash my PC before). I guess since we have sjlj is also the reason not many of the gcc devs are sold on the idea of making a 32 bit seh path since the gain would be minimal.

[mati865]

its a bit slower

In my tests with Qt 5 application few years ago the slowdown of SJLJ compared to Dwarf-2 was 11-20%. For some uses it's quite a lot.
I haven't built 32-bit apps for long time though.

dwarf throwing an exception will crash your PC

Never heard of such reproducible case.

I guess since we have sjlj is also the reason not many of the gcc devs are sold on the idea of making a 32 bit seh path since the gain would be minimal.

32-bit SEH was patented until 2014 when 32-bits were already considered not worth putting so much work into.

sjlj is pretty close to what seh on 32 bit is

It's true that SJLJ exception work way better across across different libraries.

There is also compatibility factor.
Many binaries that use their own exceptions handling (especially compilers other than GCC/Clang) support only Dwarf unwinding.

[revelator]

it might crash your PC (newer said it will by default :) ) and i had a few cases where that happened, if i come across one of them again ill send you the executable or dll that causes it.

well in my case i newer had such huge slowdowns because of sjlj, maybe a few microseconds but newer anything in the vicinity of 20% Oo thats just insane.

not sure where the linux world is these days but i hardly think 32 bit is going anywhere anytime soon with windows, if it does i feel pretty bad for all those games who would need some serious emulation to even run xD

hmm only binaries i can think of using dwarf are either from the nix or android world, in the windows world watcom can use it but i think even it defaults to seh ?.

[mati865]

hmm only binaries i can think of using dwarf are either from the nix or android world, in the windows world watcom can use it but i think even it defaults to seh ?.

Rust only supports Dwarf for i686 Windows and is required by librsvg so the dependency tree is rather large here.
The same thing goes for some Objective C compiler IIRC.

[revelator]

ment more like standard C or C++ compilers, but ok.

[revelator]

so to debunk a few things dwarf is a little not a lot faster than sjlj when no exceptions are thrown, sjlj is a lot faster (<> 20%) when they are.

prof has been posted at the winbuilds github ;).

as for the TDM plugin, it works with either (dwarf or sjlj) if a dwarf build is the only thing accepted it will work with that also. SEH does not need the shared memory hack.
exceptions have been tested now and work as expected any oddities are due to statically linking the runtimes and applies to normal gcc builds as well so it works as it should. for those edge cases it is possible to use --shared-libgcc --shared-libstdc++ to get around fails.

ASLR is a bit wonky with the TDM compiler due to internal address randomization.

[revelator]

the original mutex code newer worked it turns out this was also what broke aslr / dep because it passed invalid data to the exception handler ouch... aslr and dep work now :) but damn it took some doing.

[revelator]

started building all packages with the TDM based gcc toolchain (this will take some time), a few packages need workarounds as they do not work to well linked to the static runtimes (ogre3d "zziplib has some troubles configuring correctly with the static runtimes"+ some gdk-pixbuf dependencies "libtiff for one") wxwidgets also has a bug but a patch has been hanging about on the wxwidgets dev forums which was slated for wxwidgets-3.2 release but someone forgot to apply it... so far everything else seems to work. I had most packages allready built before i noticed that ASLR and DEP was actually working with it xD so now i have to recompile everything again bummer...

[revelator]

i can now provide a build of the latest codeblocks from svn built with the new TDM compiler :) and it works again.
it is ASLR and DEP enabled to. I created an archive intended for use with msys2 with support for the standard compilers and also for the TDM builds if they at some point make it in to the packages.

[revelator]

a load of work has gone into the shared memory patch at this point... ASLR and DEP are now fully working, the named mutex use for the shared memory code now uses the same mechanism as libgccjit for permissions. I backported some code from the original shared memory patch to avoid having to use macros as pointers in C++ (caused a heap of problems even if it compiled without warnings).
my personal build still uses sjlj for 32 bit but seh for 64 bit so no rust in 32 bit but i managed to build the 64 bit version though i had to use a few hacks like making an empty libgcc_eh.a archive since the TDM compiler does not have that (by default it uses libgcc.a as if the gcc compiler itself was built statically and a shared runtime build eg. -shared-libgcc does not use libgcc_eh at all). rust works fine though.

[revelator]

done and good riddance... this was a project that literally took years but i can now provide 98 % of all packages built against the static gcc runtimes with working exceptions. the remaining 2 % have some quirks that does not allow them to be built against the static runtimes atm only 2 packages have problems (ogre3d and to a degree wxwidgets) everything else just works. my compiler is far removed from the TDM build i started out with as you still need -static-libgcc -static-libstdc++ to link to the static runtimes, this was done because of some quirks with TDM where the static libgcc runtime was built like gcc itself was static and so had no unwinder library (libgcc_eh.a) this broke building packages like rust which we seem to rely on more and more. rust does build now and yes it works :)