Friday, February 19, 2021

Cross compiling for Windows target with cmake

This is not directly related to Smalltalk, but it is a common source of trouble: how to compile a third-party package for windows platform especially when it comes with cmake files?

My latest example: recompile an upgraded version of lapack from netlib. Lapack sources are now conveniently located on github.

I do not want to use mingw directly (mingw stands for minimal gnu for windows), because I did spend too much time for installing the various tools on mingw in the past.

Instead, I either use cygwin cross-compilation toolchain for mingw target, or linux equivalent thru wsl (windows subsystem for linux). The main advantage of cygwin or linux distributions is the large amount of packages ready to use and easy to install (FORTRAN compilers and other niceties upgraded from the past).

Usually, we find many half-working solutions on the web, like setting a bunch of environment variable. For example I first tried this little shell after cloning the git source in /home/nice/git/lapack.

#!/bin/bash
#
# my own compilation of LAPACK dll for mingw target via cygwin cross-compilation
#
[ -d build_win64 ] || mkdir build_win64
cd build_win64
CC=x86_64-w64-mingw32-gcc \
FC=x86_64-w64-mingw32-gfortran \
AR=x86_64-w64-mingw32-ar \
NM=x86_64-w64-mingw32-nm \
DLLTOOL=x86_64-w64-mingw32-dlltool \
DLLWRAP=x86_64-w64-mingw32-dllwrap \
OBJCOPY=x86_64-w64-mingw32-objcopy \
STRP=x86_64-w64-mingw32-strip \
cmake .. -DBUILD_SHARED_LIBS=ON -DBUILD_INDEX64=ON
make

The worse thing is that it somehow worked...

Except that the generated dll had a ugly and incorrect cyg prefix. And except that when I tried to add the C interface with additionnal options -DCBLAS=ON -DLAPACKE=ON that miserably failed with this message:

CMake Error at /usr/share/cmake-3.17.3/Modules/FortranCInterface.cmake:383 (message):
 The Fortran compiler:

   /usr/bin/x86_64-w64-mingw32-gfortran.exe

 and the C compiler:

   /usr/bin/cc

 failed to compile a simple test project using both languages.  The output
 was:

   Change Dir: /home/nice/git/lapack/build_win64/CMakeFiles/FortranCInterface/VerifyC

Ouch this sticky /usr/bin/gcc is annoying, that's not what I specified! I tried to have more sticky environment variable thru export CC=... but that failed - possibly because I did not want to reconstruct the whole thing from scratch (it takes hours).

I knew that I should have instructed cmake about the toolchain instead. But how?

The wrong answer: spend some times in the terrific, horribilific, unhelpful cmake documentation. I'm not exagerating. Tell me how much are your skill increased on a scale of 10 after reading this toolchain reference. For me, the answer is very close to zero ! I've never seen such a poor documentation in my already long life!

The good answer: it's on the web of course, somewhere... In order to increase the probability of hitting one, I decided to duplicate the already existing answers and share my findings.

Here is the little toolchain file /home/nice/toolchain_x86_64-w64-mingw32.cmake that I assembled after an hour or more of browsing:

# For cross compiling for a windows target
# using cygwin or linux gcc toolchain
#
# The target System
set(CMAKE_SYSTEM_NAME Windows)

# Factor out the prefix which is common to our set of tools
set(TOOLCHAIN_PREFIX x86_64-w64-mingw32)

# The various compilers to use
set(CMAKE_C_COMPILER ${TOOLCHAIN_PREFIX}-gcc)
set(CMAKE_Fortran_COMPILER ${TOOLCHAIN_PREFIX}-gfortran)
set(CMAKE_RC_COMPILER ${TOOLCHAIN_PREFIX}-windres)

# and where to find includes etc...
SET(CMAKE_FIND_ROOT_PATH /usr/${TOOLCHAIN_PREFIX})

Then I just have to change my script like this:

#!/bin/bash
#
# my own compilation of LAPACK dll for mingw target via cygwin cross-compilation
#
[ -d build_win64 ] || mkdir build_win64
cd build_win64
cmake .. -DBUILD_SHARED_LIBS=ON -DBUILD_INDEX64=ON -DCBLAS=ON -DLAPACKE=ON \
  -DCMAKE_TOOLCHAIN_FILE=../toolchain_x86_64-w64-mingw32.cmake
make

Et voilĂ ... This snapshot is somehow a proof of concept:

 

It's simple and magic! Don't ask why ortran is lowercase, nor any other question about this masterpiece.

Magic is exactly the main grief I have toward cmake. As I always tell, cmake is great, as long as someone writes guesses the files for you...

There are other griefs, like not recompiling what is strictly necessary, but that's the IT trend, waste cycles of our sophisticated machines for repeating the same compilation again and again... Bad for the planet!

If this post can help someone, that will be already something. At least, I'm pretty sure that it will help myself in the near future, because with the age, I developped a new capability: forget the un-important things (we can delegate that to google, no matter the cost).