The Basics of Compiling Programs Under Linux | Linux 程序编译基础
You may have noticed that almost 70 per cent of the GAAS repository on GitHub is C++ code. Even though in the tutorials we are programming our drones mainly with Python, it is C++ that does all the heavy lifting behind the scene.
As you may have already known, C++ is a compiled language, i.e. C++ code needs to be translated into binary format to run. This process is not particularly straightforward and may remain a mystery even for some veteran coders. So here is a quick-and-dirty, tl;dr style of introduction to the basics of compiling C++ programs under Linux, especially those relevant to GAAS.
You should have some entry-level experience with Linux, and be relatively comfortable with the command line interface. Basic C/C++ programming experience would be greatly helpful too, even if it was not on Linux.
Compiler, Build Tool, and CMake
If you find the names
CC confusing, it may help to know a bit of history: the earliest C compilers in the 1970s were plainly named
cc and could be found under
/usr/bin/cc on Unix systems. Then GNU made their own C compiler in the late 80s, named it GCC (GNU C Compiler, later renamed to GNU Compiler Collection as it supports a lot more than just C), which is normally soft-linked to
/usr/bin/cc on GNU/Linux, as well as some other Unix-like systems. Similar story for C++: the early C++ compiler in the mid 80s is simply
c++, GNU named their version as
g++ and soft-linked to
GCC might be the most popular compiler on Linux, but it is not the only choice. There is Clang/LLVM or Intel C++ Compiler. It is possible to have more than just one C++ compiler installed on your system (and it is also possible that a system does not have a compiler installed at all), so by convention, environment variables
CXX, if defined, point to the compiler that should be used for the compilation. It is possible that when you compile a program by calling
cc mycode.c under command line, the
/usr/bin/gcc is used, but since your
CC environment variable is set to
/usr/loca/bin/clang, when you call
make, Clang will be used instead of GCC. Similar problems may happen if you have multiple versions of the same compiler too.
The simplest way to compile the simplest programs is a direct call to
cc for C or
c++ for C++ programs ( Actually, nowadays
g++ is nothing more than just a faster way of calling
gcc -xc++ -lstdc++ -shared-libgcc).
cc helloworld.c # compiles helloworld.c, generates 'a.out' executable cc helloworld.c -o helloworld # compiles helloworld.c, generates 'hellworld' executable
However, the calls to compilers quickly get messy if the project gets more complex than just several single files. Below is a random excerpt of real world GCC call:
/usr/loca/gcc-4.9/bin/gcc -maix32 -c -DPERL_CORE -D_ALL_SOURCE -D_ANSI_C_SOURCE \ -D_POSIX_SOURCE -DUSE_NATIVE_DLOPEN -fno-strict-aliasing -pipe -I/usr/local/include \ -D_LARGE_FILES -D_FORTIFY_SOURCE=2 -std=c89 -O -Wall -Werror=declaration-after-statement \ -Wextra -Wc++-compat -Wwrite-strings caretx. /usr/loca/gcc-4.9/bin/gcc -maix32 -o miniperl -Wl,-brtl -Wl,-bdynamic -L/usr/local/lib -Wl,-b32 \ -Wl,-bmaxdata:0x80000000 perlmini.o opmini.o miniperlmain.o gv.o toke.o perly.o pad.o regcomp.o \ dump.o util.o mg.o reentr.o mro_core.o keywords.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o \ pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o \ mathoms.o locale.o pp_pack.o pp_sort.o caretx.o -lpthread -lbind -lnsl -lgdbm -ldbm -ldl -lld -lm -lcrypt -lc
Since it is unreasonable to manually type in these commands every time you want to compile a program, build tools are introduced to generate and run these commands automatically.
Make is a build tool invented in the late 1970s, almost as old as
cc itself. The idea of Make is very simple: the programer writes down some targets, as well as the commands that generate these targets, in a file called
Makefile. A target defined in Makefile can either depend on another target or depend on files;
make will read this Makefile, check if the targets are generated / up-to-date, and run the commands if they are not. Here is an example:
app : main.o utils.o cc -o app main.o utils.o main.o : main.c defs.h cc -c main.c utils.o : utils.c defs.h cc -c utils.c clean : rm app main.o utils.o
So the Makefile above defines four targets:
clean, while the target
app depends on the targets
main.o depends on
utils.o depends on
clean depends on nothing. Targets and their dependencies are written in the same line, and the rules about how to generate them are written directly under it. For example, if we modify
utils.c and invoke
make will notice that the timestamp of
utils.c is newer than that of
utils.o, thus calling automatically
cc -c utils.c to update
utils.o, then re-compile
utils.o is now newer than
Makefile is a primitive tool. As a program becomes larger, especially when it has to be compiled on different machines or even different platforms, maintaining a flexible Makefiles could easily become a pain. A standardized way to solve this problem was established in the early 90s, that is, using a shell script called
configure to generate/update the
Makefile automatically. This is how the famous
./configure && make started, or as the GNU calls it, a GNU Build System was born. Here is an example from GNU, showing the Build System in action:
~ % tar zxf amhello-1.0.tar.gz ~ % cd amhello-1.0 ~/amhello-1.0 % ./configure … config.status: creating Makefile config.status: creating src/Makefile … ~/amhello-1.0 % make … ~/amhello-1.0 % make check … ~/amhello-1.0 % su Password: /home/adl/amhello-1.0 # make install … /home/adl/amhello-1.0 # exit ~/amhello-1.0 % make installcheck …
The GNU Coding Standards specifies some common targets, like
check, etc. for the Makefile, as well as some parameter or variables that should be accepted by the
configure script. If you implement them, other people can simply build your code by following muscle memory. This convention is easy to learn, and flexible enough for many Unix developers, so it is still popular almost thirty years later today. It is not uncommon to see projects that maintain the
Makefile (or its template,
Makefile.in) by hand. Yet some developers still don’t feel the process automated enough, so they went a further step forward and wrote the Autotools, which are basically tools that generate
Makefile for you. So the whole source code distribution process with Autotools can be summarized as the following steps:
- The developer writes/generates
- The developer calls
- The developer calls
- The developer distributes the generated
Makefile.inas part of the source code
- The user calls
configurewhich examines the system and converts
- The user calls
makethat build the source code
GNU Autotools has its limitations, one being that it is not straightforward to learn, but the major problem is that it is (almost) Unix-only. So in the year 1999, CMake was invented to support real multi-platform build for C/C++ projects.
Despite the name, CMake has little to do with Make, the only relationship between the two is that CMake can be used to generate
Makefiles on Unix, i.e. it is a tool which generates files that can be used to build C/C++ projects: Makefile or Ninja for Linux, Xcode projects for macOS, and Microsoft Visual Studio projects for Windows. It is a separated step to actually start the build process by opening/calling these generated files, you can either do this by hand or as the CMake to call them for you. Here is an example:
cmake_minimum_required(VERSION 2.8.9) project (prj) add_executable(app main.c)
By adding three lines, we defined a project named
prj, with a target
app which should be an executable, and it should be compiled from source
main.c. Now if you save these three lines into a file named
CMakeLists.txt and put it together with a reasonable
main.c, then run
cmake . on a Linux machine with GCC and build tools properly installed, you will see an output similar to the following:
-- The C compiler identification is GNU 4.9.2 -- The CXX compiler identification is GNU 4.9.2 -- Check for working C compiler: /usr/local/gcc-4.9.2/bin/gcc -- Check for working C compiler: /usr/local/gcc-4.9.2/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working CXX compiler: /usr/local/gcc-4.9.2/bin/g++ -- Check for working CXX compiler: /usr/local/gcc-4.9.2/bin/g++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Configuring done -- Generating done -- Build files have been written to: /home/metaphox/code/gaas-example
If you check the directory, among other files, a
Makefile is created. If you run
make in this directory, an
app executable will be built.
What happens if we run
cmake . for exactly the same code on Windows? The following is the output of
cmake . on my Windows with Visual Studio 2015 installed:
-- Building for: Visual Studio 14 2015 -- The C compiler identification is MSVC 19.0.24210.0 -- The CXX compiler identification is MSVC 19.0.24210.0 -- Check for working C compiler using: Visual Studio 14 2015 -- Check for working C compiler using: Visual Studio 14 2015 -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working CXX compiler using: Visual Studio 14 2015 -- Check for working CXX compiler using: Visual Studio 14 2015 -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Configuring done -- Generating done -- Build files have been written to: C:/repo/cmaketest
And if we check the current directory:
C:\repo\cmaketest>dir Datenträger in Laufwerk C: ist System Volumeseriennummer: 1AAA-3821 Verzeichnis von C:\repo\cmaketest 22.08.2019 12:50 <DIR> . 22.08.2019 12:50 <DIR> .. 22.08.2019 12:50 55.795 ALL_BUILD.vcxproj 22.08.2019 12:50 272 ALL_BUILD.vcxproj.filters 22.08.2019 12:50 65.773 app.vcxproj 22.08.2019 12:50 545 app.vcxproj.filters 22.08.2019 12:50 11.679 CMakeCache.txt 22.08.2019 12:50 <DIR> CMakeFiles 22.08.2019 12:49 82 CMakeLists.txt 22.08.2019 12:50 1.268 cmake_install.cmake 22.08.2019 12:49 71 main.c 22.08.2019 12:50 3.132 prj.sln 22.08.2019 12:50 39.654 ZERO_CHECK.vcxproj 22.08.2019 12:50 509 ZERO_CHECK.vcxproj.filters 11 Datei(en), 178.780 Bytes 3 Verzeichnis(se), 59.097.427.968 Bytes frei
So CMake generated
prj.sln solution file, as well as the
app project files. These are Visual C++ project files that can be open and built with Visual Studio, but we can also simply build them with the
msbuild command-line tool shipped with Visual Studio by calling
msbuild app.vcxproj, and an
app.exe will be compiled.
You can specify what kind of build tools you would like to use with CMake, these are called “Generators” and can be passed with
-G option, for example
cmake . -G "Unix Makefiles" will generate Makefiles (which is the default option on Linux),
cmake . -G "Visual Studio 14 2015 Win64" will generate Visual Studio solution files for x86_64 architecture,cmake . -G "XCode"
will generate Apple XCode project files, andcmake . -G “Eclipse CDT4 - Ninja”` will generate Ninja-based Eclipse CDT4 project files (whatever that is).
Do notice that, although we ran
msbuild by hand, it was just for illustration purpose. CMake does support calling the build tool for you, which is simply
cmake --build <build-dir>.
As you may see, CMake is powerful for its ability to generate build-tool specific files across multiple platforms, which is of great importance for portable C++ projects, and that is why it has become the de-facto standard for such projects.
A common practice with CMake worth mentioning here is out-of-source build. Citing a documentation we recommend you to read:
“Out-of-source build” is a good practice of keeping separately generated files from binary tree and source files from source tree. CMake do support contrary “in-source build” layout but such approach has no real benefits and unrecommended.
That is, if you are using CMake, it is better to create a dedicated directory for just the build process, so the source code would not be littered with unnecessary files produced by the build process. This directory needs not to be really “out” of the source tree, you can just create a
build directory directly under the root of the source code. For example, in our tutorial E06, we built the Kernelized Correlation Filters (KCF) algorithms with:
cd (GAAS-Object-Tracking_PATH)/KCF mkdir build cd build cmake .. make -j6
So here we are doing out-of-source build, and calling
make in a separated step for clarity.
Headers and Libs
A major and maybe the most common headache of compiling C++ projects is missing header files and/or missing libraries.
The GNU Coding Standards define a set of standard configuration variables used during the build. Here are some:
CC: C compiler command
CFLAGSC compiler flags
CXXC++ compiler command
CXXFLAGSC++ compiler flags
CPPFLAGSC/C++ preprocessor flags