Analyzing Boost on Linux, a remarkable story to learn.

This story is for a developer who either:

– wants to learn how to get a project set up with no error on Linux,

– is interested in analyzing the Boost library (http://www.Boost.org),

The article has two parts. The first part is to get you there quickly with minimum explanation. The second part reviews all the steps in detail.

Why do I think this article can help you? I am one of the active participants working on the open-source initiative at SciTools. As part of it, I wanted to offer a pre-made analysis for the Boost library. However, as I started to work, I quickly realized how complex it would be before getting a complete and accurate analysis. So I decided to summarize all the pointers you can use if you are trying to analyze a specific piece of the Boost library or open source yourself.

Instead of analyzing all of the Boost libraries, I focus on the Boost filesystem library to illustrate creating an accurate Scitools Understand project. Contact me at [email protected]if you need other modules analyzed.

All modules will soon be part of our Open-Source Web Site repository.

Just get me there…

Part 1: The “brainless” recipe to get there step by step

Here is a list of steps to follow to analyze the Boost filesystem library v 1.76.0 with 0 errors. I provide minimal explanations.

Step
1

Start on Linux (I used
CentOS 8) with GCC and SciTools Understand installed. I start in a directory
named ~/boost/1.76.0, so please modify the paths accordingly as you go along.

Step
2

Then, we install boost, check
version 1.76 out, and build the headers and filesystem library.

git clone –recursive https://github.com/boostorg/boost.git

cd boost

git checkout boost-1.76.0

./bootstrap.sh

./b2 headers

cd libs/filesystem/build

../../../b2

Step
3

We will need to know the
path of stdarg.h on your system, so execute this and save the result for
later:

find  /usr -name stdarg.h

On my system, I found stdarg.h
in /usr/lib/gcc/x86_64-redhat-linux/8/include

Step
4

In ~/boost/1.76.0, create four
files
MACRO.txt, DIR.txt,
SYSDIR.txt and OPTION.txt

Remember to get back into ~/boost/1.76.0

Step
5

In MACRO.txt, add a single line:

BOOST_FILESYSTEM_NO_CXX20_ATOMIC_REF

Step
6

In DIR.txt, add the following (or the equivalent for
your system):

~/boost/1.76.0/boost

~/boost/1.76.0/boost/libs/filesystem/src

Step
7

In SYSDIR.txt, add the following (or
the equivalent for your system; For the last line, use the path you found
from Step 3):

/usr/include/c++/8

/usr/include

/usr/lib/gcc/x86_64-redhat-linux/8/include

Step
8

In OPTION.txt, add the following:

-C++UseStrict on

-EnforcePortability off

-C++Target x86_64-unknown-linux

-C++CLanguageStandard c11

-C++C++LanguageStandard c++11

-C++CreateReferencesInInactiveCode off

-C++SaveMacroExpansionText on

-C++AddFoundFilesToProject on

-C++AddFoundSystemFiles on

-C++SearchForIncludesInProject off

-C++IgnoreDirectory off

-C++CompareIncludesByContent off

-C++AnalyzeHeadersInIsolation on

-C++CaseInsensitiveLookup off

Step
9

In ~/boost/1.76.0, execute
the following commands:

und create -db boost.und -languages C++

und settings @OPTION.txt -db boost.und

und settings -c++_includesAdd @DIR.txt -db boost.und

und settings -c++_macrosAdd @MACRO.txt -db boost.und

und settings -c++_SystemIncludesAdd @SYSDIR.txt -db boost.und

Step
10

In ~/boost/1.76.0, execute
the following commands:

und add  ./boost/libs/filesystem
-db boost.und

Step
11

In ~/boost/1.76.0, execute
the following commands:

und remove ./boost/libs/filesystem/config/has_bcrypt.cpp -db boost.und

und remove ./boost/libs/filesystem/config/has_cxx20_atomic_ref.cpp
-db boost.und

und remove ./boost/libs/filesystem/config/is_windows_ce.cpp -db boost.und

und remove ./boost/libs/filesystem/example/tchar.cpp -db boost.und

und remove ./boost/libs/filesystem/test/windows_attributes.cpp -db boost.und

und remove ./boost/libs/filesystem/config/has_stat_st_birthtim.cpp
-db boost.und

und remove ./boost/libs/filesystem/config/has_stat_st_birthtimensec.cpp
-db boost.und

und remove
./boost/libs/filesystem/config/has_stat_st_birthtimespec.cpp -db boost.und

und remove ./boost/libs/filesystem/config/has_stat_st_mtimensec.cpp
-db boost.und

und remove ./boost/libs/filesystem/config/has_stat_st_mtimespec.cpp
-db boost.und

und remove ./boost/libs/filesystem/test/issues/3332/test.cpp -db boost.und

und remove
./boost/libs/filesystem/test/issues/recurse_dir_iter_5403.cpp -db boost.und

und remove ./boost/libs/filesystem/test/issues/11166-remove-race.cpp
-db boost.und

und remove
./boost/libs/filesystem/test/issues/fchmodat_AT_SYMLINK_NOFOLLOW_6659.cpp  -db boost.und

und remove ./boost/libs/filesystem/src/windows_tools.hpp -db boost.und

Step
12

Open boost.und in the Understand’s
GUI, and do “Project >> Analyze all files” to analyze the project. You
should get 17 errors!

Why the errors? I firmly believe there is a bug in their examples. The errors are in the following files:

~/boost/1.76.0/boost/libs/filesystem/example/mbpath.hpp
~/boost/1.76.0/boost/libs/filesystem/example/mbcopy.cpp
~/boost/1.76.0/boost/libs/filesystem/example/mbpath.cpp
~/boost/1.76.0/boost/libs/filesystem/test/equivalent.cpp
~/boost/1.76.0/boost/libs/filesystem/test/path_times.cpp

To get to 0 error, you can either remove the 5 files as we did in Step 11, or you can trust the suggested changes in the following table. Always remember that some code may not compilable “as-is” in some open-source projects.

mbpath.hpp

        Line 21 uses a deprecated class. Replace the
line with:
typedef boost::filesystem::path mbpath;

        Comment Line 40 to 44 out.

mbpath.cpp

         Comment Line 45 to 48 out and 64 to 67 out as they
use a deprecated expression for exceptions.

        Add on line 19:
#include <boost/throw_exception.hpp>
#include <boost/filesystem.hpp>

equivalent.cpp

        Comment line 19 out. The Boost library removed
the function ‘default_name_check.’

path_times.cpp

         Put () around long double on line 91 and 93
(so that you get something like (long double)(s))

mbcopy.cpp

        Replace line 95 with:
::copy_file(*it, target_dir / it->path().filename());

        Replace line 24 with:
#include “libs/filesystem/include/boost/filesystem/detail/utf8_codecvt_facet.hpp”

        Because class wpath  is deprecated, add at line 27 the following:
namespace boost::filesystem { typedef
boost::filesystem::path wpath; }
namespace boost::filesystem { typedef boost::filesystem::directory_iterator
wdirectory_iterator; }

        Add at line 25:
#include
<boost/filesystem/directory.hpp>

Now, re-analyze all the files, and you will get 0 errors!

Perfect analysis it is!

Not quite sure how we go there? Let me explain in Part 2.

Need more explanations? Coming up in part 2!

Part 2: How did we get there? And why did we make the choice we did?

Step1 and Step2 follow the suggested steps from the Boost library’s website.

Before analyzing a project, a good practice is to “build it” using the provided configuration and building tools. Building the library may generate config header files needed for a successful build. Therefore, you must include those generated files in your Understand project directly or indirectly.

Step3 is important because SciTools Understand, when “analyzing,” will look for the std lib headers, such as stdarg.h. Unfortunately, SciTools Understand will fall back on its internal “Clang” include directories if it can’t find them, potentially creating conflicts between implementation. Your project has most likely similar issues if you get errors around “size_t” not being defined.

As you will notice, we will add /usr/lib/gcc/x86_64-redhat-linux/8/includeto the system includes in step 7 and step 9. I have tested this with the “C++ Strict” build only.

Step 4 is creating files; Step 5, 6, 7, and 8 fill the files. Finally, step 9 uses the files. The files all contain “configurations” injected into the GUI through scripting. Alternatively, we could have chosen to show you about 100s screenshots, but we decided it was easier to introduce you to the und command-line interface.

Step 5 contains the macros we defined. We only add one, BOOST_FILESYSTEM_NO_CXX20_ATOMIC_REF, as C++20 std::atomic_ref extension conflicted with the C++17 support of SciTools Understand.

Step 6 contains the includes for the project. The Boost library is simple as all the needed headers are under a boost folder (you created the required symlinks when you ran ./b2 headers in Step2).

We also add ~/boost/1.76.0/boost/libs/filesystem/src because it contains a required header file used to build the filesystem library. Boost is a header library, but seldom components, like the filesystem’s library, require building. So we did it in Step2 when calling b2 in libs/filesystem/build.

Step 7 contains the system includes, including /usr/lib/gcc/x86_64-redhat-linux/8/include.

Step 8 is where we set up the SciTools Database options.

– We use C++ strict (-C++UseStrict on)

– We set the target to Linux (-C++Target x86_64-unknown-linux).

– We add all the headers included by the analysis (-C++AddFoundFilesToProject on and -C++AddFoundSystemFiles on). These flags also add all the system headers used (like <vector>, <stdlib>…). You may decide to turn them off.

– We analyze all the headers in the project files, even if no .cpp file includes them (-C++AnalyzeHeadersInIsolation). However, doing so may not always be a good idea as it can analyze header files that are not part of what you are trying to research and create many errors that you can’t fix.

Step 9 creates and sets up the project with the und command line. For information, “und” is in the same directory as the “understand” binary. To learn more, you can run the command “und help.”

Step 10 adds the directory containing all the files related to the filesystem library. Essentially, We tell SciTools Understand what to analyze. Sometimes, the best approach may be adding files individually instead of the entire content of the folder, as we will see in Step 11.

Step 11 removes problematic files. For this specific library, it was easier to add the complete directory and remove some files with issues. The dismissed files are source files (in config) containing compilation errors by design. The compiler’s characterization process makes use of them. Some other ignored files are specific to the Windows Operating System (tchar.cpp and windows_attributes.cpp). Finally, we also remove examples of past or existing issues.

Be First to Comment

Leave a Reply

Your email address will not be published.