This is part 2 of my C++11 data structures and algorithms blog series. You can see the first part here. I have also finally uploaded the code to my bitbucket account which you can see here:
https://bitbucket.org/Dauntless/shi
So as I mentioned in the previous post, I have been writing a small C++ library for data structures and algorithms. The only C++ I've done recently was to make a little ioctl library for one of the drivers at work, but it wasn't that much code. Also, I really want to get into the habit of doing Test Driven Development. I actually went out and bought two books on TDD both by Pragmatic Programmers:
Modern C++ Programming with TestDriven Development: Code Better, Sleep Better by Jeff Langr
Test Driven Development for Embedded C by James W Greening
One of the first things I've been tackling is a build system. I think one of the most confusing parts to building native apps is the build system. Makefiles aren't really taught all that well in school, nor for that matter is the idea of logically breaking up a large project into modules. No matter the build system, they are either ugly and super hard to debug when you don't get them right, or are a mysterious black box that you have no clue what it's doing. And there are a ton of build solutions out there, so which one should you use anyway? Do you care about cross-platform capability? Ease of use? Ability to debug?
I considered learning CMake as it seems to be pretty popular nowadays, but I decided on learning waf instead. Why waf? Because waf is just a set of python libraries, and since it's python, that means it is Turing complete. Having a build system that is also a library offers several advantages.
For example, my project requires the Google Mock framework. I am writing the waf build script so that if it doesn't find the Google Mock library on your system, it can download and install it for you. Try doing that with a Makefile, Visual Studio Solution, Eclipse project, or CMake. I could also programmatically run the GMock tests upon the finish of a build. In my opinion, it's probably the closest a C(++) build system can get to a Java maven build. Admittedly, with more elbow grease required, but at least it's possible.
The waf build system is actually relatively easy to pick up...for the basics at least. Like most build systems, it breaks up a software project into different tasks. For example, there's a configuration phase where you can set up various compiler options or dependency checks, a build phase to actually generate the binaries, and an install phase where you can install the binary to the user's system. You can also generate your own commands, and this is what I will do to hook the build with running the GMock tests.
The key concept to understand is that waf uses a wscript file as the build script. This is a python script but without a .py extension. Because the script is being called by waf, you don't even have to import anything.
My actual project directory looks like this:
/home/sean/Projects
/shi
/src
/gui
/algos
wscript
/include
/templates
wscript
Wait! why are there two wscripts? Normally, you only want one entry point for your build script, but it may not make sense to have to change into a particular directory to run the build script. For example, the configuration doesn't need to be in a source directory. Notice the wscript I displayed above has a function called build(), and this function calls bld.recurse(). That's where the second wscript comes in. When you call the recurse() function, the parameter is a directory, and it will call the same function in the wscript as the one from which recurse was called. Since recurse() is being called from the build() function, it will look for
./src/wscript
and call the build() method defined in _that_ wscript. So that being said, let's look at the wscript in the toplevel project directory folder.
Hopefully all of the above seems reasonable. But if not, in essense the waf system needs to configure the environment in which to do the build, including checking for dependencies, and it has to actually perform a build. The configuration of your project is done in the configure function. When you actually run this phase you would call it like this:
./waf configure
Or, if you do not have gmock in a standard library location, you could run the configure stage like this:
./waf configure --gmock=/home/sean/Downloads/gmock-1.6.0 --compiler=g++
Currently, my build doesn't download and install gmock if you don't have it, but that's one of the nice things about waf. If you wanted to, you could. There are some other things it doesn't check for, like CMake (to build gmock), or actual verification of the directory passed in from --gmock. Creating a dependency system is no small task (trust me, I've done it before), but it's an interesting one involving some fun computer science
skills (graph traversals, transactions, and cyclic detection for example).
The configuration is very similar to running ./configure in a typical autotools C(++) program. It makes sure that your build environment has all the required development tools, and sets up any environment variables required for building.
Once your system has been configured, you'll obviously want to do a build. This is the step that actually generates your binaries. Technically waf can differentiate between debug builds and release builds, but I'm doing a release build here. In order to do a build, you just run the command like this:
./waf -v build
The -v is just a verbose flag, but it's useful if a compile fails. The actual binary gets generated in a folder that you specify in your wscript from a variable named out. It will generate sub-folders depending on the location of the recursively called wscript. For example, the first level wscript calls ./src/wscript. So the actual binary gets put into ./build/src. If you had your second wscript in ./source, the binary would be in ./build/source for example.
So speaking of the second wscript, what does it look like?
Hopefully, that doesn't look too bad if you're familiar with makefiles. Currently, none of the programs require other object files, mainly because I am using templates. Because I am using templates, I have to include the source rather than generate an object file that another object uses. It's just one of the limitations of templates, and perhaps somewhat unintuitively, I have made my templates .hpp files rather than .cpp files (since they are getting included into other source rather than being compiled and linked against other objects). If you do have objects to compile, you can use bld.object() instead of bld.program(), and other objects that use it would include a keyword argument called uses and the value would be target. For example:
bld.object(
source = "some_file.cpp",
target = "some_obj", ## generates some_obj.o
includes = INCLUDES + ["includes/some_file.hpp"],
lib = ['math'],
cxxflags = ['-Wall', '-std=c++11'],
dflags = ['-g'])
bld.object(
source = "another_file.cpp",
uses = ["some_obj"], ## it links against some_obj.o
cxxflags = ['-Wall', '-std=c++11'],
dflags = ['-g'])
Just as the make program builds binaries, sometimes you want to clean everything. In waf, you just specify:
./waf clean
And it will delete the contents (recursively) of your build folder. There's also a distclean, which also forces you to run configure again. This is because the waf build will cache some data to prevent rebuilding everything.
So that's it for now on waf. As I make improvements to waf, I'll cover them in a future blog post.
https://bitbucket.org/Dauntless/shi
So as I mentioned in the previous post, I have been writing a small C++ library for data structures and algorithms. The only C++ I've done recently was to make a little ioctl library for one of the drivers at work, but it wasn't that much code. Also, I really want to get into the habit of doing Test Driven Development. I actually went out and bought two books on TDD both by Pragmatic Programmers:
Modern C++ Programming with TestDriven Development: Code Better, Sleep Better by Jeff Langr
Test Driven Development for Embedded C by James W Greening
One of the first things I've been tackling is a build system. I think one of the most confusing parts to building native apps is the build system. Makefiles aren't really taught all that well in school, nor for that matter is the idea of logically breaking up a large project into modules. No matter the build system, they are either ugly and super hard to debug when you don't get them right, or are a mysterious black box that you have no clue what it's doing. And there are a ton of build solutions out there, so which one should you use anyway? Do you care about cross-platform capability? Ease of use? Ability to debug?
I considered learning CMake as it seems to be pretty popular nowadays, but I decided on learning waf instead. Why waf? Because waf is just a set of python libraries, and since it's python, that means it is Turing complete. Having a build system that is also a library offers several advantages.
For example, my project requires the Google Mock framework. I am writing the waf build script so that if it doesn't find the Google Mock library on your system, it can download and install it for you. Try doing that with a Makefile, Visual Studio Solution, Eclipse project, or CMake. I could also programmatically run the GMock tests upon the finish of a build. In my opinion, it's probably the closest a C(++) build system can get to a Java maven build. Admittedly, with more elbow grease required, but at least it's possible.
The waf build system is actually relatively easy to pick up...for the basics at least. Like most build systems, it breaks up a software project into different tasks. For example, there's a configuration phase where you can set up various compiler options or dependency checks, a build phase to actually generate the binaries, and an install phase where you can install the binary to the user's system. You can also generate your own commands, and this is what I will do to hook the build with running the GMock tests.
The key concept to understand is that waf uses a wscript file as the build script. This is a python script but without a .py extension. Because the script is being called by waf, you don't even have to import anything.
My actual project directory looks like this:
/home/sean/Projects
/shi
/src
/gui
/algos
wscript
/include
/templates
wscript
Wait! why are there two wscripts? Normally, you only want one entry point for your build script, but it may not make sense to have to change into a particular directory to run the build script. For example, the configuration doesn't need to be in a source directory. Notice the wscript I displayed above has a function called build(), and this function calls bld.recurse(). That's where the second wscript comes in. When you call the recurse() function, the parameter is a directory, and it will call the same function in the wscript as the one from which recurse was called. Since recurse() is being called from the build() function, it will look for
./src/wscript
and call the build() method defined in _that_ wscript. So that being said, let's look at the wscript in the toplevel project directory folder.
''' This is the waf build script for shi ''' import os import urllib2 top = '.' out = "build" def notImpl(fn): def wrapper(): print "TODO: {0} is not yet implemented".format(fn.__name__) return wrapper def find_gmock(ctx): ''' Find the gmock/gtest library ''' print "trying to find Google gmock" if "GMOCK_HOME" in ctx.env: print "GMOCK_HOME is in env" return True has_gmock = False if ctx.options.gmock and \ os.path.exists(ctx.options.gmock): has_gmock = True ctx.env['GMOCK_HOME'] = ctx.options.gmock else: print "ctx.options.gmock is ", ctx.options.gmock if not has_gmock: getGmock() ctx.fatal("Could not find gmock/gmock.h") return has_gmock @notImpl def getGmock(version): ''' Will retrieve the google gmock source ''' pass @notImpl def find_boost(ctx): ''' Searches for boost library ''' pass def configure(ctx): HOME = os.environ['HOME'] has_gmock = find_gmock(ctx) if has_gmock: ctx.env.GMOCK_INC = ctx.env.GMOCK_HOME + "/include" ctx.env.GMOCK_LIB = ctx.env.GMOCK_HOME + "/lib" ctx.env.GTEST_HOME = ctx.env.GMOCK_HOME + "/gtest" ctx.env.GTEST_INC = ctx.env.GTEST_HOME + "/include" ctx.env.GTEST_LIB = ctx.env.GTEST_HOME + '/lib' ctx.find_program(ctx.options.compiler, var="CLANG", mandatory=True) ctx.env['CXX'] = [ctx.options.compiler] ctx.load("compiler_cxx") def options(conf): conf.add_option("--compiler", action="store", default="clang++", help="compiler to use (clang++ or g++)") conf.add_option("--gmock", type="string", dest="gmock", help="location of google gmock library or GMOCK_HOME env var") conf.load("compiler_cxx") def build(bld): bld.recurse('src') ## call build from ./src/wscript
Hopefully all of the above seems reasonable. But if not, in essense the waf system needs to configure the environment in which to do the build, including checking for dependencies, and it has to actually perform a build. The configuration of your project is done in the configure function. When you actually run this phase you would call it like this:
./waf configure
Or, if you do not have gmock in a standard library location, you could run the configure stage like this:
./waf configure --gmock=/home/sean/Downloads/gmock-1.6.0 --compiler=g++
Currently, my build doesn't download and install gmock if you don't have it, but that's one of the nice things about waf. If you wanted to, you could. There are some other things it doesn't check for, like CMake (to build gmock), or actual verification of the directory passed in from --gmock. Creating a dependency system is no small task (trust me, I've done it before), but it's an interesting one involving some fun computer science
skills (graph traversals, transactions, and cyclic detection for example).
The configuration is very similar to running ./configure in a typical autotools C(++) program. It makes sure that your build environment has all the required development tools, and sets up any environment variables required for building.
Once your system has been configured, you'll obviously want to do a build. This is the step that actually generates your binaries. Technically waf can differentiate between debug builds and release builds, but I'm doing a release build here. In order to do a build, you just run the command like this:
./waf -v build
The -v is just a verbose flag, but it's useful if a compile fails. The actual binary gets generated in a folder that you specify in your wscript from a variable named out. It will generate sub-folders depending on the location of the recursively called wscript. For example, the first level wscript calls ./src/wscript. So the actual binary gets put into ./build/src. If you had your second wscript in ./source, the binary would be in ./build/source for example.
So speaking of the second wscript, what does it look like?
top = '.' import os def build(bld): ''' Generates the node_test executable ''' PRJ_DIR = os.getcwd() INCLUDES = [".", "..", PRJ_DIR + "/includes", PRJ_DIR + "/templates", bld.env.GMOCK_INC, bld.env.GTEST_INC] LIBPATH = ["/usr/local/lib", bld.env.GMOCK_LIB, bld.env.GTEST_LIB] bld.program( source = 'gtests/main.cpp', target = 'node_test', includes = INCLUDES, lib = ['pthread'], libpath = LIBPATH, stlib = ['gtest', 'gmock'], stlibpath = ['/usr/local/lib'], cxxflags = ['-Wall', '-std=c++11'], dflags = ['-g']) bld.program( source = 'gtests/bintree_test.cpp', target = 'bintree_test', includes = INCLUDES, lib = ['pthread'], libpath = LIBPATH, stlib = ['gtest', 'gmock'], stlibpath = ['/usr/local/lib'], cxxflags = ['-g', '-Wall', '-std=c++11'], dflags = ['-g']) bld.program( source = 'gui/firstSFML.cpp', target = 'ogl-gui', includes = [PRJ_DIR, "/usr/local/include"], lib = ["sfml-graphics", "sfml-window", "sfml-system"], libpath = ["/usr/local/lib"], cxxflags = ['-Wall', '-std=c++11'], dflags = ['-g'])
Hopefully, that doesn't look too bad if you're familiar with makefiles. Currently, none of the programs require other object files, mainly because I am using templates. Because I am using templates, I have to include the source rather than generate an object file that another object uses. It's just one of the limitations of templates, and perhaps somewhat unintuitively, I have made my templates .hpp files rather than .cpp files (since they are getting included into other source rather than being compiled and linked against other objects). If you do have objects to compile, you can use bld.object() instead of bld.program(), and other objects that use it would include a keyword argument called uses and the value would be target. For example:
bld.object(
source = "some_file.cpp",
target = "some_obj", ## generates some_obj.o
includes = INCLUDES + ["includes/some_file.hpp"],
lib = ['math'],
cxxflags = ['-Wall', '-std=c++11'],
dflags = ['-g'])
bld.object(
source = "another_file.cpp",
uses = ["some_obj"], ## it links against some_obj.o
cxxflags = ['-Wall', '-std=c++11'],
dflags = ['-g'])
Just as the make program builds binaries, sometimes you want to clean everything. In waf, you just specify:
./waf clean
And it will delete the contents (recursively) of your build folder. There's also a distclean, which also forces you to run configure again. This is because the waf build will cache some data to prevent rebuilding everything.
So that's it for now on waf. As I make improvements to waf, I'll cover them in a future blog post.
No comments:
Post a Comment