Installing Boost 1.56 with MPI (Ubuntu 12.04, 13.04, 14.04, Fedora)

Finally, one of the my favourite aspects of C++. Boost is one of the most well received additions to the C++ standard library that is out there. So much so in fact that you can see a lot of its implementation absorbed into the new standard, C++11; an example in point been shared pointers, initialisation of static containers such as std::map, the foreach iterator, boost::array is now std::array, addition of unordered sets and of course this is just a drop in the ocean of what else is now included. Boost is well developed and has been well supported over the years. Covering boost would take forever and a day and I am certainly no authority on the subject, however, to follow on from the promise in the last blog, and show you how to install this wonderful toolkit.

Boost is extremely simple to install, as usual you can try the “sudo apt-cache search boost” (Ubuntu) or “yum search boost” (Fedora) first and see if there are libraries already pre-compiled for your distro. As you may have guessed by now I prefer the manual install as you get the latest release where usually the repo is a few versions behind. For a full install of everything you will also need python libraries and icu libraries (if you want regex pattern matching) which are easily got via the “apt-get” method and in this script. Navigate to the Boost homepage and download the release, copy the below once inside the downloaded directory and run the script, it will install Boost in your /usr/local area as per usual. Please read the script NEVER EVER install something if you are unsure, however it is quite simple to follow and follows the basic instructions on the Boost webpages!! As of 25/07/2013 the script worked for Ubuntu 13.04.

# Get the version of Boost that you require. This is for 1.54 but feel free to change or manually download yourself
wget -O boost_1_56_0.tar.gz http://sourceforge.net/projects/boost/files/boost/1.56.0/boost_1_56_0.tar.gz/download
tar xzvf boost_1_56_0.tar.gz
cd boost_1_56_0/
# Now we are inside the boost directory we can get the installation script and execute it.
# NOTE: READ THE SCRIPT FIRST
wget https://dl.dropboxusercontent.com/u/88131281/install_boost.sh
chmod +x install_boost.sh
./install_boost.sh

The script is given here if you wish to cut and paste it.

#!/bin/bash
# Matthew M Reid 10/01/2013. Please use and distribute under GNU licence.
# This script will compile boost on the maximum number of physical cores.
# If you plan to build the Parallel Graph Libraries you may find the
# following warning: Graph library does not contain MPI-based parallel components.
# note: to enable them, add "using mpi ;" to your user-config.jam. The script does this for you.

# Get the required libraries, main ones are icu for boost::regex support
echo "Getting required libraries..."
sudo apt-get update
sudo apt-get install build-essential g++ python-dev autotools-dev libicu-dev libbz2-dev libzip-dev

installDir="/usr/local/boost-1.56.0"
./bootstrap.sh --prefix=$installDir

# pipe "using mpi ;" to the config file so that mpi is enabled
user_configFile=`find $PWD -name project-config.jam`
mpicc_exe=`which mpic++` # required incase the mpi path is not in root
echo "using mpi : $mpicc_exe ;" >> $user_configFile

# Build using maximum number of physical cores
n=`cat /proc/cpuinfo | grep "cpu cores" | uniq | awk '{print $NF}'`

# begin install
sudo ./b2 --with=all -j $n cxxflags="-std=c++11" --target=shared,static install 

sudo echo "$installDir/lib" >> /etc/ld.so.conf.d/boost-1.56.0.conf"
sudo ldconfig -v

This will install Boost.MPI. If you do not want or require the MPI aspect of this installation then comment out the line starting with ‘”echo “using mpi ;”‘ by placing a hash, #, before the echo command. That’s it! One point to note is to make an entry in your .bashrc for easy compilation; export BOOSTROOT=/usr/local/boost-1.56.0. For those who installed with the MPI option, you can run a test by using the code I provide below. So like I mentioned in a previous blog serialisation is one of the coolest things about Boost.MPI, more on that once I get a minute to write something interesting.

// M. Reid - Boost.cpp - Test of boost mpi interface

#include <iostream>

// The boost headers
#include "boost/mpi.hpp"

int main(int argc, char* argv[])
{
    // Allows you to query the MPI environment
    boost::mpi::environment env( argc, argv );
    std::string processor_name( env.processor_name() );
    
    // permits communication and synchronization among a set of processes
    boost::mpi::communicator world;
    unsigned int rank( world.rank() ), numprocessors( world.size() );
    
    if ( rank == 0 ) {
        std::cout << "Processor name: " << processor_name << "\n";
	std::cout << "Master (" << rank << "/" << numprocessors << ")\n";
    } else {
	std::cout << "Slave  (" << rank << "/" << numprocessors << ")\n";
    }
    return 0;
}

To compile you can check any required library names by looking in your /usr/local/lib/libboost_xxxx, from there you find the one you want and remove the “lib” and any subsequent numbers at the end. To check the installation has worked, try the compiling the code posted above “Boost.cpp” with the following,

mpic++ -W -Wall Boost.cpp -o Boost -lboost_mpi -lboost_serialization -lboost_system -lboost_filesystem -lboost_graph_parallel -lboost_iostreams
./Boost
matt@matt-W250ENQ-W270ENQ:$ mpirun -np 4 Boost
Processor name: matt-W250ENQ-W270ENQ
Master (0/4)
Slave  (1/4)
Slave  (2/4)
Slave  (3/4)

Or stick that in a makefile or cmake (I will eventually get around to doing something on cmake, ping me if you are interested) or which ever your favourite compiler aid may be, clearly several of these libraries are redundant in this example since we do not require linking to the graph, filesystem, serialization, iostreams or system libraries in order for this code to work. Enjoy, more to come in due course.

38 comments on “Installing Boost 1.56 with MPI (Ubuntu 12.04, 13.04, 14.04, Fedora)

  1. al3x88 says:

    ./install.sh should be ./install_boost.sh in the first code snippet

    • mattreid9956 says:

      Thanks, I have just corrected that typo! As with anyone reading my posts please correct me is you see a mistake, I value feedback! Cheers and hope it worked ok for you otherwise.

  2. Philip says:

    I get the following error when I run the final command, ‘./install_boost.sh,’ to try to install boost:
    ./install_boost.sh: line 14: ./bootstrap.sh: No such file or directory
    sudo: ./b2: command not found

    Also, when I attempt to compile your test code, Boost.cpp, I get the following error:
    g++: error: Boost.cpp: No such file or directory

    • Philip says:

      Also, during the installation, why did it do the following?

      The following packages will be REMOVED:
      lib32z1 libqt3-mt linux-headers-3.2.0-24 linux-headers-3.2.0-24-generic
      lsb-core lsb-cxx lsb-desktop lsb-graphics lsb-printing ncurses-term pax
      0 upgraded, 0 newly installed, 11 to remove and 0 not upgraded.
      After this operation, 79.1 MB disk space will be freed.
      Do you want to continue [Y/n]? Y
      (Reading database … 353547 files and directories currently installed.)
      Removing lsb-printing …
      Removing lsb-desktop …
      Removing lsb-graphics …
      Removing lsb-cxx …
      Removing lsb-core …
      Removing lib32z1 …
      Removing libqt3-mt …
      Removing linux-headers-3.2.0-24-generic …
      Removing linux-headers-3.2.0-24 …
      Removing ncurses-term …
      Removing pax …

      • mattreid9956 says:

        The command “sudo apt-get autoremove” is responsible for this. I use it often as it removes depreciated libraries that are no longer needed, for instance I had various 32 bit libs that were converted to 64 bit. I have never had any issues but I shall take it out of the script from now on. Anyone wishing to do that can just execute it themselves.You were given the option to say no however and I would urge anyone, for good practice, to understand what they’re about to run before doing so. I myself have grabbed scripts on-line and then trashed my libraries… Learn from my mistake people!

    • mattreid9956 says:

      Are you running the script from inside the boost directory that you downloaded?

      • Philip says:

        Yes. Any idea why this would be occurring?

      • mattreid9956 says:

        I checked the post again and I think it may be the script. For some reason when I pasted the code displayed above some characters were changed into ampersands. I have corrected the above script you can do a “diff” command to make sure you have a file that looks like the current one above. I think I will link to my public Dropbox area from now on and leave any shell scripts in there. Please try


        cd boost_directory_1.5X/
        wget https://dl.dropboxusercontent.com/u/88131281/install_boost.sh
        chmod +x install_boost.sh
        ./install_boost.sh

        Please note though that if you do not have MPI installed (see previous blog), then this may not work out of the box anyway. You should comment out the line beginning with “echo” in the script. It also means that the example code below will not compile. Let me know how it goes. I can send you a couple of simple boost lines if you think the installation worked and you want to try it.

  3. pierdealdo says:

    Hello,

    when doing the last commande : sudo ./b2 etc … I have this error message :
    ” MPI auto-detection failed: unknown wrapper compiler mpic++
    Please report this error to the Boost mailing list: http://www.boost.org

    I have 5/6 times.

    is that normal ?

    • mattreid9956 says:

      Hi there. This is certainly not normal. I assume you have open mpi installed?do you have any more debug information?i am not near a computer right now to check myself. Check my last comment above though that stresses the correct usage if you do not have MPI installed.

      • pierdealdo says:

        I tried to install it using : “sudo apt-get install -y autotools-dev autoconf-dev automake-dev g++ build-essential openmpi1.5-bin openmpi1.5-doc libopenmpi1.5-dev”
        I had the error : unable to locate package autoconf-dev and unable to locate package automake-dev.
        but I didn’t have any error for the openmpi package.

      • mattreid9956 says:

        What happens if in a terminal you type “mpic++”. If that command is recognised then you have correctly installed open mpi. Else we may need to investigate further.

      • pierdealdo says:

        I have the following error message:
        “g++: fatal error: no input files
        compilation terminated”

  4. pierdealdo says:

    I just restarted the computer. did everything again and it seems to work better. don’t know exactly what happened but it seems to be solved …
    so I don’t care about the two packages autoconf-dev and automake-dev ? is that ok if it didn’t install them ?

    • mattreid9956 says:

      Hmmm… Sometimes that is just the way the universe works! but no you do not need automake-dev and autoconf-dev, I think these names changed recently anyway and cannot see them in the repository. autoconf and automake are useful tools but they come under the umbrella of autotools anyway which is in the code i posted. As long as you can run the Boost.MPI example then all is good! Have fun boost is great.

  5. […] Installing Boost 1.53 (Ubuntu 12.04, Fedora) « Particle Physics and Code […]

  6. Hani says:

    Thanks for the Well-organized tutorial for installation. I am going to compile pcl-trunk under ubuntu 12.04, there is some problem like “../../lib/libpcl_common.so.1.7.0: error: undefined reference to ‘boost::thread::join_noexcept()”, and I found out there is no libboost_thread_mt in the /usr/local/lib! would u please help me if possible?

    • mattreid9956 says:

      Hi Hani, I am unsure why you still require xxx_mt? I was under the impression that it was made obsolete (could be wrong..). If you project really does depend on libboost_threat_mt.so and you have compiled version 1.53 of boost, then you could make a symbolic link in the /usr/local/lib directory to that name. It simply does not exist in the latest versions since there was no point not compiling the multi-thread libs without the -mt flag. So I suppose “cd /usr/local/lib; sudo ln -s libboost_thread.so libboost_thread_mt.so”. Now personally I would not advise adding in extra links like this since in the future you could mess up a different installation. But you could then see if it compiles. I think without more information it is hard to tell what the problem is, if you do “ldconfig -p” can you see the boost libs at all? Let me know. I can always give it a go at compiling it myself just a matter of when.

      • Hani says:

        Dear mattreid9956,

        Thanks for your reply. finally I re-installed the boost1.48! now it somehow works (with no boost-related errors) but there is stil some problems with vtk.

        actually I am working on pcl-kinfu during last few month, it was working on my last PC, last week I changed my PC to a better one (GPU: GTX480) I install everything needed to compile pcl-trunk under ubuntu 12.04 (as I did before on my previous PC and it worked) but now after one week I am really getting creazy 😦 …. I can not compile it!

        ldconfig -p and also ‘synaptic package manager’ I have boost but i can not compile pcl-trunk!

        I have tried to reinstall every needed kibrary for pcl-trunk and still there are some problem!

  7. […] Installing Boost 1.54 (Ubuntu 12.04, Fedora) « Particle Physics and Code […]

  8. Ruslan says:

    Hello author and everyone,

    My system is Ubuntu 12.04. I have downloaded Boost 1_55_0

    Then I have follwed instructions and ran script from the article. Everything was built okay.

    Then I ran an example from the article and got an error:

    “/usr/local/include/boost/mpi/config.hpp:20:17: fatal error: mpi.h: No such file or directory”

    could you please help me to solve it?

    Thank you.

    PS: other boost examples run without any errors

    • Ruslan says:

      mpi.h files I have found using “locate mpi.h”

      /usr/src/linux-headers-3.8.0-29/include/linux/mpi.h
      /usr/src/linux-headers-3.8.0-29-generic/include/config/usb/serial/siemens/mpi.h
      /usr/src/linux-headers-3.8.0-29-generic/include/linux/mpi.h

      • mattreid9956 says:

        Firstly, do you want to use mpi? What example is causing you the issue given that it works with other boost tests?i take it you installed mpi using sudo apt-get? Please give me some more info and I will try to help.

  9. Ruslan says:

    Sorry, I was not precise with my comment about examples which work well. By other examples I meant other boost examples, for example containers. As for mpi examples: so far I have tried just your example (from the article you wrote) and it did not work for me.

    I am new to mpi but want to use it.

    Yes, I have installed mpi package using sudo apt-get.

    • mattreid9956 says:

      No problem, its always difficult to debug things online. So I think you should check the example posted here https://particlephysicsandcode.wordpress.com/2012/11/ , can you run that simple example at the bottom of the post? That will indicate that mpi is working at least. Then we know it is definitely a boost installation issue. We may have to install boost again you see and you can tell me what the output is but before that, lets ascertain that you can run mpi.

      • Ruslan says:

        I should have tried to check dependences. mpi was not properly installed. The problem was with dependencies. Instructions on the page you recommended have helped. Thanks for being patient with me. It took me a while to recompile source code.

      • mattreid9956 says:

        Not a problem, so it is now working?if so great, if not feel free to write back. What is your plan for using mpi btw just out of curiosity?

  10. Ruslan says:

    Matthew,
    thank you for your answer.
    1. We have an application which consists of several instances. Sometimes they have to communicate between each other by sending messages according to some protocol. Sometimes they should exchange data between them again by some protocol. My goal is to find a way of communicating between different instances of one application instead of using special protocols for that purpose. I have taken a look at mpi in boost.
    2. Still was not able to build:
    mpic++ -W -Wall main.cpp -o Boost -lboost_mpi -lboost_serialization -lboost_system -lboost_filesystem -lboost_graph_parallel -lboost_iostreams /usr/bin/ld: cannot find -lboost_mpi
    /usr/bin/ld: cannot find -lboost_graph_parallel
    collect2: ld returned 1 exit status
    Could you please help with that? I found no libs mentioned (boost_graph_parallel, boost_mpi).

    • mattreid9956 says:

      Hi Ruslan. Sounds like a cool project! Ok so first I will assume that after you installed mpi, you then reinstalled boost using the above script. If you did this then in theory you should have lots of different boost libs in /usr/local/lib all starting libboost_xxxx.so. If you have boost in there let me know, if not did you change any paths in the script above?or have you also previously installed boost via sudo apt-get?

      • Ruslan says:

        Thanks, Matthew! We did that! Now I have an instrument to research. Thanks a lot for instructions. Can I do anything for you?

      • mattreid9956 says:

        Hey Ruslan, thanks for the offer unless you can help me with my PhD thesis or offer me a job after March then you’re ok :). Feel free to just enjoy Boost.MPI. Actually, if you find anything useful and want to share on here or there are a few other places for Boost.mpi online that I know of then give me a shout. I am happy to take a look if you have an svn/git repo. Good luck!

  11. Ruslan says:

    Thanks Matthew. I was once pursuing my doctorate degree, and then decided to go in another direction 🙂 Will be glad to share my experience if you let me know your email.

  12. Onkar says:

    Hi mattreid9956

    Excellent blog very helpful.I am using a 64 bit machine with Ubuntu 12.04.The boost version i have is Version: 1.48.0.2.As mentioned here “http://wiki.apache.org/thrift/ThriftInstallation” and “http://thrift.apache.org/docs/install/” i am trying to install rhbase which requires thrift and thrift requires pre-requisite of boost 1.53.0 on Ubuntu 12.04. I have a questions is it possible to configure boost 1.53.0 on Ubuntu 12.04?.Have you tried this installation for Ubuntu 12.04?
    I got a message “Boost installation complete.”after executing the code.Does this mean that Boost 1.53.0 is installed?.When i checked for the version info it was displaying Version: 1.48.0.2.Or is there something i am missing.Do you have any relevant information on why 1.53.0 is required.?

    I followed your methods which i was able to execute successfully but i got an error after executing command
    ./Boost

    [server2:19185] [[INVALID],INVALID] ORTE_ERROR_LOG: A system-required executable either could not be found or was not executable by this user in file ../../../../../../orte/mca/ess/singleton/ess_singleton_module.c at line 357
    [server2:19185] [[INVALID],INVALID] ORTE_ERROR_LOG: A system-required executable either could not be found or was not executable by this user in file ../../../../../../orte/mca/ess/singleton/ess_singleton_module.c at line 230
    [server2:19185] [[INVALID],INVALID] ORTE_ERROR_LOG: A system-required executable either could not be found or was not executable by this user in file ../../../orte/runtime/orte_init.c at line 132
    ————————————————————————–
    It looks like orte_init failed for some reason; your parallel process is
    likely to abort. There are many reasons that a parallel process can
    fail during orte_init; some of which are due to configuration or
    environment problems. This failure appears to be an internal failure;
    here’s some additional information (which may only be relevant to an
    Open MPI developer):

    orte_ess_set_name failed
    –> Returned value A system-required executable either could not be found or was not executable by this user (-127) instead of ORTE_SUCCESS
    ————————————————————————–
    ————————————————————————–
    It looks like MPI_INIT failed for some reason; your parallel process is
    likely to abort. There are many reasons that a parallel process can
    fail during MPI_INIT; some of which are due to configuration or environment
    problems. This failure appears to be an internal failure; here’s some
    additional information (which may only be relevant to an Open MPI
    developer):

    ompi_mpi_init: orte_init failed
    –> Returned “A system-required executable either could not be found or was not executable by this user” (-127) instead of “Success” (0)
    ————————————————————————–
    *** The MPI_Init() function was called before MPI_INIT was invoked.
    *** This is disallowed by the MPI standard.
    *** Your MPI job will now abort.
    [server2:19185] Abort before MPI_INIT completed successfully; not able to guarantee that all other processes were killed!

    Thank you.

    • mattreid9956 says:

      Sorry for the delay! Did you figure this out or are you still having trouble? I have installed Boost 1.5x on Ubuntu 12.04 myself in the past using this script so it should work for you too. Quick question, do you have OpenMPI installed?

  13. Thanks for the helpful post! I’m running Ubuntu 12.04 and require a couple different versions of Boost, so this was just what I needed. Btw, I wrote a quick-and-dirty Makefile for your Boost.cpp, which you’re of course welcome to use: https://gist.github.com/tensorjack/a5d96f797ab4a8973f88

  14. Harith says:

    Hi, I have installed ubuntu 14.04 in oracle virtual box. I tried installing Boost 1.56. Every command u said went fine except this:
    sudo apt-get install build-essential g++ python-dev autotools-de

    I am getting following error:
    E: Unable to locate package autotools-de

    I tried the commands that followed the above command and everything worked. please help me to resolve this.

    And really thanks for this post. Its really useful.

    • mattreid9956 says:

      Hi it should be autotools-dev no de. My bad! Hopefully that will resolve your issue. Let me know if not. Thanjs have a good weekend.

      • Harith says:

        Hi, it worked. Thanks. Its showing autotools-dev, build-essential, g++ & python-dev are of the newest version. i have already run the commands that followed it. So should i rerun it? I am trying to install revkit , so istalling boost.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s