Skip to content

lucasz93/BinaryBuilder

 
 

Repository files navigation

*************************************************************************
1. INTRODUCTION

A. Stereo Pipeline

The NASA Ames Stereo Pipeline (ASP) is a suite of free and open source
automated geodesy and stereogrammetry tools designed for processing
planetary and Earth imagery captured from satellites and robotic
rovers. It produces cartographic products, including digital elevation
models (DEMs), ortho-projected imagery, and 3D models. These data
products are suitable for science analysis, mission planning, and
public outreach.

B. Binary Builder

The Binary Builder is a collection of scripts for building the release
binaries for the NASA Ames Stereo Pipeline (ASP). Binary Builder
downloads and builds all dependencies of ASP including the
dependencies of Vision Workbench and USGS's Integrated Software for
Imagers and Spectrometers (ISIS).

Building release binaries is a very difficult and fragile
process. Never can we guarantee that this code works as it will break
on changes made by our dependencies. This code will break on server
URL changes, build system changes, and just generally unsupported
environments. You use this software at your own risk.

************************************************************************
2. LICENSE

A. Copyright and License Summary

Copyright (c) 2009-2012, United States Government as represented by
the Administrator of the National Aeronautics and Space
Administration. All rights reserved.

ASP is licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License. You
may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License.

************************************************************************
3. DOCUMENTATION

A. Produce a Complete Binary Build

All the ASP dependencies can be fetched with conda. If necessary, conda
can also be used to rebuild them. This is described at

  https://github.com/NeoGeographyToolkit/StereoPipeline/blob/master/INSTALLGUIDE.rst

and

  https://github.com/NeoGeographyToolkit/StereoPipeline/blob/master/docs/conda_build.rst

Then, VisionWorkbench and Stereo Pipeline can be built either as
described there, or using the the scripts in this repository. 
The last step will be packaging the build, which is described later
in this section.

Building VisionWorkbench and Stereo Pipeline using build.py
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Having installed the asp_deps conda environment per the above
documentation, clone this repository in the directory
~/projects/BinaryBuilder and run the build script:

    cd ~; mkdir projects; cd projects
    git clone https://github.com/NeoGeographyToolkit/BinaryBuilder.git
    cd BinaryBuilder
    conda activate asp_deps
    source ./auto_build/utils.sh
    ./build.py 

One may need to set some paths in ``./auto_build/utils.sh`` to get
things to work.
   
One can specify the compilers as::

    ./build.py --cc=/path/to/gcc --cxx=/path/to/g++ --gfortran=/path/to/gfortran

If the conda packages were installed in a location other than
``$HOME/miniconda3/envs/asp_deps``, the path to that directory should be
set via ``--asp-deps-dir``.

Due to the amount of code that must be downloaded and built,
BinaryBuilder will take quite a while to finish.  If you see the
message "All done!" then it has succeeded.  Otherwise something has
gone wrong and must be fixed before the build can continue.

If the build failed and you need to restart it after finding a fix,
the name of the individual package that needs to be built can be
passed to ``build.py`` as an argument. Note that this tool keeps track of
built packages in::

    build_asp/done.txt

so to force one to rebuild one can remove its entry from there.

Packaging the build
~~~~~~~~~~~~~~~~~~~

Once the build is successful you should make a distribution tarball to
store the completed build. 

ASP has to ship a full Python runtime which ISIS depends on. That has
to be prepared before the build is packaged. Create it with conda as
follows::

    conda create -n python3.6 python=3.6

Then copy the obtained directory, which may be named::

    $HOME/miniconda3/envs/python3.6

to the BinaryBuilder directory.
        
Then package the build by running from the BinaryBuilder directory::

    ./make-dist.py last-completed-run/install

If the conda packages were installed in a location other than
``$HOME/miniconda3/envs/asp_deps``, the path to that directory should be
set via ``--asp-deps-dir``.

The files to be shipped as part of the distribution should be listed
in the file named ``whitelist``.

For packaging to succeed on Linux, the additional tool named
``chrpath`` is needed, which can be installed in your conda
environment with the command::

    conda install -c conda-forge chrpath

We strongly encourage you to read the output from ./build.py --help and
./make-dist.py --help. There are more features not talked about here.

B. Produce a Partial Build or Dev Environment

We usually like to build the dependencies once and then build the
release binaries with every ASP release. Here's how to build the
dependencies as an archive tarball:

 ./build.py --build-goal 1
 ./make-dist.py --include all --set-name BaseSystem last-completed-run/install

Then, when you are ready to build the ASP release tarballs:

 ./build.py --base BaseSystem-*.tar.bz2 visionworkbench stereopipeline
 ./make-dist.py last-completed-run/install

However, if you just want to call and build VW and ASP yourself so
that you can modify the code, instead of the last two commands use:

 ./deploy-base.py BaseSystem-*.tar.bz2 <INSTALL DIR for 3rd Party software>

Inside the <INSTALL DIR> that you specified you'll find the
appropriate 'bin' and 'lib' directories for all of the used third
party libraries. You will also find example config.option files for VW
and ASP. Copy those files into your respective directories for VW and
ASP. Then you can build VW. Modify the config.options for ASP so that
it points to where VW is.

C. Produce a VW only build

BinaryBuilder now supports the option to build only for Vision Workbench,
saving the time of building the extensive libraries needed only for ASP.
These options work similarly to the ASP methods described above.

For a complete VW build, run build.py with the argument: --build-goal 2

For a VW only development environment, run build.py with the argument: --build-goal 3

Other than using these flag values you can use the make-dist.py and
deploy-base.py tools in the same manner as if building ASP.

************************************************************************
4. ADDITIONAL BUILD OPTIONS

The list of packages that were already built is stored in
<build-root>/done.txt (default: ./build_asp/done.txt). On each line is
the package name and the hash tag for that package. To force a package
to be rebuilt, remove its entry from the list. A package will be
rebuilt in either case if the hash tag changes.

To avoid building a package even if it was not built yet, invoke
build.py with the option _<package name>, i.e., ./build.py _isis.

About

Script for generating the binary builds of Stereo Pipeline

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 60.2%
  • CMake 19.8%
  • Shell 19.7%
  • M4 0.3%