1. Introduction

Welcome to the installation instructions for LSDTopoTools, which is a collection of programs for analysing topography and modelling hydrology, landscape evolution and soil development.

LSDTopoTools is built for Linux operating systems. Wait stop!! Where are you going? Come back!! We are going to get this to work on your computer, honest!

Most users will not use a Linux operating system as their day-to-day operating system; Windows and MacOS are much more popular. However, through the either virtualisation or containers you can build a Linux environment within any operating system. In fact, building either a virtual machine or a container has the advantage of ensuring that everyone using these systems has the same environment. This precludes the "works on my machine" problem that is common amongst scientific software packages.

In these documents you will find four flavours of LSDTopoTools installation instructions:

  • Installation using Docker. This works for MacOS and Linux well. It works for Windows 10 Enterprise edition. If you don’t have those operating systems you will need to use VirtualBox + Vagrant (see below).

  • Native installation on a Linux system. If your native operating system is Linux you can just install LSDTopoTools after getting all the libraries you need. The documents are made under the assumption that you use Ubuntu, but if you are a Linux user we are confident you’ll be able to port the instructions to your native system (usually this involves replacing sudo apt-get with sudo yum).

  • University of Edinburgh installation. This software is developed at the University of Edinburgh and is used in teaching. We include instructions specific for Edinburgh students for use on the School of GeoSciences servers.

  • Installation using VirtualBox + Vagrant. This method is well tested but it requires two bits of software instead of one for [Docker], and instead of a lightweight container you need a full blown Ubuntu system as a virtual machine, meaning that this takes up more space on your hard disk. Also, on Windows machines, it is difficult to install a 64-bit guest operating system: you will probably be limited to a 32-bit system that limits your memory if you use this approach. So if you have MacOS, Linux, or Windows 10 Enterprise you should probably use the Docker installation.

2. Installing LSDTopoTools using Docker

As of October 2018, this is the preferred method to install LSDTopoTools. We have various containers on the docker hub which can be run with simple pull commands. The only software you need to run these is Docker.

Docker and LSDTopoTools quickstart
  1. Download and install Docker. We will assume this actually is working.

  2. Create an LSDTopoTools directory on your host operating system that you will share with the LSDTopoTools docker containers.

    1. We’ll assume this is in C:\LSDTopoTools on Windows

    2. OR \LSDTopoTools on MacOS and Linux.

  3. Pull the full LSDTopoTools container and run it with a linked volume:

    1. For windows:

      $ docker run --rm -it -v C:/LSDTopoTools:/LSDTopoTools lsdtopotools/lsdtt_pcl_docker
    2. For MacOS or Linux:

      $ docker run --rm -it -v /LSDTopoTools:/LSDTopoTools lsdtopotools/lsdtt_pcl_docker
    3. Or if you have a different directory to LSDTopoTools data on your host machine:

      $ docker run --rm -it -v /PATH/TO/YOUR/DATA:/LSDTopoTools lsdtopotools/lsdtt_pcl_docker
  4. Once you run this, you will need to run the script:

    # Start_LSDTT.sh

2.1. Should I use Docker or Vagrant?

Short answer: If you have MacOS, Linux or Windows 10 Enterprise, use Docker. Otherwise use Vagrant.

Long answer: There are many, many, many articles comparing vagrant to docker. The docker setup is probably a bit simpler than the vagrant setup but (and this is a big but) it won’t work on a Windows machine unless you have Windows 10 Enterprise. It does work on any recent version of Linux and MacOS. If you have MacOS, native Linux or Windows 10 Enterprise you should consider using Docker since it involves fewer steps than our VirtualBox+Vagrant setup.

2.2. Installing Docker

Docker is software for creating and managing containers. These containers are not complete operating systems, like ones created using VirtualBox and Vagrant, so the boot time is quicker (usually almost straight away). Docker is perhaps simpler to use but does not work on every operating system.

To set up LSDTopoTools using Docker:

  1. Download and install Docker for Windows (only works with Windows 10 enterprise), Docker for Mac, or Docker for Ubuntu or Debian.

    1. On MacOS we recommend installing docker using brew: brew cask install docker

  2. We will henceforth assume that you actually have a functioning version of Docker on your host machine. If you want more details about how to use docker, or getting it set up (particularly in Windows, in Linux and MacOS this is more straightforward), see our section on docker notes.

2.3. The four LSDTopoTools images

LSDTopoTools has four different containers. If you are an LSDTopoToools developer you will probably use all four. Most users will only use two of them. The four containers are:

  1. lsdtt-alpine: This is basic LSDTopoTools container. It uses a very lightwight linux distrubution called the alpine linux distribution. For many users, this is the only container you will need. It is an 88 Mb download and when built, it is a but under 300 Mb.

  2. lsdtt-pcl: A few of our routines (notably the terrace extraction routines and any routines using swath profiling) require something called the point cloud library (PCL). This library is ENORMOUS and a massive pain to install. It does everything the lsdtt-alpine container does, and more. It is big: the download is 644 Mb and built it is over 2 Gb. However, if you need this one you can use it instead of the lsdtt-alpine container.

  3. lsdtt-viz: This container installs all the python routines you need if you want to use our visualisation tools. Our tools require multiple python packages, meaning that this container is also quite large. If you use a GIS to look at our data, you won’t need this. However we do all our visualisation in python and have written lots of automation scripts, so if you think you will used LSDTopoTools routinely you might want to grab this dockerfile.

  4. lsdtt-docs: Our documentation is all written in asciidoc format, and processed with the extremely awesome asciidoctor. It runs in the Ruby programming language, and requires some stuff to be installed. We have a container for building the documentation so you don’t need to worry about installing the correct stuff. It is unlikely you will need this unless you are an LSDTopoTools developer.

Which container do I use?
  • Pick either lsdtt-alpine or lsdtt-pcl for the main programs. The former is a small container but can’t do the terrace algorithms. If you anticipate extracting terraces or floodpains use the latter. It is much bigger: lsdtt-alpine is an 88 Mb download and ~240Mb unpacked. lsdtt-pcl is a 664 Mb download and ~2 Gb unpacked.

  • lsdtt-viz is for our python plotting routines. If you just intend to use a GIS to look at LSDTopoTools, you don’t need this. But if you want to reproduce the figures we have in our papers you will need this container. LSDMappingTools needs a lot of python packages so the container is large, I’m afraid.

  • lsdtt-docs: he containes the software needed to compile our documentation. The documentation is available on a website (you are reading it) so you only need this container if you are actually going to be writing LSDTopoTools documentation.

How do I get the containers

There are two options for getting the containers!

  • Option 1: Download the images from Docker hub: .

  • Option 2: Build the containers from the raw dockerfiles.

Option 1 is better. Option 2 is only for enthusiasts.

2.4. Before you do anything else: Set up a directory to share with docker containers.

When you run LSDTopoTools, or our visualisation routines, you will want to see the resulting files. Docker containers have self contained filesystems that cannot be read on your host operating system unless you tell docker to "share" the directories. These are called "volumes" in docker speak.

  1. First, make a directory in your host operating system where you will put all the files associated with LSDTopoTools. I put mine in C:\DockerBoxes\LSDTopoTools\

  2. To share a directory in docker you use the -v flag.

    1. Following the -v flag you need to tell it which directories to share.

    2. The format is: /Host/directory:/Container/directory

  3. When you run docker (see below) you will need to ensure that the host directory name is the same as the directory you just created for LSDTopoTools.

2.5. First option: pull the docker images

  1. Okay, this should be fairly easy.

  2. If you want to run the full docker installation from the get go, simply run:

    $ docker run --rm -it -v C:/LSDTopoTools:/LSDTopoTools lsdtopotools/lsdtt_pcl_docker

    This will pull the repo and run it at the same time. Warning! This container is 663 Mb so if you just want to do some initial testing use the lightweight alpine version!

  3. If you want the lightweight version (but it can’t do floodplains and terraces) use:

    $ docker run --rm -it -v C:/LSDTopoTools:/LSDTopoTools lsdtopotools/lsdtt_alpine_docker
  4. After that, run the install script:

    # Start_LSDTT.sh
  5. You are now ready to start using LSDTopoTools!

2.6. Second option: build the docker containers

Unless you are a docker purist, please ignore this section and use option 1 instead.
  1. Create a folder where you are going to put files associated with docker. In the examples I will assume these are in C:\Docker\LSDTopoTools\ (this is a windows path, since the majority of organisations use the Windows operating system).

    1. If you have git on your computer, you can clone our dockerfiles into this directory.

      $ git clone https://github.com/LSDtopotools/lsdtt_alpine_docker.git
      $ git clone https://github.com/LSDtopotools/lsdtt_pcl_docker.git
    2. If you don’t have git, download and unzip the contents of the repository.

  2. If you are only running LSDTopoTools for routine analysis and looking at the results in GIS software, you will only need one docker container.

  3. We are only going to work with the lsdtt-alpine container. But the steps are basically the same for all the other containers.

  4. Start a terminal (MacOS or Linux) or powershell (Windows) with docker enabled.

  5. Go into the directory inside you docker directory (in my case C:\Docker\LSDTopoTools\) and go into the lsdtt_alpine directory. In windows, if your directory is called the same thing as my directory, you can get there with:

    PS> cd C:\Docker\LSDTopoTools\lsdtt_alpine_docker
  6. Build the docker container:

    PS> docker build -t lsdtt-alpine .
  7. The -t flag in docker stands for "tag". It is the name of the container. You could call it anything you want but all documentation henceforth assumes you have called it lsdtt-alpine.

    1. If you are building a different container, use a different tag, e.g., lsdtt-pcl for the lsdtt-pcl container.

  8. Now you need to run this container:

    $ docker run -it -v C:/LSDTopoTools:/LSDTopoTools lsdtt_alpine_docker
  9. After that, run the install script:

    # Start_LSDTT.sh
  10. You are now ready to start using LSDTopoTools!

2.7. Some notes on Docker

You might want to read the docker documentation, but docker documentation is similar to documentation for the turbo encabulator. Below are some very simple notes to get you started.

Docker quick reference

Here are some shortcuts if you just need a reminder of how docker works.

List all containers

$ docker ps -a

List containsers with size

$ docker ps -as

Remove all unused conainers

$ docker system prune

Complete cleanup: stop everything, remove containers, then remove images.

$ docker stop $(docker ps -aq)
$ docker rm $(docker ps -aq)
$ docker rmi $(docker images -q)

2.7.1. Docker on Linux

After you install docker on Linux, you will need to add users to the docker permissions:

$ sudo usermod -a -G docker $USER

Once you have done this you will need to log out and log back in again.

2.7.2. Docker for Windows

I have not made any scientific study of this but most LSDTopoTools users are on Windows operating systems.

Firstly, you need to have Windows 10 Enterprise. It will not work otherwise. If you don’t have Windows 10 Enterprise but are on Windows you probably should use the Vagrant instructions. If you do have Windows 10 enterprise then you can download and install Docker for Windows CE. After you install this you will need to restart your computer not once but twice: once after intall and a second time to activate the hyper-V feature, which allows you to have 64 bit guest operating systems.

Second, if you have that and have it installed, you also need to add yourself to the docker-users group. To do that, do this (instructions from here: https://github.com/docker/for-win/issues/868):

  1. Logon to Windows as Administrator

  2. Go to Windows Administrator Tools

  3. Look for Windows Computer Management and click on it.

  4. Or you can skip steps 1, right mouse clicking Computer Management, go to more, and select run as administrator and provide Administrator password.

  5. Double click docker-users group and add your account as member.

  6. Also add your account to Hyper-V Administrator. This was added when you installed docker for Windows.

  7. Log off from Windows and log back on.

  8. Click on Windows icon on bottom left and start Docker for Windows. This will start docker windows service.

  9. Start Windows Powershell and type docker --version. It will show Docker version 17.09.1-ce, build 19e2cf6. This is the latest version.

2.7.3. Building a base image

By the time these notes go online, there should be a functioning docker image that you can download from the docker hub. The instructions will be clear and users can just start running LSDTopoTools from within docker. These notes exists as something of a history or cheat sheet of how we made the base image.

Before we do anything, you should look at some notes about running a functioning image of ubuntu: http://phusion.github.io/baseimage-docker/

I have no idea if that stuff is important! But we will try to test images that both include and don’t incude a fully formed startup environment.

The other big issue with docker is the volume syncing. This is more difficult to understand syncing in vagrant.

A simple docker image

So basically for LSDTopoTools functionality we might want a few different base images:

  • An image just for the c++ code.

  • An image just for the python scripts.

  • An image just for building documentation.

  • An image with everything installed.

The reason why we might separate the first three images is because the python image will be HUGE whereas the LSDTopoTools source code is small. So people might want to be offered a slimm version.

Lets start with a very simple python image. The docker documentation for getting you started up is not so useful because they are written to get you running an app in the docker box. For our purposes we want:

  1. To log into to the docker container via ssh or access it as though you have a little linux box running

  2. To be able to access you file system (we don’t want to be having to ftp files in an out of the container each time or worse yet losing all our work when the docker session ends).

For python, we could go with a full python build but that results in very large containers (over a GB). We will follow advice for a minimal python installation. For this we will use the Alpine base box.

  1. First, create a directory that will contain the files you need. I’ll call it python_test.

  2. Go into this directory. You will keep something called a dockerfile in this directory. It has instructions for what stuff is included in the container. We will start with an incredibly simple one. Create the follwoing textfile, called dockerfile:

    FROM python:3.6-alpine
    
    COPY . /app
    WORKDIR /app
  3. Before we do anything we need to build the container. You build it with some name, like:

    $ docker build -t chumbucket .

    In this command, the -t tells docker to tag the container, the chumbucket gives the container its tag, and the . means that you want to call the Dockerfile from this directory.

  4. This downloads some stuff and then hopefully tells you the container is sucessfully built. You can look at it with docker image ls.

  5. You need to actually run the image if you want to do anything with it:

    $ docker run -it chumbucket sh
  6. The -it means interactive and tag and you give it the tag. You then tell it to run the shell (sh). To get out of the shell simply type ctrl-d.

  7. This gets you into a python container, and in addition it has a (very old version of) pip installed. It breaks if you try to install even simple things like pandas. This makes it totally unusable so we are going to have to add some stuff to the docker file. Unfortunately the alpine distro has all sorts of fundamental problems, for example you can’t install scipy.

A slightly more complicated python image

Okay, that container only has a very simple version of python on it. You can then install pip and try to get some packages working, but this is not always straightforward. Your friendly LSDTopoTools developers prefer the miniconda environment, so we are going to build a container on top of the miniconda3 image.

  1. The dockerfile to start looks like this:

    FROM conda/miniconda3
    
    COPY . /app
    WORKDIR /app
  2. Again, we need to build this container in order to use it:

    $ docker build -t chumbucket2 .
  3. We can see what images are available with the call

    $ docker images -a
  4. You will notice the conda image is 228 MB vs the 78.7 MB: the conda version has more bells and whistles involved in getting miniconda to work. But the end result is that you can install things using conda. Let’s update the image with a conda install. Here is the dockerfile:

    # Use the miniconda distribution.
    FROM conda/miniconda3
    
    # Add the app directory and set app to the working directory
    COPY . /app
    WORKDIR /app
    
    # Now add some conda packages
    RUN conda install -y numpy
  5. And we can build this once more (with the same build call so we don’t create a totally new container):

    $ docker build -t chumbucket2 .
  6. Now we can open this container in the interactive mode:

    $ docker run -it chumbucket2 sh
  7. If you run python inside this container and then import numpy as np you can see that it has been installed.

  8. From here you should be able to see that we can install all sorts of stuff from the dockerfile using the RUN conda install command.

    Some Gotchas in simple containers

    So far it seems like smooth sailing? Well there are a few gotchas.

    Firstly, in the command docker run -it chumbucket2 sh you can see we are using sh: this is in most of the "getting started" docker documentation. However, using sh really sucks since it does not feature any of the Linux command line shortcuts you are used to. Instead, you should us bash.

    Secondly even through you have copied the ./app directory into the container, it does not communicate with your storage. So if you update anything in that directory on your home computer, you need to completely rebuild the container!

Mounting volumes in Docker
  1. What we really want is exchange of files between your host operating system and your docker container.

  2. You might start by reading about docker volumes. But like most Docker documenattion that is a bit impenetrable so you might move on to some plain english summaries: summary 1 and summary 2.

  3. The upshot of this is that you don’t really want a docker volume container, but rather you want to link your docker container to your host file system.

  4. On MacOS and Linux this works automatically, but on Windows 10 you need to activate it. Right click on the docker icon in the system tray, go to Settings and then to Shared Drives and activate drive sharing.

  5. Once you have done this you can activate the drive sharing with the -v flag in a docker run command (see below).

An LSDMappingTools container
  1. Okay, lets make a container for LSDMappingTools.

  2. The dockerfile looks like this:

    # This is the LSDMappingTools dockerfile. It is currently set up to install the
    # latest versions of the various required python packages
    
    # To run this you will need to connect it to your host directories if you want to be able to retrieve
    # your figures and get data in and out of the container.
    # On windows you need to allow docker access to the host drive
    # Right click on the docker icon in your system tray
    # Then select "settings"
    # Then "Shared Drives" and activate the drive you want
    # In MacOS and Linux this is automatically set up.
    # After you build the container
    # e.g.
    # docker build -t lsdmt .
    #
    # Then you need to run it with the volume attached
    # e.g.
    # docker run C:/Docker_containers/python_containers/LSDMappingTools:/LSDTopoTools/LSDMappingTools lsdmt bash
    
    # Use the miniconda distribution.
    FROM conda/miniconda3
    
    LABEL MAINTAINERS = "Simon Marius Mudd <simon.m.mudd@ed.ac.uk>, Fiona J Clubb <clubb@uni-potsdam.de>"
    
    # Set LSDTopoTools to be the working directory
    # This will create a directory in the root of the container.
    # When you start the container you will be placed in this directory.
    WORKDIR /LSDTopoTools
    
    # Now add some conda packages
    RUN conda config --add channels conda-forge
    RUN conda install -y numpy matplotlib pandas scipy
    RUN conda install -y gdal shapely fiona rasterio pyproj
    RUN conda install -c conda-forge descartes
    This takes a very long time to set up!
  3. This doesn’t include the actual python scripts! You need to copy them into a directory where docker can access them (on my machine this means withing the C: drive) and then add them using a volume command:

  4. First build the container:

    $ docker build -t lsdmt .
  5. Then run the container in interactive mode with the volume attached:

    $ docker run -it -v C:/Docker_containers/python_containers/python_test1/LSDMappingTools:/LSDTopoTools/LSDMappingTools lsdmt bash
  6. Note that this container has many python packages so it is rather larege: nearly 4GB (to see, type docker ps -as).

An LSD Documentation container
  1. We use asciidoctor for our documentation. I am pleased to say that there is a dockerfile for asciidoctor.

  2. Simply copy the asciidoctor dockerfile and add two lines:

        awesome_print \
        bundler \

    these come after the line tile \ on line 55.

  3. Now build the container

    $ docker built -t asc .
  4. Then open that container up linking to the proper directory. I use:

    $ docker run -it -v C:/Docker_containers/ruby_containers/LSDTT_documentation/master:/documents/LSDTT_documents asc bash
  5. You can find this dockerfile within the LSDTT Documentation repository.

  6. When that is running, go into the documentation directory and run our python script:

    # cd LSDTT_documentation
    # python compile_LSDTTDocs.py
  7. The resulting website will be in the directory html_build.

The LSDTT core code container
  1. We actually have two containers for this since a subset of our code uses PCL which is massive.

  2. The simpler version, used in most of our code, is the following:

    # Pull base image.
    FROM ubuntu:18.10
    
    RUN apt-get update
    RUN apt-get install -y build-essential git gdal-bin python-gdal libfftw3-dev cmake
    
    WORKDIR /LSDTopoTools
  3. We can also use an Alpine linux version, which is much smaller:

    FROM alpine
    MAINTAINER Simon Mudd (simon.m.mudd@ed.ac.uk) and Fiona Clubb (clubb@uni-potsdam.de)
    
    # install essential packages
    RUN apk upgrade -U && \
        apk update && \
        apk add --virtual build-dependencies build-base gcc wget git && \
        apk add --repository http://dl-cdn.alpinelinux.org/alpine/edge/main libressl2.7-libcrypto && \
        apk add gdal --update-cache --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing && \
        apk add fftw-dev cmake && \
        rm -rf /var/cache/apk/* && \
        rm -rf /tmp/*
    
    # update to avoid weird apk error
    RUN apk update
    
    # clone LSDTT repo
    WORKDIR /LSDTopoTools/
  4. After we build this container we can run it with:

    $ docker run -it -v C:/Docker/cpp_containers/LSDTopoTools:/LSDTopoTools/ lsdtt-alpine
The LSDTT PCL container

Some of our routines need the [Point Cloud Library]. This is a HUGE library that needs millions of additional bits of software. The resulting container is enormous, and takes an age to build so you should only use this if you need the PCL versions of LSDTopoTools.

One quite annoying thing is that for anything later than Ubuntu 14 you need to build PCL from source. This takes a long time. The whole installation process takes an age. Sorry.

This is the dockerfile:

# Pull base image.
FROM ubuntu:16.04

# These are the basic build tools
RUN apt-get update
RUN apt-get install -y build-essential git gdal-bin python-gdal libfftw3-dev cmake

# And this is the point cloud library

# install pcl
RUN apt-get install -y libpcl-dev

# We need this as well
RUN apt-get install -y libproj-dev

WORKDIR /LSDTopoTools

3. Installing natively on a Linux system

Our software is designed for linux operating systems but we do recommend you use either our VirtualBox+Vagrant setup or our Docker setup because in those two cases you will be using the same environment as the developers so it will be easier to answer questions when you have problems. However, if you just want LSDTopoTools on your Linux machine, follow these steps:

  1. Start a terminal session.

  2. Install some essential softare (we will use Ubuntu installation, you need to modify these if you are on another version of Linux). This assumes you have the GNU c++ tools:

    $ sudo apt-get install -y git
    $ sudo apt-get install -y gdal-bin
    $ sudo apt-get install -y python-gdal
    $ sudo apt-get install -y libfftw3-dev
    $ sudo apt-get install -y cmake
  3. Go the the directory where you want to install LSDTopoTools.

  4. Download our setup script.

    $ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py
  5. Run the setup tool:

    $ python LSDTopoToolsSetup.py -id 1
    The -id flag tells LSDTopoToolsSetup.py where to put the files. In vagrant it always starts in the root directory (this is for file syncing purposes). However, if your native operating system is Linux, then you can either install in your home directory (-id 0) or in the directory where you called LSDTopoToolsSetup.py (-id 1).
  6. If wget doesn’t work, you can follow the link: https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py
    Copy the text, paste it into a text editor and save it as LSDTopoToolsSetup.py.

  7. The LSDTopoToolsSetup.py tool has a number of options: if you want a preview of what it does, you can call its help options:

    $ python LSDTopoToolsSetup.py -h

4. Installing LSDTopoTools at the University of Edinburgh.

These instructions are for students and staff within the School of GeoSciences at the University of Edinburgh.

Quick Setup for Edinburgh Geoscience users

If it is the first time you are using LSDTopoTools

  1. In the windows search bar, look for MobaXterm personal edition.

  2. When you open this, click on the button that says start local terminal.

  3. In the terminal window, type ssh UUN@ssh.geos.ed.ac.uk and enter your password. UUN is your university user name.

  4. This puts you into the university ssh server, which is slow. Move into a better one by typing ssh burn.

  5. Go into your T: drive by entering: cd /exports/csce/datastore/geos/users/UUN

  6. Now enter: git clone https://github.com/LSDtopotools/LSDTopoTools2.git

  7. Now enter: cd LSDTopoTools2/src/

  8. Now enter: sh build.sh

  9. You will need to wait a bit for this step.

  10. Now enter: cd ..

  11. Finally enter: sh lsdtt2_terminal.sh

If you have already followed the instructions above once

  1. In the windows search bar, look for MobaXterm personal edition.

  2. When you open this, click on the button that says start local terminal.

  3. In the terminal window, type ssh UUN@ssh.geos.ed.ac.uk and enter your password. UUN is your university user name.

  4. This puts you into the university ssh server, which is slow. Move into a better one by typing ssh burn

  5. Go into your T: drive by entering: cd /exports/csce/datastore/geos/users/UUN

  6. Now enter: cd LSDTopoTools2

  7. Finally enter: sh lsdtt2_terminal.sh

4.1. Mapping your network drive

You will need access to your data on the network drive to use LSDTopoTools. You only need to do this one in any computer lab (and possibly only once ever):

  1. Open a file explorer window and under computer select the "University drive":

    The U drive
    Figure 1. The university drive
  2. Then go into the Science and Engineering folder ("SCE"):

    SCE
    Figure 2. The science and engineering folder
  3. Then into the GeoSciences folder ("GEOS"):

    GEOS
    Figure 3. The geosciences folder
  4. Then into the Drive Mapping folder ("drive_mapping"):

    Drive mapping
    Figure 4. The drive mapping folder
  5. And finally double click the shortcut "Access Netdata":

    Access netdata
    Figure 5. The access netdata shortcut

4.2. Getting on to our servers with MobaXterm

  1. Use the windows search to find the program MobaXterm.

  2. When you open this, click on the button that says start local terminal.

  3. In the terminal window, type ssh UUN@ssh.geos.ed.ac.uk and enter your password. UUN is your university user name.

  4. This puts you into the university ssh server, which is slow. Move into a better one by typing ssh burn.

  5. Go into your T: drive by entering: cd /exports/csce/datastore/geos/users/UUN

4.3. Cloning the LSDTopoTools2 repository

LSDTopoTools2 lives in a github repository and you use a program names git to get it. Don’t worry, this is easy.

  1. Type (or copy and paste): git clone https://github.com/LSDtopotools/LSDTopoTools2.git

  2. It will download some stuff. Now you need to build the software.

4.4. Building the software

  1. Type: cd LSDTopoTools2/src

  2. Type: sh build.sh

  3. Wait a bit until it finishes. It should take a minute or two.

4.5. Making sure the software works

  1. Type cd .. (this puts you into the main LSDTopoTools2 directory)

  2. Now type: sh lastt2_terminal.sh

  3. You can test the softare by typing: lsdtt-basic-metrics and see if it actually runs the program (it won’t do any analysis yet but it will tell you to give it some files).

5. Installing LSDTopoTools using VirtualBox and Vagrant

Important note
Starting in October 2018, we have moved to Docker as the default installation method. If you use MacOS, Linux, or Windows 10 Enterprise, we suggest you use Docker. Otherwise, you should use this installation method.

Our most tested method uses two software packages, which are both free to download:

  • VirtualBox is software for creating virtual machines, which are implementations of operating systems that live in your host operating system (e.g., you can have a virtual Linux system operating within your Windows or MacOS operating system).

As of 01-Nov-2017 Vagrant does not work with VirtualBox 5.2. Make sure you download Virtualbox 5.1 instead: https://www.virtualbox.org/wiki/Download_Old_Builds_5_1
  • Vagrant is software for managing and provisioning virtual machines.

As of 22-Dec-2017 Vagrant 2.0.1 has conflicts with some ruby packages. Make sure you download Vagrant 2.0.0: https://releases.hashicorp.com/vagrant/2.0.0/

5.1. Quick instructions

Quick instructions for starting a Vagrant server with LSDTopoTools
  1. Download and install virtualbox. As of 01-Nov-2017 Vagrant does not work with VirtualBox 5.2. Make sure you download Virtualbox 5.1 instead: https://www.virtualbox.org/wiki/Download_Old_Builds_5_1

  2. Download and install vagrant. As of 22-Dec-2017 Vagrant 2.0.1 has conflicts with some ruby packages. Make sure you download Vagrant 2.0.0: https://releases.hashicorp.com/vagrant/2.0.0/ You might have to restart your computer after this. You need to install virtualbox before you install vagrant, or you might get errors.

  3. If you are on Windows, download putty.exe. If you are on Linux or MacOS you can skip this because they have built-in equivalents.

  4. Make a directory for your vagrant box. We tend to put ours in a directory called VagrantBoxes.

  5. Inside that directory make two new directories: LSDTopoTools and Ubuntu. The second directory’s name doesn’t matter, it is just for holding a vagrant file (see below). However you MUST have the LSDTopoTools directory. The directory name is case sensitive!

  6. Download one of our vagrantfiles: https://github.com/LSDtopotools/LSDTT_vagrantfiles into the Ubuntu directory (again, the name of the directory doesn’t matter).

  7. Rename the vagrantfile from the repo (either Vagrantfile_32bit_FFTW or Vagrantfile_64bit_FFTW) simply vagrantfile. Your operating system is almost certainly 64 bit, but on most computers you need to select 32 bit because the default setting is to disable 64 bit guest operating systems. This can be changed but only by expert users.

  8. Open a terminal or powershell window and navigate to the directory with the vagrantfile.

  9. Run vagrant up from the command line.

    If you are running vagrant up for the first time it can take some time to download the base box. They are several hundred Mb each!
  10. Run vagrant provision after the box has started.

  11. If on Windows, you should now be able to use putty.exe to ssh into your LSDTopoTools server. The host name is almost always 127.0.0.1 and the port is almost always 2222.

  12. On Windows, you will need to give a username and password after connecting using putty.exe. The machine is running locally on your computer so nothing is being sent over the internet. The username is always vagrant and the password is also vagrant.

  13. If you are on MacOS or Linux you do not need putty.exe; all you need to do is type vagrant ssh into the command line. See the vagrant instructions.

Quick instructions for setting up directories in your vagrant box
  1. Use putty.exe (in Windows) or vagrant ssh (on MacOS/Linux) to log into your vagrant server.

  2. Go to the LSDTopoTools directory by typing cd /LSDTopoTools

  3. Type python LSDTopoToolsSetup.py -id 0

  4. You are ready to go for some basic analysis!

5.2. Full instructions on setting up our Vagrant servers

5.2.1. First steps: Starting a Vagrant box

You will need sufficient space on your hard disk to host a guest operating system. You also need room for the LSDTopoTools dependencies. You will struggle if you have less than 5Gb free.

Vagrant is software that automates the creation and provisioning of virtual machines. What does that mean? It means that you will create a Linux server that runs inside of your day-to-day computer. This server will run even if you are using a different operating system (e.g., Windows). Vagrant machines can be configured using a vagrantfile, so you download our vagrantfile and you simply point vagrant to it and should get a working server that can run LSDTopoTools.

  1. You need software for running virtual machines. We recommend virtualbox since it is both well supported and free. Download and install. Our instructions assume you are using virtual box. As of 01-Nov-2017 Vagrant does not work with VirtualBox 5.2. Make sure you download Virtualbox 5.1 instead: https://www.virtualbox.org/wiki/Download_Old_Builds_5_1

  2. Download and install Vagrant. As of 22-Dec-2017 Vagrant 2.01 has conflicts with some ruby packages. Make sure you download Vagrant 2.0.0: https://releases.hashicorp.com/vagrant/2.0.0/

  3. Vagrant works via command line, so you will need to know how to open a terminal on OS X, Linux (usually you can open one using ctrl-alt-T, but if you use Linux that means you were born knowing how to open a terminal), or a Windows powershell.

  4. If you are working on Windows, you will probably have to restart after installing Vagrant so that Windows can register the path to Vagrant.

  5. Okay, we now assume you have installed everything and are in a terminal or powershell. You need to make a directory where you keep information about your vagrant boxes. I made a folder names vagrantboxes and then subfolders for different boxes.

  6. If you are in Windows, you will need an ssh utility to communicate with your vagrant box. You should download putty.exe from the putty website. In Linux and MacOS ssh utilities are already installed.

  7. Now you should fetch one of our vagrantfiles from our git repo: https://github.com/LSDtopotools/LSDTT_vagrantfiles

    Get the Vagrant Files
    1. The 32 bit file is here: https://raw.githubusercontent.com/LSDtopotools/LSDTT_vagrantfiles/master/Vagrantfile_32bit_FFTW

    2. The 64 bit file is here: https://raw.githubusercontent.com/LSDtopotools/LSDTT_vagrantfiles/master/Vagrantfile_64bit_FFTW

    3. Although most computers today are 64 bit, most of them, as default, disable guest 64 bit operating systems. Unless you know how to turn this on (for example see this link), you should choose the 32 bit vagrantfile.

    4. Save one of these files into the directory for your vagrant files. See below for the appropriate directory structure.

  8. Rename the vagrantfile from the repo (either Vagrantfile_32bit_FFTW or Vagrantfile_64bit_FFTW) simply vagrantfile

  9. If you use our vagrant files, you will need to make a directory LSDTopoTools in the same directory as your folders for different vagrant boxes. For example, you might make a directory C:\VagrantBoxes\, and in that directory you can put both LSDTopoTools and Ubuntu32_FFTW (or some such name) directories. You will put the vagrant file in the Ubuntu32_FFTW directory. Your tree might look a bit like this:

    C:\vagrantboxes\
    |--Ubuntu32_FFTW
       |-- vagrantfile
    |--Ubuntu64_FFTW
       |-- vagrantfile
    |--LSDTopoTools
    It is ESSENTIAL that the LSDTopoTools folder is present and is one directory level lower than the vagrant file. If this is not true, the vagrant machine will NOT WORK.
    In the above file structures the vagrantfiles have been renamed from the vagrant files in our repository.
  10. Go into the folder with the operating system you want (e.g. Ubuntu32_FFTW):

    PS: > cd C:\VagrantBoxes
    PS: > cd C:\Ubuntu32_FFTW
  11. Now start your vagrant box (this might take some time since it has to fetch stuff from the internet):

    PS: > vagrant up
    You do not need to download a "base box" (that is a Linux operating system, in this case 32 bit Ubuntu) before you run vagrant up: Vagrant does this for you. However if you are running vagrant up for the first time Vagrant will download the box for you which will take some time (it is ~400Mb). You will only need to download the base box once.
  12. Congratulations! You now have a functioning Vagrant box!! Now you need to log on to the box.

    If you want to update the base box you can use vagrant box update command from the powershell or terminal windows.

5.2.2. Logging on to your Vagrant box

  1. All right! Your Vagrant box is running. Other than a sense of vague accomplishment, this doesn’t really help you run LSDTopoTools. You need to log on to the box. You will operate your vagrant box as a server: you log into the machine and run code on it, but you won’t have pretty windows to look at. You will run everything through an ssh terminal, using a command line interface.

  2. We do this using ssh.

    A note on your vagrant ssh server
    • ssh allows you to communicate securely with a server on an unsecured connection (it encrypts communication between you and the server).

    • You will use ssh to communicate with your Vagrant server. This server is not on the internet but rather is living on your computer.

    • Vagrant is clever in that it sets up an IP address for your vagrant server (in other words your Linux machine living on your host computer, which could be Windows, Linux or OSX), and as such ssh can establish a connection to this machine via ssh.

    • Vagrant’s default settings are to set your server up to sit on host 127.0.0.1 and port 2222. You will need to use these settings in putty.exe

    • When you first log in, putty or other ssh clients will ask you to cache a new host key.

  3. If you are starting from a Linux or OSX machine, an ssh client is built into your command prompt and you can just type vagrant ssh into the command prompt.

Windows login
  1. If you are on Windows, you need to download putty.exe and run it.

  2. In putty, set the host to 127.0.0.1 and the port to 2222. These are vagrant’s default settings.

Windows using the old version (precise pangolin)
  1. You will need to add the RSA key to your cache (just say yes: remember you are not connecting to the internet where baddies can spy on you but rather a server running on your own computer).

  2. Now you need to log in. Your vagrant box has a username of vagrant and a password of vagrant.

Windows using the new version (precise pangolin)
  1. Using the new version (which has a Precise Pangolin box: this is supported until 2021).

  2. This seems to upset putty.exe. Here is how to fix that.

  3. Run vagrant up.

  4. Then download puttygen.exe from the putty website.

  5. Run puttygen and load an existing key. Vagrant makes a private key called private_key.

    1. It will be in your vagrantboxes folder. Somewhere like C:/Vagrant/Ubuntu32/.vagrant/machines/default/virtualbox/

    2. Click on load in puttygen.exe. You need to make sure it is looking for all files and not putty key files (ppk).

  6. Once you load that you can put in your own private passphrase.

  7. Then save this key as something sensible (like private_key_putty.ppk).

  8. Now in putty look on the right with all the setting options and click on Connection → ssh.

  9. Then click on Auth and then browse for your private key.

  10. Then click on session and save this as Vagrant_xenial or something like that.

  11. When you ssh in using putty you will need to use vagrant as the username but then you use your private passphrase to get on the system.

5.2.3. Your Vagrant box and file syncing

  1. So you are logged in. Now what? It turns out Vagrant has done some clever things with your files.

  2. Vagrant can sync folders across your Vagrant box and your host computer (that is, the computer you started vagrant from).

  3. When you log in to your vagrant box, you will not be in the same folder where I have built the LSDTopoTools file structures. You need to navigate down to this:

    $ pwd
    /STUFF
    $ cd ..
    $ cd ..
    $ pwd
    /STUFF
    $ cd LSDTopoTools
    $ ls
    STUFF

    You can also jump directly there:

    $ cd /LSDTopoTools
  4. As you can see above, the LSDTopoTools folder contains folders for different LSDTopoTools packages, for topographic datasets.

  5. Here is the amazing thing: the files that are in LSDTopoTools folder in your vagrant box ARE ALSO visible, and synced, in your host computer. So if you use LSDTopoTools to do some analysis within your vagrant box, you will be able to see the files within your host computer as well. This means that you can, for example, do a Linux-based LSDTopoTools analysis and then plot that analysis in a GIS on your host windows box without having to transfer files. Not only that, but you can modify the code, update python scripts, change parameter files, etc., with your favourite text editor in Windows (or MacOS, or whatever) and those files will be visible to your Vagrant box. Fantastic!

5.2.4. Updating to the latest versions of the software

To check out the latest version of our software you can run the vagrant provision command

PS: > vagrant up
PS: > vagrant provision

5.2.5. Shutting things down

When you are finished with your session, you just need to go into the powershell or a terminal and type:

PS: > vagrant halt

5.2.6. If you want to start from scratch

If you want to remove the virtual machine, start it up and then run vagrant destroy:

PS: > vagrant up
PS: > vagrant destroy

5.3. Troubleshooting a vagrant server

There are a few common problems people have when running a vagrant server.

5.3.1. Vagrant up doesn’t work

  • The first time you run vagrant up you will need to be connected to the internet as you need to download the base box (which is ~700Mb).

  • If you are on an old computer, sometimes vagrant times out before the virtual machine boots. This most frequently happens the first time you boot a vagrant machine. The most effective way to fix this is with the canonical IT solution: turning it off and on again. To do that run vagrant halt and vagrant up in succession.

5.3.2. I closed the window running vagrant: how do I shut it down?

  • Open virtualbox: you will see the vagrant machine. Simply power it off (from within virtualbox).

5.3.3. Vagrant hangs up in powershel or terminal window

  • If vagrant hangs up in the powershell or terminal window and does not give you back the command prompt, turn it off and on again by typing ctrl-c and then running the vagrant command again.

5.3.4. File syncing doesn’t work

  • If your files are not syncing across your host and vagrant machine, it is probably because there is some misspelling in your LSDTopoTools folder on the host machine. Make sure that folder is in the correct place and is spelled correctly (remember it should be case sensitive!!).

  • This also has emerged because of a conflict with the latest VirtualBox versions. Make sure you have VirtualBox 5.1 or lower. As of December 2017 VirtualBox 5.2 doesn’t work.

5.3.5. I get some crazy conflict about Ruby when I run vagrant up

  • This is an error associated with the latest Vagrant version. You need version 2.0.0 or lower. As of January 2018 Vagrant 2.0.1 doesn’t work.

5.3.6. I ran vagrant up. Now what?

  • You need to log on to the vagrant machine with either putty.exe (Windows) or by typing vagrant ssh (Linux, MacOS) into the command line. See instructions for more details.

5.4. Technical details on vagrant

Here are some technical details on vagrant. Hopefully you will never have to read any of this.

5.4.1. Brief notes for setting up your own Vagrant server

This section is for customising your vagrant environment (or rather, your Ubuntu environment that vagrant sets up for you) and can be safely ignored by 95% of LSDTopoTools users. We include the below notes for obsessive hackers who have nothing better to do.

We have written Vagrant files for you so you don’t have to customise your working environment, but if you want to set up your own Vagrant boxes with your own software here are some notes.

Initiating a Vagrant box
  1. Go into an empty folder to start a new Vagrant box.

  2. Initiate Vagrant with:

    PS> C:\> vagrant init

    Alternatively you can initiate with a base box. In this example we use the Ubuntu precise 32 base box:

    PS> C:\vagrant init ubuntu/precise32
  3. This command (init) will simply make a vagrant file. To get the server up and running you need to up it. Before you do that you probably want to modify the vagrant file.

  4. One of the things you probably need to modify is the memory assigned to your guest vagrant box. In the vagrant file you should have:

      config.vm.provider "virtualbox" do |vb|
        # Customize the amount of memory on the VM:
        vb.memory = "3000"
      end

    The default memory is something small, and the problem with it is that it will take the guest operating system too long to boot, and vagrant will time out. I would give the vagrant box 3-4 Gb of memory.

  5. Now you can up your vagrant box. In the folder with the vagrant file, type:

    PS> vagrant up
  6. If this is the first time booting the linux machine, this will take a while.

Notes on the base box

Vagrant sets up a Linux server living in your computer (it is called the Host computer). The server will run a Linux operating system, and you need to choose a functioning base system vagrant base boxes. Here we have started with ubuntu/precise32. You might want to try other base boxes, they can be found at the atlas website.

If you choose a base box that you do not already have (and you start with none), vagrant will download it. They are big!! Usually over 500Mb (each is a fully operational linux operating system). You will either need a fast internet connection or a lot of time. Make sure you also have enough room on your hard disk.

You do need to be careful with base boxes!

Not all base boxes work! On many windows machines, you can only run a 32 bit version of linux, even though you are almost certainly running 64 bit windows. You can change this by going into your BIOS and changing the settings, but that is dangerous and if you do not know what your BIOS is do not even think about attempting to change these settings.

In testing, I found many bases boxes did not work at all. The one that worked well for me was the ubuntu/precise32 box. You can just vagrant init an empty vagrant instance and change the box in the vagrantfile with config.vm.box = "ubuntu/precise32".

You can update your base box with the command vagrant box update.

Details of provisioning

If you change your vagrantfile with the box still running, you can run the new provisioning with:

PS> vagrant provision

If you have downloaded our vagrant files, the provisioning of your virtual server should be automatic. However, you may wish to know what is happening during the provisioning, so here are some notes.

To install software, we use the shell provisioning system of vagrant. This should go into the vagrantfile and will look a bit like this:

  config.vm.provision "shell", inline: <<-SHELL
    sudo apt-get update
    sudo apt-get install -y git
  SHELL

In the above shell command, we are installing git. The -y flag is important since apt-get will ask if you actually want to download the software and if you do not tell it -y from the shell script it will just abort the installation.

You sync folders like this:

  config.vm.synced_folder "../LSDTopoTools", "/LSDTopoTools"

Were the first folder is the folder on the host machine and the second is the folder on the Vagrant box.

6. Extra dependencies for swaths and point clouds

Do I need to read this section?

If you have followed our docker installation you don’t need to read any of this: all the tools mentioned below are included in the LSDTopoTools PCL container.

Some of our tools require the point cloud library. To get this working you need a few different utilities and libraries (take a deep breath):

  1. The cmake utility. This is like make but is required for our tools that examine point clouds, since it is required by something called the point cloud library.

  2. pcl: The Point Cloud Library.

  3. libLAS: a library for working with LAS format data.

Our Vagrant installation installs these automatically. The docker installation (since it is 3rd party) does not. However you can install on an Ubuntu system (and the Docker installation is Ubuntu) using the following instructions.

6.1. Install the Point Cloud Library

If you are using an older version of ubuntu there is a package. Newer versions of Ubuntu don’t have it, so you will unfortunately need to install by hand.

6.1.1. Ubuntu 14 and 12

This is relatively painless (we hope):

$ sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl
$ sudo apt-get update
$ sudo apt-get install libpcl-all

6.1.2. Ubuntu 16.04 LTS

First, try this. It is possible it will only work on 64 bit systems:

$ sudo apt-get update
$ sudo apt-get install libpcl-dev

If that doesn’t work you need to install from source. I am very sorry about this.

To install the pcl library from source you use the following steps:

This will take at least an hour!! It will also download an insane amount of stuff onto your machine. You must ensure that you follow the steps in the correct order.

Install oracle-java8-jdk:

$ sudo add-apt-repository -y ppa:webupd8team/java && sudo apt update && sudo apt -y install oracle-java8-installer

Install various dependencies and pre-requisites:

$ sudo apt -y install g++ cmake cmake-gui doxygen mpi-default-dev openmpi-bin openmpi-common libusb-1.0-0-dev libqhull* libusb-dev libgtest-dev
$ sudo apt -y install git-core freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libphonon-dev libphonon-dev phonon-backend-gstreamer
$ sudo apt -y install phonon-backend-vlc graphviz mono-complete qt-sdk libflann-dev libflann1.8 libboost1.58-all-dev

Install the eigen linear algebra package:

$ cd ~/Downloads
$ wget http://launchpadlibrarian.net/209530212/libeigen3-dev_3.2.5-4_all.deb
$ sudo dpkg -i libeigen3-dev_3.2.5-4_all.deb
$ sudo apt-mark hold libeigen3-dev

Install VTK (this has some plotting functionality, this takes AGES.):

$ wget http://www.vtk.org/files/release/7.1/VTK-7.1.0.tar.gz
$ tar -xf VTK-7.1.0.tar.gz
$ cd VTK-7.1.0 && mkdir build && cd build
$ cmake ..
$ make
$ sudo make install

Now install the Point Cloud Library v1.8:

$ cd ~/Downloads
$ wget https://github.com/PointCloudLibrary/pcl/archive/pcl-1.8.0.tar.gz
$ tar -xf pcl-1.8.0.tar.gz
$ cd pcl-pcl-1.8.0 && mkdir build && cd build
$ cmake ..
$ make
$ sudo make install

Do some clean up:

$ cd ~/Downloads
$ rm libeigen3-dev_3.2.5-4_all.deb VTK-7.1.0.tar.gz pcl-1.8.0.tar.gz
$ sudo rm -r VTK-7.1.0 pcl-pcl-1.8.0

You should now have pcl installed on your system! You can do a small test of the installation if you want to:

$ cd ~
$ mkdir pcl-test && cd pcl-test

Create a CMakeLists.txt file and save this in the directory pcl-test:

cmake_minimum_required(VERSION 2.8 FATAL_ERROR)
project(pcl-test)
find_package(PCL 1.2 REQUIRED)

include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})

add_executable(pcl-test main.cpp)
target_link_libraries(pcl-test ${PCL_LIBRARIES})

SET(COMPILE_FLAGS "-std=c++11")
add_definitions(${COMPILE_FLAGS})

Create a file and call it main.cpp:

#include <iostream>

int main() {
    std::cout << "hello, world!" << std::endl;
    return (0);
}

Compile the cpp file and then test it :

$ mkdir build && cd build
$ cmake ..
$ make
$ ./pcl-test

Output should be hello, world!

6.2. Extra libraries for the floodplain and terrace extraction

As of January 2018 liblas does not compile on our vagrant server since liblas3 is not available for Ubuntu 12. We have updated the vagrantfiles to Ubuntu 14 so you should use the new vagrantfiles (this will require downloading a new vagrant box if you were using the old vagrantfile). We don’t use Ubuntu 16 since it is such an incredible pain to install the point cloud library on it.

You might find you need to install a couple of other libraries for the floodplain and terrace drivers to compile properly. If you are using Ubuntu, you can do this using the following commands:

$ sudo apt-get update
$ sudo apt-get install libgoetiff-dev
$ sudo apt-get install liblas-dev

7. Installing the LSDTopoTools python toolchain

LSDTopoTools is written in C++ but also includes a number of automation and visualisation rountines written in python.

Because LSDTopoTools handles primarily spatial data, a number of python packages for handling spatial data are needed in the LSDTopoTools python toolchain. We have attempted to make setting up this toolchain as painless as possible using miniconda and conda environments.

Please see the chapter on LSDTopoTools visualisation for instructions on how to install the LSDTopoTools python toolchain.

8. Get a text editor!

A text editor isn’t part of LSDTopoTools but you need one to do anything. You will need to modify parameter files and you might even get to the stage of opening Pass:[c++] or Python scripts and editing them so you will need an editor.

If you are Linux native you probably chortle superiorly if you lean someone doesn’t use vi to modify files, but most students have usually only interacted with text files via a word processor.

A word processor (e.g., Word, or whatever fasionable software is on your fruit-based computer) will not work to modify code: these convert your text into binary files and add a bunch of formatting.

You need to work with raw ascii files, and for that you need a text editor. We like these:

  • Brackets is a multi platform text editor (it works on all operating systems) and is free. It has nice file navigation. This documentation was written using Brackets.

  • Atom is another multi platform editor. Similar to Brackets. Also free.

  • PSPad is a freeware text editor (it will ask you occasionally for a donation). I quite like it for Pass:[c++] code and other code since it has a quite nice built-in function navigator.

Appendix A: Using the LSDTopoToolsSetup.py script

This script has been replaced in LSDTopoTools version 2 with several scripts that are described in previous sections.

To help you set up LSDTopoTools, we have written a series of modules in python that automate the creation of our standard directory structure, downloading of the source code and compiling the source code into programs. The script is called LSDTopoToolsSetup.py.

A.1. Getting the LSDTopoTools setup script

LSDTopoTools works in all popular operating systems (Windows, MacOS, Linux) by using Vagrant to create a Linux server within your host operating system: if you are not in a native Linux environment you should start a Vagrant server by following these directions.

Once you have done that, you should be able to ssh into a Linux session using putty.exe on Windows or the vagrant ssh command in MacOS and Linux.

Once you are in a Linux operating system, the easiest way to get the script is to use the utility wget:

$ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py

A.1.1. Alternatives to wget

You can also get the script by cloning the LSDAutomation repository:

$ git clone https://github.com/LSDtopotools/LSDAutomation.git

or you can got to the raw text of the file, then copy the text and paste into your favourite text editor (i.e. Brackets). You should save the file as LSDTopoToolsSetup.py.

A.2. Running the LSDTopoTools setup script

The setup script is a python script so you need python to run it. It should be run in Linux, so if you do not have a native Linux operating system the script should be run in your Vagrant server. Almost all flavours of Linux come with python installed as default (the Ubuntu system installed by our vagrantfiles is one example) so we will assume you have python.

The script requires parameters; if you run the script with no parameters, it will give you a message telling you what to do:

$ python LSDTopoToolsSetup.py

=======================================================================
Hello there, I am the going to help you set up LSDTopoTools!
You will need to tell me what to do.
If you are in vagrant, LSDTopoTools will always be installed in the root directory (\).
If you are not in vagrant,
LSDTopoTools will be installed in your home directory if you run this with:
   python LSDTopoToolsSetup.py -id 0

Or your current directory.
(i.e., the directory from which you called this program)

if you run with this:
   python LSDTopoToolsSetup.py -id 1

For help type:
   python LSDTopoToolsSetup.py -h

=======================================================================

A.2.1. Looking at all the LSDTopoToolsSetup options

You can call help for the LSDTopoToolsSetup.py script with:

$ python LSDTopoToolsSetup.py -h

A.2.2. A default setup

In fact, there is no default setup, since you have to tell the tool where to install LSDTopoTools. You do this by using the -id flag.

  • If you select 0, LSDTopoTools will be installed in your home directory, which you can get to with the command cd ~.

  • If you select 1, LSDTopoTools will be installed in the current directory (i.e. the directory where the script is located).

  • If you are running a vagrant machine, these instructions will be ignored and LSDTopoTools will be installed in the root directory (you can get there with the command cd \).

This will install a directory LSDTopoTools with subdirectories Git_projects and Topographic_projects. Each of these folders will clone subfolders from our github pages, and will compile the Analysis_driver as well as the chi_tool.

The directories will look like this:

/LSDTopoTools
  |--Git_projects
  |----LSDTopoTools_AnalysisDriver
  |----LSDTopoTools_ChiMudd2014
  |--Topographic_projects
  |----LSDTT_workshop_data
  |----Test_data

A.2.3. Other setup options

There are a number of other options on LSDTopoToolsSetup.py, which we will list here, as well as in the table below.

  • -CE True Clone and compile the channel extraction programs used in Clubb et al, WRR, 2014.

  • -MChi True Clone and compile the tools for performing the chi analysis of Mudd et al., JGR-ES, 2014.

  • -CRN True Clone and compile the tools for calculating basinwide denudation rates from the cosmogenic nuclides 10Be and 26Al Mudd et al., ESURF, 2016.

  • -cp True Check the path names in all the subfolders in Topographic_projects to reflect your own directory structure.

A.2.4. If you are running on native Linux

If your native operating system is Linux, you should be able to get LSDTopoTools working very quickly using LSDTopoToolsSetup.py without having to use vagrant. However, you do need to ensure that you have the utilities make, the gnu Pass:[C++] compiler installed, and also you need FFTW installed. If you are in Ubuntu installing FFTW is as easy as: sudo apt-get install -y libfftw3-dev. If you are in another flavour of Linux you will need to follow the full setup, which is described on the FFTW website. Once you have that stuff working, you can use LSDTopoToolsSetup.py to get everything working in a jiffy.

A.3. Table of LSDTopoToolsSetup.py options

Table 1. Installation directory options
Flag Input type Description

-id

Integer, either 0 or 1

0 == home directory, 1 == current directory. If you are in vagrant it will ignore this option and install in the root directory.

-CRN

Boolean (either True or False)

If this is True, installs the CAIRN CRN package for calculating basinwide denudation rates from the cosmogenic nuclides 10Be and 26Al Mudd et al., ESURF, 2016.

-CE

Boolean (either True or False)

If this is True, installs the channel extraction programs used in Clubb et al, WRR, 2014.

-MChi

Boolean (either True or False)

If this is True, installs the tools for performing the chi analysis of Mudd et al., JGR-ES, 2014.

-cp

Boolean (either True or False)

If this is True, checks the pathnames of the parameter files .driver and .LSDTT_driver in all the subfolders of Topographic_projects and corrects them so they reflect the user’s directory structure.

A.4. The LSDTTParamfileUpdater.py script for updating directory names

We also provide a script for updating directory write and read names, called LSDTTParamfileUpdater.py. LSDTopoTools analyses require you to provide a parameter file that contains the instructions for running analyses. In that parameter file there are always instrutions for where you want to read the data from, and to where you want to write new data. These lines in the parameter file look like this:

read path: /my/directory/
write path: /my/directory/

It is tedious to keep updating these every time you move data to a new folder, so you run the LSDTTParamfileUpdater.py to ensure that the path names are the same as the data directory. The script assumes you want to put your results in the same directory and your data.

You can download this script with:

$ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTTParamfileUpdater.py

This script can take 0, 1 or 2 arguments.

  • With 0 arguments, the script updates the read and write path of every file in the current directory with the extensions .driver, .LSDTT_driver, and .param.

  • With 1 argument, the script assumes the argument is the directory of the data and updates the read and write path of every file in that directory with the extensions .driver, .LSDTT_driver, and .param.

  • With 2 arguments, the script assumes the first argument is the directory of the data and updates the read and write path of the file named by the second argument.

A.5. The LSDTT_path_adjuster.sh script for updating paths

We have included a bash script for modifying your terminal enviroment so that it "sees" the LSDTopoTools directories. This means that you can call LSDTopoTools programs from the data folders.

Calling this script only updates the path for the terminal session from which it is called. It doesn’t not permanently modify your terminal environment. You need to run the script every time you start a terminal session.

You can download this script with:

$ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTT_path_adjuster.sh

You run the script with:

$ bash LSDTT_path_adjuster.sh

If you get a permissions denied error, you can change these with:

$ chmod 0744 LSDTT_path_adjuster.sh

and then run the bash command once again.