1. Introduction

Welcome to the installation instructions for LSDTopoTools, which is a collection of programs for analysing topography and modelling hydrology, landscape evolution and soil development.

LSDTopoTools is built for Linux operating systems. Wait stop!! Where are you going? Come back!! We are going to get this to work on your computer, honest!

Most users will not use a Linux operating system as their day-to-day operating system; Windows and MacOS are much more popular. However, through the either virtualisation or containers you can build a Linux environment within any operating system. In fact, building either a virtual machine or a container has the advantage of ensuring that everyone using these systems has the same environment. This precludes the "works on my machine" problem that is common amongst scientific software packages.

In these documents you will find four flavours of LSDTopoTools installation instructions:

  • Installation using VirtualBox + Vagrant. This method is the most tested. You might consider this the default method.

  • Installation using Docker. This has fewer steps than the VirtualBox+Vagrant method. The Docker method and the VirtualBox method are not compatible on Windows machines (using Docker on Windows breaks VirtualBox). It only works on Windows 10 Enterprise and MacOS. If you have one of these systems you might consider Docker.

  • Native installation on a Linux system. If your native operating system is Linux you can just install LSDTopoTools after getting all the libraries you need. The documents are made under the assumption that you use Ubuntu, but if you are a Linux user we are confident you’ll be able to port the instructions to your native system (usually this involves replacing sudo apt-get with sudo yum).

  • University of Edinburgh installation. This software is developed at the University of Edinburgh and is used in teaching. We include instructions specific for Edinburgh students for use on the School of GeoSciences servers.

2. Installing LSDTopoTools using VirtualBox and Vagrant

Our most tested method uses two software packages, which are both free to download:

  • VirtualBox is software for creating virtual machines, which are implementations of operating systems that live in your host operating system (e.g., you can have a virtual Linux system operating within your Windows or MacOS operating system).

As of 01-Nov-2017 Vagrant does not work with VirtualBox 5.2. Make sure you download Virtualbox 5.1 instead: https://www.virtualbox.org/wiki/Download_Old_Builds_5_1
  • Vagrant is software for managing and provisioning virtual machines.

As of 22-Dec-2017 Vagrant 2.0.1 has conflicts with some ruby packages. Make sure you download Vagrant 2.0.0: https://releases.hashicorp.com/vagrant/2.0.0/

2.1. Quick instructions

Quick instructions for starting a Vagrant server with LSDTopoTools
  1. Download and install virtualbox. As of 01-Nov-2017 Vagrant does not work with VirtualBox 5.2. Make sure you download Virtualbox 5.1 instead: https://www.virtualbox.org/wiki/Download_Old_Builds_5_1

  2. Download and install vagrant. As of 22-Dec-2017 Vagrant 2.0.1 has conflicts with some ruby packages. Make sure you download Vagrant 2.0.0: https://releases.hashicorp.com/vagrant/2.0.0/ You might have to restart your computer after this. You need to install virtualbox before you install vagrant, or you might get errors.

  3. If you are on Windows, download putty.exe. If you are on Linux or MacOS you can skip this because they have built-in equivalents.

  4. Make a directory for your vagrant box. We tend to put ours in a directory called VagrantBoxes.

  5. Inside that directory make two new directories: LSDTopoTools and Ubuntu. The second directory’s name doesn’t matter, it is just for holding a vagrant file (see below). However you MUST have the LSDTopoTools directory. The directory name is case sensitive!

  6. Download one of our vagrantfiles: https://github.com/LSDtopotools/LSDTT_vagrantfiles into the Ubuntu directory (again, the name of the directory doesn’t matter).

  7. Rename the vagrantfile from the repo (either Vagrantfile_32bit_FFTW or Vagrantfile_64bit_FFTW) simply vagrantfile. Your operating system is almost certainly 64 bit, but on most computers you need to select 32 bit because the default setting is to disable 64 bit guest operating systems. This can be changed but only by expert users.

  8. Open a terminal or powershell window and navigate to the directory with the vagrantfile.

  9. Run vagrant up from the command line.

    If you are running vagrant up for the first time it can take some time to download the base box. They are several hundred Mb each!
  10. Run vagrant provision after the box has started.

  11. If on Windows, you should now be able to use putty.exe to ssh into your LSDTopoTools server. The host name is almost always 127.0.0.1 and the port is almost always 2222.

  12. On Windows, you will need to give a username and password after connecting using putty.exe. The machine is running locally on your computer so nothing is being sent over the internet. The username is always vagrant and the password is also vagrant.

  13. If you are on MacOS or Linux you do not need putty.exe; all you need to do is type vagrant ssh into the command line. See the vagrant instructions.

Quick instructions for setting up directories in your vagrant box
  1. Use putty.exe (in Windows) or vagrant ssh (on MacOS/Linux) to log into your vagrant server.

  2. Go to the LSDTopoTools directory by typing cd /LSDTopoTools

  3. Type python LSDTopoToolsSetup.py -id 0

  4. You are ready to go for some basic analysis!

2.2. Full instructions on setting up our Vagrant servers

2.2.1. First steps: Starting a Vagrant box

You will need sufficient space on your hard disk to host a guest operating system. You also need room for the LSDTopoTools dependencies. You will struggle if you have less than 5Gb free.

Vagrant is software that automates the creation and provisioning of virtual machines. What does that mean? It means that you will create a Linux server that runs inside of your day-to-day computer. This server will run even if you are using a different operating system (e.g., Windows). Vagrant machines can be configured using a vagrantfile, so you download our vagrantfile and you simply point vagrant to it and should get a working server that can run LSDTopoTools.

  1. You need software for running virtual machines. We recommend virtualbox since it is both well supported and free. Download and install. Our instructions assume you are using virtual box. As of 01-Nov-2017 Vagrant does not work with VirtualBox 5.2. Make sure you download Virtualbox 5.1 instead: https://www.virtualbox.org/wiki/Download_Old_Builds_5_1

  2. Download and install Vagrant. As of 22-Dec-2017 Vagrant 2.01 has conflicts with some ruby packages. Make sure you download Vagrant 2.0.0: https://releases.hashicorp.com/vagrant/2.0.0/

  3. Vagrant works via command line, so you will need to know how to open a terminal on OS X, Linux (usually you can open one using ctrl-alt-T, but if you use Linux that means you were born knowing how to open a terminal), or a Windows powershell.

  4. If you are working on Windows, you will probably have to restart after installing Vagrant so that Windows can register the path to Vagrant.

  5. Okay, we now assume you have installed everything and are in a terminal or powershell. You need to make a directory where you keep information about your vagrant boxes. I made a folder names vagrantboxes and then subfolders for different boxes.

  6. If you are in Windows, you will need an ssh utility to communicate with your vagrant box. You should download putty.exe from the putty website. In Linux and MacOS ssh utilities are already installed.

  7. Now you should fetch one of our vagrantfiles from our git repo: https://github.com/LSDtopotools/LSDTT_vagrantfiles

    Get the Vagrant Files
    1. The 32 bit file is here: https://raw.githubusercontent.com/LSDtopotools/LSDTT_vagrantfiles/master/Vagrantfile_32bit_FFTW

    2. The 64 bit file is here: https://raw.githubusercontent.com/LSDtopotools/LSDTT_vagrantfiles/master/Vagrantfile_64bit_FFTW

    3. Although most computers today are 64 bit, most of them, as default, disable guest 64 bit operating systems. Unless you know how to turn this on (for example see this link), you should choose the 32 bit vagrantfile.

    4. Save one of these files into the directory for your vagrant files. See below for the appropriate directory structure.

  8. Rename the vagrantfile from the repo (either Vagrantfile_32bit_FFTW or Vagrantfile_64bit_FFTW) simply vagrantfile

  9. If you use our vagrant files, you will need to make a directory LSDTopoTools in the same directory as your folders for different vagrant boxes. For example, you might make a directory C:\VagrantBoxes\, and in that directory you can put both LSDTopoTools and Ubuntu32_FFTW (or some such name) directories. You will put the vagrant file in the Ubuntu32_FFTW directory. Your tree might look a bit like this:

    C:\vagrantboxes\
    |--Ubuntu32_FFTW
       |-- vagrantfile
    |--Ubuntu64_FFTW
       |-- vagrantfile
    |--LSDTopoTools
    It is ESSENTIAL that the LSDTopoTools folder is present and is one directory level lower than the vagrant file. If this is not true, the vagrant machine will NOT WORK.
    In the above file structures the vagrantfiles have been renamed from the vagrant files in our repository.
  10. Go into the folder with the operating system you want (e.g. Ubuntu32_FFTW):

    PS: > cd C:\VagrantBoxes
    PS: > cd C:\Ubuntu32_FFTW
  11. Now start your vagrant box (this might take some time since it has to fetch stuff from the internet):

    PS: > vagrant up
    You do not need to download a "base box" (that is a Linux operating system, in this case 32 bit Ubuntu) before you run vagrant up: Vagrant does this for you. However if you are running vagrant up for the first time Vagrant will download the box for you which will take some time (it is ~400Mb). You will only need to download the base box once.
  12. Congratulations! You now have a functioning Vagrant box!! Now you need to log on to the box.

    If you want to update the base box you can use vagrant box update command from the powershell or terminal windows.

2.2.2. Logging on to your Vagrant box

  1. All right! Your Vagrant box is running. Other than a sense of vague accomplishment, this doesn’t really help you run LSDTopoTools. You need to log on to the box. You will operate your vagrant box as a server: you log into the machine and run code on it, but you won’t have pretty windows to look at. You will run everything through an ssh terminal, using a command line interface.

  2. We do this using ssh.

    A note on your vagrant ssh server
    • ssh allows you to communicate securely with a server on an unsecured connection (it encrypts communication between you and the server).

    • You will use ssh to communicate with your Vagrant server. This server is not on the internet but rather is living on your computer.

    • Vagrant is clever in that it sets up an IP address for your vagrant server (in other words your Linux machine living on your host computer, which could be Windows, Linux or OSX), and as such ssh can establish a connection to this machine via ssh.

    • Vagrant’s default settings are to set your server up to sit on host 127.0.0.1 and port 2222. You will need to use these settings in putty.exe

    • When you first log in, putty or other ssh clients will ask you to cache a new host key.

  3. If you are starting from a Linux or OSX machine, an ssh client is built into your command prompt and you can just type vagrant ssh into the command prompt.

Windows login
  1. If you are on Windows, you need to download putty.exe and run it.

  2. In putty, set the host to 127.0.0.1 and the port to 2222. These are vagrant’s default settings.

Windows using the old version (precise pangolin)
  1. You will need to add the RSA key to your cache (just say yes: remember you are not connecting to the internet where baddies can spy on you but rather a server running on your own computer).

  2. Now you need to log in. Your vagrant box has a username of vagrant and a password of vagrant.

Windows using the new version (precise pangolin)
  1. Using the new version (which has a Precise Pangolin box: this is supported until 2021).

  2. This seems to upset putty.exe. Here is how to fix that.

  3. Run vagrant up.

  4. Then download puttygen.exe from the putty website.

  5. Run puttygen and load an existing key. Vagrant makes a private key called private_key.

    1. It will be in your vagrantboxes folder. Somewhere like C:/Vagrant/Ubuntu32/.vagrant/machines/default/virtualbox/

    2. Click on load in puttygen.exe. You need to make sure it is looking for all files and not putty key files (ppk).

  6. Once you load that you can put in your own private passphrase.

  7. Then save this key as something sensible (like private_key_putty.ppk).

  8. Now in putty look on the right with all the setting options and click on Connection → ssh.

  9. Then click on Auth and then browse for your private key.

  10. Then click on session and save this as Vagrant_xenial or something like that.

  11. When you ssh in using putty you will need to use vagrant as the username but then you use your private passphrase to get on the system.

2.2.3. Your Vagrant box and file syncing

  1. So you are logged in. Now what? It turns out Vagrant has done some clever things with your files.

  2. Vagrant can sync folders across your Vagrant box and your host computer (that is, the computer you started vagrant from).

  3. When you log in to your vagrant box, you will not be in the same folder where I have built the LSDTopoTools file structures. You need to navigate down to this:

    $ pwd
    /STUFF
    $ cd ..
    $ cd ..
    $ pwd
    /STUFF
    $ cd LSDTopoTools
    $ ls
    STUFF

    You can also jump directly there:

    $ cd /LSDTopoTools
  4. As you can see above, the LSDTopoTools folder contains folders for different LSDTopoTools packages, for topographic datasets.

  5. Here is the amazing thing: the files that are in LSDTopoTools folder in your vagrant box ARE ALSO visible, and synced, in your host computer. So if you use LSDTopoTools to do some analysis within your vagrant box, you will be able to see the files within your host computer as well. This means that you can, for example, do a Linux-based LSDTopoTools analysis and then plot that analysis in a GIS on your host windows box without having to transfer files. Not only that, but you can modify the code, update python scripts, change parameter files, etc., with your favourite text editor in Windows (or MacOS, or whatever) and those files will be visible to your Vagrant box. Fantastic!

2.2.4. Updating to the latest versions of the software

To check out the latest version of our software you can run the vagrant provision command

PS: > vagrant up
PS: > vagrant provision

2.2.5. Shutting things down

When you are finished with your session, you just need to go into the powershell or a terminal and type:

PS: > vagrant halt

2.2.6. If you want to start from scratch

If you want to remove the virtual machine, start it up and then run vagrant destroy:

PS: > vagrant up
PS: > vagrant destroy

2.3. Troubleshooting a vagrant server

There are a few common problems people have when running a vagrant server.

2.3.1. Vagrant up doesn’t work

  • The first time you run vagrant up you will need to be connected to the internet as you need to download the base box (which is ~700Mb).

  • If you are on an old computer, sometimes vagrant times out before the virtual machine boots. This most frequently happens the first time you boot a vagrant machine. The most effective way to fix this is with the canonical IT solution: turning it off and on again. To do that run vagrant halt and vagrant up in succession.

2.3.2. I closed the window running vagrant: how do I shut it down?

  • Open virtualbox: you will see the vagrant machine. Simply power it off (from within virtualbox).

2.3.3. Vagrant hangs up in powershel or terminal window

  • If vagrant hangs up in the powershell or terminal window and does not give you back the command prompt, turn it off and on again by typing ctrl-c and then running the vagrant command again.

2.3.4. File syncing doesn’t work

  • If your files are not syncing across your host and vagrant machine, it is probably because there is some misspelling in your LSDTopoTools folder on the host machine. Make sure that folder is in the correct place and is spelled correctly (remember it should be case sensitive!!).

  • This also has emerged because of a conflict with the latest VirtualBox versions. Make sure you have VirtualBox 5.1 or lower. As of December 2017 VirtualBox 5.2 doesn’t work.

2.3.5. I get some crazy conflict about Ruby when I run vagrant up

  • This is an error associated with the latest Vagrant version. You need version 2.0.0 or lower. As of January 2018 Vagrant 2.0.1 doesn’t work.

2.3.6. I ran vagrant up. Now what?

  • You need to log on to the vagrant machine with either putty.exe (Windows) or by typing vagrant ssh (Linux, MacOS) into the command line. See instructions for more details.

2.4. Technical details on vagrant

Here are some technical details on vagrant. Hopefully you will never have to read any of this.

2.4.1. Brief notes for setting up your own Vagrant server

This section is for customising your vagrant environment (or rather, your Ubuntu environment that vagrant sets up for you) and can be safely ignored by 95% of LSDTopoTools users. We include the below notes for obsessive hackers who have nothing better to do.

We have written Vagrant files for you so you don’t have to customise your working environment, but if you want to set up your own Vagrant boxes with your own software here are some notes.

Initiating a Vagrant box
  1. Go into an empty folder to start a new Vagrant box.

  2. Initiate Vagrant with:

    PS> C:\> vagrant init

    Alternatively you can initiate with a base box. In this example we use the Ubuntu precise 32 base box:

    PS> C:\vagrant init ubuntu/precise32
  3. This command (init) will simply make a vagrant file. To get the server up and running you need to up it. Before you do that you probably want to modify the vagrant file.

  4. One of the things you probably need to modify is the memory assigned to your guest vagrant box. In the vagrant file you should have:

      config.vm.provider "virtualbox" do |vb|
        # Customize the amount of memory on the VM:
        vb.memory = "3000"
      end

    The default memory is something small, and the problem with it is that it will take the guest operating system too long to boot, and vagrant will time out. I would give the vagrant box 3-4 Gb of memory.

  5. Now you can up your vagrant box. In the folder with the vagrant file, type:

    PS> vagrant up
  6. If this is the first time booting the linux machine, this will take a while.

Notes on the base box

Vagrant sets up a Linux server living in your computer (it is called the Host computer). The server will run a Linux operating system, and you need to choose a functioning base system vagrant base boxes. Here we have started with ubuntu/precise32. You might want to try other base boxes, they can be found at the atlas website.

If you choose a base box that you do not already have (and you start with none), vagrant will download it. They are big!! Usually over 500Mb (each is a fully operational linux operating system). You will either need a fast internet connection or a lot of time. Make sure you also have enough room on your hard disk.

You do need to be careful with base boxes!

Not all base boxes work! On many windows machines, you can only run a 32 bit version of linux, even though you are almost certainly running 64 bit windows. You can change this by going into your BIOS and changing the settings, but that is dangerous and if you do not know what your BIOS is do not even think about attempting to change these settings.

In testing, I found many bases boxes did not work at all. The one that worked well for me was the ubuntu/precise32 box. You can just vagrant init an empty vagrant instance and change the box in the vagrantfile with config.vm.box = "ubuntu/precise32".

You can update your base box with the command vagrant box update.

Details of provisioning

If you change your vagrantfile with the box still running, you can run the new provisioning with:

PS> vagrant provision

If you have downloaded our vagrant files, the provisioning of your virtual server should be automatic. However, you may wish to know what is happening during the provisioning, so here are some notes.

To install software, we use the shell provisioning system of vagrant. This should go into the vagrantfile and will look a bit like this:

  config.vm.provision "shell", inline: <<-SHELL
    sudo apt-get update
    sudo apt-get install -y git
  SHELL

In the above shell command, we are installing git. The -y flag is important since apt-get will ask if you actually want to download the software and if you do not tell it -y from the shell script it will just abort the installation.

You sync folders like this:

  config.vm.synced_folder "../LSDTopoTools", "/LSDTopoTools"

Were the first folder is the folder on the host machine and the second is the folder on the Vagrant box.

3. Installing LSDTopoTools using Docker

3.1. Should I use Docker or Vagrant?

There are many, many, many articles comparing vagrant to docker. The docker setup is probably a bit simpler than the vagrant setup but (and this is a big but) it won’t work on a Windows machine unless you have Windows 10 Enterprise. It does work on any recent version of Linux and MacOS. However the developers of LSDTopoTools don’t have access to Windows 10 Enterprise so can’t test the Docker versions on Windows, so we can’t offer support of this installation option. However if you have MacOS, native Linux or Windows 10 Enterprise you should consider using Docker since it involves fewer steps than our VirtualBox+Vagrant setup.

3.2. The LSDTopoTools docker container

Docker is software for creating and managing containers. These containers are not complete operating systems, like one creates using VirtualBox and Vagrant, so the boot time is quicker. Docker is perhaps simpler to use but does not work on every operating system.

The instructions here are a slight modification of instructions from Aaron Friel.

To set up LSDTopoTools using Docker:

  1. Download and install Docker for Windows (only works with Windows 10 enterprise), Docker for Mac, or Docker for Ubuntu or Debian.

  2. On Mac we recommend installing docker using brew: brew cask install docker

  3. Create a folder where you are going to put files associated with docker. In the examples I will assume these are in C:\DockerBoxes\LSDTopoTools\.

  4. Run the following command in a command prompt or terminal for Mac or Linux (if you use Linux I am sure you don’t need instruction on how to open a terminal):

    docker run --rm -it -v C:\DockerBoxes\LSDTopoTools\:/LSDTopoTools afriel/lsdtopotools

    Note that you will need to modify the path (here C:\DockerBoxes\LSDTopoTools\) to reflect your own directory where you want your LSDTopoTools files.

  5. You should get a prompt that looks like this, perhaps with some different letters and numbers after the @:

root@9f73030d8598:/LSDTopoTools#
  1. If this is your first time using the tools and your LSDTopoTools directory is empty, you will need to run the install script. In the prompt, enter: install.sh and hit enter.

  2. Some users have reported permissions problems with running install.sh on a fresh docker install. If this happens you need to run this command chmod 0744 /bin/install.sh to update the permissions on the install script and then run install.sh as normal.

  3. Follow docker documentation for your operating system to start and shut down docker containers.

4. Installing natively on a Linux system

Our software is designed for linux operating systems but we do recommend you use either our VirtualBox+Vagrant setup or our Docker setup because in those two cases you will be using the same environment as the developers so it will be easier to answer questions when you have problems. However, if you just want LSDTopoTools on your Linux machine, follow these steps:

  1. Start a terminal session.

  2. Install some essential softare (we will use Ubuntu installation, you need to modify these if you are on another version of Linux). This assumes you have the GNU c++ tools:

    $ sudo apt-get install -y git
    $ sudo apt-get install -y gdal-bin
    $ sudo apt-get install -y python-gdal
    $ sudo apt-get install -y libfftw3-dev
    $ sudo apt-get install -y cmake
  3. Go the the directory where you want to install LSDTopoTools.

  4. Download our setup script.

    $ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py
  5. Run the setup tool:

    $ python LSDTopoToolsSetup.py -id 1
    The -id flag tells LSDTopoToolsSetup.py where to put the files. In vagrant it always starts in the root directory (this is for file syncing purposes). However, if your native operating system is Linux, then you can either install in your home directory (-id 0) or in the directory where you called LSDTopoToolsSetup.py (-id 1).
  6. If wget doesn’t work, you can follow the link: https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py
    Copy the text, paste it into a text editor and save it as LSDTopoToolsSetup.py.

  7. The LSDTopoToolsSetup.py tool has a number of options: if you want a preview of what it does, you can call its help options:

    $ python LSDTopoToolsSetup.py -h

5. Installing LSDTopoTools at the University of Edinburgh.

These instructions are for students and staff within the School of GeoSciences at the University of Edinburgh.

Quick Setup for Edinburgh Geoscience users
  1. Start an nx session.

  2. Open a terminal window.

  3. Go to your home directory: cd ~

  4. Download the setup tool:

    $ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py
  5. Run the setup tool by typing

    $ python LSDTopoToolsSetup.py -id 0
  6. See instructions below for testing the tool.

5.1. Mapping your network drive

You will need access to your data on the network drive to use LSDTopoTools. You only need to do this one in any computer lab (and possibly only once ever):

  1. Open a file explorer window and under computer select the "University drive":

    The U drive
    Figure 1. The university drive
  2. Then go into the Science and Engineering folder ("SCE"):

    SCE
    Figure 2. The science and engineering folder
  3. Then into the GeoSciences folder ("GEOS"):

    GEOS
    Figure 3. The geosciences folder
  4. Then into the Drive Mapping folder ("drive_mapping"):

    Drive mapping
    Figure 4. The drive mapping folder
  5. And finally double click the shortcut "Access Netdata":

    Access netdata
    Figure 5. The access netdata shortcut

5.2. Getting on to our servers with NX

  1. We will be working on the University of Edinburgh servers.

  2. First you should start an NX session. NX is installed on all the computer labs in GeoScience (and in many other computer labs) Follow these instructions: https://www.ed.ac.uk/geosciences/intranet/it/linux-servers/nx

  3. You will get something that looks like a desktop. Right click anywhere in the desktop and select "open terminal".

5.3. Using the LSDTopoTools setup script

  1. Before you do anything you need to move onto a decent server (the ssh server is only for logging in). Type either:

    $ ssh burn

    OR

    $ ssh achray
  2. In this window type cd ~. This takes you to your home directory.

  3. Download our setup tool by typing:

    $ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py
  4. Run the setup tool by typing

    $ python LSDTopoToolsSetup.py -id 0
  5. You should now have LSDTopoTools directories installed in your home directory. You can always get to your home directory by typing cd ~.

5.4. Testing the tool on Edinburgh servers

Now, if you want to test the tool:

  1. Go to the example data directory: cd ~/LSDTopoTools/Topographic_projects/LSDTT_workshop_data/

  2. You want to add the path to the main program:

    $ PATH=$PATH:~/LSDTopoTools/Git_projects/LSDTopoTools_AnalysisDriver/Analysis_driver;export PATH
    This update to your path is only valid for your current terminal session. You will need to enter this command each time you start a new session.
  3. Download the path modification tool:

    $ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTTParamfileUpdater.py
  4. run the tool

    $ python LSDTTParamfileUpdater.py
  5. Okay, now tes the program:

    $ LSDTT_BasicMetrics.out . WA_BasicMetrics01.driver
  6. If the program does some things and then says I’m all finished! Have a nice day. it means it has worked!

6. Extra dependencies for swaths and point clouds

Some of our tools require the point cloud library. To get this working you need a few different utilities and libraries (take a deep breath):

  1. The cmake utility. This is like make but is required for our tools that examine point clouds, since it is required by something called the point cloud library.

  2. pcl: The Point Cloud Library.

  3. libLAS: a library for working with LAS format data.

Our Vagrant installation installs these automatically. The docker installation (since it is 3rd party) does not. However you can install on an Ubuntu system (and the Docker installation is Ubuntu) using the following instructions.

6.1. Install the Point Cloud Library

If you are using an older version of ubuntu there is a package. Newer versions of Ubuntu don’t have it, so you will unfortunately need to install by hand.

6.1.1. Ubuntu 14 and 12

This is relatively painless (we hope):

$ sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl
$ sudo apt-get update
$ sudo apt-get install libpcl-all

6.1.2. Ubuntu 16.04 LTS

First, try this. It is possible it will only work on 64 bit systems:

$ sudo apt-get update
$ sudo apt-get install libpcl-dev

If that doesn’t work you need to install from source. I am very sorry about this.

To install the pcl library from source you use the following steps:

This will take at least an hour!! It will also download an insane amount of stuff onto your machine. You must ensure that you follow the steps in the correct order.

Install oracle-java8-jdk:

$ sudo add-apt-repository -y ppa:webupd8team/java && sudo apt update && sudo apt -y install oracle-java8-installer

Install various dependencies and pre-requisites:

$ sudo apt -y install g++ cmake cmake-gui doxygen mpi-default-dev openmpi-bin openmpi-common libusb-1.0-0-dev libqhull* libusb-dev libgtest-dev
$ sudo apt -y install git-core freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libphonon-dev libphonon-dev phonon-backend-gstreamer
$ sudo apt -y install phonon-backend-vlc graphviz mono-complete qt-sdk libflann-dev libflann1.8 libboost1.58-all-dev

Install the eigen linear algebra package:

$ cd ~/Downloads
$ wget http://launchpadlibrarian.net/209530212/libeigen3-dev_3.2.5-4_all.deb
$ sudo dpkg -i libeigen3-dev_3.2.5-4_all.deb
$ sudo apt-mark hold libeigen3-dev

Install VTK (this has some plotting functionality, this takes AGES.):

$ wget http://www.vtk.org/files/release/7.1/VTK-7.1.0.tar.gz
$ tar -xf VTK-7.1.0.tar.gz
$ cd VTK-7.1.0 && mkdir build && cd build
$ cmake ..
$ make
$ sudo make install

Now install the Point Cloud Library v1.8:

$ cd ~/Downloads
$ wget https://github.com/PointCloudLibrary/pcl/archive/pcl-1.8.0.tar.gz
$ tar -xf pcl-1.8.0.tar.gz
$ cd pcl-pcl-1.8.0 && mkdir build && cd build
$ cmake ..
$ make
$ sudo make install

Do some clean up:

$ cd ~/Downloads
$ rm libeigen3-dev_3.2.5-4_all.deb VTK-7.1.0.tar.gz pcl-1.8.0.tar.gz
$ sudo rm -r VTK-7.1.0 pcl-pcl-1.8.0

You should now have pcl installed on your system! You can do a small test of the installation if you want to:

$ cd ~
$ mkdir pcl-test && cd pcl-test

Create a CMakeLists.txt file and save this in the directory pcl-test:

cmake_minimum_required(VERSION 2.8 FATAL_ERROR)
project(pcl-test)
find_package(PCL 1.2 REQUIRED)

include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})

add_executable(pcl-test main.cpp)
target_link_libraries(pcl-test ${PCL_LIBRARIES})

SET(COMPILE_FLAGS "-std=c++11")
add_definitions(${COMPILE_FLAGS})

Create a file and call it main.cpp:

#include <iostream>

int main() {
    std::cout << "hello, world!" << std::endl;
    return (0);
}

Compile the cpp file and then test it :

$ mkdir build && cd build
$ cmake ..
$ make
$ ./pcl-test

Output should be hello, world!

6.2. Extra libraries for the floodplain and terrace extraction

As of January 2018 liblas does not compile on our vagrant server since liblas3 is not available for Ubuntu 12. We have updated the vagrantfiles to Ubuntu 14 so you should use the new vagrantfiles (this will require downloading a new vagrant box if you were using the old vagrantfile). We don’t use Ubuntu 16 since it is such an incredible pain to install the point cloud library on it.

You might find you need to install a couple of other libraries for the floodplain and terrace drivers to compile properly. If you are using Ubuntu, you can do this using the following commands:

$ sudo apt-get update
$ sudo apt-get install libgoetiff-dev
$ sudo apt-get install liblas-dev

7. Installing the LSDTopoTools python toolchain

LSDTopoTools is written in C++ but also includes a number of automation and visualisation rountines written in python.

Because LSDTopoTools handles primarily spatial data, a number of python packages for handling spatial data are needed in the LSDTopoTools python toolchain. We have attempted to make setting up this toolchain as painless as possible using miniconda and conda environments.

Please see the chapter on LSDTopoTools visualisation for instructions on how to install the LSDTopoTools python toolchain.

8. Get a text editor!

A text editor isn’t part of LSDTopoTools but you need one to do anything. You will need to modify parameter files and you might even get to the stage of opening Pass:[c++] or Python scripts and editing them so you will need an editor.

If you are Linux native you probably chortle superiorly if you lean someone doesn’t use vi to modify files, but most students have usually only interacted with text files via a word processor.

A word processor (e.g., Word, or whatever fasionable software is on your fruit-based computer) will not work to modify code: these convert your text into binary files and add a bunch of formatting.

You need to work with raw ascii files, and for that you need a text editor. We like these:

  • Brackets is a multi platform text editor (it works on all operating systems) and is free. It has nice file navigation. This documentation was written using Brackets.

  • Atom is another multi platform editor. Similar to Brackets. Also free.

  • PSPad is a freeware text editor (it will ask you occasionally for a donation). I quite like it for Pass:[c++] code and other code since it has a quite nice built-in function navigator.

Appendix A: Using the LSDTopoToolsSetup.py script

To help you set up LSDTopoTools, we have written a series of modules in python that automate the creation of our standard directory structure, downloading of the source code and compiling the source code into programs. The script is called LSDTopoToolsSetup.py.

A.1. Getting the LSDTopoTools setup script

LSDTopoTools works in all popular operating systems (Windows, MacOS, Linux) by using Vagrant to create a Linux server within your host operating system: if you are not in a native Linux environment you should start a Vagrant server by following these directions.

Once you have done that, you should be able to ssh into a Linux session using putty.exe on Windows or the vagrant ssh command in MacOS and Linux.

Once you are in a Linux operating system, the easiest way to get the script is to use the utility wget:

$ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py

A.1.1. Alternatives to wget

You can also get the script by cloning the LSDAutomation repository:

$ git clone https://github.com/LSDtopotools/LSDAutomation.git

or you can got to the raw text of the file, then copy the text and paste into your favourite text editor (i.e. Brackets). You should save the file as LSDTopoToolsSetup.py.

A.2. Running the LSDTopoTools setup script

The setup script is a python script so you need python to run it. It should be run in Linux, so if you do not have a native Linux operating system the script should be run in your Vagrant server. Almost all flavours of Linux come with python installed as default (the Ubuntu system installed by our vagrantfiles is one example) so we will assume you have python.

The script requires parameters; if you run the script with no parameters, it will give you a message telling you what to do:

$ python LSDTopoToolsSetup.py

=======================================================================
Hello there, I am the going to help you set up LSDTopoTools!
You will need to tell me what to do.
If you are in vagrant, LSDTopoTools will always be installed in the root directory (\).
If you are not in vagrant,
LSDTopoTools will be installed in your home directory if you run this with:
   python LSDTopoToolsSetup.py -id 0

Or your current directory.
(i.e., the directory from which you called this program)

if you run with this:
   python LSDTopoToolsSetup.py -id 1

For help type:
   python LSDTopoToolsSetup.py -h

=======================================================================

A.2.1. Looking at all the LSDTopoToolsSetup options

You can call help for the LSDTopoToolsSetup.py script with:

$ python LSDTopoToolsSetup.py -h

A.2.2. A default setup

In fact, there is no default setup, since you have to tell the tool where to install LSDTopoTools. You do this by using the -id flag.

  • If you select 0, LSDTopoTools will be installed in your home directory, which you can get to with the command cd ~.

  • If you select 1, LSDTopoTools will be installed in the current directory (i.e. the directory where the script is located).

  • If you are running a vagrant machine, these instructions will be ignored and LSDTopoTools will be installed in the root directory (you can get there with the command cd \).

This will install a directory LSDTopoTools with subdirectories Git_projects and Topographic_projects. Each of these folders will clone subfolders from our github pages, and will compile the Analysis_driver as well as the chi_tool.

The directories will look like this:

/LSDTopoTools
  |--Git_projects
  |----LSDTopoTools_AnalysisDriver
  |----LSDTopoTools_ChiMudd2014
  |--Topographic_projects
  |----LSDTT_workshop_data
  |----Test_data

A.2.3. Other setup options

There are a number of other options on LSDTopoToolsSetup.py, which we will list here, as well as in the table below.

  • -CE True Clone and compile the channel extraction programs used in Clubb et al, WRR, 2014.

  • -MChi True Clone and compile the tools for performing the chi analysis of Mudd et al., JGR-ES, 2014.

  • -CRN True Clone and compile the tools for calculating basinwide denudation rates from the cosmogenic nuclides 10Be and 26Al Mudd et al., ESURF, 2016.

  • -cp True Check the path names in all the subfolders in Topographic_projects to reflect your own directory structure.

A.2.4. If you are running on native Linux

If your native operating system is Linux, you should be able to get LSDTopoTools working very quickly using LSDTopoToolsSetup.py without having to use vagrant. However, you do need to ensure that you have the utilities make, the gnu Pass:[C++] compiler installed, and also you need FFTW installed. If you are in Ubuntu installing FFTW is as easy as: sudo apt-get install -y libfftw3-dev. If you are in another flavour of Linux you will need to follow the full setup, which is described on the FFTW website. Once you have that stuff working, you can use LSDTopoToolsSetup.py to get everything working in a jiffy.

A.3. Table of LSDTopoToolsSetup.py options

Table 1. Installation directory options
Flag Input type Description

-id

Integer, either 0 or 1

0 == home directory, 1 == current directory. If you are in vagrant it will ignore this option and install in the root directory.

-CRN

Boolean (either True or False)

If this is True, installs the CAIRN CRN package for calculating basinwide denudation rates from the cosmogenic nuclides 10Be and 26Al Mudd et al., ESURF, 2016.

-CE

Boolean (either True or False)

If this is True, installs the channel extraction programs used in Clubb et al, WRR, 2014.

-MChi

Boolean (either True or False)

If this is True, installs the tools for performing the chi analysis of Mudd et al., JGR-ES, 2014.

-cp

Boolean (either True or False)

If this is True, checks the pathnames of the parameter files .driver and .LSDTT_driver in all the subfolders of Topographic_projects and corrects them so they reflect the user’s directory structure.

A.4. The LSDTTParamfileUpdater.py script for updating directory names

We also provide a script for updating directory write and read names, called LSDTTParamfileUpdater.py. LSDTopoTools analyses require you to provide a parameter file that contains the instructions for running analyses. In that parameter file there are always instrutions for where you want to read the data from, and to where you want to write new data. These lines in the parameter file look like this:

read path: /my/directory/
write path: /my/directory/

It is tedious to keep updating these every time you move data to a new folder, so you run the LSDTTParamfileUpdater.py to ensure that the path names are the same as the data directory. The script assumes you want to put your results in the same directory and your data.

You can download this script with:

$ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTTParamfileUpdater.py

This script can take 0, 1 or 2 arguments.

  • With 0 arguments, the script updates the read and write path of every file in the current directory with the extensions .driver, .LSDTT_driver, and .param.

  • With 1 argument, the script assumes the argument is the directory of the data and updates the read and write path of every file in that directory with the extensions .driver, .LSDTT_driver, and .param.

  • With 2 arguments, the script assumes the first argument is the directory of the data and updates the read and write path of the file named by the second argument.

A.5. The LSDTT_path_adjuster.sh script for updating paths

We have included a bash script for modifying your terminal enviroment so that it "sees" the LSDTopoTools directories. This means that you can call LSDTopoTools programs from the data folders.

Calling this script only updates the path for the terminal session from which it is called. It doesn’t not permanently modify your terminal environment. You need to run the script every time you start a terminal session.

You can download this script with:

$ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTT_path_adjuster.sh

You run the script with:

$ bash LSDTT_path_adjuster.sh

If you get a permissions denied error, you can change these with:

$ chmod 0744 LSDTT_path_adjuster.sh

and then run the bash command once again.