LSDTopoTools for Geomorphology, Hydrology, Ecology and Environmental Sciences

You will be manipulating text files so you need a decent text editors. There are many options but we like Brackets. Atom is also a nice alternative.

Git is version control software. Version control software helps you keep track of changes to your scripts, notes, papers, etc. It also facilitates communication and collaboration through the online communities github and bitbucket.

We post updated versions of our software to the Github site https://github.com/LSDtopotools. We also post version of the software used in publications on the CSDMS github site: https://github.com/csdms.

It is possible to simply download the software from these sites but if you want to keep track of our updates or modify the software it will be better if you have git installed on your computer.

You will need a compiler to build the software, as it is written in c++. In addition you will need a few tools to go along with the compiler. If you use our Vagrant setup these are installed for you. The things you really need are:

In addition the TNT library is required, but this doesn’t require installation and we package it with our software releases. If you are wondering what it is when you download our software, it is used to do linear algebra and handle matrices.

In addition, there are a few isolated bits of the code that need these other components. Most users will not need them, but for complete functionality they are required. First, some of our makefiles include flags for profiling and debugging. We try to remove these before we release the code on Github, but every now and then one sneaks through and the code won’t compile if you don’t have a debugger or profiler. It might save you some confusion down the line if you install:

Next, there are a few specialized tools that are only required by some of our more advanced components.

The Geospatial Data Abstraction Library has fantastic tools for preparing your data. It performs operations like clipping data, patching data together, resampling data, reprojecting data and doing coordinate transformations. If you don’t know what those things are, don’t worry, we explain these things in the preliminary steps chapter.

You can install all of GDAL if you want, but really you will only need their utilities.

This is included in the vagrant distribution.

Python is a programming language used by many scientists to visualize data and crunch numbers. It is NOT included in our Vagrant setup and our python scripts will not work in the vagrant server (because they make figures, and the server does not come with a windowing system). Therefore, you need to intall python on your host operating system. We use it for visualization, and also for automating a number of tasks associated with topographic analysis.

Instructions on how to install python are in this section: Getting python running.

You will need:

The industry standard GIS is ArcGIS, and if you are at a university you might have a site license for this software. It is not so easy to get on a personal computer, however, so there are a number of open source options that can be used as an alternative.

One alternative, and the one that will be used in these tutorials, is QGIS.

If you are familiar with ArcMap, you should be able to become proficient at QGIS in a few days. In my experience, it also has the advantage of being more stable (i.e., it crashes less) than ArcMap.

One thing that is quite nice about QGIS is the number of plugins that are available.

You should download and install QGIS from their website, and click on the `Plugins tab to get some plugins. the OpenLayers plugin, which allows you to quickly load satellite and map information from all sorts of vendors.

This book, and various other notes and websites associated with the LSDTopoTools project, have been built using something called asciidoctor. Asciidoctor is used to produce cross-linked documents and documentation, and has been designed to simplify the tool chain that takes one from writing technical documentation to producing a book rather simple. You can read about its rationale here: http://asciidoctor.org/docs/what-is-asciidoc/. The software has worked well for us.

If you want to get asciidoctor working, you will need to get some packages working in Ruby. The instructions can be found in the appendices.

Vagrant is software that automates the creation and provisioning of virtual machines. What does that mean? It means that you will create a Linux server that runs inside of your day-to-day computer. This server will run even if you are using a different operating system (e.g., Windows). Vagrant machines can be configured using a vagrantfile, so you download our vagrantfile and you simply point vagrant to it and should get a working server that can run LSDTopoTools.

+ . As you can see above, the LSDTopoTools folder contains folders for different LSDTopoTools packages, for topographic datasets.

+ . Here is the amazing thing: the files that are in LSDTopoTools folder in your vagrant box ARE ALSO visible, and synced, in your host computer. So if you use LSDTopoTools to do some analysis within your vagrant box, you will be able to see the files within your host computer as well. This means that you can, for example, do a Linux based LSDTopoTools analysis and then plot that analysis in a GIS on your host windows box without having to transfer files. Not only that, but you can modify the code, update python scripts, change parameter files, etc., with your favourite text editor in Windows (or OSX, or whatever) and those files will be visible to your Vagrant box. Fantastic!

To check out the latest version of our software you can run the vagrant provision command

When you are finished with your session, you just need to go into the powershell or a terminal and type:

If you want to remove the virtual machine, start it up and than run vagrant destroy:

We have written Vagrant files for you so you don’t have to customise your working environment, but if you want to set up your own Vagrant boxes with your own software here are some notes.

We have found the best way to install python is miniconda. We will use Python 2.7, so use the Python 2.7 installer.

Once you have installed that, you can go into a powershell or terminal window and get the other stuff you need:

The only difference in Windows is that your prompt in powershell will say PS>.

To run spyder you just type spyder at the command line.

If you don’t want to use miniconda, it is quite straightforward to install these on a Linux or OSX system:

or

In OSX, you need a package manager such as Homebrew, and you can follow similar steps (but why not use miniconda?).

After that, you need:

You can get all this with a combination of pip and sudo, yum or homebrew, depending on your operating system.

For example, with an Ubuntu system, you can use:

The GDAL python tools are a bit harder to install; see here: https://pypi.python.org/pypi/GDAL/.

Everything in a computer is organised into files. These are collections of numbers and/or text that contain information and sometimes programs. These files need to be put somewhere so they are organised into directories which can be nested: i.e., a directory can be inside of another directory. Linux users are born with this knowledge, but users of more intuitive devices and operating systems are not.

Our software is distributed as source code. These are files with all the instructions for the programs. They are not the programs themselves! Most of the instructions live in objects that have both a cpp and hpp file. They are stored in a directory together. Contained alongside these files are a few other directory. There is always a TNT folder for storing some files for matrix and vector operations. There will always be another directory with some variant of the word driver in it. Then there might be some other directories. Do not move these files around! Their location relative to each other is important!!.

Inside the folder that has driver in the name there will be yet more cpp files. There will also be files with the extension make. These make files are instructions to change the source code into a program that you can run. The make file assumes that files are in specific directories relative to each other so it is important not to move the relative locations of the cpp files and the various directories such as the TNT directory. If you move these files the you will not be able to make the programs.

We tend to keep data separate from the programs. So you should make a different set of folders for your raw data and for the outputs of our programs.

If you use our vagrantfiles then you will need some specific directories. Our instructions assume this directory structure so if you have your own Linux box and you want to copy and paste things from our instructions then you should replicate this directory structure.

You can download our various packages using git, but many of them will be downloaded automatically by our vagrantfiles. Suppose you were looking at the LSDTopoTools_ChiMudd2014 package. In our Linux vagrant system, the path to this would be /LSDTopoTools/Git_projects/LSDTopoTools_ChiMudd2014.

If you type:

You will see a large number of files ending with .cpp and .hpp. These are files containing the instructions for computation. But they are not the program! You need to translate these files into something the computer can understand and for this you use a compiler. But there is another layer because we use lots of files so we need to compile lots of files, so we use something called a makefile which is a set of instructions about what bits of source code to mash together to make a program.

The *makefiles are in a folder called driver_functions_MuddChi2014. All of our packages have a directory with some variation of driver in the name. You need to go into this folder to get to the makefiles. You can see them with:

This will list a bunch of makefiles. By running the command make -f and then the name of a makefile you will compile a program. The -f just tells make that you are using a makefile with a specific name and not a file called make.

Calling make with a makefile results in a program, that has the extension .out or .exe. The extension doesn’t really matter. We could have told the makefile to give the program the extension .hibswinthecup and it should still work. BUT it will only work with the operating system within which it was compiled: in this case the Ubuntu system that vagrant set up.

The crucial thing to realise here is that the program is located in the driver_functions_MuddChi2014 directory. If you want to run this program you need to be in this directory. In Linux you can check what directory you are in by typing pwd.

When you run a makefile it will create a program sitting in some directory. Your data, if you are being clean and organised, will sit somewhere else.

Most of our programs need to look for another file, sometimes called a driver file and sometimes called a parameter file. We probably should use a consistent naming convention for these files but I’m afraid you will need to live with our sloppiness. You get what you pay for, after all.

The programs will be in your source code folders, so for example, you might have a program called get_chi_profiles.exe in the /LSDTopoTools/Git_projects/LSDTopoTools_ChiMudd2014/driver_functions_MuddChi2014 directory. You then have to tell this program where the driver or parameter file is:

In the above examples, ./git_get_profiles.exe is calling the program.

/LSDTopoTools/Topographic_projects/Test_data/ is the folder where the driver/parameter file is. We tend to keep the topographic data and parameter files together. The final / is important: some of our programs will check for it but others won’t (sorry) and they will not run properly without it.

Example.driver is the filename of the driver/parameter file.

In the above example it means that the parameter file will be in the folder /LSDTopoTools/Topographic_projects/Test_data/ even though your program is in a different folder (/LSDTopoTools/Git_projects/LSDTopoTools_ChiMudd2014/driver_functions_MuddChi2014/).

The LSDTopoToolbox works predominantly with raster data; if you don’t know what that is you can read about it here: http://en.wikipedia.org/wiki/Raster_data. In most cases, the raster data you will start with is a digital elevation model (DEM). Digital elevation models (and rasters in general) come in all sorts of formats. LSDTopoToolbox works with three formats:

Below you will find instructions on how to get data into the correct format: data is delivered in a wide array of formats (e.g., ESRI bil, DEM, GeoTiff) and you must convert this data before it can be used by LSDTopoTools.

If you want to analyze topography, you should get some topographic data! The last decade has seen incredible gains in the availability and resolution of topographic data. Today, you can get topographic data from a number of sources. The best way to find this data is through search engines, but below are some common sources:

One of the most frequent operations in GDAL is just to see what sort of data you have. The tool for doing this is gdalinfo which is run with the command line:

where filename.ext is the name of your raster.

This is used mostly to:

This utility can read Arc formatted rasters but you need to navigate into the folder of the raster and use the .adf file as the filename. There are sometimes more than one .adf files so you’ll just need to use ls -l to find the biggest one.

Say you have a raster but it is in the wrong format (LSDTopoToolbox at the moment only takes .bil, .flt and .asc files) and in the wrong projection.

You will need to be able to both change the projection of your rasters and change the format of your raster. The two utilities for this are:

You might also want to clip your raster to a smaller area. This can sometimes take ages on GUI-based GISs. An alternative is to use gdalwarp for clipping:

or you can change the output format:

Since this is a gdalwarp operation, you can add all the bells and whistles to this, such as:

The main thing to note about the -te operation is that the clip will be in the coordinates of the source raster (input.tif). You can look at the extent of the raster using gdalinfo.

Often websites serving topographic data will supply it to you in tiles. Merging these rasters can be a bit of an annoyance if you use a GIS, but is a breeze with GDAL. The gdal_merge.py program allows you to feed in a file with the names of the rasters you want to merge, so you can use a Linux pipe to merge all of the rasters of a certain format using just two commands:

The above command works for ascii files but tif or other file formats would work as well.

The exception to this is ESRI files, which have a somewhat bizarre structure which requires a bit of extra work to get the file list. Here is an example python script to process tiles that have a common directory name:

If you use this you will need to modify the directory structure to reflect your own files.

If you would like something really lightweight, you can use our python mapping tools, available here: https://github.com/LSDtopotools/LSDMappingTools. These have been designed for internal use for our group, so at this point they aren’t well documented. However if you know a bit of python you should be able to get them running. You will need python with numpy and matplotlib.

To look at a DEM, you will need to download LSDMappingTools.py and TestMappingTools.py from the Gitub repository. The latter program just gives some examples of usage. At this point all the plotting functions do are plot the DEM and plot a hillshade, but if you have python working properly you can plot something in a minute or two rather than having to set up a GIS.

The software is delivered as C++ source code. Before you can run any analyses, you need to compile it, using something called a compiler. You can think of a compiler as a translator, that translates the source code (which looks very vaguely like English) into something your computer can understand, that is in the form of 1s and 0s.

The C++ source code has the extensions .cpp and .hpp. In addition, there are files with the extension .make, which give instructions to the compiler through the utility make.

Don’t worry if this all sounds a bit complex. In practice you just need to run make (we will explain how to do that) and the code will compile, leaving you with a program that you can run on your computer.

LSDTopoTools consists of three distinct components:

When you download the code, the objects will sit in a root directory. Within this directory there will be driver_function_* directories as well as a TNT directory.

We start with the assumption that you have installed git on your computer. If it isn’t installed, you should consult the appendices for instructions on how to install it.

You can call git with:

If it is your first time using git, you should configure it with a username and email:

Now, if you are the kind of person who cares what the internet thinks of you, you might want to set your email and username to be the same as on your Github account (this is easily done online) so that your contributions to open source projects will be documented online.

You can config some other stuff as well, if you feel like it, such as your editor and merge tool. If you don’t know what those are, don’t bother with these config options:

Okay, once you have set up git, you are ready to get some code! To get the code, you will need to clone it from a repository. Most of our code is hosted on Github, and the repository https://github.com/LSDtopotools, but for now we will run you through an example.

First, navigate to a folder where you want to keep your repositories. You do not need to make a subfolder for the specific repository; git will do that for you.

Go to Github and navigate to a repository you want to grab (in git parlance, you will clone the repository). Here is one that you might try: https://github.com/LSDtopotools/LSDTopoTools_ChiMudd2014.

If you look at the right side of this website there will be a little box that says HTTPS clone URL. Copy the contents of this box. In your powershell or terminal window type

The repository will be cloned into the subdirectory LSDTopoTools_ChiMudd2014. Congratulations, you just got the code!

If you start to modify our code, or want to start keeping track of your own scripts, you might create your own repositories using git and host them on Github, Bitbucket or some other hosting website.

First, you go to a directory where you have some files you want to track. You will need to initiate a git repository here. This assumes you have git installed. Type:

to initiate a repository. If you are downloading an LSDTopoTools repository from github, you won’t need to init a repository.

So now you gave run git init in some folder to initiate a repository. You will now want to add files with the add command:

Git adds all files in a folder, including all the files in a named subdirectoy.

If you want to add a specific file(s), you can do something like this:

We post the latest release versions of the software on GitHub.

The github site is: https://github.com/LSDtopotools. This site contains a number of offerings: documentation, notes, automation scripts, plotting scripts and other goodies.

When we publish papers that use new algorithms, we tend to post the code on the Community Surface Dynamics Modeling System website, found here: http://csdms.colorado.edu/wiki/Main_Page.

These versions are the ones used in the publications, so they are representative of the code used in the papers but not the latest versions. Currently our CSDMS offerings are:

Now, clone the repository from GitHub. The repository in the first example is here: https://github.com/LSDtopotools/LSDTopoTools_AnalysisDriver. The command to clone is:

Perhaps you feel like being difficult and have decided not to use git, because you find its name offensive or because Linus Torvalds once threw an apple core at your cat.

In that case you can download a zipped version of the repository, and unzip it

If you are starting with our vagrant box, you can go directly to the correct folder with:

If you have cloned from git on your own, you will still be sitting in the directory /LSDTopoTools/Git_projects/, so navigate up to the directory LSDTopoTools_AnalysisDriver/Analysis_driver/.

You can now compile the code with

I am afraid there will be a lot of warnings. We apologize for being naughty programmers. Note if there is an error then that is a problem! You’ll need to contact Simon or post a comment on the github repo. However, after all of those warnings you should be able to type ls and see a program called LSDTT_analysis_from_paramfile.out.

The last message from the of the compilation should say this:

Congratulations! You have compiled your first LSDTopoTools program. You are now ready to do some analysis.

The example data is automatically downloaded by vagrant, we will use the WA.bil dataset in LSDTopoTools/Topographic_projects/Test_data.

Navigate into your data folder and download the data using the wget tool We have placed several example datasets on a github repository. Today we will be working with a topographic map from Scotland, you can get it with:

This data is in .tif format! Quite a lot of the data you might download from the web is in this format. LSDTopoTools doesn’t read tif files (that is a job for the future), so you need to convert to a valid file format. We will convert the data using GDAL: see the section Translating your raster into something that can be used by LSDTopoToolbox.

Our preference is for the data to be in UTM WGS1984 coordinates. You can look up the UTM zones on this map compiled by Alan Morton. The Whiteadder catchement is close to Edinburgh Scotland in zone UTM zone 30N To convert the data to ENVI bil format (which is our preferred format) type:

Now, see if the file is there:

If you looked at the file in a GIS you might have additional files with the extension .aux.xml. The important thing is that you now have files with the extensions bil and hdr.

Important: There are two formats that use the file extension bil: the ENVI format (which is the one we use) and an ESRI format. Make sure your bil files are in ENVI format. You can always check using gdalinfo.

The code is flexible enough that the parameter file can be in a different location from the data, but I think it is good practice to keep the parameter files with the data. The parameter file not only runs the software, but more importantly it is a reproducible record of your analyses! So if you are doing research you should save these files. The software is designed so that if you send someone else the parameter file and the DEM they can reproduce your analysis exactly. This differentiates our software from GUI driven software like ArcMap and QGIS.

If you use vagrant there are two example parameter files. One is in the source code repository (the one with all the .cpp files) and the other is in the directory with the test data. The one in the source code directory is for people who are not using the test data. If you do use the test data the example parameter file is /LSDTopoTools/Topographic_projects/Test_data/Vagrant_Example.LSDTT_driver

Before you run the program you need to modify the parameter file. The parameter file is just plain text, so you will need a text editor to modify it. You can modify it in your favorite text editor, but DO NOT use a program that inserts a bunch of stupid formatting like Wordpad or Word.

In fact most text editors in Windows systems have the unfortunate habit of inserting diabolical hidden characters, called control characters that you will never see or notice if you just look at text but will completely screw up the program. We have endeavoured to remove these characters within our code, but I highly recommend editing the parameter file either in Linux, or using a text editor on Windows that won’t insert these characters.

A great text editor that works on all operating systems and is open source is Brackets.

For Windows, we use Pspad or Atom

For Linux, we use either emacs or vim

You can also use brackets for both Windows and Linux.

These Linux text editors take a bit of getting used to, so unless you are going to start writing code, you should probably stick with Brackets, Pspad or Atom.

In many text editors, you can select the text formatting. It turns out there are different formattings for different operating systems. You should use the magic of the internet to determine how to change the text formatting. Many editors have the options MAC, DOS, and UNIX formatting. You want UNIX formatting.

Go into the driver function folder, and make the program:

This will compile into a prgram called DEM_preprocessing.exe

You give DEM_preprocessing.exe a text file with information about what you want it to do. You need to tell the program where this file is and its name. If the file name is DEMPP.driver and it is sitting with your data files in the directory /LSDTopoTools/Topographic_projects/Test_data then you would call the program with

The .driver file looks a bit like this:

The components of this file are hopfully self explanitory: you will need to update the read and write paths and the read and write fnames (these are the prfixes of your DEM) to reflect your own messed up DEM.

To see all the options you should refer to our appendix on DEM preprocessing Options.

If you haven’t run our vagrant setup, you need to clone the repository. First navigate to the folder where you will keep the GitHub repository. In this example it is called /LSDTopoTools/Git_projects/. To navigate to this folder in a UNIX terminal use the cd command:

You can use the command pwd to check you are in the right folder. Once you are in this folder, you can clone the repository from the GitHub website:

Navigate to this folder again using the cd command:

If you don’t want to use git, you can download a zipped version of the code:

We have provided some example datasets which you can use in order to test the channel extraction algorithms. In this tutorial we will work using a lidar dataset from Indian Creek, Ohio.

The DEM is called indian_creek.bil and indian_creek.hdr, and the relevant parameter file is Vagrant_ChannelExtraction.driver.

This dataset is already in the preferred format for use with LSDTopoTools (the ENVI bil format). The figure below shows a shaded relief map of part of the Indian Creek DEM which will be used in these examples.

The parameter file has keywords followed by a value. The format of this file is similar to the files used in the LSDTT_analysis_from_paramfile program, which you can read about in the section Running your first analysis.

The parameter file has keywords followed by the : character. After that there is a space and the value.

Below are options for the parameter files. Note that all DEMs must be in ENVI bil format (DO NOT use ArcMap’s bil format: these are two different things. See the section What data does LSDTopoToolbox take? if you want more details). The reason we use bil format is because it retains georeferencing which is essential to our file output since many of the files output to csv format with latitude and longitude as coordinates.

Below is an exaple parameter file. This file is included in the repository along with the driver functions.

One of the simplest ways of extracting channel networks from DEMs uses a contributing area threshold. This method is useful for coarse resolution (e.g. >10m) DEMs, where topographic features of the channel heads themselves cannot be reliably identified from the DEM. The user has to specify the threshold area, which represents the upstream area that must drain to a pixel before it is considered as part of the channel.

We will work through an example using the Indian Creek example dataset that you downloaded.

For higher-resolution DEMs a number of different methods have been developed to extract channel networks more accurately. This section details how to extract channels using methods relying on geometric signatures of channel incision, primarily how it affects planform curvature. Although many methods have been developed that use a variety of planform curvature to detect channel heads, we will discuss three methods: the Geonet method the Geonet method (developed by Passalacqua et al. 2010a, b, 2012), a method developed by Pelletier (2013) and implemented in LSDTopoTools, and a similar geometric method available within LSDTopoTools.

This file contains all of the basin average values for each basin, these files contain a large number of columns, providing a wealth of basin average data. The columns in the file, from left to right are as follows:

This file can be loaded and the data visualized using these python scripts.

This file contains hillslope metrics calculated for every hilltop pixel in the dataset which was routed successfully to a stream pixel. The columns in the file, from left to right are as follows:

This file can be loaded and the data visualized using these python scripts.

An optional switch can be set within the code to print out the coordinates of the path of each trace, allowing hilltop flow paths to be visualized. This option is not exposed at the command line as it will considerably slow the execution of the algorithm.

To enable this feature open the driver file LH_Driver.cpp and find the following parameters which should be located around line 140:

These control the path file printing and give several options to limit the scope of the path printing to ensure a manageable number of files are generated. The possible values for each parameter are:

Once these parameters have been set, re-compile the driver following the steps in Compile The Driver and run the code. Once the code has executed a large number of files will have been generated in the supplied path. They are plain text, space delimited files which have the following headings:

A python script is provided to process these files into a shapefile, which can be viewed in any GIS package, or plotted using other python scripts provided. The python script to process these files is located here and is called trace_process_1_1.py. To run this file, alter the input path to point to where the trace files are stored, and then set the output path to a new directory which must already exist.

The line:

is used to thin the dataset to speed up processing, in case this was not performed earlier. Again, a value of 1 will keep all of the data, and any other integer will keep every nth file. Once the code has been executed a single shapefile will be produced in the user defined output directory.

The data is located in the ExampleTopoDatasets repository. Firstly, we must create a new directory to store our topogrpahic data:

We will only take the data we need, which is the Gabilan Mesa DEM and the associated DrEICH channel head file, so we can use wget to download the data:

You should now have the following files in your data folder:

Next we need to download the code from the CSDMS repository where the latest stable version of the hilltop flow routing algorithm is located. This can again be downloaded using wget and it will come as a zipfile, which should be extracted into a separate folder to the data:

Finally we need to get the visualization python scripts so we can explore the data:

Now we should have 3 folders, one with the data, one with the main c++ code and a third with the python code.

We need to check that the filenames for our input and output data make sense in the LH_Driver.cpp file. Open the file in a text editor and look at the line which loads the DEM:

We are working with bil files, so need to change the "flt" to "bil", which can be done with a simple search and replace within your text editor.

The code also expects our data to be tagged *_dem so lets rename our data files to make life simpler:

Now we can navigate back to the directory where the driver and makefile are stored and make the driver:

Some warnings may appear which can be ignored as long as the final few lines look something like they do above and the file LH_Driver.out is created.

The binary file can then be run using the desired input arguments

Once the code is completed it will print some data to the screen about the success rate of the traces. Due to the nature of real topography there will always be a small number of failed traces, traces which hit the edge of the DEM should also be rare, but these are excluded from analysis as they are truncated. If these values are very large relative to the stream count, which denotes traces which have successfully completed, there may be a problem with your input data. In the case of this worked example we get the following results:

Returning to the data folder we can see the the two data files described earlier in this chapter have been written.

We can now load the data files into the python scripts to visualize the data. Firstly, we can make a histogram to view the distribution of hillslope length values for our landscape using the RAW_LH_Hist.py script. We need to update the plotting parameters to reflect our data, the following lines can be edited within the script using any text editor:

The plot style parameters require a bit of trial and error to get the the correct axis limits, so the code may need to be executed several times. Once the parameters have been set the script can be run at the command line:

Which produces a histogram within the data folder.

This process can be repeated to run any of the plotting scripts provided with this package, as each has a similar interface.

First navigate to the folder where you will keep the GitHub repository. In this example it is called /home/LSDTT_repositories. To navigate to this folder in a UNIX terminal use the cd command:

You can use the command pwd to check you are in the right folder. Once you are in this folder, you can clone the repository from the GitHub website:

Navigate to this folder again using the cd command:

If you don’t want to use git, you can download a zipped version of the code:

In addition to the topographic analysis code, some python code is provided to handle the generation of the E* R* data and its visualization. This code is stored in a separate GitHub repository which can be checked out in the same manner as before. It is a good idea to place the python code into a separate directory to avoid confusion later on.

Navigate to this folder again using the cd command:

or if you prefer to avoid git:

For the code to run correctly the following packages must be installed with a version number greater than or equal to the version number listed below. The code has only been tested on Python 2.7 using the listed versions of these packages, so if you experience unexpected behavior on a higher version, try installing the specified version.

To test if you have a package installed, launch python at the terminal and try to import each package in turn. For example, to test if we have numpy installed:

If importing the package does nothing, that means it has worked, and we can now check the version of numpy

In this case my version of numpy is new enough to run Plot_ER_Data.py without any problems. Repeat this test for each of the 4 packages and if any of them are not installed or are too old a version, it can be installed by using pip at the unix terminal or upgraded by using the --upgrade switch.

We have provided some example datasets which you can use in order to test this algorithm. In this tutorial we will work using a LiDAR dataset and accompanying channel heads from Gabilan Mesa, California. You can get it from our ExampleTopoDatasets repository using wget and we will store the files in a folder called data:

This dataset is already in the preferred format for use with LSDTopoTools (the ENVI bil format). However the filenames are not structured in a manner which the code expects. Both the hillslope length driver and the E*R* driver expect files to follow the format <prefix>_<filetype>.bil so we should rename these four files to follow this format.

Now the prefix for our data is gabilan and we are ready to look at the code itself.

Firstly we will generate a density plot from the raw E* R* data. To do this we must update the path to our data files, the prefix and the basin order so that our files can be loaded. These modifications can be done in any text editor.

As we want to display a density plot we must also place a value other than 0 for the DensityFlag parameter, and ensure that all other parameters in the second parameter table are set to 0 or in the case of BinFlag, an empty string ''.

We will set the critical gradient to a value of 0.8, to avoid running the bootstrapping calculations. ErrorBarFlag should be set to False, along with the example data options, and the file format can be left as the default value.

The below complete settings file has the comments removed for clarity, as these are not needed by the program.

Once the settings file has been generated, the code can be run from the terminal:

This will write a file called gabilan_E_R_Star.png to the data folder, which should look like this:

This plot shows the density of the E* R* measurements for this test datatset, the data is quite sparse due to the small size of the input DEM, but the majority of data points still plot close to the steady state curve.

Having completed the first example plot it becomes very simple to re-run the code to generate different plots. In this example we will plot the hilltop patch data points with error bars. To do this we need to change our settings file as follows:

This set of parameters generates a small number of hilltop patch data points which plot in similar locations as the raw data.

To plot the basin average data, the same set of paramters would be used

To bin the raw data, we need to set the BinFlag parameter to 'raw' and select a number of bins to place our data into:

In this case, the result is fairly meaningless, as most of the bins have too few data points to be plottted, but on larger datasets this method can highlight landscape transience very clearly.

The final example for this section is how to use the code to estimate the critical gradient of a landscape. This is performed by configuring the bootstrapping parameters and in this case we will use the patch data and 1000 iterations to compute the best fit critical gradient.

It should be noted that on a small dataset such as this the fitting will not be very robust as there are too few data points, but this example should demonstrate how to run the code in this manner on real data. The best fit critical gradient will be printed at the top of the final plot.

Sorry the section is under construction! But if you want to read more about chi analysis, have a look at these papers: Perron and Royden, 2013; Royden and Perron, 2013; Mudd et al., 2014.

Now, clone the repository from GitHub:

If you have used our vagrantfiles and vagrant, you can update the existing repository with:

Okay, now you should have the code. If you have downloaded or cloned the code yourself, you will still be sitting in the directory /LSDTopoTools/Git_projects, so navigate up to the directory LSDTopoTools_ChiMudd2014/driver_functions_MuddChi2014/.

If you are using our vagrant distribution the code will be automatically cloned and you can get it by going directly to the appropriate folder:

There are a number of makefiles (thse with extension .make in this folder). You can have a look at all of them by running the command:

These makefiles do a number of different things that will be explained later in this chapter. We will compile them as we go through the various different types of chi analysis.

We are going to use example data from the Mandakini River in Northern India. This river was the focus of a study by Rahul Devrani and others, you can find it here: http://onlinelibrary.wiley.com/doi/10.1002/2015GL063784/full (the paper is open access).

Again, we want to make sure our data is arranged somewhere sensible. Make a directory for datasets and perhaps a folder specific to India. Again, you don’t need to follow the same naming conventions as in these examples, but you will have to remember the directory names!

I would open a second terminal window (one should already be open in the driver_functions_MuddChi2014 folder) and navigate to the data folder:

If you use vagrant, the data is already there. If you don’t, then you can grab the data from the web. The SRTM data from the catchment is stored on the data repository at GitHub: https://github.com/LSDtopotools/ExampleTopoDatasets.

You probably don’t want to clone this repository since it contains a lot of DEMs, so why don’t you just download the relevant files directly:

We are also going to use a parameter file, which comes with the cloned LSDTopoTools_ChiMudd2014 repository. It is called Example_Mudd2014.driver and is loaced in the driver_functions_MuddChi2014 directory.

In this section we will extract a channel network, and from this channel network we will choose a junction (or junctions) from which we will run chi analyses. Later in this chapter we will go over tools for running chi analysis across all the channels in a basin.

The channel extraction code requires two steps. In the first step, the toolkit takes the raw DEM and prints several derived datasets from it. The main dataset used for the next step is the junction index dataset. The second step involves selecting a junction from which the chi analysis proceeds.

After preparing the data (see section part 1: getting the channel profiles), performing the channel analysis involves 3 steps, including visualization of the data.

Once the profile data has been converted into a .chan file, the data can then be processed to determine the most likely m/n ratio for individual channels and also via the collinearity test (see Mudd et al (2014)).

The .tree file has as many rows as there are nodes in the channel network. There will be more nodes in the .tree file than in the .chan file because the code extends all tributaries to the outlet. Each row has 23 columns. The columns are

We have also provided python scripts for visualizing the data (that is "LSDVisualisation": our code uses British spelling).

Link to the full-size version of the flowchart

The chi gradient mapping tool can be compiled by navigating to the driver function folder and running make. If you are using our vagrant setup you can go directly to the correct folder by typing cd /LSDTopoTools/Git_projects/LSDTopoTools_ChiMudd2014/driver_functions_MuddChi2014/. Once there, run the command:

The program is now ready to run with the correct inputs. The function takes two inputs

So if the parameter file is located at /LSDTopoTools/Topographic_projects/Test_data and it is called test_chi_map.param, you run the program with:

As we will see momentarily, the data an the parameter file can be in different locations, although in general it might be sensible to place the parameter file in the sample directory as the data.

The code is run using a parameter file, within which users can set the data they want to print to file. Regardless of which data they choose to print to file, a file will be printed with the extension _Input.param which prints out all the parameters used in the analysis (including the default parameters). This file can be used to verify if the correct parameters have been used in the analysis.

The parameter file has keywords followed by a value. The format of this file is similar to the files used in the LSDTT_analysis_from_paramfile program, which you can read about in the section Running your first analysis.

The parameter file has keywords followed by the : character. After that there is a space and the value.

Below are options for the parameter files. Note that all DEMs must be in ENVI bil format (DO NOT use ArcMap’s bil format: these are two different things. See the section What data does LSDTopoToolbox take? if you want more details). The reason we use bil format is because it retains georeferencing which is essential to our file output since many of the files output to csv format with latitude and longitude as coordinates.