LSDTopoTools workshop

The first thing you need to do is get our python program LSDTopoToolsSetUp.py. It lives here: https://github.com/LSDtopotools/LSDAutomation/blob/master/LSDTopoToolsSetup.py

$ wget https://raw.githubusercontent.com/LSDtopotools/LSDAutomation/master/LSDTopoToolsSetup.py

Now, in your terminal window run this script. It has some options:

It turns out the LSDTopoToolsSetup.py tool has a number of options but we will explain these later. If you want a preview of what it does, you can call its help options:

$ python LSDTopoToolsSetup.py -h

If you have just started an ssh or terminal window, you can find out where you are, it will look something like this:

$ pwd
$ /home/MyUserName/

Make a directory called LSDTopoTools using the mkdir command and then go into it with the cd command:

$ mkdir LSDTopoTools
$ cd LSDTopoTools
$ pwd
$ /home/MyUserName/LSDTopoTools/

Now make two new directories in the LSDTopoTools directory, called Git_projects and Topographic_projects. The only reason for these names is that the rest of the documentation uses these directory names.

$ pwd
$ /home/MyUserName/LSDTopoTools/
$ mkdir Git_projects
$ mkdir Topographic_projects

We are going to clone some data into the Topographic_projects directory. Even if you are going to use your own DEM this is handy because the cloned data folder also includes all the parameter files that we will use in this workshop:

$ cd Topographic_projects
$ git clone https://github.com/LSDtopotools/LSDTT_workshop_data.git

This will make a folder LSDTT_workshop_data into which you can place your own data. It also contains two test DEMs from Scotland that we will use in the Eroding Landscapes course.

Now go back into your Git_projects directory. If you have followed the above directions you will be sitting in the Topographic_projects directory so go up a level and then down into the Git_projects directory:

$ cd ..
$ cd Git_projects

Now you need to clone some other repositories that have our code:

$ git clone https://github.com/LSDtopotools/LSDTopoTools_AnalysisDriver.git
$ git clone https://github.com/LSDtopotools/LSDTopoTools_CRNBasinwide.git
$ git clone https://github.com/LSDtopotools/LSDTopoTools_ChannelExtraction.git
$ git clone https://github.com/LSDtopotools/LSDTopoTools_ChiMudd2014.git

Great! Now you have everything you need for the workshop.

Navigate into the LSDTopoTools_AnalyisDriver directory, and then into the subdirectory Analysis_driver.

$ cd /path/to/LSDTopoTools/
$ cd Git_projects/LSDTopoTools_AnalysisDriver/Analysis_driver

Then use make to compile the programs. You will get lots of warnings that can be ignored.

$ make -f Drive_analysis_from_paramfile.make

Once you have done this, you should see a file called LSDTT_analysis_from_paramfile.out in this directory. That is the program we will use to make some new topographic data!

At this stage you should have a terminal window or secure shell window open in the Analysis_driver directory.

Use the SECOND terminal window to go to the data folder (if you’ve not got another window, or secure shell, open one now).

Use this second window to go to the LSDTT_workshop_data directory:

$ cd /path/to/LSDTopoTools/Topographic_projects/LSDTT_workshop_data/

We have provided an example parameter file in this directory. You can see all the stuff in the LSDTT_workshop_data directory by using the ls command:

$ ls
README.asc  WAWater.bil                       Workshop_ChiTool.driver
WA.bil      WAWater.hdr
WA.hdr      Workshop_AD_example.LSDTT_driver

The file you want is Workshop_AD_example.LSDTT_driver. If you are using our Vagrant setup then this doesn’t need to be edited. But if you are running on an Edinburgh server or are using your own data, this file will need to be modified.

Hopefully you know what a text editor is (if not read the box above!). Open Workshop_AD_example.LSDTT_driver in a text editor. It looks like this

# This is a driver file for LSDTopoTools
# Any lines with the # symbol in the first row will be ignored

# File information
# IMPORTANT: you will need to modify the file path to reflect your directory structure
# NOTE: If you want to use a smaller test DEM use WA instead of WAWater
dem read extension: bil
dem write extension: bil
read path: /LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
write path: /LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
read fname: WAWater
write fname: WAWater

# Parameters for various topographic analysis
min_slope_for_fill: 0.0001

# Parameters for polyfit and slope calculations
polyfit_window_radius: 19
slope_method: polyfit

# The different analyses to be run
write hillshade: true
write slope: true
write curvature: true

Whoa! What is all this stuff? There are instructions to our program telling the computer what to do. Most of this you can ignore (for now), but for us the important lines are:

read path: /LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
write path: /LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
read fname: WAWater
write fname: WAWater

The two lines with path tell the program where to read and write the data.

The read fname is the name of the file on which you will perform analysis. The read fname is the PREFIX of the DEM you will use. So if your DEM is called WA.bil then read fname: will be WA.

To see where your data is, type pwd in the LSDTT_workshop_data:

$ pwd
homes/smudd/LSDTopoTools/Topographic_projects/LSDTT_workshop_data/

YOUR directory will be different! Copy your directory into the file:

read path: /my/directory/LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
write path: /my/directory/LSDTopoTools/Topographic_projects/LSDTT_workshop_data/

You can also change the name of the data file. It can be your own data, or it can be the much smaller DEM WA.bil.

Okay, at this point you should have modified the Workshop_AD_example.LSDTT_driver file so that it reflects the directory of your data and the name of your data. Don’t touch anything else.

Go back to the terminal window where you ran the make command (remember, use two terminal or SSH windows: it makes things easer).

In that window make sure you have the program: type

$ ls LSDTT_analysis_from_paramfile.out

If you have that file, type

$ ./LSDTT_analysis_from_paramfile.out /my/directory/LSDTopoTools/Topographic_projects/LSDTT_workshop_data/ Workshop_AD_example.LSDTT_driver

When you run this command, the program will work for a little while and then spit out several rasters that will sit in your data folder. The raster that it prints are determined by these lines in the LSDTT_driver:

# The different analyses to be run
write hillshade: true
write slope: true
write curvature: true

$ ls
README.asc   Workshop_AD_example_curvature.bil  Workshop_AD_example_slope.bil
WA.bil       Workshop_AD_example_curvature.hdr  Workshop_AD_example_slope.hdr
WA.hdr       Workshop_AD_example_hs.bil         Workshop_ChiTool.driver
WAWater.bil  Workshop_AD_example_hs.hdr
WAWater.hdr  Workshop_AD_example.LSDTT_driver

The rasters have filenames that reflect what they are (HS, for example, denotes a hillshade raster). Okay, we can now move on to looking at the data!

First, go into your data directory and make a another copy of the driver file:

$ cp Workshop_AD_example.LSDTT_driver Mask_example.LSDTT_driver

Now you just need to adjust a few parameters:

# This is a driver file for LSDTopoTools
# Any lines with the # symbol in the first row will be ignored

# File information
# IMPORTANT: you will need to modify the file path to reflect your directory structure
# NOTE: If you want to use a smaller test DEM use WA instead of WAWater
dem read extension: bil
dem write extension: bil
read path: /LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
write path: /LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
read fname: WA_curvature
write fname: WA_curvature

# Parameters for masking
mask_threshold: 0
mask_nodataisbelowthreshold: 0

# The different analyses to be run
write mask threshold: true

The differences here are that you are using the WA_curvature raster instead of the DEM, that you are trying to write a mask threshold, and you are going to turn anything that is over the threshold value of 0 into a nodata (that is, you tell the computer it doesn’t have any data there).

You can change this around a bit: if you run mask_nodataisbelowthreshold: 1 it means anything below the threshold is masked, and you can change the threshold value by changing mask_threshold:.

Once you have done this try loading into QGIS and looking at this data over the hillshade using transparency (in the properties menu: right-click on the layer).

If you followed the directions on setting up directories the easy way, you will already have the chi tools set up!

If you haven’t already set things up with this file, download it and run EITHER:

$ python LSDTopoToolsSetup.py -id 0

$ python LSDTopoToolsSetup.py -id 1

Doing this will set up LSDTopoTools in the root directory (in Vagrant). If you are not in vagrant, running it with -id 0 it put LSDTopoTools in your root directory (in any other Linux system), whereas -id 1 will put it in the directory from which you called LSDTopoToolsSetup.py. In vagrant, to get to the root just type:

$ cd /

$ cd ~

Navigate into the LSDTopoTools_ChiMudd2014 directory, and then into the subdirectory driver_functions_MuddChi2014.

$ cd /path/to/LSDTopoTools/
$ cd Git_projects/LSDTopoTools_ChiMudd2014/driver_functions_MuddChi2014

Then use make to compile the programs. You will get lots of warnings that can be ignored.

$ make -f chi_mapping_tool.make

Once you have done this, you should see a file called LSDTT_analysis_from_paramfile.out in this directory. That is the program we will use to make some new topographic data!

The lines that control the location of the files to read and write are here:

read path: /LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
write path: /LSDTopoTools/Topographic_projects/LSDTT_workshop_data/
read fname: WA

These lines control how dense the channel network will be when you run the code:

threshold_contributing_pixels: 2000
minimum_basin_size_pixels: 500
test_drainage_boundaries: false

# Parameters for chi analysis
A_0: 1
m_over_n: 0.45
threshold_pixels_for_chi: 2000

# The data that you want printed to file
only_check_parameters: false
print_stream_order_raster: true
print_DrainageArea_raster: true
print_segmented_M_chi_map_to_csv: true
print_simple_chi_map_to_csv: true

If you’ve done the analysis driver exercise, this should be familiar.

Go into the directory where you compiled the chi tool and run:

$ ./chi_mapping_tool.exe /path/to/LSDTopoTools/Topographic_projects/LSDTT_workshop_data/ Workshop_ChiTool.driver

The program should spit out a bunch of text, and afterwards you will be left with some new files, which we will look at in QGIS in the next section.

It is a bit more difficult to add csv data (csv stands for comma separated values). In QGIS, you need to click on the layer menu and then scroll to the add text delimeted layer button:

Figure 12. Add csv data

We will add the WA_MChiSegmented.csv layer. Once you select that file (using browse) you will see something like this:

Figure 13. Create layer menu

Hit okay. You will then be asked what coordinate system you want. All csv data from LSDTopoTools is in the WGS84 coordinate system! Your rasters are typically in a local coordinate system but we convert point data to WGS84 so that they can be loaded onto web mapping platforms (i.e., Google Earth). The upshot is that you need to use WGS84 for the coordinate system.

All coordinate systems have a code, called an EPSG code, and the EPSG code for WGS84 is 4326: search for this (in the Filter form) and select it:

Figure 14. Choose WGS 84 coordinate system with EPSG:4325

You should get something that looks like this:

Figure 15. Point data

Okay, this doesn’t look so great, right click on properties of the layer, choose graduated symbols, select the m_chi column (this is chi slope) pick a nice colour ramp, and then click on classify:

Figure 16. Chi map coloured by chi steepness

Make a directory to hold the plotting tools. I usually call the directory for git projects "Git_projects".

In Windows, right click on this directory and select "Git bash here". For this to work you will have needed to have installed git for Windows first.

In Linux or MacOS, open a terminal and go to this folder.

Clone our plotting repositories from github (this grabs the files from github):

Getting files from github. In MacOS and linux the prompt will be $, in powershell it will be PS>.

$ git clone https://github.com/LSDtopotools/LSDMappingTools.git
$ git clone https://github.com/LSDtopotools/LSDAutomation.git
$ git clone https://github.com/LSDtopotools/LSDPlotting.git

You should have all the python files you need now.