Test inventory

What lsdtt3 is tested with, by evidence (how a test knows its answer) and level (unit or system). The categories are defined in lsdtt3_architecture.md and testing_and_benchmarking.md. A test with several evidence labels is counted under each, so rows sum to more than the number of tests. . means none.

CTest counts are for the configuration LSDTT_ENABLE_PYTHON_TESTS=ON, LSDTT_ENABLE_PYTHON_GEO_TESTS=ON, LSDTT_ENABLE_NETWORK_TESTS=ON, LSDTT_ENABLE_BENCHMARKS=OFF, LSDTT_ENABLE_PERF_TESTS=OFF; the benchmark and performance tiers are not included. Python counts are test functions and methods (a parametrised function counts once).

Regenerate with python3 tools/generate_test_inventory.py --build-dir build --update after adding or relabelling tests; the CTest test unit_test_inventory_is_current fails while this file is stale.

Totals

Suite	Tests	exact_solution	property	metamorphic	differential	characterisation	smoke	negative	performance	conformance	unit	system
CTest	882	208	138	131	19	176	386	245	.	43	217	665
pytest	240	78	59	47	.	10	27	62	.	61	237	3

CTest by domain

A test in several domains (for example lithology, which cuts across compartments) is counted in each.

Domain	Tests	exact	prop	meta	diff	char	smoke	neg	perf	conf
basin_stats	4	.	.	1	.	1	4	.	.	.
basin_surface_metrics	5	1	.	1	.	2	4	.	.	.
cellular	7	2	4	4	.	.	2	.	.	.
channel	2	2	1	.	.	.	.	.	.	.
chi	19	10	1	7	1	2	4	.	.	.
compatibility	1	.	.	.	.	1	1	.	.	.
conventions	9	2	.	.	.	.	.	1	.	9
core	36	14	2	3	.	4	1	8	.	16
cosmo	56	7	11	9	1	18	20	9	.	3
cosmo_realworld	8	.	.	.	.	8	.	.	.	.
cross_sections	2	.	.	.	.	.	2	.	.	.
depression_hierarchy	2	1	1	.	1	.	.	.	.	.
distributed_hydrology	32	9	12	2	3	4	10	18	.	.
divide_diagnostics	1	.	1	1	1	.	.	1	.	.
divide_migration	13	7	.	2	.	3	8	4	.	.
docs	3	.	.	.	.	.	.	.	.	3
erodibility_calibration	30	8	1	4	1	1	12	10	.	.
fetch	80	11	4	6	1	5	17	47	.	8
flow	47	6	7	7	3	25	28	13	.	.
fluvial_surfaces	94	20	19	18	1	28	53	18	.	2
gdal	7	.	3	.	.	2	.	1	.	1
giuh	1	.	1	1	.	.	.	.	.	.
hillslope	26	11	17	10	2	1	7	4	.	.
hilltop_flow_routing	14	1	3	2	.	4	8	4	.	.
hydraulic	4	.	1	1	.	.	1	2	.	.
image_ops	2	1	2	1	2	1	.	.	.	.
internal_drainage	14	4	1	2	.	2	9	4	.	.
jenks	1	1	1	.	.	.	.	1	.	.
landscape_equilibration	13	.	2	.	.	.	9	3	.	.
landslide	22	6	7	2	.	5	9	4	.	.
lem	115	25	6	11	.	4	47	37	.	3
linear_stats	2	.	.	1	.	1	2	.	.	.
lithology	63	17	1	9	.	3	22	24	.	.
marsh	1	.	.	.	.	.	1	.	.	.
mask	1	1	.	.	.	.	.	1	.	.
mountain_front	1	1	.	.	.	.	.	1	.	.
network	288	74	45	48	8	68	144	79	.	3
outlet_matching	1	.	.	.	.	.	1	.	.	.
packaging	3	.	.	.	.	.	1	.	.	2
paired_flowpaths	4	1	.	.	.	3	3	.	.	.
parameters	15	1	.	.	.	.	.	.	.	14
point_input	3	2	.	2	.	1	.	2	.	.
quartz	1	1	.	.	.	.	.	.	.	.
raster	59	20	12	8	2	12	24	16	.	1
raster_algebra	1	1	.	.	.	.	.	.	.	.
ridgeline	5	1	.	.	.	4	5	.	.	.
slope_area	1	.	.	.	.	.	1	.	.	.
snapping	3	2	1	1	.	.	2	1	.	.
statistics	2	.	.	.	.	2	.	.	.	.
step_change	5	2	3	2	.	.	1	.	.	.
stream_burning	14	3	5	4	.	4	8	2	.	.
streamlines	1	1	.	.	.	.	.	.	.	.
surface_metrics	49	13	7	12	.	16	33	5	.	4
synthetic_networks	31	2	4	3	1	6	17	9	.	4
water	1	1	.	.	.	.	.	.	.	.

Domains with no exact-solution, property, metamorphic or differential test

These are checked only by smoke, negative, characterisation or conformance tests: they show that the code runs, refuses bad input or has not changed, but not that its answers are right.

• compatibility (1 test)
• cosmo_realworld (8 tests)
• cross_sections (2 tests)
• docs (3 tests)
• marsh (1 test)
• outlet_matching (1 test)
• packaging (3 tests)
• slope_area (1 test)
• statistics (2 tests)

Command-line programs

CTest tests whose command runs the program, directly or through the harness. Unit tests of the routines a CLI calls are not counted here; see the domain table.

CLI	Tests	exact	prop	meta	diff	char	smoke	neg	perf	conf
lsdtt-advection-diffusion	8	.	.	2	.	2	4	4	.	.
lsdtt-bankfull-width	6	.	1	1	.	4	6	.	.	.
lsdtt-basin-extraction	9	.	.	1	.	3	8	1	.	.
lsdtt-basin-stats	5	.	.	1	.	2	3	2	.	.
lsdtt-basin-surface-metrics	7	.	.	1	.	3	6	1	.	.
lsdtt-channel-extraction	14	4	.	1	.	2	10	3	.	1
lsdtt-channel-network-analysis	21	3	.	1	.	3	9	12	.	.
lsdtt-chi-analysis	33	1	.	2	.	8	17	16	.	.
lsdtt-cosmo-column-tool	4	1	1	1	.	2	1	.	.	.
lsdtt-cosmo-landslides	16	.	1	2	.	6	9	4	.	.
lsdtt-cosmo-prediction	16	1	5	4	.	2	7	2	.	1
lsdtt-cosmo-tool	19	1	1	2	.	10	4	4	.	.
lsdtt-cross-sections	3	.	.	1	.	3	3	.	.	.
lsdtt-distributed-hydrology	28	4	8	2	.	5	12	16	.	.
lsdtt-divide-migration	12	6	.	2	.	3	8	4	.	.
lsdtt-drainage-analysis	18	3	3	3	.	7	16	1	.	1
lsdtt-feature-finder	13	3	4	3	.	5	10	1	.	1
lsdtt-fetch-erosion-rates	12	2	1	1	.	.	7	5	.	.
lsdtt-fetch-lithology	6	.	.	1	.	1	3	3	.	.
lsdtt-fetch-raster	57	.	1	.	.	9	5	36	.	8
lsdtt-fetch-river-lines	3	1	.	1	.	1	1	1	.	.
lsdtt-flat-surface-finder	12	.	1	3	.	4	9	2	.	.
lsdtt-fluvial-surfaces	53	4	9	10	.	20	35	12	.	.
lsdtt-graphflood	3	.	.	.	.	.	1	2	.	.
lsdtt-hillslope-channel-coupling	11	.	2	2	.	4	7	4	.	.
lsdtt-internal-drainage	11	2	.	1	.	2	8	3	.	.
lsdtt-lem	78	2	4	1	.	2	46	29	.	3
lsdtt-linear-stats	1	.	.	1	.	1	1	.	.	.
lsdtt-lithology-erodibility	14	1	.	.	.	.	7	7	.	.
lsdtt-marsh-topography	1	.	.	1	.	1	1	.	.	.
lsdtt-mountain-front	5	.	.	3	.	4	4	.	.	1
lsdtt-outlet-matching	6	.	1	1	.	3	3	3	.	.
lsdtt-raster-calculator	4	.	.	1	.	2	3	1	.	.
lsdtt-raster-preprocessing	39	12	3	2	.	7	25	13	.	1
lsdtt-ridgeline-analysis	8	1	.	.	.	7	8	.	.	.
lsdtt-sediment-routing	1	.	.	.	.	1	1	.	.	.
lsdtt-slope-area	9	.	.	2	.	4	4	5	.	.
lsdtt-spectral-analysis	7	.	.	1	.	2	5	.	.	2
lsdtt-stream-burning	12	.	4	3	.	5	9	2	.	.
lsdtt-surface-metrics	11	3	.	1	.	.	7	2	.	2
lsdtt-swath-profile	1	1	.	1	.	.	1	.	.	.
lsdtt-synthetic-networks	22	2	3	1	.	4	12	9	.	1
lsdtt-synthetic-surfaces	13	1	.	1	.	2	9	.	.	4
lsdtt-valley-centreline	11	1	1	1	.	3	8	1	.	1
lsdtt-water-centreline	1	.	.	1	.	1	1	.	.	.
lsdtt-water-line-repair	6	5	.	1	.	.	5	1	.	.

CLIs with no system test

None.

CLIs whose system tests are only smoke, negative, characterisation or conformance

• lsdtt-graphflood
• lsdtt-sediment-routing

Python binding tests by file

File	Tests	exact	prop	meta	diff	char	smoke	neg	perf	conf	unit	system
test_analysis_closure.py	4	2	1	.	.	1	1	.	.	.	4	.
test_api_examples.py	1	.	.	.	.	.	.	.	.	1	1	.
test_api_naming.py	1	.	.	.	.	.	.	.	.	1	1	.
test_api_reference_contract.py	1	.	.	.	.	.	.	.	.	1	1	.
test_binding_source_contract.py	2	.	.	.	.	.	.	.	.	2	2	.
test_burn_to_corridor.py	5	.	1	2	.	1	.	1	.	.	5	.
test_channel_point_attributes.py	4	2	.	1	.	.	.	1	.	1	4	.
test_cli_packaging.py	6	3	.	.	.	.	.	1	.	2	6	.
test_contributing_threshold_migrations.py	2	.	.	2	.	.	.	2	.	2	2	.
test_cosmo_basin.py	2	.	2	.	.	.	.	.	.	.	2	.
test_cosmo_column.py	2	.	1	1	.	.	.	.	.	.	2	.
test_cosmo_raster.py	2	.	1	.	.	.	1	.	.	.	2	.
test_crs_semantic_matching.py	3	.	1	.	.	.	1	1	.	.	3	.
test_feature_finder.py	3	.	.	.	.	.	3	1	.	.	3	.
test_flat_surfaces.py	3	2	.	.	.	.	1	.	.	.	3	.
test_flow_info.py	7	4	2	.	.	.	6	.	.	.	7	.
test_fluvial_surfaces.py	18	10	5	1	.	.	3	8	.	1	18	.
test_geospatial_runtime.py	3	1	.	.	.	.	1	1	.	.	3	.
test_hillslope_transport.py	4	.	2	.	.	.	1	1	.	.	4	.
test_hydraulic.py	12	5	6	3	.	1	.	3	.	1	12	.
test_internal_drainage.py	5	3	2	1	.	.	1	1	.	1	5	.
test_keyword_compatibility.py	3	2	.	.	.	.	.	2	.	2	3	.
test_landscape_model.py	11	3	3	3	.	1	.	2	.	.	11	.
test_landslides.py	3	1	2	1	.	.	.	1	.	1	3	.
test_llm_helper.py	9	.	.	.	.	.	.	1	.	9	9	.
test_point_snapping.py	19	8	6	8	.	.	.	8	.	3	19	.
test_public_api_manifest.py	6	.	.	.	.	.	.	.	.	6	6	.
test_public_docstrings.py	13	.	.	1	.	.	.	.	.	13	13	.
test_public_keyword_migrations.py	6	1	.	3	.	.	1	5	.	4	6	.
test_raster.py	14	11	8	4	.	.	1	2	.	.	14	.
test_raster_conditioning.py	10	6	4	1	.	.	.	3	.	.	10	.
test_raster_fetch.py	8	2	.	.	.	.	.	6	.	.	8	.
test_remaining_unit_keyword_migrations.py	3	.	.	3	.	.	.	2	.	3	3	.
test_rotation_frames.py	18	4	3	7	.	4	.	4	.	2	15	3
test_sediment_routing.py	3	.	1	1	.	.	.	2	.	1	3	.
test_spectral.py	3	.	3	.	.	.	1	.	.	.	3	.
test_surface_keyword_migrations.py	5	.	.	3	.	.	.	2	.	4	5	.
test_swath.py	3	2	.	.	.	.	.	1	.	.	3	.
test_synthetic_workflows.py	3	.	2	1	.	.	.	.	.	.	3	.
test_terrain_features.py	2	1	1	.	.	.	2	.	.	.	2	.
test_tidal_workflows.py	2	1	1	.	.	.	.	.	.	.	2	.
test_valley_cross_sections.py	1	.	.	.	.	.	1	.	.	.	1	.
test_valley_mask_migration.py	2	2	.	.	.	2	.	.	.	.	2	.
test_vector_hydrology.py	3	2	1	.	.	.	2	.	.	.	3	.

Abbreviations: exact = exact_solution, prop = property, meta = metamorphic, diff = differential, char = characterisation, smoke = smoke, neg = negative, perf = performance, conf = conformance.