02SpatialML - Spatial Machine Learning Lab

User manual, version 0.1.1 (2026-11-16). QGIS 3.28 to 4.x. GPL-2.0-or-later. Yusuf Eminoğlu (Geophilo).

1. What the plugin does

02SpatialML carries one predictive-mapping study from samples to a map with known limits inside QGIS. The input is a point or polygon layer with an observed target (a number for regression, a class for classification) and covariates from the attribute table or from a stack of covariate rasters. The study runs in seven steps: build the sample table, explore the data, choose a validation design, cross-validate the models on the same folds, explain them on held-out data, predict a map with an interval and the area of applicability, and write a model card with a run manifest that replays the study.

The same study is available in two ways: the lab dock (one page per step, with charts and styled layers) and the Processing provider (one algorithm per step plus a whole-study algorithm, usable in models and batch runs). Every engine is written in NumPy: the histogram tree builder shared by the random forest and gradient boosting, TreeSHAP, GWR and MGWR, the variogram fit, kNNDM and the area of applicability. SciPy and scikit-learn are used when present; nothing has to be installed beyond QGIS.

Why it exists

Predictive mapping with machine learning has a well-documented failure mode: models are validated with random cross-validation on clustered samples, the error estimate is far too optimistic, and the map is then extrapolated into covariate space the model has never seen (Ploton et al. 2020; Meyer & Pebesma 2022). The R ecosystem has the remedies (CAST with the area of applicability and kNNDM, blockCV, spatialsample, mlr3spatiotempcv, sperrorest, GWmodel, SpatialML) and Python has PySAL mgwr, shap, verde and spatial-kfold, but none of these runs as a study inside QGIS. QGIS plugins cover single slices: image classification into land cover (dzetsaka, GeoClassify, the Semi-Automatic Classification Plugin), imaging spectroscopy (EnMAP-Box), one random forest (ml4qgis), (M)GWR on a vector table (mgwr_plugin), one model each with external packages (GPBoost Spatial Predictor, Downscale Anything) or species distribution only (QMaxent). None computes an area of applicability, compares validation designs by their nearest-neighbour distance distributions, or writes a replayable model record.

Hot spots, LISA, spatial econometrics, kriging, clustering and other learners (SVM, MLP, EBM) are not part of 02SpatialML; they are in GeoStats Lab, linked from the Help page.

2. Install

  1. QGIS: Plugins > Manage and Install Plugins, search 02SpatialML, install. Or Install from ZIP with a release zip.
  2. A toolbar button and the menu Plugins > 02SpatialML open the lab; the Processing toolbox lists the provider 02SpatialML.
  3. Nothing else is required. NumPy and GDAL ship with QGIS. Optional packages, shown on the Settings page:
    PackageUsed forWhen missing
    SciPyk-d tree neighbour queries, linear solvesNumPy fallback with identical results (to 10−1010^{-10})
    scikit-learnengine scikit-learn for the random forest on large sample sets; its trees are converted to the native format, so quantile intervals, TreeSHAP and the AOA still worknative engine
    shap, mgwrdeveloper cross-checks onlychecks skipped
    The plugin never installs a package itself. The Settings page shows the command "<QGIS Python>" -m pip install --user scikit-learn with the path of the Python that runs QGIS filled in, to copy into the OSGeo4W Shell (or a terminal); restart QGIS afterwards.

Limits of the native engines: a sample table above 50 000 rows or a target with more than 30 classes is refused with a readable message; GWR is skipped (and logged) above 20 000 samples and MGWR above 10 000; MGWR standard errors are computed up to 3 000 samples; GWR and MGWR raster prediction is limited to 250 000 valid cells unless the covariates are aggregated.

3. Units and working CRS

Every distance (block size, buffer radius, GWR fixed bandwidth, variogram lags, nearest-neighbour distances) is in metres in a projected CRS. The working CRS is the CRS of the first covariate raster when it is projected in metres, else the CRS of the sample layer when it is projected in metres, else the UTM zone of the centre of the sample extent (EPSG:326xx north, 327xx south). Sample coordinates are transformed into it. Covariate rasters that differ from the template (the first raster) in CRS, cell size, extent or alignment are resampled onto its grid with GDAL Warp into the run folder: bilinear for continuous rasters, nearest neighbour for categorical rasters, nodata merged. The Data page badge, the run log, the manifest and the model card record the working CRS, the template grid and every resampling step.

4. The lab dock, page by page

The lab is a dock on the right of the QGIS window. A vertical icon rail switches between seven numbered pages; Settings and Help sit at the bottom of the rail. Each page runs one step as a QGIS background task with a progress bar and a Cancel button, so QGIS stays responsive. Changing the settings of an earlier step clears the results of every later step (order: data, explore, design, validate, explain, predict).

1 Data

Choose the samples layer (points or polygons, one row per observation) and the target field. The task is detected from the field and shown as a chip: numbers give a regression, text or an integer field with at most 10 distinct values gives a classification (at most 30 classes); the chips Regression and Classification override it. Attribute covariates are numeric fields of the samples layer; covariate rasters are raster layers of the project, each with a Categorical toggle. For polygon samples choose the polygon statistic (mean of cells, majority of cells, centroid cell); for point samples the point buffer (0 reads the cell under the point, a radius in metres averages the cells within it). The CRS badge shows the working CRS and why it was chosen; the grid badge shows the template grid and whether the rasters must be harmonised. Load demo landscape builds the synthetic landscape of section 15. Build sample table reads the covariates at every sample; KPI cards show the samples kept, the rows dropped with their reasons, the number of covariates and the target range (or the classes). Run whole study runs every step with the settings of every page.

2 Explore

The target histogram; the Pearson correlation between covariates as a heat map with the variance inflation factor of each covariate, VIFj=1/(1−Rj2)\mathrm{VIF}_j = 1/(1 - R_j^2), where Rj2R_j^2 is the coefficient of determination of covariate jj regressed on the others (values above 5 are marked, and ALE is then preferred over PDP); the empirical variogram of the target and of the OLS residuals with the fitted model and its practical range, shown as the suggested block size.

3 Validation

Design chips (random k-fold, spatial blocks, hexagonal blocks, coordinate clusters, buffered leave-one-out, kNNDM), the number of folds kk, the block size or buffer radius (prefilled from the variogram range when Size from the variogram range is ticked) and Also run random k-fold, which cross-validates with random folds too so that the optimism gap can be reported. Make folds builds the folds of the chosen design (and random folds) and the distance diagnostics; Preview folds on map adds the block or cluster polygons with their fold id. The chart shows the nearest-neighbour distance distributions of section 7 (samples, prediction locations, each design) and a table gives the Wasserstein distance WW per design; lower is better. A hint appears when random k-fold has the lower WW (samples spread like the prediction locations). KPI cards: folds of the chosen design, samples per fold, smallest and largest test fold.

4 Models

A checklist of the five models (OLS, random forest, gradient boosting, GWR, MGWR), each with its formula and reference as a tooltip; GWR and MGWR are regression only. The engine chip (native or scikit-learn) is enabled only when scikit-learn is installed. Tune hyperparameters (nested CV) runs the grids of section 9. MGWR minimum bandwidth sets the lower bound of each MGWR term's bandwidth search (0 keeps the default of 42 neighbours; the demo sets 100, see section 8). Run validation tunes and cross-validates every ticked model on the same folds and refits each on all samples. The leaderboard has one row per model and design, sorted by the spatial-CV error. KPI cards for the best model: spatial-CV R2R^2 (or balanced accuracy), spatial RMSE, interval coverage and residual Moran's II. A bar per model shows the optimism gap (spatial-CV RMSE minus random-CV RMSE; for classes random minus spatial balanced accuracy). Show residuals on map styles the samples by their held-out residual.

5 Explain

Choose the model (the best one by default) and the budget: permutation repeats RR, the rows and the first trees used by TreeSHAP, and the grid points of the effect curves. Run explanations computes the permutation importance (bars with whiskers for the spread over folds), the TreeSHAP summary (mean ∣ϕj∣|\phi_j| bars and a strip plot of the per-sample values) and the effect plot of any covariate with the chips PDP, ICE and ALE (the covariate picker lists the six most important). For GWR and MGWR the local coefficient layer is added, a picker maps any coefficient, and a table lists the bandwidth of every term (neighbours for the adaptive kernel, metres for the fixed kernel).

6 Predict

Prediction target: Raster grid (every valid cell of the harmonised covariate grid), Prediction layer (a copy of a vector layer receives the prediction fields; covariates are read the same way as for the samples) or Skip the prediction. Settings: interval level (nominal central coverage), area of applicability, tile size (edge of the raster window read and predicted at once), GWR aggregation factor (GWR and MGWR raster prediction aggregates the covariates by a block mean of factor ×\times factor cells; it is refused above 250 000 valid cells without aggregation). Predict uses the best model and writes Float32 GeoTIFFs into the run folder. KPI cards: cells predicted, share inside the AOA, median interval width. The layers are added to a 02SpatialML group: mean (viridis), interval width (magma), AOA (inside teal, outside amber) and the dissimilarity index.

7 Report

Choose the output folder and a file prefix, tick the outputs and press Write outputs: the model card (HTML with KaTeX), the run manifest (JSON), the CV predictions (CSV: observed, predicted, interval and fold of every held-out prediction, per model and design) and the layers (GeoPackage: samples with CV predictions, fold polygons, local coefficients). Replay manifest takes a manifest written by this plugin, rebuilds the table, checks the hashes, refits and compares the metrics, and shows a pass or fail card with any mismatch.

Settings and Help

Settings lists the optional packages (found, version, not found) with the copyable install command and Check again, the default seed (one NumPy generator per random step) and the run folder for harmonised rasters and predictions. Help gives a short guide, the method references and links to this manual, the repository and GeoStats Lab.

5. Samples and covariates

Point samples read the covariate value of the cell under the point, or the mean of the cells whose centre lies within the point buffer. The radius is in ground metres also when the template raster is geographic (degrees) or in feet: it is converted with the length of one degree of longitude and latitude on the WGS 84 ellipsoid at the sample, or with the linear unit. Polygon samples take the mean (or the majority, for categorical covariates and on request) of the cells whose centre lies inside the polygon, or the value at the centroid. Each kept sample is placed at its point or polygon centroid in the working CRS. Rows with a NULL target, a NULL attribute covariate or nodata in any raster are dropped and counted by reason. Categorical covariates are stored as integer codes 0,…,L−10,\dots,L-1 of their sorted levels; trees split on the codes, while OLS, GWR and MGWR expand them to one-hot columns with the first level dropped. A level not seen in training gets the code −1-1.

6. Variogram

The empirical semivariogram uses the Matheron estimator in 15 distance classes up to half the largest sample distance, where N(h)N(h) is the number of pairs in class hh:

Matheron semivariogram

γ^(h)=12N(h)∑∥si−sj∥∈h(zi−zj)2(1)\hat{\gamma}(h) = \frac{1}{2N(h)} \sum_{\lVert s_i - s_j \rVert \in h} \left(z_i - z_j\right)^2 \tag{1}

A spherical and an exponential model with nugget c0c_0, partial sill cc and practical range aa are fitted,

γsph(h)=c0+c(3h2a−h32a3) for h≤a,γexp(h)=c0+c(1−e−3h/a),(2)\gamma_{\mathrm{sph}}(h) = c_0 + c\left(\tfrac{3h}{2a} - \tfrac{h^3}{2a^3}\right) \text{ for } h \le a,\qquad \gamma_{\mathrm{exp}}(h) = c_0 + c\left(1 - e^{-3h/a}\right), \tag{2}

by the weighted least squares of Cressie (1985), and the model with the lower residual is kept:

Cressie weighted least squares

min⁡θ∑kN(hk)(γ^(hk)γ(hk;θ)−1)2(3)\min_{\theta} \sum_k N(h_k)\left(\frac{\hat{\gamma}(h_k)}{\gamma(h_k;\theta)} - 1\right)^2 \tag{3}

The variogram is computed on the target and on the OLS residuals. The practical range of the residual variogram (of the target for classification) is the default block size and buffer radius (Roberts et al. 2017; Valavi et al. 2019): samples closer than the range are correlated, so testing on them overstates accuracy.

7. Validation designs and distance diagnostics

Designs

DesignRuleReference
Random k-foldsamples assigned to kk folds at random-
Spatial blockssquare grid of side ss anchored at the minimum of the sample extent (so a translation of all coordinates gives the same folds); whole blocks are dealt to kk folds by a seeded greedy balance of the sample countsValavi et al. 2019
Hexagonal blocksthe same with hexagons of area s2s^2Valavi et al. 2019
Coordinate clustersk-means++ on the coordinates into kk clusters, one fold eachBrenning 2012
Buffered leave-one-outeach sample ii is tested alone; its training set excludes every sample within radius rr of iiLe Rest et al. 2014; Pohjankukka et al. 2017
kNNDMkk-fold assignment from a hierarchical clustering of the coordinates whose cut minimises WW (equation (4)) between the CV distances and the prediction distancesLinnenbrink et al. 2024

kNNDM follows CAST::knndm: the number of clusters is searched on 100 log-spaced values from kk to n−2n-2; clusters are ordered along the first principal axis of their centroids, clusters larger than n/kn/k are kept whole and the rest are dealt round-robin to the folds, with no fold above half of the samples. When a one-sided Kolmogorov-Smirnov test does not show that the prediction distances exceed the sample distances, the samples already cover the prediction area and random k-fold is returned (the log says so).

Distance diagnostics

Three empirical cumulative distribution functions of nearest-neighbour distances are compared (Milà et al. 2022): sample to nearest other sample, G^j\hat{G}_j; prediction location to nearest sample, G^ij\hat{G}_{ij}; and test sample to nearest training sample under the design, G^j∗\hat{G}^{*}_{j}. Prediction locations are the valid cells of the template raster (a regular subsample of at most 20 000 cells) or the centroids of the prediction layer. A design whose G^j∗\hat{G}^{*}_{j} lies on G^ij\hat{G}_{ij} tests at the distances the map predicts at. The gap is the Wasserstein-1 distance (in metres):

Wasserstein distance between distance distributions

W=∫0∞∣G^ij(d)−G^j∗(d)∣ dd(4)W = \int_0^{\infty} \left|\hat{G}_{ij}(d) - \hat{G}^{*}_{j}(d)\right|\,\mathrm{d}d \tag{4}

Random k-fold on clustered samples gives a G^j∗\hat{G}^{*}_{j} far to the left of G^ij\hat{G}_{ij} and a large WW; spatial designs move it right. WW is reported for every design run.

8. Models

All five models are fitted on the same folds. In the formulas XX is the n×kn \times k design matrix (with intercept), yy the target, TT the number of trees and α=1−\alpha = 1 - the interval level.

OLS (baseline)

Fitted by QR decomposition, with adjusted R2R^2, AICc and the VIF per covariate; the prediction interval is analytic:

Ordinary least squares

β^=(X⊤X)−1X⊤y,y^0±t1−α/2, n−k s1+x0⊤(X⊤X)−1x0(5)\hat{\beta} = (X^\top X)^{-1} X^\top y,\qquad \hat{y}_0 \pm t_{1-\alpha/2,\,n-k}\, s\sqrt{1 + x_0^\top (X^\top X)^{-1} x_0} \tag{5}

Every other model must beat this baseline under the spatial design to be worth its complexity.

Random forest and quantile regression forest

Bootstrap samples, mtry=⌈p/3⌉m_{\mathrm{try}} = \lceil p/3 \rceil covariates per split for regression and ⌈p ⌉\lceil \sqrt{p}\, \rceil for classification, minimum leaf size 5 (regression) or 1 (classification), 300 trees, squared-error or Gini impurity. Splits are searched on at most 64 quantile bins per covariate (histogram builder, as in LightGBM, Ke et al. 2017); categorical codes are split after ordering the levels by their mean target (Fisher 1958). The out-of-bag score is reported; impurity importance is reported but not used for ranking.

Random forest

f^(x)=1T∑t=1Tft(x)(6)\hat{f}(x) = \frac{1}{T}\sum_{t=1}^{T} f_t(x) \tag{6}

Intervals come from the quantile regression forest (Meinshausen 2006): each in-bag sample ii gets a weight from the leaves Lt(x)L_t(x) that xx falls into, with ctic_{ti} its bootstrap count in tree tt, and the interval bounds are quantiles of the weighted distribution of yy:

Quantile regression forest weights

wi(x)=1T∑t=1Tcti 1[i∈Lt(x)]∑jctj 1[j∈Lt(x)],F^(y∣x)=∑iwi(x) 1[yi≤y](7)w_i(x) = \frac{1}{T}\sum_{t=1}^{T} \frac{c_{ti}\,\mathbf{1}[i \in L_t(x)]}{\sum_{j} c_{tj}\,\mathbf{1}[j \in L_t(x)]},\qquad \hat{F}(y \mid x) = \sum_i w_i(x)\,\mathbf{1}[y_i \le y] \tag{7}

For classes the probability is the mean leaf class share, and the uncertainty is the normalised entropy H(x)=−∑cpcln⁡pc/ln⁡CH(x) = -\sum_{c} p_c \ln p_c / \ln C of the CC class probabilities.

Gradient boosting

Squared-error or binary logistic loss with gradients gig_i and Hessians hih_i, Newton leaf values, shrinkage ν=0.05\nu = 0.05, depth 4, subsample 0.8, λ=1\lambda = 1 and the number of rounds chosen by early stopping on an inner split that respects the outer design (Friedman 2001, 2002). Multiclass boosting is not available; a target with more than two classes is refused for this model.

Gradient boosting with Newton leaves

Fm(x)=Fm−1(x)+ν γl(x),γl=∑i∈lgi∑i∈lhi+λ(8)F_m(x) = F_{m-1}(x) + \nu\,\gamma_{l(x)},\qquad \gamma_l = \frac{\sum_{i \in l} g_i}{\sum_{i \in l} h_i + \lambda} \tag{8}

Split-conformal intervals

Gradient boosting, GWR and MGWR get split-conformal intervals from the absolute cross-validation residuals ∣r∣|r| under the chosen design (Lei et al. 2018). The half-width qq is the ⌈(n+1)(1−α)⌉\lceil (n+1)(1-\alpha) \rceil-th smallest absolute residual:

Split-conformal interval

y^±q,q=∣r∣(⌈(n+1)(1−α)⌉)(9)\hat{y} \pm q,\qquad q = |r|_{(\lceil (n+1)(1-\alpha) \rceil)} \tag{9}

When ⌈(n+1)(1−α)⌉>n\lceil (n+1)(1-\alpha) \rceil > n the interval is unbounded and is written as NULL (nodata), never as infinity. The guarantee assumes exchangeable residuals, which spatial autocorrelation can break; the model card states this caveat.

GWR

Geographically weighted regression (Brunsdon, Fotheringham & Charlton 1996; Fotheringham et al. 2002) fits a local regression at every location uiu_i. The default kernel is adaptive bisquare with dkd_k the distance to the kk-th nearest neighbour; the alternative is the fixed Gaussian kernel wij=exp⁡ ⁣(−12(dij/b)2)w_{ij} = \exp\!\left(-\tfrac{1}{2}(d_{ij}/b)^2\right) with bandwidth bb in metres.

Geographically weighted regression

β^(ui)=(X⊤WiX)−1X⊤Wiy,wij=(1−(dij/dk)2)21[dij<dk](10)\hat{\beta}(u_i) = (X^\top W_i X)^{-1} X^\top W_i y,\qquad w_{ij} = \left(1 - (d_{ij}/d_{k})^2\right)^2 \mathbf{1}[d_{ij} < d_{k}] \tag{10}

The bandwidth is chosen by golden-section search on the corrected AIC (Hurvich, Simonoff & Tsai 1998), where SS is the hat matrix, or on the CV score:

Corrected AIC for GWR

AICc=2nln⁡σ^+nln⁡2π+n n+tr⁡Sn−2−tr⁡S(11)\mathrm{AIC_c} = 2n\ln\hat{\sigma} + n\ln 2\pi + n\,\frac{n + \operatorname{tr}S}{n - 2 - \operatorname{tr}S} \tag{11}

Outputs per location: coefficients, local tt-values judged against the adjusted α\alpha of da Silva & Fotheringham (2016), local R2R^2 and the condition number. Prediction at new locations uses the same kernel on the training samples. Regression only.

MGWR

Multiscale GWR (Fotheringham, Yang & Kang 2017; Oshan et al. 2019) gives each covariate jj its own bandwidth bjb_j. XX and yy are standardised; back-fitting updates one term at a time with a golden-section AICc search per term until the score of change falls below 10−510^{-5} or 200 iterations are reached (the model card states whether it converged). Each term's bandwidth is searched between 42 neighbours (40+2k40 + 2k with k=1k = 1, as PySAL mgwr) and nn; the MGWR minimum bandwidth raises that lower bound. With clustered samples a kernel that stays inside one cluster makes the intercept and the slopes nearly collinear, and back-fitting then creeps towards the tolerance without reaching it: on the demo landscape (400 samples in 8 clusters of 50, a one-hot categorical covariate) the default bound stops at 200 iterations, while a bound of 100 neighbours (two clusters) converges. Loading the demo sets 100; the model card states the bound used. Covariate effective numbers of parameters and standard errors use the operator method of Yu et al. (2020) for the refit on all samples (n≤3 000n \le 3\,000); coefficients are on the standardised scale.

Multiscale GWR

yi=∑j=0kβbjj(ui) xij+εi(12)y_i = \sum_{j=0}^{k} \beta_{b_j j}(u_i)\, x_{ij} + \varepsilon_i \tag{12}

9. Tuning and metrics

Nested tuning

Small documented grids are evaluated with nested cross-validation, so the reported score never sees the test fold: inside each outer training set an inner design of the same kind (k=3k = 3 folds by default) scores every grid point. Grids: random forest mtry∈{⌈p ⌉,⌈p/3⌉,⌈p/2⌉}m_{\mathrm{try}} \in \{\lceil\sqrt{p}\,\rceil, \lceil p/3\rceil, \lceil p/2\rceil\} and minimum leaf size ∈{1,5,10}\in \{1, 5, 10\} (100 trees while tuning); gradient boosting depth ∈{2,4,6}\in \{2, 4, 6\} and ν∈{0.03,0.1}\nu \in \{0.03, 0.1\} (at most 300 rounds while tuning); GWR kernel ∈\in {adaptive bisquare, fixed Gaussian}. kNNDM uses coordinate clusters as the inner design, and so do blocks when there are too few blocks; buffered leave-one-out is not tuned (defaults, logged). The final hyperparameters are the grid point with the lowest mean inner loss over the outer folds; the random-CV comparison uses them without nesting.

Metrics

All metrics are computed on the pooled held-out predictions. Regression: RMSE=1n∑i(yi−y^i)2\mathrm{RMSE} = \sqrt{\tfrac{1}{n}\sum_i (y_i - \hat{y}_i)^2}, MAE=1n∑i∣yi−y^i∣\mathrm{MAE} = \tfrac{1}{n}\sum_i |y_i - \hat{y}_i|, bias =1n∑i(y^i−yi)= \tfrac{1}{n}\sum_i (\hat{y}_i - y_i) (mean error),

Coefficient of determination

R2=1−∑i(yi−y^i)2∑i(yi−yˉ)2(13)R^2 = 1 - \frac{\sum_i (y_i - \hat{y}_i)^2}{\sum_i (y_i - \bar{y})^2} \tag{13}

and Lin's concordance correlation between observed and predicted values (Lin 1989):

Lin's concordance correlation

ρc=2sxysx2+sy2+(xˉ−yˉ)2(14)\rho_c = \frac{2 s_{xy}}{s_x^2 + s_y^2 + (\bar{x} - \bar{y})^2} \tag{14}

Classification: accuracy, balanced accuracy (mean recall over classes), Cohen's κ\kappa, macro F1F_1, log loss −1n∑iln⁡p^i,yi-\tfrac{1}{n}\sum_i \ln \hat{p}_{i,y_i} and Brier score 1n∑i∑c(p^ic−1[yi=c])2\tfrac{1}{n}\sum_i \sum_c (\hat{p}_{ic} - \mathbf{1}[y_i = c])^2. The interval coverage is the share of held-out values inside the nominal interval. Residual Moran's II (Moran 1950) uses 8 nearest neighbours with row-standardised weights wijw_{ij}, S0=∑i∑jwijS_0 = \sum_i\sum_j w_{ij}, residuals ziz_i centred on their mean, and a 999-permutation seeded pp-value:

Moran's I of residuals

I=nS0 ∑i∑jwijzizj∑izi2(15)I = \frac{n}{S_0}\,\frac{\sum_i \sum_j w_{ij} z_i z_j}{\sum_i z_i^2} \tag{15}

The optimism gap (Ploton et al. 2020) is the difference between the spatial-CV and the random-CV error of the same model and hyperparameters:

Optimism gap

Δ=RMSEspatial CV−RMSErandom CV(16)\Delta = \mathrm{RMSE}_{\text{spatial CV}} - \mathrm{RMSE}_{\text{random CV}} \tag{16}

A positive gap means random cross-validation overstates the map accuracy. Spatial cross-validation estimates the error of predicting at the distances of the chosen design; it is not a design-based estimate of map accuracy (Wadoux et al. 2021).

10. Explanations

Permutation importance

For each covariate jj and each test fold, the covariate is permuted inside the test fold and the increase of the held-out loss LL (RMSE, or log loss for classes) is averaged over RR repeats (Breiman 2001; Fisher, Rudin & Dominici 2019). It is computed on held-out data under the chosen design, so it describes what the model uses to predict new places. Whiskers show the spread over folds.

Permutation importance

Ij=1R∑r=1R[L(y,f(X(j,r)))−L(y,f(X))](17)I_j = \frac{1}{R}\sum_{r=1}^{R} \left[L\big(y, f(X^{(j,r)})\big) - L\big(y, f(X)\big)\right] \tag{17}

TreeSHAP

Exact path-dependent TreeSHAP (Lundberg et al. 2020, Algorithm 2) for the random forest and gradient boosting gives Shapley values ϕj(x)\phi_j(x) that add up to the model output with the base value ϕ0\phi_0 (local accuracy):

SHAP local accuracy

f(x)=ϕ0+∑j=1pϕj(x)(18)f(x) = \phi_0 + \sum_{j=1}^{p} \phi_j(x) \tag{18}

To keep the cost bounded it runs on a stratified subset of rows (500 by default) and on the first trees of the ensemble (100 by default); the model card states the numbers used. For OLS and GWR the linear SHAP value is exact (Lundberg & Lee 2017):

Linear SHAP

ϕj(x)=βj(u) (xj−xˉj)(19)\phi_j(x) = \beta_j(u)\,(x_j - \bar{x}_j) \tag{19}

Partial dependence, ICE and ALE

Partial dependence (Friedman 2001) sets covariate jj to a grid value zz for every sample and averages the predictions; ICE curves (Goldstein et al. 2015) show the same for 100 single samples:

Partial dependence

fˉj(z)=1n∑i=1nf(z,xi,−j)(20)\bar{f}_j(z) = \frac{1}{n}\sum_{i=1}^{n} f(z, x_{i,-j}) \tag{20}

Accumulated local effects (Apley & Zhu 2020) accumulate the mean finite differences inside quantile bins (zk−1,zk](z_{k-1}, z_k] with nkn_k samples, centred by cc to mean zero. They stay valid when covariates are correlated (VIF above 5), where partial dependence evaluates the model on unrealistic combinations:

Accumulated local effects

f^j(z)=∑k=1k(z)1nk∑i: xij∈(zk−1,zk][f(zk,xi,−j)−f(zk−1,xi,−j)]−c(21)\hat{f}_j(z) = \sum_{k=1}^{k(z)} \frac{1}{n_k}\sum_{i:\,x_{ij} \in (z_{k-1}, z_k]} \left[f(z_k, x_{i,-j}) - f(z_{k-1}, x_{i,-j})\right] - c \tag{21}

For GWR and MGWR the explanations also include the local coefficient layer (section 14).

11. Prediction and area of applicability

Area of applicability

Following Meyer & Pebesma (2021) and CAST, covariates are scaled by the training mean and standard deviation and multiplied by weights wjw_j, the permutation importances of the model (negative values set to 0; equal weights if none is positive). The dissimilarity index of a location is the distance to the nearest training sample in this weighted space, divided by the mean distance dˉ\bar{d} between training samples. For the training samples themselves the nearest sample is searched outside the sample's own CV fold, so the AOA changes when the design changes. The threshold is the upper whisker of the training DI, capped at its maximum: min⁡(Q3+1.5 IQR, max⁡DItrain)\min(Q_3 + 1.5\,\mathrm{IQR},\ \max \mathrm{DI}_{\mathrm{train}}), with type-7 quantiles as in R.

Dissimilarity index

DI(x)=min⁡id(x,xi)dˉ,AOA={x:DI(x)≤Q3+1.5 IQR}(22)\mathrm{DI}(x) = \frac{\min_i d(x, x_i)}{\bar{d}},\qquad \text{AOA} = \{x : \mathrm{DI}(x) \le Q_3 + 1.5\,\mathrm{IQR}\} \tag{22}

Inside the AOA the cross-validation error applies; outside it the model extrapolates into covariate combinations it has not seen and the map should not be read.

Prediction outputs

Raster prediction reads the harmonised stack in windows of tile ×\times tile cells; a cell with nodata in any covariate is nodata in every output. Outputs are Float32 GeoTIFFs (deflate, tiled) on the template grid. Regression: mean, lower, upper, width (upper minus lower). Classification: class (Int16), prob_max, entropy. With the AOA option: di, aoa (Byte: 1 inside, 0 outside, 255 nodata) and the prediction masked to the AOA, mean_aoa (regression) or class_aoa (classification), whose cells outside the AOA are nodata: the map to publish, as Meyer & Pebesma (2021) recommend. The dock shows the masked map on top and keeps the full prediction, on the same colour scale, hidden below it. Layer prediction writes the same values as fields of a copy of the layer (section 14).

12. Model card, manifest and replay

The model card is one self-contained HTML file in the structure of Mitchell et al. (2019): intended use, data (samples kept and dropped, covariates, working CRS and grid), validation design with the distance diagnostics, the leaderboard with the optimism gap, explanations, the AOA share, limitations (including the caveat of Wadoux et al. 2021) and every reference. Its formulas are typeset with the bundled KaTeX and use the same LaTeX as this manual, so it renders offline.

The run manifest (JSON, schema docs/manifest.schema.json) records the plugin, QGIS, Python, NumPy and GDAL versions; the working CRS and template grid; the SHA-256 hash of the sample table and of each harmonised raster (first 64 MB, size and header); every parameter and seed; the fold vector and block rings of each design; the chosen and per-fold hyperparameters; and the metrics. The sample table is saved beside it as <name>.table.npz, so studies built from memory layers replay too. Replay reloads the table, checks the hashes, reruns every recorded cross-validation with the recorded folds, seeds and per-fold hyperparameters and compares every metric: exact equality for the native engines, 10−910^{-9} relative for scikit-learn. Any difference is listed by name.

13. Processing algorithms

Provider 02SpatialML. Every algorithm calls the same core functions as the dock. Inputs are copied into private memory layers; results go to feature sinks with real field types (doubles keep full precision in GeoPackage) and to GeoTIFF files in an output folder; outputs are not styled by the algorithms. Model objects pass between algorithms as a run manifest path. Each algorithm's help panel in QGIS repeats its method summary and references. Parameters marked advanced are under Advanced parameters; optional parameters may stay empty.

Shared parameter groups, used by several algorithms:

0 Whole study

Spatial machine learning lab (whole study) zero2spatialml:spatial_ml_lab

The whole study in one run, through the same staged functions as the dock: sample table and harmonised stack, variogram, the chosen design plus random k-fold with distance diagnostics, every chosen model cross-validated on the same folds (nested tuning) and refitted, permutation importance, TreeSHAP of the best tree model, PDP / ICE / ALE of the best model, GWR / MGWR local coefficients, prediction on the raster grid or a layer with interval, DI and AOA, the run manifest and, when a path is given, the model card. Models that do not support the task are skipped and logged. Sections 5 to 12.

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
TARGETTarget field field
COV_FIELDSCovariate fields (attribute table) optionalfield
RASTERSCovariate rasters optionallayers
CATEGORICALCategorical covariates (comma-separated names) optionaltext
TASKTask advancedchoice: Detect from the target field; Regression; ClassificationDetect from the target field
POLYGON_STATPolygon samples: covariate value advancedchoice: Cell mean; Cell majority; Value at the centroidCell mean
POINT_BUFFERPoint samples: mean within this radius (m, 0 = cell value) advancednumber, at least 00
SCHEMEValidation design choice: Random k-fold; Spatial blocks; Hexagonal blocks; Coordinate clusters; Buffered leave-one-out; kNNDMSpatial blocks
KFolds k number, 2 to 1005
SIZEBlock size (m, 0 = variogram range) number, at least 00
RADIUSBuffered LOO radius (m, 0 = variogram range) advancednumber, at least 00
PREDICTION_LAYERPrediction locations for kNNDM (optional; default: valid raster cells) optionalvector layer
SEEDRandom seed advancednumber, at least 042
MODELSModels choice (several): OLS baseline; Random forest; Gradient boosting; GWR; MGWROLS baseline, Random forest, Gradient boosting, GWR
TUNETune hyperparameters by nested cross-validation yes / noyes
ENGINERandom forest engine advancedchoice: Native (numpy); scikit-learn (when installed)Native (numpy)
MGWR_MIN_BWMGWR minimum bandwidth (neighbours, 0 = default 42) advancednumber, 0 to 100000
COMPARE_RANDOMAlso run random k-fold (optimism gap) yes / noyes
REPEATSPermutation repeats advancednumber, 1 to 10010
SHAP_ROWSTreeSHAP rows advancednumber, 10 to 50000500
SHAP_TREESTreeSHAP trees advancednumber, 1 to 5000100
PREDICT_ONPredict on choice: Covariate raster grid; Prediction layer; NothingCovariate raster grid
PREDICT_LAYERPrediction layer optionalvector layer
INTERVALInterval level number, 0.5 to 0.990.9
AOADissimilarity index and area of applicability yes / noyes
TILETile size (cells) advancednumber, 16 to 8192512
AGGREGATEGWR / MGWR aggregation factor advancednumber, 1 to 101
OUTPUT_FOLDERRun folder folder output
SAMPLES_OUTCross-validated samples (best model) vector output
LEADERBOARDLeaderboard vector output
FOLDSFold polygons optionalvector output
LOCALLocal coefficients (GWR / MGWR) optionalvector output
PREDICTEDPrediction layer with predictions optionalvector output
REPORTModel card (HTML) optionalfile output

Further outputs:

OutputMeaning
BESTBest model
MANIFESTRun manifest
AOA_SHAREShare inside the AOA
MEANPrediction (mean)
LOWERInterval lower bound
UPPERInterval upper bound
WIDTHInterval width
CLASSPredicted class
PROB_MAXLargest class probability
ENTROPYNormalised entropy
DIDissimilarity index
AOA_RASTERArea of applicability
MEAN_AOAPrediction inside the AOA
CLASS_AOAPredicted class inside the AOA

1 Data

Build sample table zero2spatialml:build_sample_table

Reads the target and every covariate at the samples (section 5) and writes one point per kept sample in the working CRS (section 3). Rows dropped are counted by reason in the log.

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
TARGETTarget field field
COV_FIELDSCovariate fields (attribute table) optionalfield
RASTERSCovariate rasters optionallayers
CATEGORICALCategorical covariates (comma-separated names) optionaltext
TASKTask advancedchoice: Detect from the target field; Regression; ClassificationDetect from the target field
POLYGON_STATPolygon samples: covariate value advancedchoice: Cell mean; Cell majority; Value at the centroidCell mean
POINT_BUFFERPoint samples: mean within this radius (m, 0 = cell value) advancednumber, at least 00
SEEDRandom seed advancednumber, at least 042
OUTPUTSample table vector output

Further outputs:

OutputMeaning
N_SAMPLESSamples kept
N_DROPPEDSamples dropped
N_COVARIATESCovariates
TASK_DETECTEDTask
WORKING_CRSWorking CRS

Harmonise covariate rasters zero2spatialml:harmonise_rasters

Resamples covariate rasters onto the grid of a template raster with GDAL Warp (bilinear; nearest neighbour for the rasters named as categorical). Rasters already on the grid are returned unchanged.

ParameterLabelTypeDefault
RASTERSCovariate rasters layers
TEMPLATETemplate raster (empty: the first raster) optionalraster layer
CATEGORICALCategorical rasters (comma-separated names) optionaltext
OUTPUT_FOLDEROutput folder folder output

Further outputs:

OutputMeaning
OUTPUTHarmonised rasters
RESAMPLEDRasters resampled

2 Explore

Empirical variogram zero2spatialml:empirical_variogram

The Matheron variogram of a numeric field with spherical and exponential fits, equations (1) to (3) (section 6).

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
FIELDField field
N_LAGSDistance classes number, 3 to 10015
MAX_DISTLargest distance (m, 0 = half the largest sample distance) number, at least 00
MODELModel choice: Better of spherical and exponential; Spherical; ExponentialBetter of spherical and exponential
OUTPUTVariogram vector output

Further outputs:

OutputMeaning
MODEL_CHOSENModel
NUGGETNugget
SILLSill
RANGEPractical range (m)

3 Validation design

Make spatial folds zero2spatialml:spatial_folds

Builds the folds of one design (section 7) and, when prediction locations are given, the Wasserstein distance WW of the design. A size or radius of 0 takes the practical range of the variogram of the chosen field.

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
SCHEMEValidation design choice: Random k-fold; Spatial blocks; Hexagonal blocks; Coordinate clusters; Buffered leave-one-out; kNNDMSpatial blocks
KFolds k number, 2 to 1005
SIZEBlock size (m, 0 = variogram range) number, at least 00
RADIUSBuffered LOO radius (m, 0 = variogram range) number, at least 00
FIELDVariogram field (for a 0 size or radius) optionalfield
PREDICTION_RASTERPrediction raster (valid cells) optionalraster layer
PREDICTION_LAYERPrediction layer (centroids) optionalvector layer
SEEDRandom seed advancednumber, at least 042
OUTPUTSamples with folds vector output
BLOCKSFold polygons optionalvector output

Further outputs:

OutputMeaning
FOLDSFolds
WWasserstein distance to the prediction distances (m)
SIZE_USEDBlock size or radius used (m)
NOTEDesign

Nearest-neighbour distance diagnostics zero2spatialml:nnd_diagnostics

The three nearest-neighbour distance distributions of section 7 for an existing fold field (or buffered LOO with a radius) and the Wasserstein distance between them.

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
FOLD_FIELDFold field optionalfieldfold
RADIUSBuffered LOO radius (m, 0 = use the fold field) number, at least 00
PREDICTION_RASTERPrediction raster (valid cells) optionalraster layer
PREDICTION_LAYERPrediction layer (centroids) optionalvector layer
POINTSCurve points advancednumber, 10 to 2000100
OUTPUTDistance curves vector output

Further outputs:

OutputMeaning
W_CVW between prediction and CV distances (m)
W_SAMPLEW between prediction and sample distances (m)
MEDIAN_PREDICTIONMedian prediction-to-sample distance (m)
MEDIAN_CVMedian CV test-to-training distance (m)

4 Models

Cross-validate model zero2spatialml:cross_validate

Cross-validates one model (section 8) under the design and, optionally, under random k-fold with the same hyperparameters, so the optimism gap is visible (section 9). The model is refitted on all samples; the run manifest can be saved for replay.

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
TARGETTarget field field
COV_FIELDSCovariate fields (attribute table) optionalfield
RASTERSCovariate rasters optionallayers
CATEGORICALCategorical covariates (comma-separated names) optionaltext
TASKTask advancedchoice: Detect from the target field; Regression; ClassificationDetect from the target field
POLYGON_STATPolygon samples: covariate value advancedchoice: Cell mean; Cell majority; Value at the centroidCell mean
POINT_BUFFERPoint samples: mean within this radius (m, 0 = cell value) advancednumber, at least 00
SCHEMEValidation design choice: Random k-fold; Spatial blocks; Hexagonal blocks; Coordinate clusters; Buffered leave-one-out; kNNDMSpatial blocks
KFolds k number, 2 to 1005
SIZEBlock size (m, 0 = variogram range) number, at least 00
RADIUSBuffered LOO radius (m, 0 = variogram range) advancednumber, at least 00
PREDICTION_LAYERPrediction locations for kNNDM (optional; default: valid raster cells) optionalvector layer
SEEDRandom seed advancednumber, at least 042
MODELModel choice: OLS baseline; Random forest; Gradient boosting; GWR; MGWRRandom forest
TUNETune hyperparameters by nested cross-validation yes / nono
ENGINERandom forest engine advancedchoice: Native (numpy); scikit-learn (when installed)Native (numpy)
MGWR_MIN_BWMGWR minimum bandwidth (neighbours, 0 = default 42) advancednumber, 0 to 100000
COMPARE_RANDOMAlso run random k-fold (optimism gap) yes / noyes
INTERVALInterval level number, 0.5 to 0.990.9
OUTPUTCross-validated samples vector output
LEADERBOARDScores optionalvector output
MANIFESTRun manifest optionalfile output

Further outputs:

OutputMeaning
RMSERMSE
MAEMAE
R2R squared
BIASBias (mean error)
CCCLin's concordance correlation
ACCURACYAccuracy
BALANCED_ACCURACYBalanced accuracy
KAPPACohen's kappa
MORAN_IResidual Moran's I
COVERAGEInterval coverage
OPTIMISM_GAPOptimism gap

5 Explanation

Explain model (importance, SHAP, PDP / ALE) zero2spatialml:explain_model

Cross-validates one model, refits it and explains it (section 10): permutation importance on held-out folds, TreeSHAP for tree models, PDP and ALE curves, local coefficients for GWR / MGWR.

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
TARGETTarget field field
COV_FIELDSCovariate fields (attribute table) optionalfield
RASTERSCovariate rasters optionallayers
CATEGORICALCategorical covariates (comma-separated names) optionaltext
TASKTask advancedchoice: Detect from the target field; Regression; ClassificationDetect from the target field
POLYGON_STATPolygon samples: covariate value advancedchoice: Cell mean; Cell majority; Value at the centroidCell mean
POINT_BUFFERPoint samples: mean within this radius (m, 0 = cell value) advancednumber, at least 00
SCHEMEValidation design choice: Random k-fold; Spatial blocks; Hexagonal blocks; Coordinate clusters; Buffered leave-one-out; kNNDMSpatial blocks
KFolds k number, 2 to 1005
SIZEBlock size (m, 0 = variogram range) number, at least 00
RADIUSBuffered LOO radius (m, 0 = variogram range) advancednumber, at least 00
PREDICTION_LAYERPrediction locations for kNNDM (optional; default: valid raster cells) optionalvector layer
SEEDRandom seed advancednumber, at least 042
MODELModel choice: OLS baseline; Random forest; Gradient boosting; GWR; MGWRRandom forest
TUNETune hyperparameters by nested cross-validation yes / nono
ENGINERandom forest engine advancedchoice: Native (numpy); scikit-learn (when installed)Native (numpy)
MGWR_MIN_BWMGWR minimum bandwidth (neighbours, 0 = default 42) advancednumber, 0 to 100000
REPEATSPermutation repeats number, 1 to 10010
FEATURESEffect covariates (comma-separated, empty = six most important) optionaltext
GRIDPartial dependence grid points advancednumber, 5 to 20025
SHAP_ROWSTreeSHAP rows advancednumber, 10 to 50000500
SHAP_TREESTreeSHAP trees advancednumber, 1 to 5000100
IMPORTANCEPermutation importance vector output
EFFECTSEffect curves optionalvector output
SHAPSHAP values optionalvector output
LOCALLocal coefficients (GWR / MGWR) optionalvector output

Further outputs:

OutputMeaning
TOPMost important covariate

6 Prediction

Fit and predict raster zero2spatialml:fit_predict_raster

Cross-validates one model, refits it on all samples and predicts every cell of the covariate grid tile by tile, with interval, DI and AOA (section 11). The run manifest is written into the output folder.

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
TARGETTarget field field
RASTERSCovariate rasters layers
CATEGORICALCategorical covariates (comma-separated names) optionaltext
TASKTask advancedchoice: Detect from the target field; Regression; ClassificationDetect from the target field
POLYGON_STATPolygon samples: covariate value advancedchoice: Cell mean; Cell majority; Value at the centroidCell mean
POINT_BUFFERPoint samples: mean within this radius (m, 0 = cell value) advancednumber, at least 00
SCHEMEValidation design choice: Random k-fold; Spatial blocks; Hexagonal blocks; Coordinate clusters; Buffered leave-one-out; kNNDMSpatial blocks
KFolds k number, 2 to 1005
SIZEBlock size (m, 0 = variogram range) number, at least 00
RADIUSBuffered LOO radius (m, 0 = variogram range) advancednumber, at least 00
SEEDRandom seed advancednumber, at least 042
MODELModel choice: OLS baseline; Random forest; Gradient boosting; GWR; MGWRRandom forest
TUNETune hyperparameters by nested cross-validation yes / nono
ENGINERandom forest engine advancedchoice: Native (numpy); scikit-learn (when installed)Native (numpy)
MGWR_MIN_BWMGWR minimum bandwidth (neighbours, 0 = default 42) advancednumber, 0 to 100000
INTERVALInterval level number, 0.5 to 0.990.9
AOADissimilarity index and area of applicability yes / noyes
TILETile size (cells) advancednumber, 16 to 8192512
AGGREGATEGWR / MGWR aggregation factor advancednumber, 1 to 101
OUTPUT_FOLDEROutput folder folder output

Further outputs:

OutputMeaning
MEANPrediction (mean)
LOWERInterval lower bound
UPPERInterval upper bound
WIDTHInterval width
CLASSPredicted class
PROB_MAXLargest class probability
ENTROPYNormalised entropy
DIDissimilarity index
AOA_RASTERArea of applicability
MEAN_AOAPrediction inside the AOA
CLASS_AOAPredicted class inside the AOA
AOA_SHAREShare of cells inside the AOA
MANIFESTRun manifest
RMSERMSE
MAEMAE
R2R squared
BIASBias (mean error)
CCCLin's concordance correlation
ACCURACYAccuracy
BALANCED_ACCURACYBalanced accuracy
KAPPACohen's kappa
MORAN_IResidual Moran's I
COVERAGEInterval coverage

Area of applicability zero2spatialml:area_of_applicability

The dissimilarity index and the AOA on the covariate grid (section 11), with the permutation importance of a model under the design as weights, or equal weights. Also writes the training samples with their DI.

ParameterLabelTypeDefault
SAMPLESSamples (points or polygons) vector layer
TARGETTarget field field
RASTERSCovariate rasters layers
CATEGORICALCategorical covariates (comma-separated names) optionaltext
TASKTask advancedchoice: Detect from the target field; Regression; ClassificationDetect from the target field
POLYGON_STATPolygon samples: covariate value advancedchoice: Cell mean; Cell majority; Value at the centroidCell mean
POINT_BUFFERPoint samples: mean within this radius (m, 0 = cell value) advancednumber, at least 00
SCHEMEValidation design choice: Random k-fold; Spatial blocks; Hexagonal blocks; Coordinate clusters; Buffered leave-one-out; kNNDMSpatial blocks
KFolds k number, 2 to 1005
SIZEBlock size (m, 0 = variogram range) number, at least 00
RADIUSBuffered LOO radius (m, 0 = variogram range) advancednumber, at least 00
SEEDRandom seed advancednumber, at least 042
WEIGHTSCovariate weights choice: Permutation importance of the model; Equal weightsPermutation importance of the model
MODELModel choice: OLS baseline; Random forest; Gradient boosting; GWR; MGWRRandom forest
TUNETune hyperparameters by nested cross-validation yes / nono
ENGINERandom forest engine advancedchoice: Native (numpy); scikit-learn (when installed)Native (numpy)
MGWR_MIN_BWMGWR minimum bandwidth (neighbours, 0 = default 42) advancednumber, 0 to 100000
REPEATSPermutation repeats advancednumber, 1 to 10010
TILETile size (cells) advancednumber, 16 to 8192512
OUTPUT_FOLDEROutput folder folder output
TRAININGTraining samples with DI optionalvector output

Further outputs:

OutputMeaning
DIDissimilarity index
AOA_RASTERArea of applicability
THRESHOLDDI threshold
AOA_SHAREShare of cells inside the AOA

7 Record

Replay run manifest zero2spatialml:replay_manifest

Replays a run manifest (section 12): checks the hashes, reruns every recorded cross-validation and compares every metric.

ParameterLabelTypeDefault
MANIFESTRun manifest file
OUTPUTReplayed metrics optionalvector output

Further outputs:

OutputMeaning
OKReplay passed (1) or failed (0)
MISMATCHESMismatches
DETAILSMismatch details

14. Output layers, rasters and fields

OutputFields or bands
Sample tablefid, target, one field per covariate (raw values; categorical levels as their value)
Samples (CV) / cross-validated samplesthe sample table plus fold (buffered LOO: the row number; −1-1 never tested), cv_pred, residual (observed minus predicted), lower, upper; classes: cv_class, prob_max
Fold polygonsblock squares or hexagons, or cluster hulls, with fold
Distance curvesdistance (m), g_sample, g_prediction, g_cv
Variogramlag (m), gamma (semivariance), npairs, fitted (model value at the lag)
Leaderboard / scoresmodel, scheme, params (JSON of the hyperparameters) and one field per metric: rmse, mae, r2, bias, ccc, accuracy, balanced_accuracy, kappa, macro_f1, log_loss, brier, moran_i_residual, moran_p, coverage
Permutation importancecovariate, mean, std, rank
Effect curveskind (pdp or ale), feature, class, x, value
SHAP valuesfid, class, base, output, one phi_ field per covariate
Local coefficients (GWR, MGWR)fid, b_<term> per term (one-hot terms as c4_2), t_<term> when standard errors are available, and for GWR local_r2 and cond (condition number)
Prediction rastersmean.tif, lower.tif, upper.tif, width.tif or class.tif, prob_max.tif, entropy.tif; di.tif, aoa.tif, mean_aoa.tif or class_aoa.tif (the prediction masked to the AOA)
Prediction layerthe layer's own fields plus pred_mean, pred_lower, pred_upper, pred_width, pred_di, aoa; classes: pred_class, pred_prob_max, pred_entropy
Replayed metricsrun (model:design), metric, saved, replayed, equal

15. Worked example: the demo landscape

Load demo landscape on the Data page builds a deterministic synthetic study (seed 7) in EPSG:32635, with the origin at (500 000, 4 250 000) so that coordinates are far from the projection origin. The grid has 240×240240 \times 240 cells of 50 m (12 km by 12 km) and five covariate rasters:

The target has a planted spatially varying coefficient β1(u)=1+2u/L\beta_1(u) = 1 + 2u/L, a spatially correlated term εs\varepsilon_s (range 3 km, standard deviation 0.5) and noise ε∼N(0,0.32)\varepsilon \sim N(0, 0.3^2):

y=2+3sin⁡(πc2)+β1(u) c1+1.5 1[c4=2]+εs+εy = 2 + 3\sin(\pi c_2) + \beta_1(u)\,c_1 + 1.5\,\mathbf{1}[c_4 = 2] + \varepsilon_s + \varepsilon

The 400 samples lie in 8 clusters of radius 700 m spread over the western two thirds, so random k-fold tests a sample next to its own cluster while the map must predict up to several kilometres away.

Run with the default settings

Tick c4 as categorical and press Run whole study (defaults: spatial blocks, k=5k = 5, size from the variogram, random k-fold for comparison, OLS, random forest, gradient boosting and GWR with nested tuning, 90 % intervals, AOA). On a desktop i7-13700 under QGIS 3.44 the run takes about two minutes, most of it the nested tuning of the random forest. The numbers below are those of this run.

  1. Data. 400 samples kept, none dropped, 5 covariates, regression. The working CRS is EPSG:32635 (the CRS of the first raster, projected in metres); no harmonisation is needed.
  2. Explore. The variogram of the OLS residuals is spherical with nugget 0, sill 4.78 and practical range 599 m, which becomes the block size: 57 blocks of 599 m dealt to 5 folds.
  3. Validation. The median sample-to-sample distance is 85 m, while the median prediction-to-sample distance is 1 963 m (up to 7.1 km). WW is 2 112 m for random k-fold and 1 987 m for the 599 m blocks: blocks the size of the residual range barely move the CV distances towards the prediction distances. Larger designs do better on this sample layout: 2 km blocks give W=1 500W = 1\,500 m, coordinate clusters 820 m and kNNDM 856 m (values recorded in tests/fixtures/reference_v1/metrics.json). For a map of the whole landscape, kNNDM or coordinate clusters are the designs that mimic the prediction task.
  4. Models. The leaderboard (spatial blocks of 599 m, and random k-fold):
    ModelTuned parametersRMSE blocksRMSE randomGapR2R^2 blocksCCC blocks90 % coverage blocksMoran's II blocks
    Gradient boostingdepth 2, ν=0.1\nu = 0.10.7640.5880.1760.8930.9390.8980.32
    Random forestmtry=⌈p ⌉m_{\mathrm{try}} = \lceil\sqrt{p}\,\rceil, min leaf 11.0000.7170.2830.8160.8860.9050.35
    OLS-2.4152.1670.247−0.07-0.070.1150.8800.55
    GWRadaptive bisquare2.4392.0490.390−0.09-0.090.1630.9000.47
    Every model looks better under random k-fold (positive optimism gap). Gradient boosting is the best model. OLS and GWR cannot represent sin⁡(πc2)\sin(\pi c_2) and fall below the mean predictor (R2<0R^2 < 0) under spatial blocks. All residuals keep positive spatial autocorrelation (permutation p≤0.001p \le 0.001), the trace of εs\varepsilon_s. The interval coverage stays near the nominal 90 %.
  5. Explain. Permutation importance of gradient boosting (increase in held-out RMSE): c2 2.01, c1 0.48, c4 0.42, c3 0.28, c5 −0.001-0.001. The noise covariate gets no importance and the planted sin⁡(πc2)\sin(\pi c_2) term ranks first. TreeSHAP ran on all 400 samples with the first 100 of the 382 boosting rounds. The PDP of c2 follows the planted sine: maxima near c2=−1.5c_2 = -1.5 and c2=0.5c_2 = 0.5, minima near c2=−0.4c_2 = -0.4 and c2=1.4c_2 = 1.4. ALE is drawn for the continuous covariates; the categorical c4 has PDP and ICE only. The GWR local coefficient layer (bandwidth 214 neighbours, AICc 1 700.6) carries one b_ field per term. On this landscape c1 is itself a west-east trend, so its local coefficient cannot be separated from the local intercept and b_c1 does not trace β1(u)\beta_1(u); the recovery of a planted coefficient field by GWR is checked on a separate problem in the test suite.
  6. Predict. The mean, interval bounds and width, DI and AOA are written for all 57 600 cells. The DI threshold is 0.269; 54.8 % of the cells are inside the AOA. Every cell of the eastern third is outside, because c3 there lies beyond every training value and c3 carries importance weight; in the western two thirds 82 % of the cells are inside.
  7. Report. Write the model card and the manifest, then press Replay manifest: the table hash and every metric are reproduced exactly.

In Processing, the same study is one call of Spatial machine learning lab (whole study) with the demo samples, target y, the five rasters and c4 as categorical covariate.

16. Limitations

17. References

  1. Apley, D. W., & Zhu, J. (2020). Visualizing the effects of predictor variables in black box supervised learning models. Journal of the Royal Statistical Society B, 82(4), 1059-1086.
  2. Breiman, L. (2001). Random forests. Machine Learning, 45, 5-32.
  3. Brenning, A. (2012). Spatial cross-validation and bootstrap for the assessment of prediction rules in remote sensing: the R package sperrorest. IGARSS 2012, 5372-5375.
  4. Brunsdon, C., Fotheringham, A. S., & Charlton, M. E. (1996). Geographically weighted regression: a method for exploring spatial nonstationarity. Geographical Analysis, 28(4), 281-298.
  5. Cressie, N. (1985). Fitting variogram models by weighted least squares. Mathematical Geology, 17(5), 563-586.
  6. da Silva, A. R., & Fotheringham, A. S. (2016). The multiple testing issue in geographically weighted regression. Geographical Analysis, 48(3), 233-247.
  7. Fisher, A., Rudin, C., & Dominici, F. (2019). All models are wrong, but many are useful. Journal of Machine Learning Research, 20(177), 1-81.
  8. Fisher, W. D. (1958). On grouping for maximum homogeneity. Journal of the American Statistical Association, 53(284), 789-798.
  9. Fotheringham, A. S., Brunsdon, C., & Charlton, M. (2002). Geographically Weighted Regression. Wiley.
  10. Fotheringham, A. S., Yang, W., & Kang, W. (2017). Multiscale geographically weighted regression (MGWR). Annals of the American Association of Geographers, 107(6), 1247-1265.
  11. Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of Statistics, 29(5), 1189-1232.
  12. Friedman, J. H. (2002). Stochastic gradient boosting. Computational Statistics & Data Analysis, 38(4), 367-378.
  13. GDAL/OGR contributors (2025). GDAL/OGR Geospatial Data Abstraction software Library. Open Source Geospatial Foundation. https://gdal.org
  14. Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015). Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation. Journal of Computational and Graphical Statistics, 24(1), 44-65.
  15. Hurvich, C. M., Simonoff, J. S., & Tsai, C.-L. (1998). Smoothing parameter selection in nonparametric regression using an improved Akaike information criterion. Journal of the Royal Statistical Society B, 60(2), 271-293.
  16. Ke, G., et al. (2017). LightGBM: a highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems 30.
  17. Le Rest, K., Pinaud, D., Monestiez, P., Chadoeuf, J., & Bretagnolle, V. (2014). Spatial leave-one-out cross-validation for variable selection in the presence of spatial autocorrelation. Global Ecology and Biogeography, 23(7), 811-820.
  18. Lei, J., G'Sell, M., Rinaldo, A., Tibshirani, R. J., & Wasserman, L. (2018). Distribution-free predictive inference for regression. Journal of the American Statistical Association, 113(523), 1094-1111.
  19. Lin, L. I.-K. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45(1), 255-268.
  20. Linnenbrink, J., Milà, C., Ludwig, M., & Meyer, H. (2024). kNNDM CV: k-fold nearest-neighbour distance matching cross-validation for map accuracy estimation. Geoscientific Model Development, 17, 5897-5912.
  21. Lundberg, S. M., & Lee, S.-I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems 30.
  22. Lundberg, S. M., Erion, G., Chen, H., et al. (2020). From local explanations to global understanding with explainable AI for trees. Nature Machine Intelligence, 2, 56-67.
  23. Matheron, G. (1963). Principles of geostatistics. Economic Geology, 58(8), 1246-1266.
  24. Meinshausen, N. (2006). Quantile regression forests. Journal of Machine Learning Research, 7, 983-999.
  25. Meyer, H., & Pebesma, E. (2021). Predicting into unknown space? Estimating the area of applicability of spatial prediction models. Methods in Ecology and Evolution, 12(9), 1620-1633.
  26. Meyer, H., & Pebesma, E. (2022). Machine learning-based global maps of ecological variables and the challenge of assessing them. Nature Communications, 13, 2208.
  27. Milà, C., Mateu, J., Pebesma, E., & Meyer, H. (2022). Nearest neighbour distance matching leave-one-out cross-validation for map validation. Methods in Ecology and Evolution, 13(6), 1304-1316.
  28. Mitchell, M., Wu, S., Zaldivar, A., et al. (2019). Model cards for model reporting. Proceedings of FAT* 2019, 220-229.
  29. Moran, P. A. P. (1950). Notes on continuous stochastic phenomena. Biometrika, 37(1/2), 17-23.
  30. Oshan, T. M., Li, Z., Kang, W., Wolf, L. J., & Fotheringham, A. S. (2019). mgwr: a Python implementation of multiscale geographically weighted regression. ISPRS International Journal of Geo-Information, 8(6), 269.
  31. Ploton, P., Mortier, F., Réjou-Méchain, M., et al. (2020). Spatial validation reveals poor predictive performance of large-scale ecological mapping models. Nature Communications, 11, 4540.
  32. Pohjankukka, J., Pahikkala, T., Nevalainen, P., & Heikkonen, J. (2017). Estimating the prediction performance of spatial models via spatial k-fold cross validation. International Journal of Geographical Information Science, 31(10), 2001-2019.
  33. QGIS Development Team (2026). QGIS Geographic Information System. Open Source Geospatial Foundation Project. https://qgis.org
  34. Roberts, D. R., Bahn, V., Ciuti, S., et al. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography, 40(8), 913-929.
  35. Valavi, R., Elith, J., Lahoz-Monfort, J. J., & Guillera-Arroita, G. (2019). blockCV: an R package for generating spatially or environmentally separated folds for k-fold cross-validation of species distribution models. Methods in Ecology and Evolution, 10(2), 225-232.
  36. Wadoux, A. M. J.-C., Heuvelink, G. B. M., de Bruin, S., & Brus, D. J. (2021). Spatial cross-validation is not the right way to evaluate map accuracy. Ecological Modelling, 457, 109692.
  37. Yu, H., Fotheringham, A. S., Li, Z., Oshan, T., Kang, W., & Wolf, L. J. (2020). Inference in multiscale geographically weighted regression. Geographical Analysis, 52(1), 87-106.

18. Citation, licence, links

Eminoğlu, Y. (2026). 02SpatialML - Spatial Machine Learning Lab (version 0.1.1) [QGIS plugin]. https://gitlab.com/geophilo1/zero2spatialml. https://doi.org/10.5281/zenodo.24263652. The DOI 10.5281/zenodo.24263652 always resolves to the latest version; version 0.1.1 has DOI https://doi.org/10.5281/zenodo.24263653. Please also cite the references of the designs, models and explanations you use.

Licence: GPL-2.0-or-later. Copyright (C) 2026 Yusuf Eminoğlu (Geophilo). KaTeX (bundled in resources/katex) is MIT licensed.

Repository: gitlab.com/geophilo1/zero2spatialml · Issues: work items · Homepage: geophilo.com/zero2spatialml