Date accepted: 2023-02-27
Submitting Author Name: Mike Mahoney
Submitting Author Github Handle: @mikemahoney218
Repository: https://github.com/mikemahoney218/waywiser
Version submitted:
Submission type: Stats
Badge grade: silver
Editor: @Paula-Moraga
Reviewers: @becarioprecario, @jakub_nowosad, @Nowosad

Due date for @jakub_nowosad: 2023-02-06
Due date for @Nowosad: 2023-02-06
Archive: TBD
Version accepted: TBD
Language: en

• Paste the full DESCRIPTION file inside a code block below:

Type: Package
Package: waywiser
Title: Ergonomic Methods for Assessing Spatial Models
Version: 0.2.0.9000
Authors@R: c(
    person("Michael", "Mahoney", , "mike.mahoney.218@gmail.com", role = c("aut", "cre"),
           comment = c(ORCID = "0000-0003-2402-304X")),
    person("Lucas", "Johnson", , "lucas.k.johnson03@gmail.com", role = c("ctb"),
           comment = c(ORCID = "0000-0002-7953-0260")),
    person("RStudio", role = c("cph", "fnd"))
  )
Description: Assessing predictive models of spatial data can be challenging, 
    both because these models are typically built for extrapolating outside the
    original region represented by training data and due to potential spatially
    structured errors, with "hot spots" of higher than expected error
    clustered geographically due to spatial structure in the underlying
    data. Methods are provided for assessing models fit to spatial data, 
    including approaches for measuring the spatial structure of model errors,
    assessing model predictions at multiple spatial scales, and evaluating where 
    predictions can be made safely. Methods are particularly useful for models 
    fit using the 'tidymodels' framework. Methods include Moran's I
    ('Moran' (1950) <doi:10.2307/2332142>), Geary's C 
    ('Geary' (1954) <doi:10.2307/2986645>), Getis-Ord's G
    ('Ord' and 'Getis' (1995) <doi:10.1111/j.1538-4632.1995.tb00912.x>),
    agreement coefficients from 'Ji' and Gallo (2006) 
    (<doi: 10.14358/PERS.72.7.823>), agreement metrics from 'Willmott' (1981)
    (<doi: 10.1080/02723646.1981.10642213>) and 'Willmott' 'et' 'al'. (2012)
    (<doi: 10.1002/joc.2419>), an implementation of the area of applicability 
    methodology from 'Meyer' and 'Pebesma' (2021) 
    (<doi:10.1111/2041-210X.13650>), and an implementation of
    multi-scale assessment as described in 'Riemann' 'et' 'al'. (2010)
    (<doi:10.1016/j.rse.2010.05.010>).
License: MIT + file LICENSE
URL: https://github.com/mikemahoney218/waywiser,
    https://mikemahoney218.github.io/waywiser/
BugReports: https://github.com/mikemahoney218/waywiser/issues
Depends: 
    R (>= 3.6)
Imports: 
    dplyr,
    fields,
    FNN,
    glue,
    hardhat,
    Matrix,
    purrr,
    rlang,
    rsample,
    sf (>= 1.0-0),
    spdep (>= 1.1-9),
    stats,
    tibble,
    tidyselect,
    yardstick
Suggests: 
    applicable,
    caret,
    CAST,
    covr,
    ggplot2,
    knitr,
    modeldata,
    recipes,
    rmarkdown,
    spatialsample,
    spelling,
    testthat (>= 3.0.0),
    tidymodels,
    tidyr,
    tigris,
    vip,
    whisker,
    withr
Config/testthat/edition: 3
Config/testthat/parallel: true
Encoding: UTF-8
LazyData: true
Roxygen: list(markdown = TRUE, roclets = c("namespace", "rd", "srr::srr_stats_roclet"))
RoxygenNote: 7.2.3
Language: en-US
VignetteBuilder: knitr

Scope

• Please indicate which of our statistical package categories this package falls under. (Please check one appropriate box below):

  Statistical Packages

  • Bayesian and Monte Carlo Routines
  • Dimensionality Reduction, Clustering, and Unsupervised Learning
  • Machine Learning
  • Regression and Supervised Learning
  • Exploratory Data Analysis (EDA) and Summary Statistics
  • Spatial Analyses
  • Time Series Analyses

Pre-submission Inquiry

• A pre-submission inquiry has been approved in issue#565

General Information

• Who is the target audience and what are scientific applications of this package?

Anyone fitting models to spatial data, particularly (but not exclusively) people working within the tidymodels ecosystem. This includes a number of domains, and we've already been using it in our modeling practice.

• Paste your responses to our General Standard G1.1 here, describing whether your software is:

  • The first implementation of a novel algorithm; or
  • The first implementation within R of an algorithm which has previously been implemented in other languages or contexts; or
  • An improvement on other implementations of similar algorithms in R.

  Please include hyperlinked references to all other relevant software.

The waywiser R package makes it easier to measure the performance of models fit to 2D spatial data by implementing a number of well-established assessment methods in a consistent, ergonomic toolbox; features include new yardstick metrics for measuring agreement and spatial autocorrelation, functions to assess model predictions across multiple scales, and methods to calculate the area of applicability of a model.

Relevant software implementing similar algorithms include CAST for ww_area_of_applicability(). Several yardstick metrics implemented directly wrap spdep in a more consistent interface. Willmott's D is also implemented in hydroGOF. Other functions have (as far as I am aware) not been implemented elsewhere, such as ww_multi_scale() which implements the procedure from Riemann et al 2010, or ww_agreement_coefficient() which implements metrics from Ji and Gallo 2006.

• (If applicable) Does your package comply with our guidance around Ethics, Data Privacy and Human Subjects Research?

N/A

Badging

• What grade of badge are you aiming for? (bronze, silver, gold)

Silver

• If aiming for silver or gold, describe which of the four aspects listed in the Guide for Authors chapter the package fulfils (at least one aspect for silver; three for gold)

Have a demonstrated generality of usage beyond one single envisioned use case. Software is frequently developed for one particular use case envisioned by the authors themselves. Generalising the utility of software so that it is readily applicable to other use cases, and satisfactorily documenting such generality of usage, represents another aspect which may be considered sufficient for software to attain a silver grade.

This is the primary aspect which I believe merits the silver status. The waywiser package implements routines which are useful for a wide variety of spatial models and integrates well with the tidymodels ecosystem, making it (hopefully!) of interdisciplinary interest.

Depending on what the editors think, I'd also potentially submit this for gold, based upon the following two aspects:

Compliance with a good number of standards beyond those identified as minimally necessary. This will require reviewers and authors to agree on identification of both a minimal subset of necessary standards, and a full set of potentially applicable standards. This aspect may be considered fulfilled if at least one quarter of the additional potentially applicable standards have been met, and should definitely be considered fulfilled if more than one half have been met.

Internal aspects of package structure and design. Many aspects of the internal structure and design of software are too variable to be effectively addressed by standards. Packages which are judged by reviewers to reflect notably excellent design choices, especially in the implementation of core statistical algorithms, may also be considered worthy of a silver grade.

But I'm not familiar enough with the system to know if waywiser is likely to be in compliance with these two aspects, and am comfortable submitting for "silver" status if waywiser does not obviously meet both.

Technical checks

Confirm each of the following by checking the box.

• I have read the rOpenSci packaging guide.
• I have read the author guide and I expect to maintain this package for at least 2 years or have another maintainer identified.
• I/we have read the Statistical Software Peer Review Guide for Authors.
• I/we have run autotest checks on the package, and ensured no tests fail. (Sorry, both the release and CRAN versions of autotest fail immediately on my machine with internal errors -- that is, from autotest itself and not from my package -- and therefore I have not been able to use it).
• The srr_stats_pre_submit() function confirms this package may be submitted.
• The pkgcheck() function confirms this package may be submitted - alternatively, please explain reasons for any checks which your package is unable to pass.

This package:

• does not violate the Terms of Service of any service it interacts with.
• has a CRAN and OSI accepted license.
• contains a README with instructions for installing the development version.

Publication options

• Do you intend for this package to go on CRAN?
• Do you intend for this package to go on Bioconductor?

Code of conduct

• I agree to abide by rOpenSci's Code of Conduct during the review process and in maintaining my package should it be accepted.