Revised documentation system (#631)

* include md instead of Rmd

* don't commit engine md files

* remove extra files

* additional templates

* move some argument code to a function

* move legacy code to generate engine doc lists and seealso

* rename file

* refactor code since all files reside within parsnip

* better printing of the engine list

* fix bug in param object call

* avoid deprecation warning

* added missing mode

* add files for SDA model

* new multilevelmod engine docs

* prototypes for engine extension package notes

* some brulee files

* add a file with engine/model information

* keep extension package name

* use superscripted numbers for default engine and extensions

* Refactor the engine and mode information in the model files

* remove files no longer used

* temp remove check for extenstion pkgs

* move setup.Rmd to aaa.Rmd

* remove unused engine doc functions

* added a note with PR number

* more info on models with extension packages

* add a link to the "find" page for models

* updated model database

* fix bug related to engine names with underscores

* brulee engine files

* combined replicate files

* Wrote a readme document and added notes about it in engine files

* more code cleanup

* fix some extension package doc issues

* add a readme for the documentation system

* change path to readme doc

* add two functions back

* also merge model info by mode.

* mode-specific dependencies for rpart

* avoid failing for "Packages unavailable to check Rd xrefs"

* Edits to README-DOCS, mainly so I think I will be able to come back to this and understand it

* Apply suggestions from code review

Co-authored-by: Julia Silge <julia.silge@gmail.com>
Co-authored-by: Hannah Frick <hfrick@users.noreply.github.com>

* add install_engine_packages

* add list_md_problems() to find errors and warnings in engine specific docs

* document list_md_problems()

* add a few more packages

* add issue summary post-knit

* doc update

* update model about model = TRUE for survival

* list to table

* notes about mode

* remove "dynamically"

* more error trapping during knit

* changes for new brulee version

* Rework explanation of what is/isn't in our tab-delimited file

* Update R/rand_forest.R

* Update R/rule_fit.R

* Update R/svm_linear.R

* Update R/survival_reg.R

* Remove details on glmnet not using the formula interface

* Redocument

* doc update

* retain md files

* rm randomForest. See tidymodels/extratests/41

* temporarily disable windows testing (tensorflow install)

Co-authored-by: Julia Silge <julia.silge@gmail.com>
Co-authored-by: Hannah Frick <hfrick@users.noreply.github.com>
Co-authored-by: Emil Hvitfeldt <emilhhvitfeldt@gmail.com>

Author: 4 people

.github/workflows/R-CMD-check.yaml

@@ -23,7 +23,8 @@ jobs:
       matrix:
         config:
           - {os: macOS-latest,   r: 'release'}
-          - {os: windows-latest, r: 'release'}
+          # disable until tensorflow install is worked out
+          # - {os: windows-latest, r: 'release'}
           # Use older ubuntu to maximise backward compatibility
           - {os: ubuntu-18.04,   r: 'devel', http-user-agent: 'release'}
           - {os: ubuntu-18.04,   r: 'release'}

DESCRIPTION

@@ -18,6 +18,7 @@ BugReports: https://github.com/tidymodels/parsnip/issues
 Depends:
     R (>= 2.10)
 Imports: 
+    cli,
     dplyr (>= 0.8.0.1),
     generics (>= 0.1.0.9000),
     globals,
@@ -50,8 +51,8 @@ Suggests:
     mgcv,
     modeldata,
     nlme,
-    randomForest,
     ranger (>= 0.12.0),
+    remotes,
     rmarkdown,
     rpart,
     sparklyr (>= 1.0.0),
@@ -80,6 +81,7 @@ Config/Needs/website:
     tidymodels/tidymodels, 
     tidyverse/tidytemplate, 
     xgboost
+Config/rcmdcheck/ignore-inconsequential-notes: true
 Encoding: UTF-8
 LazyData: true
 Roxygen: list(markdown = TRUE)

NAMESPACE

@@ -206,7 +206,9 @@ export(glance)
 export(has_multi_predict)
 export(is_varying)
 export(keras_mlp)
+export(knit_engine_docs)
 export(linear_reg)
+export(list_md_problems)
 export(logistic_reg)
 export(make_call)
 export(make_classes)
@@ -286,6 +288,7 @@ export(tune)
 export(update_dot_check)
 export(update_engine_parameters)
 export(update_main_parameters)
+export(update_model_info_file)
 export(varying)
 export(varying_args)
 export(xgb_train)

R/C5_rules_C5.0.R

@@ -0,0 +1,13 @@
+#' C5.0 rule-based classification models
+#'
+#' [C50::C5.0()] fits a model that derives feature rules from a tree for
+#' prediction. A single tree or boosted ensemble can be used. [rules::c5_fit()]
+#' is a wrapper around this function.
+#'
+#' @includeRmd man/rmd/C5_rules_C5.0.md details
+#'
+#' @name details_C5_rules_C5.0
+#' @keywords internal
+NULL
+
+# See inst/README-DOCS.md for a description of how these files are processed

R/aaa.R

@@ -91,7 +91,7 @@ utils::globalVariables(
     "max_terms", "max_tree", "model", "name", "num_terms", "penalty", "trees",
     "sub_neighbors", ".pred_class", "x", "y", "predictor_indicators",
     "compute_intercept", "remove_intercept", "estimate", "term",
-    "call_info", "component", "component_id", "func", "tunable")
+    "call_info", "component", "component_id", "func", "pkg", ".order", "item", "tunable")
 )
 
 # nocov end

R/aaa_models.R

@@ -1081,218 +1081,3 @@ get_encoding <- function(model) {
   }
   res
 }
-
-#' Tools for dynamically documenting packages
-#'
-#' @description
-#' These are functions used to create dynamic documentation in Rd files
-#' based on which parsnip-related packages are loaded by the user.
-#'
-#' These functions can be used to make dynamic lists of documentation help
-#'  files. \pkg{parsnip} uses these along with files in `man/rmd` which
-#'  contain expanded documentation for specific model/engine combinations.
-#'  [find_engine_files()] looks for files that have the pattern
-#'  `details_{model}_{engine}.Rd` to link to. These files are generated by files
-#'  named `man/rmd/{model}_{engine}.Rmd`. `make_engine_list()` creates a
-#'  list seen at the top of the model Rd files while `make_seealso_list()`
-#'  populates the list seen in "See Also" below. See the details section.
-#'
-#' @param mod A character string for the model file (e.g. "linear_reg")
-#' @param pkg A character string for the package where the function is invoked.
-#' @return
-#' `make_engine_list()` returns a character string that creates a
-#' bulleted list of links to more specific help files.
-#'
-#' `make_seealso_list()` returns a formatted character string of links.
-#'
-#' `find_engine_files()` returns a tibble.
-#' @details
-#' The \pkg{parsnip} documentation is generated _dynamically_. Part of the Rd
-#'  file populates a list of engines that depends on what packages are loaded
-#'  *at the time that the man file is loaded*. For example, if
-#'  another package has a new engine for `linear_reg()`, the
-#'  `parsnip::linear_reg()` help can show a link to a detailed help page in the
-#'  other package.
-#'
-#' To enable this, the process for a package developer is to:
-#'
-#'   1. Create an engine-specific R file in the `R` directory with the name
-#'  `{model}_{engine}.R` (e.g. `boost_tree_C5.0.R`). This has a small amount of
-#'  documentation, as well as the directives "`@name details_{model}_{engine}`"
-#'  and "`@includeRmd man/rmd/{model}_{engine}.Rmd details`".
-#'
-#'   2. Copy the file in \pkg{parsnip} that is in `man/rmd/setup.Rmd` and put
-#'  it in the same place in your package.
-#'
-#'   3. Write your own `man/rmd/{model}_{engine}.Rmd` file. This can include
-#'  packages that are not listed in the DESCRIPTION file. Those are only
-#'  required when the documentation file is created locally (probably using
-#'  [devtools::document()]).
-#'
-#'   4. Run [devtools::document()] so that the Rmd content is included in the
-#'  Rd file.
-#'
-#' The examples in \pkg{parsnip} can provide guidance for how to organize
-#' technical information about the models.
-#' @name doc-tools
-#' @keywords internal
-#' @export
-#' @examples
-#' find_engine_files("linear_reg")
-#' cat(make_engine_list("linear_reg"))
-find_engine_files <- function(mod, pkg = "parsnip") {
-
-  requireNamespace(pkg, quietly = TRUE)
-  # Get available topics
-  topic_names <- search_for_engine_docs(mod)
-  if (length(topic_names) == 0) {
-    return(character(0))
-  }
-
-  # Subset for our model function
-  eng <- strsplit(topic_names, "_")
-  eng <- purrr::map_chr(eng, ~ .x[length(.x)])
-  eng <- tibble::tibble(engine = eng, topic = topic_names)
-
-  # Combine them to keep the order in which they were registered
-  all_eng <- get_from_env(mod) %>% dplyr::distinct(engine)
-  all_eng$.order <- 1:nrow(all_eng)
-  eng <- dplyr::left_join(eng, all_eng, by = "engine")
-  eng <- eng[order(eng$.order),]
-
-  # Determine and label default engine
-  default <- get_default_engine(mod, pkg)
-  eng$default <- ifelse(eng$engine == default, " (default)", "")
-
-  eng
-}
-
-#' @export
-#' @rdname doc-tools
-make_engine_list <- function(mod, pkg = "parsnip") {
-  eng <- find_engine_files(mod, pkg)
-
-  if (length(eng) == 0) {
-    return("No engines were found within the currently loaded packages.\n\n")
-  } else {
-    main <- paste("The engine-specific pages for this model are listed ",
-                  "below and contain the details:\n\n")
-  }
-
-  res <-
-    glue::glue("  \\item \\code{\\link[|eng$topic|]{|eng$engine|} |eng$default| }",
-               .open = "|", .close = "|")
-
-  res <- paste0(main, "\\itemize{\n", paste0(res, collapse = "\n"), "\n}")
-  res
-}
-
-get_default_engine <- function(mod, pkg= "parsnip") {
-  cl <- rlang::call2(mod, .ns = pkg)
-  rlang::eval_tidy(cl)$engine
-}
-
-#' @export
-#' @rdname  doc-tools
-make_seealso_list <- function(mod, pkg= "parsnip") {
-  requireNamespace(pkg, quietly = TRUE)
-  eng <- find_engine_files(mod, pkg)
-
-  main <- c("\\code{\\link[=fit.model_spec]{fit()}}",
-            "\\code{\\link[=set_engine]{set_engine()}}",
-            "\\code{\\link[=update]{update()}}")
-
-  if (length(eng) == 0) {
-    return(paste0(main, collapse = ", "))
-  }
-
-  res <-
-    glue::glue("\\code{\\link[|eng$topic|]{|eng$engine| engine details}}",
-               .open = "|", .close = "|")
-
-  if (pkg != "parsnip") {
-    main <- NULL
-  }
-  paste0(c(main, res), collapse = ", ")
-}
-
-# These will never have documentation and we can avoid searching them.
-excl_pkgs <-
-  c("C50", "Cubist", "earth", "flexsurv", "forecast", "glmnet",
-    "keras", "kernlab", "kknn", "klaR", "LiblineaR", "liquidSVM",
-    "magrittr", "MASS", "mda", "mixOmics", "naivebayes", "nnet",
-    "prophet", "pscl", "randomForest", "ranger", "rpart", "rstanarm",
-    "sparklyr", "stats", "survival", "xgboost", "xrf")
-
-search_for_engine_docs <- function(mod) {
-  all_deps <- get_from_env(paste0(mod, "_pkgs"))
-  all_deps <- unlist(all_deps$pkg)
-  all_deps <- unique(c("parsnip", all_deps))
-
-  all_deps <- all_deps[!(all_deps %in% excl_pkgs)]
-  res <- purrr::map(all_deps, find_details_topics, mod = mod)
-  res <- unique(unlist(res))
-  res
-}
-
-find_details_topics <- function(pkg, mod) {
-  meta_loc <- system.file("Meta/Rd.rds", package = pkg)
-  meta_loc <- meta_loc[meta_loc != ""]
-  if (length(meta_loc) > 0) {
-    topic_names <- readRDS(meta_loc)$Name
-    res <- grep(paste0("details_", mod), topic_names, value = TRUE)
-    if (length(res) > 0) {
-      res <- paste0(pkg, ":", res)
-    }
-  } else {
-    res <- character(0)
-  }
-  res
-}
-
-
-# For use in `set_engine()` docs
-generate_set_engine_bullets <- function() {
-  env <- get_model_env()
-  models <- env$models
-  info <- rlang::env_get_list(env, models)
-
-  model_engines <- purrr::map(info, get_sorted_unique_engines)
-
-  model_prefixes <- glue::glue(
-    "\\code{\\link[=.{models}.]{.{models}.()}}:",
-    .open = ".{",
-    .close = "}."
-  )
-
-  bullets <- purrr::map2(
-    .x = model_prefixes,
-    .y = model_engines,
-    .f = combine_prefix_with_engines
-  )
-
-  bullets <- glue::glue("\\item {bullets}")
-  bullets <- glue::glue_collapse(bullets, sep = "\n")
-  bullets <- paste("\\itemize{", bullets, "}", sep = "\n")
-
-  bullets
-}
-
-sort_c <- function(x) {
-  withr::with_collate("C", sort(x))
-}
-get_sorted_unique_engines <- function(x) {
-  engines <- x$engine
-  engines <- unique(engines)
-  engines <- sort_c(engines)
-  engines
-}
-combine_prefix_with_engines <- function(prefix, engines) {
-  if (length(engines) == 0L) {
-    engines <- "No engines currently available"
-  } else {
-    engines <- glue::glue_collapse(engines, sep = ", ")
-  }
-
-  glue::glue("{prefix} {engines}")
-}

R/bag_mars.R

@@ -5,10 +5,7 @@
 #' `bag_mars()` defines an ensemble of generalized linear models that use
 #' artificial features for some predictors. These features resemble hinge
 #' functions and the result is a model that is a segmented regression in small
-#' dimensions.
-#'
-#' There are different ways to fit this model. The method of estimation is
-#' chosen by setting the model _engine_.
+#' dimensions. This function can fit classification and regression models.
 #'
 #' \Sexpr[stage=render,results=rd]{parsnip:::make_engine_list("bag_mars")}
 #'

R/bag_mars_earth.R

@@ -0,0 +1,12 @@
+#' Bagged MARS via earth
+#'
+#' [baguette::bagger()] creates an collection of MARS models forming an
+#' ensemble. All models in the ensemble are combined to produce a final prediction.
+#'
+#' @includeRmd man/rmd/bag_mars_earth.md details
+#'
+#' @name details_bag_mars_earth
+#' @keywords internal
+NULL
+
+# See inst/README-DOCS.md for a description of how these files are processed

R/bag_tree.R

@@ -2,10 +2,8 @@
 #'
 #' @description
 #'
-#' `bag_tree()` defines an ensemble of decision trees.
-#'
-#' There are different ways to fit this model. The method of estimation is
-#' chosen by setting the model _engine_.
+#' `bag_tree()` defines an ensemble of decision trees. This function can fit
+#'  classification, regression, and censored regression models.
 #'
 #' \Sexpr[stage=render,results=rd]{parsnip:::make_engine_list("bag_tree")}
 #'

R/bag_tree_C5.0.R

@@ -0,0 +1,12 @@
+#' Bagged trees via C5.0
+#'
+#' [baguette::bagger()] creates an collection of decision trees forming an
+#' ensemble. All trees in the ensemble are combined to produce a final prediction.
+#'
+#' @includeRmd man/rmd/bag_tree_C5.0.md details
+#'
+#' @name details_bag_tree_C5.0
+#' @keywords internal
+NULL
+
+# See inst/README-DOCS.md for a description of how these files are processed