readme.md

R Package Structure

Happy Boxing Day! While you're dealing with your physical packages, we're looking into R packages!

The dataset this week comes from "Historical Trends in R Package Structure and Interdependency on CRAN" by Mark Padgham and Noam Ross. In that paper, they use the {pkgstats} R package to analyze the structure of R packages over time, using an archive of all packages on CRAN as of 2022-11-22. We've provided csv versions of two of the datasets from that paper.

The paper focuses on package characteristics over time. It might be interesting to look at the distribution of similar features (such as lines of code) across packages.

If you're unfamiliar with some of the terminology in this dataset, you might find the R Packages book by Hadley Wickham and Jennifer Bryan helpful.

The Data

# Option 1: tidytuesdayR package 
## install.packages("tidytuesdayR")

tuesdata <- tidytuesdayR::tt_load('2023-12-26')
## OR
tuesdata <- tidytuesdayR::tt_load(2023, week = 52)

cran_20221122 <- tuesdata$cran_20221122
external_calls <- tuesdata$external_calls
internal_calls <- tuesdata$internal_calls

# Option 2: Read directly from GitHub

cran_20221122 <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/main/data/2023/2023-12-26/cran_20221122.csv')
external_calls <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/main/data/2023/2023-12-26/external_calls.csv')
internal_calls <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/main/data/2023/2023-12-26/internal_calls.csv')

If you would like to dive deeper, you can download the larger dataset with this code:

cran_all_20221122 <- readr::read_rds("https://zenodo.org/records/7414296/files/pkgstats-CRAN-all.Rds?download=1")

How to Participate

• Explore the data, watching out for interesting relationships. We would like to emphasize that you should not draw conclusions about causation in the data. There are various moderating variables that affect all data, many of which might not have been captured in these datasets. As such, our suggestion is to use the data provided to practice your data tidying and plotting techniques, and to consider for yourself what nuances might underlie these relationships.
• Create a visualization, a model, a shiny app, or some other piece of data-science-related output, using R or another programming language.
• Share your output and the code used to generate it on social media with the #TidyTuesday hashtag.

Data Dictionary

cran_20221122.csv

variable	class	description
package	character	The name of the package
version	character	The package version
date	double	The release date of that version of the package
license	character	License information
files_R	double	Number of files in the /R directory, where numbers are recursively counted in all sub-directories
files_src	double	Number of files in the /src directory, where numbers are recursively counted in all sub-directories
files_inst	double	Number of files in the /inst/include directory, where numbers are recursively counted in all sub-directories
files_vignettes	double	Number of files in the /vignettes directory, where numbers are recursively counted in all sub-directories
files_tests	double	Number of files in the /tests directory, where numbers are recursively counted in all sub-directories
loc_R	double	Total lines of code across all files in the /R directory
loc_src	double	Total lines of code across all files in the /src directory
loc_inst	double	Total lines of code across all files in the /inst/include directory
loc_vignettes	double	Total lines of code across all files in the /vignettes directory
loc_tests	double	Total lines of code across all files in the /tests directory
blank_lines_R	double	Total numbers of blank lines across all files in the /R directory
blank_lines_src	double	Total numbers of blank lines across all files in the /src directory
blank_lines_inst	double	Total numbers of blank lines across all files in the /inst directory
blank_lines_vignettes	double	Total numbers of blank lines across all files in the /vignettes directory
blank_lines_tests	double	Total numbers of blank lines across all files in the /tests directory
comment_lines_R	double	Total numbers of comment lines across all files in the /R directory
comment_lines_src	double	Total numbers of comment lines across all files in the /src directory
comment_lines_inst	double	Total numbers of comment lines across all files in the /inst directory
comment_lines_vignettes	double	Total numbers of comment lines across all files in the /vignettes directory
comment_lines_tests	double	Total numbers of comment lines across all files in the /tests directory
rel_space	double	Measure of relative white space across all files in the /R, /src, and /inst directories
rel_space_R	double	Measure of relative white space across all files in the /R directory
rel_space_src	double	Measure of relative white space across all files in the /src directory
rel_space_inst	double	Measure of relative white space across all files in the /inst directory
rel_space_vignettes	double	Measure of relative white space across all files in the /vignettes directory
rel_space_tests	double	Measure of relative white space across all files in the /tests directory
indentation	double	The number of spaces used to indent code, with values of -1 indicating indentation with tab characters
nexpr	double	The median number of nested expression per line of code, counting only those lines which have any expressions
num_vignettes	double	Number of vignettes
num_demos	double	Number of demos
num_data_files	double	Number of data files
data_size_total	double	Total size of all package data
data_size_median	double	Median size of package data files
translations	character	List of translations where package includes translations files, given as a comma-separated list of (spoken) language codes
urls	character	Package URL(s)
bugs	character	URL for BugReports
desc_n_aut	double	Number of contributors with role of author
desc_n_ctb	double	Number of contributors with role of contributor
desc_n_fnd	double	Number of contributors with role of funder
desc_n_rev	double	Number of contributors with role of reviewer
desc_n_ths	double	Number of contributors with role of thesis advisor
desc_n_trl	double	Number of contributors with role of translator (relating to translation between computer and not spoken languages)
depends	character	Comma-separated character entries for all depends packages
imports	character	Comma-separated character entries for all imports packages
suggests	character	Comma-separated character entries for all suggests packages
enhances	character	Comma-separated character entries for all enhances packages
linking_to	character	Comma-separated character entries for all linking_to packages
n_fns_r	double	Numbers of functions in R
n_fns_r_exported	double	Numbers of exported R functions
n_fns_r_not_exported	double	Numbers of non-exported R functions
n_fns_src	double	Number of functions (or objects) in other computer languages, including functions in both src and inst/include directories
n_fns_per_file_r	double	Number of functions (or objects) per individual file in /R
n_fns_per_file_src	double	Number of functions (or objects) per individual file in source directories other than /R
npars_exported_mn	double	Mean number of parameters per exported R function
npars_exported_md	double	Median number of parameters per exported R function
loc_per_fn_r_mn	double	Mean lines of code per function in /R
loc_per_fn_r_md	double	Median lines of code per function in /R
loc_per_fn_r_exp_mn	double	Mean lines of code per exported function in /R
loc_per_fn_r_exp_md	double	Median lines of code per exported function in /R
loc_per_fn_r_not_exp_mn	double	Mean lines of code per non-exported function in /R
loc_per_fn_r_not_exp_md	double	Median lines of code per non-exported function in /R
loc_per_fn_src_mn	double	Mean lines of code per in other source directories
loc_per_fn_src_md	double	Median lines of code per in other source directories
languages	character	languages
doclines_per_fn_exp_mn	double	Mean lines of documentation per exported function in /R
doclines_per_fn_exp_md	double	Median lines of documentation per exported function in /R
doclines_per_fn_not_exp_mn	double	Mean lines of documentation per non-exported function in /R
doclines_per_fn_not_exp_md	double	Median lines of documentation per non-exported function in /R
doclines_per_fn_src_mn	double	Mean lines of code per in other source directories
doclines_per_fn_src_md	double	Median lines of code per in other source directories
docchars_per_par_exp_mn	double	Mean number of documentation characters per parameter of exported R functions
docchars_per_par_exp_md	double	Median number of documentation characters per parameter of exported R functions
n_edges	double	Number of edges connecting functions (and other objects) across all languages in package code
n_edges_r	double	Number of edges connecting R functions (and other objects)
n_edges_src	double	Number of edges connecting functions (and other objects) in other languages
n_clusters	double	Number of distinct clusters in package network
centrality_dir_mn	double	Mean centrality of all network edges, calculated from directed representation of network
centrality_dir_md	double	Median centrality of all network edges, calculated from directed representation of network
centrality_dir_mn_no0	double	Mean centrality of all network edges, calculated from directed representation of network, excluding edges with centrality of zero
centrality_dir_md_no0	double	Median centrality of all network edges, calculated from directed representation of network, excluding edges with centrality of zero
centrality_undir_mn	double	Mean centrality of all network edges, calculated from undirected representation of network
centrality_undir_md	double	Median centrality of all network edges, calculated from undirected representation of network
centrality_undir_mn_no0	double	Mean centrality of all network edges, calculated from undirected representation of network, excluding edges with centrality of zero
centrality_undir_md_no0	double	Median centrality of all network edges, calculated from undirected representation of network, excluding edges with centrality of zero
num_terminal_edges_dir	double	Numbers of terminal edges, calculated from directed representation of network
num_terminal_edges_undir	double	num_terminal_edges_undir, calculated from undirected representation of network
node_degree_mn	double	Mean node degree
node_degree_md	double	Median node degree
node_degree_max	double	Maximum node degree
cpl_instability_pkg	double	Coupling instability, a measure of the extent to which packages depend on external functionality without other packages in turn depending on them

external_calls.csv

variable	class	description
package_from	character	The package that makes the call
package_to	character	The package that the source package calls
n_total	double	The total number of calls from package_from to package_to
n_unique	double	The number of unique calls from package_from to package_to

internal_calls.csv

variable	class	description
package	character	The package being evaluated
n_total	double	The total number of calls from functions in one file to functions in another file within the package
n_unique	double	The number of unique calls from functions in one file to functions in another file within the package

Cleaning Script

The authors provided mostly [clean data](https://zenodo.org/records/7414296. We chose one of their datasets, lightly cleaned the data, and saved it as a CSV. We also split the external_calls data into two files, one for calls to functions in other files in the same package (internal_calls.csv) and one for calls to functions in other packages (external_calls.csv).

library(tidyverse)
library(here)
library(fs)

working_dir <- here::here("data", "2023", "2023-12-26")

cran_20221122_url <- "https://zenodo.org/records/7414296/files/pkgstats-CRAN-current.Rds?download=1"
cran_20221122 <- readr::read_rds(cran_20221122_url) |>
  dplyr::ungroup() |>
  dplyr::mutate(
    dplyr::across(
      c(translations, depends:linking_to, languages, external_calls),
      \(x) {
        dplyr::na_if(x, "NA") |> 
          dplyr::na_if("")
      }
    )
  )
dplyr::glimpse(cran_20221122)


calls_20221122 <- cran_20221122 |>
  dplyr::select(package_from = package, external_calls) |>
  tidyr::separate_longer_delim(
    external_calls,
    ","
  ) |>
  # In at least one case, an extra "L:" prefix was picked up from a 1:10-style
  # range.
  dplyr::mutate(
    external_calls = stringr::str_remove(external_calls, "^L:")
  ) |> 
  tidyr::separate_wider_delim(
    external_calls,
    ":",
    names = c("package_to", "n_total", "n_unique")
  )

cran_20221122$external_calls <- NULL

external_calls <- calls_20221122 |> 
  dplyr::filter(package_from != package_to)
internal_calls <- calls_20221122 |> 
  dplyr::filter(package_from == package_to) |> 
  dplyr::select(package = package_from, n_total, n_unique)

readr::write_csv(
  cran_20221122,
  fs::path(working_dir, "cran_20221122.csv")
)
readr::write_csv(
  external_calls,
  fs::path(working_dir, "external_calls.csv")
)
readr::write_csv(
  internal_calls,
  fs::path(working_dir, "internal_calls.csv")
)