Skip to content

Commit

Permalink
Pre-parse YAML to avoid common issues (#744)
Browse files Browse the repository at this point in the history
* Pre-parse YAML to avoid common issues.

* Apply preprocessing test.
  • Loading branch information
jonthegeek authored Sep 10, 2024
1 parent b88e753 commit 560c772
Show file tree
Hide file tree
Showing 3 changed files with 27 additions and 7 deletions.
10 changes: 5 additions & 5 deletions data/2024/2024-09-10/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
title: Economic Diversity and Student Outcomes
title: "Economic Diversity and Student Outcomes"
article:
title: Economic diversity and student outcomes at the University of Texas at Dallas
url: https://www.nytimes.com/interactive/projects/college-mobility/university-of-texas-at-dallas
title: "Economic diversity and student outcomes at the University of Texas at Dallas"
url: "https://www.nytimes.com/interactive/projects/college-mobility/university-of-texas-at-dallas"
data_source:
title: "Opportunity Insights: College-Level Data for 139 Selective American Colleges"
url: https://opportunityinsights.org/data/
url: "https://opportunityinsights.org/data/"
images:
- file: utd-access.png
- file: "utd-access.png"
alt: >
Parent income for students attending The University of Texas at Dallas (UTD).
The median family income is $89,800, which is among the highest in Texas but
Expand Down
6 changes: 4 additions & 2 deletions static/templates/assign_week.R
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# Set these variables ----------------------------------------------------------

# src_folder_name <- "american_idol"
# target_date <- "2024-07-23"
# target_date <- "2024-09-10"

# Run these scripts ------------------------------------------------------------

Expand All @@ -18,7 +18,9 @@ fs::dir_create(target_dir)

## metadata --------------------------------------------------------------------

metadata <- yaml::read_yaml(fs::path(src_dir, "meta.yaml"))
source(here::here("static", "templates", "metadata.R"), local = TRUE)

metadata <- read_metadata(fs::path(src_dir, "meta.yaml"))

dataset_files <- fs::dir_ls(src_dir, glob = "*.csv") |> unname()
dataset_filenames <- basename(dataset_files)
Expand Down
18 changes: 18 additions & 0 deletions static/templates/metadata.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
read_metadata <- function(path) {
yaml_preprocess(path)
yaml::read_yaml
}

yaml_preprocess <- function(path) {
readLines(path, warn = FALSE) |>
yaml_quote() |>
writeLines(path)
}

yaml_quote <- function(yaml_lines) {
stringr::str_replace(
yaml_lines,
"^([^:]*:\\s+)([^>|\"'].*)$",
'\\1"\\2"'
)
}

0 comments on commit 560c772

Please sign in to comment.