mycsg SASnR 9550_run-multiple-programs-for-a-single-file

Introduction

In clinical programming, we routinely build a deliverable by running several programs in sequence — for example, one program per SDTM domain, then one per ADaM dataset, then one TFL per table.
In SAS, a shared project team usually handles this through the institutional batch submission tool (SAS Grid, a shell wrapper, or a drop-folder that a scheduler watches). There is no single portable SAS file that reliably submits every other SAS file for us, which is why this lesson has no SAS variant — the equivalent step lives outside SAS itself.
In R, we can drive the whole batch from a single R file that calls each individual program in its own fresh R session. This gives us predictable isolation, per-program log files and an explicit execution order — all the things a SAS batch setup usually provides.
In this lesson we set up a tiny worked example:
- three small R programs that each create one XPT file (DM, AE, LB),
- one driver file run_all.R that runs the three programs in order and writes one log file per program.
Folder layout we assume:
- r_batch_xpt_example/
  - run_all.R (the batch driver, at the top level)
  - programs/
    - 01_make_dm_xpt.R
    - 02_make_ae_xpt.R
    - 03_make_lb_xpt.R
  - output/xpt/ (created automatically when we run)
  - output/logs/ (created automatically when we run)
Each individual R program should be fully self-contained — it should read or build its own data, process it and write its output without depending on objects left behind by another program.
To kick off a batch run, we open a terminal in the project folder and execute Rscript run_all.R. The driver runs each program in a clean R session, captures its output to a per-program log file, and continues to the next one.
After the run, we expect to find DM.xpt, AE.xpt and LB.xpt under output/xpt/, plus one .log file per program under output/logs/.
Adding a new program is a two-step change: drop a new file like 04_make_ex_xpt.R into the programs/ folder, then add its path to the scripts vector in run_all.R.

Run multiple programs from a single file

R (tidyverse)+

#==============================================================================
# programs/01_make_dm_xpt.R — build and write DM.xpt
#==============================================================================
library(tidyverse)
library(haven)

out_dir <- file.path("output", "xpt")
dir.create(out_dir, recursive = TRUE, showWarnings = FALSE)

dm <- tribble(
  ~studyid, ~domain, ~usubjid,     ~subjid, ~sex, ~age,
  "CSG001", "DM",    "CSG001-0001", "0001", "M",   35,
  "CSG001", "DM",    "CSG001-0002", "0002", "F",   29
) %>%
  mutate(across(where(is.character), ~ str_sub(.x, 1, 200)))

write_xpt(dm, file.path(out_dir, "DM.xpt"))


#==============================================================================
# programs/02_make_ae_xpt.R — build and write AE.xpt
#==============================================================================
library(tidyverse)
library(haven)

out_dir <- file.path("output", "xpt")
dir.create(out_dir, recursive = TRUE, showWarnings = FALSE)

ae <- tribble(
  ~studyid, ~domain, ~usubjid,     ~aeseq, ~aeterm,    ~aeser,
  "CSG001", "AE",    "CSG001-0001",     1, "Headache", "N",
  "CSG001", "AE",    "CSG001-0002",     1, "Nausea",   "N"
) %>%
  mutate(across(where(is.character), ~ str_sub(.x, 1, 200)))

write_xpt(ae, file.path(out_dir, "AE.xpt"))


#==============================================================================
# programs/03_make_lb_xpt.R — build and write LB.xpt
#==============================================================================
library(tidyverse)
library(haven)

out_dir <- file.path("output", "xpt")
dir.create(out_dir, recursive = TRUE, showWarnings = FALSE)

lb <- tribble(
  ~studyid, ~domain, ~usubjid,     ~lbtestcd, ~lbtest,      ~lbstresn, ~lbstresu,
  "CSG001", "LB",    "CSG001-0001", "HGB",    "Hemoglobin", 13.2,      "g/dL"
) %>%
  mutate(across(where(is.character), ~ str_sub(.x, 1, 200)))

write_xpt(lb, file.path(out_dir, "LB.xpt"))


#==============================================================================
# run_all.R — batch driver (sits at the project root)
#==============================================================================
library(tidyverse)

setwd("C:/path/to/r_batch_xpt_example")

scripts <- c(
  file.path("programs", "01_make_dm_xpt.R"),
  file.path("programs", "02_make_ae_xpt.R"),
  file.path("programs", "03_make_lb_xpt.R")
)

log_dir <- file.path("output", "logs")
dir.create(log_dir, recursive = TRUE, showWarnings = FALSE)

walk(scripts, function(s) {
  log_file <- file.path(
    log_dir,
    paste0(tools::file_path_sans_ext(basename(s)), ".log")
  )
  system2(
    command = "Rscript",
    args    = s,
    stdout  = log_file,
    stderr  = log_file
  )
})

cat("Batch run completed\n")

To run several R programs from a single driver file in tidyverse style, we list the programs in order and iterate with walk().
scripts is an ordered character vector — the same idea as listing SAS programs in a batch submit file. The execution order is the vector order.
Inside each individual program, we include library(tidyverse) and library(haven). That is critical for batch mode: Rscript does not auto-load any packages, so each program has to be self-sufficient.
We use across(where(is.character), ~ str_sub(.x, 1, 200)) to cap every character column at 200 bytes. This matches XPT v5 limits and keeps the write step safe for SDTM-style data.
In the driver, walk() is the purrr equivalent of a for loop — we iterate over scripts and execute a block for each one. Using walk() instead of map() signals “we care about the side effect, not a return value”.
Inside the walk body, we build a log file name by stripping the directory and extension from the script name, and redirect stdout and stderr into it with system2().
system2(command = "Rscript", args = s, ...) launches a brand-new R session for each program. That gives us clean isolation — a stray object or loaded package in one program cannot leak into the next.
After the loop finishes, we print “Batch run completed” so the console shows a clear end-of-batch marker.
After running Rscript run_all.R, we check that output/xpt/ contains DM.xpt, AE.xpt and LB.xpt, and output/logs/ contains one .log file per program.
If any program fails, we open its specific log file, fix the issue inside that program, and rerun Rscript run_all.R — the driver is safe to re-run end-to-end.

Happy learning!

Search lessons

Introduction

Run multiple programs from a single file