← SASnRCompress or remove specific characters

Introduction

Raw text values coming from EDC, external vendors, lab files or manual entry are often messy — extra spaces, stray punctuation, mixed separators like slashes and dashes, or special characters that we do not want in the analysis.
Before we can match, group, merge or summarise such values reliably, we typically clean them up by dropping unwanted characters.
SAS calls this operation compressing a string — the COMPRESS() function either strips a given set of characters or, with the "k" (keep) modifier, strips everything except a given set.
R does not have a dedicated "compress" function. We use str_remove_all() (tidyverse stringr) or gsub() (base R) with a pattern. A plain string removes literal characters; a regular expression removes a class of characters.
What to expect from the examples below:
- A small input dataset with six deliberately messy text values.
- Five cleaned columns showing common tasks: drop spaces only, drop any whitespace, drop slashes and dashes, keep only digits, keep only letters and digits.
- The SAS, tidyverse and base R versions produce the same five columns so we can compare the syntax side by side.
After running, inspect compress02 (or the R equivalent) and confirm each new column reflects the intended rule. For example, digits_only on row 4 should contain only 015.

Compress or remove specific characters

SAS (Base SAS)+

*==============================================================================;
* Step 1: create a small dataset with messy text values                       ;
*==============================================================================;
data compress01;
  infile datalines dlm='|' dsd truncover;
  length raw_text $40;
  input id raw_text $;
  datalines;
1|SUBJ 001
2|SUBJ-002/A
3|  AE-TERM 123 
4|VISIT 01 / DAY 5
5|ABC@123#XYZ
6| 2024-01/15 
;
run;

*==============================================================================;
* Step 2: apply several COMPRESS variations                                   ;
*==============================================================================;
data compress02;
  set compress01;

  no_spaces     = compress(raw_text);             /* default: drops blanks only       */
  no_whitespace = compress(raw_text, , "s");      /* "s" modifier: any whitespace     */
  no_dash_slash = compress(raw_text, "-/");       /* drop listed characters           */
  digits_only   = compress(raw_text, , "kd");     /* k=keep, d=digits                 */
  alnum_only    = compress(raw_text, , "kad");    /* k=keep, a=alpha, d=digits        */
run;

To drop only blank spaces, we call compress with just the first argument — for example, compress(string).
This is the default behaviour and is often enough when the only clutter in our text is ordinary spaces.
If we want to remove a specific list of characters instead, we pass them as the second argument.
For example, compress(string, "-/") drops every hyphen and slash it finds.
The second argument is plain character matching, not a regular expression.
When we need something broader — like “drop all whitespace” or “drop all punctuation” — SAS offers single-letter modifiers in the third argument.
The useful ones are d (digits), a (alphabetic), s (whitespace), p (punctuation).
We add the k modifier to flip the meaning from “drop these” to “keep these”.
So "kd" keeps digits and drops everything else.
And "kad" keeps letters and digits, drops everything else.
One gotcha: bare compress() strips ordinary blanks but not tabs.
If the upstream file might contain tabs or newlines, we add the s modifier.

R (tidyverse)+

library(tidyverse)

#==============================================================================
# Step 1: create a small dataset with messy text values
#==============================================================================
compress01 <- tribble(
  ~id, ~raw_text,
  1, "SUBJ 001",
  2, "SUBJ-002/A",
  3, "  AE-TERM 123 ",
  4, "VISIT 01 / DAY 5",
  5, "ABC@123#XYZ",
  6, " 2024-01/15 "
)

#==============================================================================
# Step 2: apply several str_remove_all variations
#==============================================================================
compress02 <- compress01 %>%
  mutate(
    no_spaces     = str_remove_all(raw_text, " "),        # literal space only
    no_whitespace = str_remove_all(raw_text, "\\s+"),     # any whitespace run
    no_dash_slash = str_remove_all(raw_text, "[-/]"),     # character class
    digits_only   = str_remove_all(raw_text, "[^0-9]"),   # drop non-digits  -> keep digits
    alnum_only    = str_remove_all(raw_text, "[^A-Za-z0-9]") # drop non-alnum -> keep alnum
  )

To drop any run of characters from a string in tidyverse, we call str_remove_all(string, pattern).
It returns the cleaned value and is vectorised, so we can drop it straight inside mutate() and let it run across a whole column.
The second argument is a regular expression, not a plain character list.
That is the big shift from SAS — instead of single-letter modifiers like "kd", we describe the characters we want to drop with regex syntax.
Here is how each of the five cleaned columns maps back to its SAS compress() equivalent:
- For dropping just literal spaces, pattern " " matches only the ordinary space character. Same result as SAS compress(raw_text).
- For any run of whitespace (spaces, tabs, newlines), we use "\\s+". Same result as SAS compress(raw_text, , "s").
- To drop a specific set of characters, we put them in a character class like "[-/]", which means “any one of - or /”. Same result as SAS compress(raw_text, "-/").
- To keep only digits, we flip the logic — "[^0-9]" means “any character that is not a digit”, so removing those leaves the digits behind. Same result as SAS compress(raw_text, , "kd").
- To keep only letters and digits, same trick — "[^A-Za-z0-9]" removes everything that is not alphanumeric. Same result as SAS compress(raw_text, , "kad").
The ^ inside square brackets is the critical trick.
It flips the class from “match these” to “match everything except these”.
That is how we emulate the SAS k (keep) modifier in R.

Happy learning!

Search lessons

Introduction

Compress or remove specific characters