stringr and regular expressions
Rworkshop
Wednesday, 7 February 2024
stringr packagestringr introductionLearning objectives
Material: Regular expressions
"^lecture([0-9]{1,2}).*[^_].qmd$/\1.qmd/g"stringrstringrstringr?
Consistency
Less typing and looking up things
All functions in stringr start with str_
All take a vector of strings as the first argument
(“data first”)
Pipes work as expected
All functions properly vectorised
(http://stringr.tidyverse.org)
[1] "cat" "cart" "carrot" "catastrophe" "dog"
[6] "rat" "bet" [1] TRUE TRUE TRUE TRUE FALSE TRUE FALSE start end
[1,] 2 2
[2,] 2 2
[3,] 2 2
[4,] 2 2
[5,] NA NA
[6,] 2 2
[7,] NA NA[[1]]
start end
[1,] 2 2
[[2]]
start end
[1,] 2 2
[[3]]
start end
[1,] 2 2
[[4]]
start end
[1,] 2 2
[2,] 4 4
[[5]]
start end
[[6]]
start end
[1,] 2 2
[[7]]
start endWarning
Returns list objects. Usually a lot more complicated than what you wanted.
[1] "cat" "cart" "carrot" "catastrophe" "dog"
[6] "rat" "bet" [1] "cat" "cart" "carr" "cata" "dog" "rat" "bet" Check c#tastrophe!
[[1]]
[1] "c" "t"
[[2]]
[1] "c" "rt"
[[3]]
[1] "c" "rrot"
[[4]]
[1] "c" "t" "strophe"
[[5]]
[1] "dog"
[[6]]
[1] "r" "t"
[[7]]
[1] "bet"str_split() creates lists with vectors of different lengths. Harder to work on programmatically.
my_col <- c("F", "M", "female", "male", "male", "female", "female", "männlich")
convert_gender <- function(x){
case_when(
str_detect(x, "m") ~ "Male",
str_detect(x, "M") ~ "Male",
str_detect(x, "F") ~ "Female",
str_detect(x, "f") ~ "Female",
TRUE ~ x
)
}
convert_gender(my_col)[1] "Female" "Male" "Male" "Male" "Male" "Male" "Male" "Male" Need a better way to express what we want to match!
regex() – rarely necessaryprint() is giving the quoted strings and therefore misleadingcat() or writeLines() to see strings properly escaped.writeLines() preferred for writingMetacharacters
Symbols matching groups of characters.
. (dot) represents any character except for the newline character (\n)
( and )| ( logical OR ) for groups [,1] [,2] [,3]
[1,] "cat" "cat" NA
[2,] "cart" NA "cart"
[3,] NA NA NA
[4,] "cat" "cat" NA
[5,] NA NA NA
[6,] NA NA NA
[7,] NA NA NA str_match() [,1] [,2]
[1,] NA NA
[2,] "cart" "ar"
[3,] NA NA
[4,] "catast" "as"
[5,] NA NA
[6,] NA NA
[7,] NA NA Output complex when multiple capture groups are around.
? at most once.
* matched zero or more times.
+ one or more times.
{n} exactly n times.
{n,} n or more times.
{n,m} at least n times, but not more than m times.
Match the longest possible subsequence.
| Pattern | Matches | Complement | Matches |
|---|---|---|---|
\d |
Digit | \D |
No digit |
\s |
Whitespace | \S |
No whitespace |
\w |
Word chars | \W |
No work char |
\b |
Boundaries | \B |
Within words |
Note: single matches counted as one
| Expression | Description |
|---|---|
| [:upper:] | Upper-case letters. |
| [:lower:] | Lower-case letters. |
| [:alpha:] | Alphabetic characters: ‘[:lower:]’ and ‘[:upper:]’. |
| [:digit:] | Digits: ‘0 1 2 3 4 5 6 7 8 9’. |
| [:punct:] | Punctuation characters: ‘! ” # $ % &’ ( ) * + , - . / : ; < = > ? @ ’ and others. |
| [:space:] | Space characters: tab, newline, vertical tab, form feed, carriage return, and space. |
| [:blank:] | Blank characters: space and tab. |
| [:alnum:] | Alphanumeric characters: ‘[:alpha:]’ and ‘[:digit:]’. |
| [:graph:] | Graphical characters: [:alnum:] and ‘[:punct:]’. |
Built-in (stringi - stringr)
| Group | Matches |
|---|---|
| [a-z] | lowercase letters |
| [a-zA-Z] | any (ascii) letter |
| [0-9] | any number |
| [aeiou] | any vowel |
| [0-7ivx] | any of 0 to 7, i, v, and x |
Rely on built-in groups where possible
Note that the set of alphabetic characters includes accents such as ß, ç or ö which are very common in some languages. Use is more general than [A-Za-z] which ascii characters only.
We saw special characters such as ., +, * or $ having special meaning in regular expressions.
[1] "YKL045W-A" "12+45=57" "$1200.00" "ID2.2" Not what we wanted!
To match a \, our pattern must represent \\.
How to match c("a\\backslash", "nobackslash", "slash","\n)?
Note the difference when printing meta-characters.
[1] "a\\backslash" "nobackslash" "slash" "\n" a\backslash nobackslash slash Our string must contain 4 backslashes!
Raw strings are written as r”( )“ and do not require additional escape sequences.
^ is …\1, \2 and so forth refer to groups matched with ().
regexplainSimple addin for RStudio by Garrick Aden-Buie
# A tibble: 6 × 2
subject_id gender_age
<int> <chr>
1 1001 f-27
2 1002 f-27
3 1003 f-56
4 1004 m-54
5 1005 m-41
6 1006 m-21 # A tibble: 6 × 2
subject_id gender_age
<int> <glue>
1 1001 m61
2 1002 m49
3 1003 m58
4 1004 f39
5 1005 m42
6 1006 m32 # A tibble: 6 × 3
subject_id sex age
<int> <chr> <chr>
1 1001 m 61
2 1002 m 49
3 1003 m 58
4 1004 f 39
5 1005 m 42
6 1006 m 32 Separate longer exists
Check separate_longer_position() or separate_longer_delim() if you split repeating values.
unite()data_value <-
tibble(
year = c(2015, 2014, 2014),
month = c(11, 2, 4),
day = c(23, 1, 30),
value = c("high", "low", "low"))
data_value# A tibble: 3 × 4
year month day value
<dbl> <dbl> <dbl> <chr>
1 2015 11 23 high
2 2014 2 1 low
3 2014 4 30 low Demonstration only.
Use the package lubridate for actually working with dates!
lubridate functionsLubridate to the rescue!
# A tibble: 5 × 3
subject visit_date good_date
<dbl> <chr> <date>
1 1 01/07/2001 2001-07-01
2 2 01.MAY.2012 2012-05-01
3 3 12-07-2015 2015-07-12
4 4 4/5/14 2014-05-04
5 5 12. Jun 1999 1999-06-12lubridate has a range of functions for parsing ill-formatted dates and times.
patient_df <- tibble(
subject_id = 1001:1003,
visit_id = c("1,2, 3", "1|2", "1"),
measured = c("9,0, 11", "11, 3", "12"))
patient_df# A tibble: 3 × 3
subject_id visit_id measured
<int> <chr> <chr>
1 1001 1,2, 3 9,0, 11
2 1002 1|2 11, 3
3 1003 1 12 Note the incoherent white space and separators.
complete()# A tibble: 9 × 3
subject_id visit_id measured
<int> <int> <int>
1 1001 1 9
2 1001 2 0
3 1001 3 11
4 1002 1 11
5 1002 2 3
6 1002 3 0
7 1003 1 12
8 1003 2 0
9 1003 3 0Use <NA>
Don’t use 0 for missing data in real life application.
Resources
stringi – General implementation of regular expressionsstringr – Wrapper for vectorisation and convenience functionsglue – formatting complex strings - for referenceAcknowledgments
stringi implementation)Further reading
