cheat sheets / Basics
Base R · Tidyverse · data.table

R Cheat Sheet

In this R cheat sheet, you'll find every function you reach for in a real analysis session, from c() to ggsave(). Searchable, filterable by dialect, copy ready.

/ to focus
dialect:
0snippets
0sections
3dialects
0signup required

R Basics: Assignment, Operators, Help and Packages

Start here. Assignment, operators, help and workspace commands work in every R session, no packages needed.

Assignment and printing BASEcore
Standard assignment, the R convention
x <- 5
Equals sign, works but reserve it for function arguments
x = 5
Right assignment, handy at the end of a pipe
5 -> x
Assign into the parent environment from inside a function
counter <<- counter + 1
Assign and print in one step with outer parentheses
(x <- 5)
Chain one value into several names
a <- b <- 0
Print explicitly
print(x)
Formatted printing without quotes
cat("Value:", x, "\n")
Interpolate values into a string
sprintf("x is %.2f", x)
Comment a line
# this line is ignored by R
Operators BASETIDYcore
Arithmetic operators
2 + 3; 5 - 1; 4 * 2; 9 / 3; 2 ^ 10
Integer division, drops the remainder
7 %/% 2
Modulo, the remainder itself
7 %% 2
Comparison operators
x == 5; x != 3; x > 1; x <= 10
Vectorised logical AND and OR
c(TRUE, FALSE) & c(TRUE, TRUE)
Scalar short circuit AND and OR, for if conditions
if (x > 0 && x < 10) print('in range')
Negation and exclusive or
!TRUE; xor(TRUE, FALSE)
Value matching, is it in this set
"b" %in% c("a", "b", "c")
Native pipe, R 4.1 and newer
c(1, 4, 9) |> sqrt() |> sum()
Magrittr pipe with a dot placeholder
mtcars %>% lm(mpg ~ wt, data = .)
Colon builds an integer sequence
1:6
Define your own infix operator
`%+%` <- function(a, b) paste0(a, b) "data" %+% "frame"
Getting help BASETIDYcore
Open the help page for a function
?mean
Same thing, function form, needed for operators
help("[")
Fuzzy search the installed documentation
??regression
Run the examples from a help page
example(lm)
List the vignettes shipped with a package
vignette(package = "dplyr")
Open one specific vignette
vignette("dplyr")
Show only the argument list
args(round)
Print the source of a function
print(sd)
Package index page
help(package = "stats")
Inspect the structure of any object
str(iris)
Workspace and session BASEcore
List objects in the global environment
ls()
Remove one object
rm(x)
Clear the whole workspace
rm(list = ls())
Current working directory
getwd()
Change the working directory
setwd("~/projects/analysis")
R version and loaded packages, paste this into bug reports
sessionInfo()
Just the version string
R.version.string
Make random draws reproducible
set.seed(42)
Change global options
options(digits = 4, scipen = 999)
Current date and time
Sys.Date(); Sys.time()
Read an environment variable
Sys.getenv("HOME")
Where R looks for packages
.libPaths()
Trigger garbage collection
gc()
Quit without saving the workspace image
q(save = "no")
Packages BASETIDYcore
Install from CRAN
install.packages("dplyr")
Install several at once
install.packages(c("dplyr", "ggplot2", "tidyr"))
Attach a package for the session
library(dplyr)
Load the whole tidyverse
library(tidyverse)
Attach conditionally, returns TRUE or FALSE instead of an error
require(dplyr)
Call a function without attaching the package
dplyr::filter(df, x > 3)
Reach an internal, unexported function
pkg:::hidden_function()
Check which version you have
packageVersion("ggplot2")
Update everything from CRAN
update.packages(ask = FALSE)
Uninstall
remove.packages("oldpkg")
Install from GitHub
remotes::install_github("tidyverse/dplyr")
Detach without restarting
detach("package:dplyr", unload = TRUE)
List everything installed
rownames(installed.packages())

R Data Types: Vectors, Coercion, Missing Values and Factors

R has no scalars. A single number is a length one vector, which is why almost everything in R is vectorised by default.

Atomic types BASEcore
Double, the default numeric type
x <- 3.14 class(x)
Integer, note the L suffix
n <- 5L typeof(n)
Character
name <- "Ada" class(name)
Logical, always spelled out in full
flag <- TRUE class(flag)
Complex and raw, rarely needed
z <- 2+3i; r <- as.raw(255)
Internal storage type versus the class attribute
typeof(1L); class(1L); mode(1L)
Compact summary of any object
str(list(a = 1, b = "x"))
Object size in memory
object.size(1:1e6)
Type checks and conversion BASETIDYcore
Test the type
is.numeric(x); is.character(x); is.logical(x)
Test the container
is.vector(x); is.list(x); is.data.frame(x); is.function(x)
Convert to number, non numeric text becomes NA with a warning
as.numeric("42")
Convert to text
as.character(42)
Convert to integer, this truncates rather than rounds
as.integer(9.99)
Convert to logical
as.logical(c(1, 0, "TRUE", "yes"))
Implicit coercion, the most permissive type wins
c(1, "a", TRUE)
Safe numeric conversion of a factor, never skip as.character
as.numeric(as.character(f))
Type stable conversion helpers
readr::parse_number("$1,299.00")
Missing and special values BASETIDYcore
The four special values
NA; NULL; NaN; Inf
Detect missing values
is.na(c(1, NA, 3))
Count missing values in a column
sum(is.na(df$score))
Missing values per column of a data frame
colSums(is.na(df))
Drop rows that contain any NA
na.omit(df)
Keep only complete rows, more explicit
df[complete.cases(df), ]
Ignore missing values in a summary function
mean(x, na.rm = TRUE)
Replace missing values with a fallback
ifelse(is.na(x), 0, x)
First non missing value across vectors
dplyr::coalesce(x, y, 0)
Fill missing values in a data frame column
tidyr::replace_na(df, list(score = 0))
Turn a sentinel value into NA
dplyr::na_if(x, -999)
NULL removes, NA marks, know the difference
length(c(1, NULL, 3)); length(c(1, NA, 3))
Building vectors BASEcore
Combine values
c(1, 2, 3)
Sequence with a step
seq(0, 1, by = 0.25)
Sequence of a fixed length
seq(0, 1, length.out = 5)
Safe sequence for loops, handles length zero
seq_len(5); seq_along(c("a","b"))
Repeat a whole vector
rep(1:2, times = 3)
Repeat each element
rep(1:2, each = 3)
Empty typed vectors
numeric(3); character(2); logical(2); vector("list", 2)
Named vector
v <- c(a = 1, b = 2) v["b"]
Rename after the fact
names(v) <- c("first", "second")
Vector operations and indexing BASEcore
Position based, R counts from 1
x[1]
Drop elements with a negative index
x[-1]; x[-(1:3)]
Logical filter, the R workhorse
x[x > 3 & !is.na(x)]
Slice a range
x[2:4]
First and last elements
head(x, 3); tail(x, 3)
Length, and resizing by assignment
length(x); length(x) <- 10
Sort ascending or descending
sort(x); sort(x, decreasing = TRUE)
Order returns positions, use it to sort other objects
x[order(x)]
Reverse
rev(x)
Unique values and duplicate flags
unique(x); duplicated(x)
Positions that satisfy a condition
which(x > 3)
Position of the maximum and the minimum
which.max(x); which.min(x)
Vectorised arithmetic, no loop needed
c(1, 2, 3) * 10
Recycling, the shorter vector repeats
c(1, 2, 3, 4) + c(0, 10)
Cumulative helpers
cumsum(1:5); cumprod(1:5); cummax(c(1,3,2))
Set operations
union(a, b); intersect(a, b); setdiff(a, b)
Factors BASETIDYdata
Create a factor with explicit level order
f <- factor(c("low", "high", "low"), levels = c("low", "high"))
Inspect the levels
levels(f); nlevels(f)
Count each level
table(f)
Ordered factor, comparisons then work
g <- factor(c("S","L"), levels = c("S","M","L"), ordered = TRUE)
Change the reference level for modelling
f <- relevel(f, ref = "high")
Remove levels that no longer appear
droplevels(f)
Recode levels
levels(f) <- c("Low", "High")
Reorder a factor by another variable, great for plots
forcats::fct_reorder(f, value)
Order by frequency, most common first
forcats::fct_infreq(f)
Collapse rare levels into Other
forcats::fct_lump_n(f, n = 5)
Rename specific levels
forcats::fct_recode(f, Small = "S", Large = "L")
Bin a numeric variable into a factor
cut(x, breaks = c(0, 18, 65, Inf), labels = c("child", "adult", "senior"))

R Data Structures: Matrices, Lists, Data Frames and Tibbles

Matrices hold one type, lists hold anything, data frames are lists of equal length columns. Almost all analysis happens in the third one.

Matrices and arrays BASEdata
Build a matrix, filled by column
m <- matrix(1:6, nrow = 2)
Fill by row instead
matrix(1:6, nrow = 2, byrow = TRUE)
Dimensions
dim(m); nrow(m); ncol(m)
Name the rows and columns
dimnames(m) <- list(c("r1","r2"), c("a","b","c"))
Index by row, column, or cell
m[1, ]; m[, 2]; m[1, 2]
Keep the matrix shape when slicing one row
m[1, , drop = FALSE]
Bind rows or columns together
rbind(m, c(7, 8, 9)); cbind(m, 0)
Transpose
t(m)
Matrix multiplication and inversion
a %*% b; solve(a)
Identity matrix and diagonal
diag(3); diag(m)
Fast row and column summaries
rowSums(m); colMeans(m)
Apply a function over rows (1) or columns (2)
apply(m, 1, max)
Three dimensional array
arr <- array(1:24, dim = c(2, 3, 4))
Lists BASETIDYdata
Create a named list
l <- list(id = 1, tags = c("a", "b"), fit = NULL)
Extract the element itself
l$tags; l[["tags"]]
Single brackets return a sub list, not the element
class(l["tags"]); class(l[["tags"]])
Add or overwrite an element
l$score <- 0.91
Remove an element
l$score <- NULL
Names and length
names(l); length(l)
Flatten to a vector, types get coerced
unlist(list(1, 2, 3))
Attach names to an existing list
setNames(list(1, 2), c("a", "b"))
Merge two lists, second one wins
modifyList(defaults, user_opts)
Reach into a nested list
l$model$coefficients[["wt"]]
Apply a function to every element
lapply(l, length)
Pull one field out of a list of lists
purrr::map_chr(records, "name")
Data frames BASEdata
Create a data frame
df <- data.frame(id = 1:3, name = c("a", "b", "c"))
First and last rows
head(df); tail(df, 3)
Structure, dimensions and quick stats
str(df); dim(df); summary(df)
Row and column counts
nrow(df); ncol(df)
Column names, readable and writable
names(df); colnames(df)[2] <- "label"
Select a column as a vector
df$name; df[["name"]]
Select a column and keep the data frame
df["name"]; df[, "name", drop = FALSE]
Filter rows with a logical condition
df[df$id > 1, ]
Row and column at once
df[df$id > 1, c("id", "name")]
Add a computed column
df$flag <- df$id > 2
Drop a column
df$flag <- NULL
Filter and select in one base call
subset(df, id > 1, select = c(id, name))
Sort by one column, then another descending
df[order(df$id, -df$value), ]
Stack data frames and join them
rbind(df1, df2); merge(df1, df2, by = "id")
Open the spreadsheet viewer in RStudio
View(df)
Built in data frames for practice
data(iris); data(mtcars)
Tibbles TIDYdata
Create a tibble
tibble(x = 1:3, y = x ^ 2)
Convert an existing data frame
as_tibble(mtcars)
Row by row literal, readable for small tables
tribble( ~name, ~score, "a", 90, "b", 72 )
Transposed structure view, better than str for wide data
glimpse(df)
Tibbles never partially match or drop dimensions
df[, "x"] # still a tibble
Move row names into a real column
tibble::rownames_to_column(mtcars, "car")
Print more rows than the default ten
print(tb, n = 40)

Importing and Exporting Data in R

readr and data.table readers are faster than the base equivalents and never silently convert text to factors.

Reading delimited text BASETIDYDTi/o
Base CSV reader
df <- read.csv("data.csv")
Base reader with common arguments
read.csv("data.csv", header = TRUE, sep = ",", na.strings = c("", "NA"))
Any delimiter, tab separated here
read.delim("data.tsv")
readr, faster and returns a tibble
df <- readr::read_csv("data.csv")
Pin the column types, stops guessing surprises
read_csv("data.csv", col_types = cols( id = col_integer(), date = col_date("%Y-%m-%d") ))
Skip lines and limit rows while exploring
read_csv("data.csv", skip = 2, n_max = 1000)
European style decimals and separators
read_csv2("data.csv")
data.table reader, fastest for large files
DT <- data.table::fread("big.csv")
Read only selected columns from a huge file
fread("big.csv", select = c("id", "amount"))
Read raw lines
lines <- readLines("notes.txt")
Excel, stats formats and JSON BASETIDYi/o
Read an Excel sheet
readxl::read_excel("book.xlsx", sheet = "Q1")
List the sheet names first
readxl::excel_sheets("book.xlsx")
Read a specific cell range
read_excel("book.xlsx", range = "B2:F200")
SPSS, Stata and SAS files
haven::read_sav("survey.sav") haven::read_dta("panel.dta")
JSON into a list or data frame
jsonlite::fromJSON("data.json")
Write JSON
jsonlite::write_json(df, "out.json")
Read a fixed width file
read.fwf("f.txt", widths = c(3, 8, 5))
Read directly from a URL
read_csv("https://example.com/data.csv")
Download a file first, then read it
download.file(url, "local.csv", mode = "wb")
Writing files BASETIDYDTi/o
Base CSV writer, always turn off row names
write.csv(df, "out.csv", row.names = FALSE)
readr writer
readr::write_csv(df, "out.csv")
Append to an existing file
write_csv(df, "out.csv", append = TRUE)
Fast writer for large tables
data.table::fwrite(DT, "out.csv")
Excel output, no Java needed
writexl::write_xlsx(df, "out.xlsx")
Several sheets in one workbook
write_xlsx(list(raw = df1, summary = df2), "out.xlsx")
Plain text lines
writeLines(c("a", "b"), "out.txt")
R native formats and databases BASETIDYi/o
Save a single object, restore it under any name
saveRDS(model, "model.rds") model <- readRDS("model.rds")
Save several objects with their names
save(df, model, file = "session.RData") load("session.RData")
Save the entire workspace
save.image("workspace.RData")
Connect to a database
con <- DBI::dbConnect(RPostgres::Postgres(), dbname = "shop", host = "localhost", user = Sys.getenv("DB_USER"))
Run a query and get a data frame back
DBI::dbGetQuery(con, "SELECT * FROM orders LIMIT 10")
Parameterised query, avoids SQL injection
dbGetQuery(con, "SELECT * FROM users WHERE id = ?", params = list(42))
Treat a table as a lazy dplyr source
tbl(con, "orders") %>% filter(amount > 100) %>% collect()
See the generated SQL before running it
show_query(lazy_query)
Write a table and close the connection
dbWriteTable(con, "results", df) dbDisconnect(con)
Paths and files BASETIDYi/o
Build a portable path
file.path("data", "raw", "2024.csv")
Project relative paths that survive setwd
here::here("data", "raw.csv")
List matching files in a folder
list.files("data", pattern = "\\.csv$", full.names = TRUE)
Read every CSV in a folder into one table
list.files("data", full.names = TRUE) |> lapply(read.csv) |> do.call(what = rbind)
Same thing, tidyverse style
purrr::map(files, read_csv) |> purrr::list_rbind(names_to = "source")
Existence checks and file admin
file.exists("f.csv"); file.remove("f.csv") dir.create("output")
Temporary file for intermediate output
tmp <- tempfile(fileext = ".csv")
File metadata
file.info("data.csv")$size

Data Wrangling With dplyr: Verbs, Joins and Column Operations

Six verbs cover most of the work: filter rows, select columns, mutate values, arrange order, summarise groups, group_by splits.

The core verbs TIDYverbs
Keep rows that match a condition
filter(df, score > 80)
Several conditions, comma means AND
filter(df, score > 80, region == "EU")
Keep columns
select(df, id, name, score)
Drop columns with a minus
select(df, -id, -starts_with("tmp_"))
Add or overwrite a column
mutate(df, ratio = wins / games)
Refer to a column you just created
mutate(df, ratio = wins / games, pct = round(ratio * 100, 1))
Sort rows
arrange(df, score)
Sort descending, several keys
arrange(df, region, desc(score))
Collapse to summary statistics
summarise(df, avg = mean(score), n = n())
Group then summarise
df %>% group_by(region) %>% summarise(avg = mean(score), n = n())
Group inline, no ungroup needed
summarise(df, avg = mean(score), .by = region)
Always release the grouping when you are done
df %>% group_by(a) %>% mutate(z = x / sum(x)) %>% ungroup()
Selecting columns TIDYverbs
Prefix and suffix matching
select(df, starts_with("q_"), ends_with("_date"))
Substring and regular expression matching
select(df, contains("score"), matches("^x[0-9]+$"))
Select by type
select(df, where(is.numeric))
A range of adjacent columns
select(df, id:score)
Select from a character vector of names
select(df, all_of(cols)); select(df, any_of(cols))
The last column, or third from the end
select(df, last_col()); select(df, last_col(2))
Rename while selecting
select(df, customer = name)
Rename without dropping anything
rename(df, customer = name)
Rename with a function
rename_with(df, toupper, starts_with("q_"))
Reorder columns
relocate(df, id, .before = name)
Extract a single column as a vector
pull(df, score)
Rows and counting TIDYverbs
Distinct rows, or distinct on chosen columns
distinct(df); distinct(df, region, .keep_all = TRUE)
Frequency table sorted by count
count(df, region, sort = TRUE)
Count combinations and add a share column
df %>% count(region, tier) %>% mutate(share = n / sum(n))
Group size inside a verb
mutate(df, group_n = n(), .by = region)
Count distinct values
summarise(df, users = n_distinct(user_id))
First rows, last rows
slice_head(df, n = 5); slice_tail(df, n = 5)
Top and bottom by value
slice_max(df, score, n = 3) slice_min(df, score, n = 3)
Top n per group
df %>% group_by(region) %>% slice_max(score, n = 1)
Random rows
slice_sample(df, n = 100) slice_sample(df, prop = 0.1)
Row numbers by position
slice(df, 10:20)
Column calculations TIDYverbs
Apply one function to many columns, lambda form since dplyr 1.1
mutate(df, across(where(is.numeric), \(x) round(x, 2)))
Summarise many columns at once
summarise(df, across(c(height, weight), \(x) mean(x, na.rm = TRUE)))
Several functions, named output columns
summarise(df, across(score, list(avg = mean, sd = sd), .names = "{.col}_{.fn}"))
Vectorised if, keeps the type strict
mutate(df, pass = if_else(score >= 60, "yes", "no"))
Multi branch conditional
mutate(df, grade = case_when( score >= 90 ~ "A", score >= 80 ~ "B", score >= 70 ~ "C", .default = "F" ))
Recode values without writing conditions
mutate(df, region = case_match(code, "us" ~ "North America", c("de", "fr") ~ "Europe", .default = "Other"))
Range test, inclusive on both ends
filter(df, between(age, 18, 65))
Previous and next values, for time series
mutate(df, delta = value - lag(value))
Running total within groups
mutate(df, running = cumsum(amount), .by = user)
Rank and quantile buckets
mutate(df, rk = min_rank(desc(score)), quartile = ntile(score, 4))
Row index within a group
mutate(df, i = row_number(), .by = user)
Operate one row at a time
df %>% rowwise() %>% mutate(best = max(c(a, b, c)))
Joins and binds BASETIDYjoins
Keep all rows on the left
left_join(orders, customers, by = "customer_id")
Keep only matching rows
inner_join(a, b, by = "id")
Keep everything from both sides
full_join(a, b, by = "id")
Keep all rows on the right
right_join(a, b, by = "id")
Different key names on each side
left_join(a, b, by = join_by(user_id == id))
Join on several keys
inner_join(a, b, by = c("year", "region"))
Filter by presence in another table, no columns added
semi_join(orders, vip_customers, by = "id")
Filter by absence
anti_join(orders, refunds, by = "order_id")
Control the suffix on duplicated column names
left_join(a, b, by = "id", suffix = c("_2023", "_2024"))
Inequality join, rows within a range
left_join(events, windows, by = join_by(between(ts, start, end)))
Stack tables on top of each other
bind_rows(q1, q2, .id = "quarter")
Place tables side by side, order must already match
bind_cols(ids, features)
Base equivalent of a left join
merge(a, b, by = "id", all.x = TRUE)
Cleaning with janitor TIDYclean
Fix column names in one call, the first thing to run after import
df <- janitor::clean_names(df)
Choose the naming style
clean_names(df, case = "upper_camel")
Drop columns and rows that are entirely empty
janitor::remove_empty(df, which = c("rows", "cols"))
Drop constant columns that carry no information
janitor::remove_constant(df)
Frequency table with counts and percentages
janitor::tabyl(df, region)
Cross tabulation with totals and formatting
df %>% tabyl(region, tier) %>% adorn_totals("row") %>% adorn_percentages("row") %>% adorn_pct_formatting(digits = 1)
Find duplicate records by key
janitor::get_dupes(df, customer_id, order_date)
Convert Excel serial numbers to real dates
janitor::excel_numeric_to_date(45000)
Round every numeric column at once
janitor::round_half_up(df$score, digits = 2)

Reshaping Data With tidyr: Pivoting, Splitting and Nesting

Tidy data means one variable per column, one observation per row. pivot_longer and pivot_wider move between that shape and reporting layouts.

Pivoting TIDYreshape
Wide to long, the most common reshape
pivot_longer(df, cols = q1:q4, names_to = "quarter", values_to = "revenue")
Pick columns by type or pattern
pivot_longer(df, cols = where(is.numeric)) pivot_longer(df, cols = starts_with("y_"))
Strip a prefix from the new name column
pivot_longer(df, starts_with("y_"), names_prefix = "y_", names_to = "year")
Split a compound column name into two variables
pivot_longer(df, -id, names_to = c("metric", "year"), names_sep = "_")
Convert the new name column to a number or date
pivot_longer(df, -id, names_to = "year", names_transform = list(year = as.integer))
Long to wide, one column per category
pivot_wider(df, names_from = quarter, values_from = revenue)
Aggregate duplicates while widening
pivot_wider(df, names_from = k, values_from = v, values_fn = mean, values_fill = 0)
Prefix the generated column names
pivot_wider(df, names_from = year, names_prefix = "y", values_from = value)
Splitting and combining TIDYreshape
Split one column into several by a delimiter
separate_wider_delim(df, full_name, delim = " ", names = c("first", "last"))
Split by a regular expression with capture groups
separate_wider_regex(df, code, patterns = c(region = "[A-Z]{2}", "-", id = "\\d+"))
Split by fixed character positions
separate_wider_position(df, sku, widths = c(dept = 2, item = 5))
Turn a delimited column into extra rows
separate_longer_delim(df, tags, delim = ",")
Glue several columns into one
unite(df, "full_date", year, month, day, sep = "-")
Keep the source columns after uniting
unite(df, "key", a, b, remove = FALSE)
Missing values and completeness TIDYreshape
Drop rows with NA in specific columns
drop_na(df, score, region)
Drop rows with NA anywhere
drop_na(df)
Replace NA per column
replace_na(df, list(score = 0, region = "unknown"))
Carry the last value forward, then backward
fill(df, price, .direction = "downup")
Make implicit missing combinations explicit
complete(df, user, date, fill = list(sales = 0))
Every combination of two vectors as a data frame
expand_grid(year = 2020:2024, quarter = 1:4)
Only the combinations already present
expand(df, nesting(region, city))
Expand a count column back into one row each
uncount(df, weight)
Nesting and list columns BASETIDYadvanced
One data frame per group, stored in a column
nested <- df %>% nest(data = -region)
Fit a model per group
nested %>% mutate(fit = purrr::map(data, ~ lm(y ~ x, data = .x)))
Unpack a list column back into rows
unnest(nested, data)
Spread a list of vectors across rows or columns
unnest_longer(df, tags) unnest_wider(df, settings)
Tidy every model into one table
nested %>% mutate(stats = purrr::map(fit, broom::tidy)) %>% unnest(stats)
Flatten deeply nested JSON safely
jsonlite::fromJSON("f.json", flatten = TRUE)

Strings, Regular Expressions and Dates in R

stringr functions all start with str_ and take the string first, which makes them pipe friendly. Base equivalents are listed next to them.

String basics BASETIDYtext
Number of characters
str_length("analysis")
Base equivalent
nchar("analysis")
Join strings
str_c("data", "frame", sep = "_")
Base joining, with and without a separator
paste("a", "b"); paste0("a", "b")
Collapse a vector into one string
paste(c("a","b","c"), collapse = ", ")
Substring by position, negatives count from the end
str_sub("analysis", 1, 4); str_sub("analysis", -3)
Base substring
substr("analysis", 1, 4)
Change case
str_to_upper(x); str_to_lower(x); str_to_title(x)
Trim whitespace, squish also collapses inner runs
str_trim(" hi "); str_squish("a b")
Pad to a fixed width
str_pad("7", width = 3, pad = "0")
Repeat and reverse
strrep("ab", 3); stringi::stri_reverse("abc")
Insert values into a template
glue::glue("User {name} scored {score}")
Format numbers into text
sprintf("%05.2f | %s | %d", 3.1, "ok", 42L)
Thousands separators and rounding for reports
format(1234567.891, big.mark = ",", nsmall = 2)
Search and replace BASETIDYtext
Does the string contain a pattern
str_detect(x, "error")
Base equivalent, returns TRUE or FALSE
grepl("error", x)
Which elements match
which(str_detect(x, "^A")); grep("^A", x)
Keep only matching elements
str_subset(x, "\\d{4}"); grep("[0-9]{4}", x, value = TRUE)
Replace the first match, then all matches
str_replace(x, "a", "-") str_replace_all(x, "a", "-")
Base replacement
sub("a", "-", x); gsub("a", "-", x)
Replace several patterns in one call
str_replace_all(x, c("cat" = "dog", "red" = "blue"))
Extract the first match and all matches
str_extract(x, "\\d+") str_extract_all(x, "\\d+")
Extract capture groups into a matrix
str_match(x, "(\\w+)@(\\w+)\\.com")
Count occurrences
str_count(x, "a")
Split on a delimiter
str_split("a,b,c", ",") strsplit("a,b,c", ",")
Split into a fixed number of pieces
str_split_fixed(x, "-", n = 3)
Prefix and suffix tests
str_starts(x, "id_"); str_ends(x, ".csv")
Turn off regex, match the literal characters
str_detect(x, fixed("."))
Ignore case
str_detect(x, regex("error", ignore_case = TRUE))
Regex quick reference BASEregex
Digit, word character, whitespace, note the double backslash
"\\d" "\\w" "\\s"
Their negations
"\\D" "\\W" "\\S"
Any single character except a newline
"a.c"
Character class and range
"[aeiou]" "[A-Za-z0-9]" "[^0-9]"
Anchors, start and end of the string
"^start" "end$"
Quantifiers
"a*" "a+" "a?" "a{2,4}"
Lazy match, as few characters as possible
"<.+?>"
Alternation and grouping
"(cat|dog)s?"
Backreference to the first group
"(\\w)\\1"
Lookahead and lookbehind, needs perl = TRUE in base
gsub("(?<=\\d)(?=(\\d{3})+$)", ",", x, perl = TRUE)
Word boundary
"\\bcat\\b"
Common validation patterns
"^[\\w.]+@[\\w.]+\\.[a-z]{2,}$" # email "^\\d{4}-\\d{2}-\\d{2}$" # ISO date
Dates and times BASETIDYdates
Today and now
Sys.Date(); Sys.time()
Parse an ISO date
as.Date("2024-03-15")
Parse a non standard format
as.Date("15/03/2024", format = "%d/%m/%Y")
lubridate parsers, named after the field order
ymd("2024-03-15"); dmy("15/03/2024"); mdy("March 15, 2024")
Parse a timestamp with a time zone
ymd_hms("2024-03-15 14:30:00", tz = "Europe/Bucharest")
Format a date as text
format(as.Date("2024-03-15"), "%d %B %Y")
Extract components
year(d); month(d); day(d); wday(d, label = TRUE)
Base extraction without lubridate
as.integer(format(d, "%Y")); weekdays(d)
Round down to a period, the key grouping trick
floor_date(d, unit = "month")
Difference between two dates
difftime(end, start, units = "days") as.numeric(end - start)
Add periods safely across month lengths
d %m+% months(1); d + days(7)
Sequence of dates
seq(as.Date("2024-01-01"), by = "month", length.out = 12)
Convert epoch seconds
as.POSIXct(1710500000, origin = "1970-01-01", tz = "UTC")
Change the display time zone
with_tz(ts, "America/New_York")
Date format codes BASEdates
Year, four digit and two digit
%Y %y
Month number, abbreviated name, full name
%m %b %B
Day of month, day of year
%d %j
Weekday abbreviated and full
%a %A
Hour 24 and 12 clock, minute, second
%H %I %M %S
AM or PM marker and time zone
%p %Z %z
Ready made combinations
%F is %Y-%m-%d, %T is %H:%M:%S

R Functions, Control Flow, Iteration and Object Systems

Reach for a vectorised function first, an apply or map second, and an explicit for loop only when order of execution matters.

Writing functions BASETIDYcore
Basic function, the last expression is returned
add <- function(x, y) { x + y }
Default argument values
greet <- function(name, greeting = "Hello") { paste(greeting, name) }
Explicit return, useful for early exits
f <- function(x) { if (x < 0) return(NA) sqrt(x) }
Pass extra arguments through with dots
wrapper <- function(x, ...) { mean(x, ...) }
Short anonymous function, R 4.1 and newer
sapply(1:5, \(x) x ^ 2)
Formula shorthand in purrr, .x is the argument
purrr::map_dbl(1:5, ~ .x ^ 2)
Validate inputs early
f <- function(x) { stopifnot(is.numeric(x), length(x) > 0) mean(x) }
Check whether an argument was supplied
if (missing(y)) y <- 0
Restrict an argument to a set of choices
f <- function(type = c("mean", "median")) { type <- match.arg(type) }
Return without printing
invisible(result)
Return several values in a list
list(estimate = m, error = se)
A function that builds another function
power <- function(p) function(x) x ^ p square <- power(2)
Control flow BASEcore
If, else if, else
if (x > 0) { "positive" } else if (x < 0) { "negative" } else { "zero" }
Vectorised conditional, works on whole columns
ifelse(x > 0, "pos", "neg")
Multi way branch on a string
switch(type, csv = read.csv(f), xlsx = read_excel(f), stop("unsupported"))
For loop over values
for (i in 1:5) print(i ^ 2)
Loop over a vector safely, seq_along handles length zero
for (i in seq_along(x)) { results[i] <- x[i] * 2 }
Preallocate the result, never grow inside a loop
out <- vector("numeric", length(x))
While loop
while (err > 1e-6) { err <- step(err) }
Repeat until a break
repeat { x <- x - 1 if (x <= 0) break }
Skip the rest of the current iteration
for (i in 1:10) { if (i %% 2 == 0) next print(i) }
The apply family BASEiterate
Rows (1) or columns (2) of a matrix
apply(m, 2, mean)
Over a list or vector, always returns a list
lapply(1:3, function(i) i ^ 2)
Same but simplified to a vector when possible
sapply(1:3, function(i) i ^ 2)
Type safe version, declare the output shape
vapply(x, length, integer(1))
Iterate over several vectors in parallel
mapply(function(a, b) a * b, 1:3, 4:6)
Map is mapply without simplification
Map(paste, c("a", "b"), c("x", "y"))
Split by a factor and summarise
tapply(df$score, df$region, mean)
Aggregate with a formula interface
aggregate(score ~ region, data = df, FUN = mean)
Call a function with a list of arguments
do.call(rbind, list_of_dfs)
Fold a vector into a single value
Reduce(`+`, 1:5, accumulate = TRUE)
Keep elements that pass a test
Filter(function(x) x > 2, 1:5)
Split a data frame into a list of groups
split(df, df$region)
purrr iteration TIDYiterate
Always returns a list
map(1:3, ~ .x ^ 2)
Typed variants, they error instead of guessing
map_dbl(x, mean); map_chr(x, class) map_int(x, length); map_lgl(x, is.numeric)
Two inputs in parallel
map2_dbl(width, height, ~ .x * .y)
Any number of inputs from a list
pmap_dbl(list(a, b, c), function(a, b, c) a + b * c)
Index available as .y
imap_chr(v, ~ paste0(.y, ": ", .x))
Combine results into one data frame
map(files, read_csv) |> list_rbind()
Side effects only, returns the input invisibly
walk(plots, print)
Filter a list
keep(x, is.numeric); discard(x, is.null); compact(x)
Fold a list
reduce(list_of_dfs, full_join, by = "id")
Keep going when one element fails
safe_read <- possibly(read_csv, otherwise = NULL) map(files, safe_read)
Capture both the result and the error
res <- map(x, safely(log))
Extract one field from every element
map_chr(records, "email") map_dbl(records, list("stats", "score"))
Errors and debugging BASEdebug
Stop with a message
stop("input must be numeric")
Warn without stopping
warning("dropping ", n, " rows")
Informational message, goes to stderr
message("processing chunk ", i)
Handle errors and warnings
tryCatch({ risky(x) }, error = function(e) { message("failed: ", conditionMessage(e)) NA }, finally = { close(con) })
Keep going on failure, check the class afterwards
res <- try(risky(x), silent = TRUE) if (inherits(res, "try-error")) res <- NA
Silence noisy output
suppressWarnings(as.numeric(x)) suppressMessages(library(dplyr))
Always run cleanup when a function exits
on.exit(close(con), add = TRUE)
Pause execution and inspect the environment
browser()
Show the call stack after an error
traceback()
Enter the debugger on every call
debug(myfun); undebug(myfun)
Time an expression
system.time(slow_function())
Visual profile of where the time goes
profvis::profvis(slow_function())
S3, S4 and R6 classes BASEobjects
S3 is just an attribute, the informal system behind most of R
obj <- structure(list(x = 1), class = "myclass")
Define a method by naming it generic.class
print.myclass <- function(x, ...) { cat("myclass with x =", x$x, "\n") }
Create your own generic
area <- function(shape, ...) UseMethod("area") area.circle <- function(shape, ...) pi * shape$r ^ 2
Fall back to the parent class
NextMethod()
List every method for a generic, and every generic for a class
methods("summary"); methods(class = "lm")
Test and set the class
inherits(obj, "myclass"); class(obj) <- "other"
S4 is the formal system, with declared slots and validation
setClass("Person", representation( name = "character", age = "numeric" ))
Create an S4 object and reach a slot
p <- new("Person", name = "Ada", age = 36) p@name
S4 methods dispatch on the signature
setGeneric("greet", function(obj) standardGeneric("greet")) setMethod("greet", "Person", function(obj) cat("Hi", obj@name))
Check the slots of an S4 class
isVirtualClass("Person"); slotNames("Person")
R6 gives mutable objects with reference semantics
Counter <- R6::R6Class("Counter", public = list( n = 0, add = function(k = 1) { self$n <- self$n + k invisible(self) } ))
R6 objects change in place, no copy on modify
c1 <- Counter$new() c1$add()$add(5) c1$n
Environments are the low level building block
e <- new.env() assign("x", 10, envir = e) get("x", envir = e)

Plotting With ggplot2 and Base R Graphics

A plot is data plus aesthetic mappings plus at least one geom. Everything else is a layer added with a plus sign.

Plot skeleton TIDYplot
The three required pieces
ggplot(df, aes(x = wt, y = mpg)) + geom_point()
Store a plot and add to it later
p <- ggplot(df, aes(wt, mpg)) p + geom_point() + geom_smooth()
Mapping versus setting, inside aes is data driven
geom_point(aes(colour = cyl)) # mapped geom_point(colour = "steelblue") # fixed
Layer specific data
geom_point(data = outliers, colour = "red")
Global versus layer aesthetics
ggplot(df, aes(x, y)) + geom_line() + geom_point(aes(size = n))
Pipe straight into a plot
df %>% filter(year == 2024) %>% ggplot(aes(region, sales)) + geom_col()
Geoms TIDYplot
Scatter plot
geom_point(size = 2, alpha = 0.7)
Reduce overplotting
geom_jitter(width = 0.2, height = 0)
Line and step charts
geom_line(); geom_step()
Bar chart of counts versus bar chart of values
geom_bar() # counts rows geom_col() # uses a y column
Stacked, grouped or filled bars
geom_col(position = "stack") geom_col(position = position_dodge(width = 0.9)) geom_col(position = "fill")
Histogram, always set the bins yourself
geom_histogram(bins = 30) geom_histogram(binwidth = 5)
Density curve
geom_density(alpha = 0.4)
Box plot and violin plot
geom_boxplot(outlier.shape = NA) geom_violin(trim = FALSE)
Trend line, method can be lm, loess or gam
geom_smooth(method = "lm", se = TRUE)
Area and ribbon for ranges
geom_area(alpha = .5) geom_ribbon(aes(ymin = lo, ymax = hi), alpha = .2)
Text labels, ggrepel avoids overlaps
geom_text(aes(label = name), vjust = -0.6) ggrepel::geom_text_repel(aes(label = name))
Heat map tiles
geom_tile(aes(fill = value))
Error bars
geom_errorbar(aes(ymin = m - se, ymax = m + se), width = 0.2)
Reference lines
geom_hline(yintercept = 0, linetype = "dashed") geom_vline(xintercept = mean(x)) geom_abline(slope = 1, intercept = 0)
Count summary inside the plot
stat_summary(fun = mean, geom = "point", size = 3)
Scales, colour and legends TIDYplot
Set limits, breaks and labels
scale_y_continuous( limits = c(0, 100), breaks = seq(0, 100, 25))
Format axis labels as currency or percent
scale_y_continuous(labels = scales::dollar) scale_y_continuous(labels = scales::percent)
Comma separated thousands
scale_y_continuous(labels = scales::comma)
Log scale
scale_x_log10()
Date axis with custom tick format
scale_x_date(date_breaks = "1 month", date_labels = "%b %Y")
Manual colours by category
scale_colour_manual(values = c( "EU" = "#1f6feb", "US" = "#0d9488"))
Colour blind safe palettes
scale_fill_viridis_d() scale_fill_viridis_c(option = "magma")
Brewer palettes
scale_fill_brewer(palette = "Set2")
Continuous gradient between two colours
scale_fill_gradient(low = "#eef3f9", high = "#1f6feb")
Rename the legend title
labs(colour = "Region")
Hide a legend
theme(legend.position = "none") guides(size = "none")
Reverse the axis or swap x and y
scale_y_reverse(); coord_flip()
Zoom without dropping data, unlike ylim
coord_cartesian(ylim = c(0, 50))
Fix the aspect ratio
coord_fixed(ratio = 1)
Facets, labels and themes TIDYplot
One panel per level of a variable
facet_wrap(~ region, ncol = 3)
Free axis ranges per panel
facet_wrap(~ region, scales = "free_y")
Grid of two variables
facet_grid(year ~ region)
All the text in one call
labs( title = "Revenue by region", subtitle = "2024 financial year", x = "Quarter", y = "Revenue (EUR)", caption = "Source: internal")
Built in themes
theme_minimal(); theme_bw(); theme_classic() theme_light(); theme_void()
Set a theme for every plot in the script
theme_set(theme_minimal(base_size = 13))
Rotate the x axis labels
theme(axis.text.x = element_text(angle = 45, hjust = 1))
Move the legend
theme(legend.position = "bottom")
Style the title
theme(plot.title = element_text(face = "bold", size = 16))
Remove grid lines
theme(panel.grid.minor = element_blank())
Saving and combining plots TIDYplot
Save the last plot to a file
ggsave("plot.png", width = 8, height = 5, dpi = 300)
Save a named plot object as a vector file
ggsave("plot.pdf", plot = p, device = cairo_pdf)
Side by side and stacked with patchwork
library(patchwork) p1 + p2 p1 / p2 (p1 | p2) / p3
Shared title across a patchwork
p1 + p2 + plot_annotation(title = "Overview")
Interactive version of any ggplot
plotly::ggplotly(p)
Base R graphics BASEbase
Generic plot, behaviour depends on the input class
plot(mtcars$wt, mtcars$mpg, pch = 19, col = "steelblue")
Histogram and box plot
hist(x, breaks = 30) boxplot(mpg ~ cyl, data = mtcars)
Bar chart from a table
barplot(table(df$region))
Add to an existing plot
lines(x, y); points(x, y); abline(lm(y ~ x), col = "red")
Several plots in one window
par(mfrow = c(2, 2))
Legend
legend("topright", legend = c("a", "b"), fill = c(1, 2))
Write directly to a file device
png("plot.png", width = 800, height = 500) plot(x, y) dev.off()
Diagnostic plots for a fitted model
plot(model)

Statistics and Modelling in R

Formula notation reads as response on the left, predictors on the right. The same y ~ x syntax drives t tests, ANOVA, linear and generalised models.

Descriptive statistics BASEstats
Centre and spread
mean(x); median(x); sd(x); var(x)
Range and extremes
min(x); max(x); range(x)
Quantiles and the interquartile range
quantile(x, probs = c(.25, .5, .75)); IQR(x)
Five number summary of any object
summary(df)
Frequency table, one or two variables
table(df$region) table(df$region, df$tier)
Proportions, 1 means rows and 2 means columns
prop.table(table(df$region, df$tier), margin = 1)
Correlation, choose the method
cor(x, y, method = "pearson") cor(x, y, method = "spearman")
Correlation matrix, dropping missing pairs
cor(df[, sapply(df, is.numeric)], use = "complete.obs")
Covariance
cov(x, y)
Standardise to mean zero and unit variance
scale(x)
Group means without dplyr
tapply(df$score, df$region, mean)
Skewness and kurtosis
e1071::skewness(x); e1071::kurtosis(x)
Distributions and sampling BASEstats
The four prefixes, r d p q
rnorm(10) # random draws dnorm(0) # density pnorm(1.96) # cumulative probability qnorm(0.975) # quantile
Normal draws with a mean and standard deviation
rnorm(1000, mean = 100, sd = 15)
Uniform, binomial and Poisson draws
runif(10, 0, 1) rbinom(10, size = 1, prob = 0.3) rpois(10, lambda = 4)
Sample from a vector
sample(1:100, size = 10, replace = FALSE)
Shuffle rows of a data frame
df[sample(nrow(df)), ]
Weighted sampling
sample(c("a", "b"), 100, TRUE, prob = c(.7, .3))
Reproducible randomness
set.seed(2024)
Simple bootstrap of the mean
boot_means <- replicate(1000, mean(sample(x, replace = TRUE))) quantile(boot_means, c(.025, .975))
Hypothesis tests BASEtests
One sample t test
t.test(x, mu = 0)
Two sample t test with the formula interface
t.test(score ~ group, data = df)
Paired t test
t.test(before, after, paired = TRUE)
Non parametric alternatives
wilcox.test(score ~ group, data = df) kruskal.test(score ~ group, data = df)
Chi squared test of independence
chisq.test(table(df$a, df$b))
Exact test for small counts
fisher.test(matrix(c(8, 2, 1, 9), nrow = 2))
Test of proportions
prop.test(x = c(45, 60), n = c(100, 120))
Normality and equal variance checks
shapiro.test(x); var.test(a, b) bartlett.test(score ~ group, data = df)
Correlation test with a confidence interval
cor.test(x, y)
One way analysis of variance
fit <- aov(score ~ group, data = df) summary(fit)
Post hoc pairwise comparison
TukeyHSD(fit)
Adjust p values for multiple testing
p.adjust(pvals, method = "BH")
Linear and generalised models BASEmodels
Fit a linear model
model <- lm(mpg ~ wt + hp, data = mtcars)
Coefficients, r squared and p values
summary(model)
Just the coefficients
coef(model)
Confidence intervals for the coefficients
confint(model, level = 0.95)
Fitted values and residuals
fitted(model); residuals(model)
Predict on new data with an interval
predict(model, newdata = new_df, interval = "prediction")
Compare two nested models
anova(model_small, model_full)
Logistic regression
glm(bought ~ age + income, data = df, family = binomial)
Poisson regression for counts
glm(visits ~ channel, data = df, family = poisson)
Convert log odds to odds ratios
exp(cbind(OR = coef(fit), confint(fit)))
Stepwise selection by AIC
step(model, direction = "both")
Model quality scores
AIC(model); BIC(model)
Diagnostic plots in a two by two grid
par(mfrow = c(2, 2)); plot(model)
Multicollinearity check
car::vif(model)
Formula syntax BASEmodels
Response on the left, predictors on the right
y ~ x1 + x2
All remaining columns as predictors
y ~ .
Remove the intercept
y ~ x - 1
Interaction only, and main effects plus interaction
y ~ x1:x2 y ~ x1 * x2
Transform inside the formula
log(y) ~ poly(x, 2) + I(x ^ 2)
Random intercept per group, mixed model
lme4::lmer(y ~ x + (1 | school), data = df)
Interaction with a factor gives one slope per level
y ~ x * factor(group)
Tidy model output and machine learning BASETIDYmodels
Coefficient table as a data frame
broom::tidy(model, conf.int = TRUE)
One row of model level statistics
broom::glance(model)
Original data plus fitted values and residuals
broom::augment(model)
K means clustering
km <- kmeans(scale(df), centers = 3, nstart = 25) km$cluster
Principal components
pca <- prcomp(df, scale. = TRUE) summary(pca)
Hierarchical clustering
hc <- hclust(dist(scale(df)), method = "ward.D2") cutree(hc, k = 3)
Train and test split
idx <- sample(nrow(df), 0.8 * nrow(df)) train <- df[idx, ]; test <- df[-idx, ]
Random forest
randomForest::randomForest(y ~ ., data = train)
Gradient boosting on a matrix
xgboost::xgboost(data = X, label = y, nrounds = 100)
Cross validated workflow with tidymodels
library(tidymodels) folds <- vfold_cv(train, v = 5) fit_resamples(workflow, folds)
Survival analysis
survival::survfit(Surv(time, status) ~ group, data = df)
Time series decomposition and forecast
ts_data <- ts(x, frequency = 12) forecast::forecast(auto.arima(ts_data), h = 12)

data.table Syntax Reference

Read DT[i, j, by] as take rows i, compute j, grouped by. Updates with := change the table in place, without copying it.

Creating and reading DTdt
Load the package
library(data.table)
Create a data table
DT <- data.table(id = 1:3, x = c("a", "b", "c"))
Convert an existing data frame, in place and free
setDT(df)
Convert with a copy
DT <- as.data.table(df)
Fast file reader
DT <- fread("big.csv")
Fast file writer
fwrite(DT, "out.csv")
Check the class
class(DT)
Filter, select, compute DTdt
Filter rows, no need to repeat the table name
DT[score > 80]
Several conditions
DT[score > 80 & region == "EU"]
First rows
DT[1:5]
Select columns, dot is shorthand for list
DT[, .(id, score)]
Select using a character vector
DT[, cols, with = FALSE]
Compute a summary in j
DT[, .(avg = mean(score), n = .N)]
Filter and compute together
DT[region == "EU", .(avg = mean(score))]
Number of rows in the current group
DT[, .N, by = region]
Sort
DT[order(region, -score)]
Chain operations
DT[score > 50][, .(n = .N), by = region][order(-n)]
Update by reference DTdt
Add a column without copying the table
DT[, ratio := wins / games]
Add several columns at once
DT[, c("a", "b") := .(x * 2, y / 2)]
Functional form, allows spaces in names
DT[, `:=`(total = x + y, flag = x > 0)]
Update only some rows
DT[score < 0, score := 0]
Delete a column
DT[, c("tmp1", "tmp2") := NULL]
Group wise update
DT[, share := amount / sum(amount), by = region]
Rename columns in place
setnames(DT, "old", "new")
Reorder columns in place
setcolorder(DT, c("id", "region"))
Sort in place, no copy
setorder(DT, region, -score)
Take an explicit copy when you need one
DT2 <- copy(DT)
Grouping and special symbols DTdt
Group by one or several columns
DT[, .(total = sum(amount)), by = .(region, year)]
Group by an expression
DT[, .N, by = .(decade = floor(year / 10) * 10)]
keyby groups and sorts the result
DT[, .N, keyby = region]
.N is the row count, .I the row numbers
DT[, .I[which.max(score)], by = region]
.SD is the subset of data for each group
DT[, lapply(.SD, mean), by = region]
Restrict .SD to chosen columns
DT[, lapply(.SD, mean), by = region, .SDcols = c("score", "amount")]
First two rows of every group
DT[, head(.SD, 2), by = region]
Group counter and group name
DT[, .GRP, by = region]
Count distinct values fast
uniqueN(DT$user_id)
Remove duplicate rows
unique(DT, by = c("id", "date"))
Keys, joins and reshaping DTdt
Set a key, enables fast binary search
setkey(DT, id)
Right join, X rows for every Y row
DT_a[DT_b, on = "id"]
Inner join
DT_a[DT_b, on = "id", nomatch = 0]
Anti join
DT_a[!DT_b, on = "id"]
Join with different column names
DT_a[DT_b, on = c(user_id = "id")]
Update columns during a join
DT_a[DT_b, on = "id", name := i.name]
Rolling join, nearest earlier match in time
prices[trades, on = .(sym, ts), roll = TRUE]
Wide to long
melt(DT, id.vars = "id", measure.vars = c("q1", "q2"), variable.name = "quarter", value.name = "rev")
Long to wide
dcast(DT, id ~ quarter, value.var = "rev")
Aggregate while casting
dcast(DT, region ~ year, value.var = "amount", fun.aggregate = sum)
Handy data.table helpers DTdt
Fast vectorised if, type strict
fifelse(x > 0, "pos", "neg")
Multi condition version of case_when
fcase( score >= 90, "A", score >= 80, "B", default = "C")
Lag and lead
shift(x, n = 1, type = "lag") shift(x, n = 1, type = "lead")
Run length group id, for sessionising
rleid(status)
Between, with an index if a key is set
DT[amount %between% c(10, 100)]
Value matching with several options
DT[region %chin% c("EU", "US")]
Fast row bind of a list of tables
rbindlist(list_of_dts, fill = TRUE)
Control the number of threads
setDTthreads(4)

R Workflow: Projects, Reporting, Testing and Performance

Reproducibility beats cleverness. Projects, pinned package versions and scripted output make analyses that still run next year.

RStudio shortcuts BASEeditor
Run the current line or selection
Ctrl + Enter (Cmd + Enter on macOS)
Run the whole script
Ctrl + Shift + Enter
Insert the assignment arrow
Alt + -
Insert the pipe operator
Ctrl + Shift + M
Comment or uncomment the selection
Ctrl + Shift + C
Restart the R session, the real fix for weird state
Ctrl + Shift + F10
Reformat and reindent the selection
Ctrl + Shift + A
Jump to a file or function by name
Ctrl + .
Insert a code chunk in R Markdown or Quarto
Ctrl + Alt + I
Render the document
Ctrl + Shift + K
Clear the console
Ctrl + L
Show every shortcut
Alt + Shift + K
Projects and reproducibility BASETIDYsetup
Create a project folder with usethis
usethis::create_project("~/work/analysis")
Paths relative to the project root
here::here("data", "raw.csv")
Start a private package library for the project
renv::init()
Record the exact package versions in use
renv::snapshot()
Restore that library on another machine
renv::restore()
Never save or restore the workspace between sessions
usethis::use_blank_slate()
Store secrets outside the script
usethis::edit_r_environ() Sys.getenv("API_KEY")
Set up git for the project
usethis::use_git(); usethis::use_github()
R Markdown and Quarto BASEreport
Document header
--- title: "Quarterly report" format: html execute: echo: false ---
A code chunk with options
```{r} #| label: fig-sales #| fig-width: 8 #| warning: false ggplot(df, aes(x, y)) + geom_line() ```
Defaults for every chunk in the document
knitr::opts_chunk$set(echo = FALSE, message = FALSE)
Inline value inside a sentence
Total revenue was `r format(total, big.mark = ",")`.
Render from the console
rmarkdown::render("report.Rmd") quarto::quarto_render("report.qmd")
Render with parameters
rmarkdown::render("report.Rmd", params = list(region = "EU"))
Print a table that looks right in the output
knitr::kable(head(df)) gt::gt(df)
Testing, style and packages BASETIDYquality
Write a test
testthat::test_that("mean works", { expect_equal(mean(c(1, 3)), 2) })
Common expectations
expect_equal(a, b); expect_true(x) expect_error(f()); expect_length(x, 3)
Run the whole test suite
devtools::test()
Reformat code to a consistent style
styler::style_file("script.R")
Static checks for common mistakes
lintr::lint("script.R")
Start a package skeleton
usethis::create_package("~/mypkg")
Load, document and check during development
devtools::load_all() devtools::document() devtools::check()
Roxygen comment block above a function
#' Add two numbers #' #' @param x numeric #' @param y numeric #' @return numeric #' @export
Performance BASETIDYDTspeed
Time a single expression
system.time(expr)
Compare implementations properly
bench::mark(base = f1(x), tidy = f2(x))
Find the slow line visually
profvis::profvis(pipeline())
Preallocate instead of growing a vector
out <- numeric(n) for (i in seq_len(n)) out[i] <- f(i)
Prefer vectorised code over loops
x * 2 # fast for (i in ...) ... # slower
Use vapply for a known output type
vapply(x, f, numeric(1))
Parallel map across cores
future::plan(multisession) furrr::future_map(files, read_csv)
Read only the columns you need from a big file
fread("big.csv", select = cols)
Cache an expensive step to disk
if (!file.exists("cache.rds")) { saveRDS(expensive(), "cache.rds") } res <- readRDS("cache.rds")
Running R from the command line BASEcli
Run a script, the standard way to schedule R
Rscript analysis.R
Run a one liner
Rscript -e 'print(sessionInfo())'
Pass arguments to a script
Rscript report.R 2024 EU
Read those arguments inside the script
args <- commandArgs(trailingOnly = TRUE) year <- as.integer(args[1])
Proper flag parsing
library(optparse) opts <- parse_args(OptionParser(option_list = list( make_option("--year", type = "integer") )))
Exit with a status code so the scheduler notices failures
if (nrow(result) == 0) quit(status = 1)
Write progress to stderr, keep stdout for data
message("done") # stderr cat("value\n") # stdout
Fail the whole script on any warning, useful in production
options(warn = 2)
Run an R script from a cron job
0 6 * * 1 cd /srv/etl && /usr/bin/Rscript load.R >> log.txt 2>&1
Interactive session in the terminal, and batch mode
R R CMD BATCH script.R
Build and check a package from the shell
R CMD build mypkg R CMD check mypkg_0.1.0.tar.gz
Detect whether code is running interactively
if (interactive()) browser()

Time Series, Spatial Data and Shiny Apps in R

Three specialist areas with their own object types. Each one has a dominant package worth learning before anything else.

Time series objects BASETIDYtime
Build a regular time series, frequency 12 means monthly
ts_data <- ts(x, start = c(2020, 1), frequency = 12)
Inspect the span and frequency
start(ts_data); end(ts_data); frequency(ts_data)
Take a slice by date
window(ts_data, start = c(2022, 1), end = c(2023, 12))
Split into trend, seasonal and remainder
decompose(ts_data) stl(ts_data, s.window = "periodic")
Irregular series with real timestamps
xts::xts(values, order.by = timestamps)
Aggregate to a coarser frequency
xts::apply.monthly(x, mean) xts::to.period(x, period = "months")
Tidy time series, a data frame with an index column
tsibble::as_tsibble(df, index = date, key = region)
Rolling window calculations
zoo::rollmean(x, k = 7, fill = NA, align = "right") slider::slide_dbl(x, mean, .before = 6)
Time series analysis BASETIDYtime
Autocorrelation and partial autocorrelation plots
acf(ts_data); pacf(ts_data)
Stationarity test
tseries::adf.test(ts_data)
Difference the series to remove a trend
diff(ts_data, lag = 1) diff(ts_data, lag = 12) # seasonal
Fit an ARIMA model automatically
fit <- forecast::auto.arima(ts_data) summary(fit)
Forecast ahead with prediction intervals
fc <- forecast::forecast(fit, h = 12) plot(fc)
Exponential smoothing
forecast::ets(ts_data) HoltWinters(ts_data)
Accuracy against a holdout
forecast::accuracy(fc, test_set)
Modern tidy forecasting workflow
library(fable) data %>% model(arima = ARIMA(value)) %>% forecast(h = "1 year")
Anomaly and changepoint detection
changepoint::cpt.mean(x)
Spatial data with sf TIDYspatial
Read a shapefile or GeoPackage
library(sf) shp <- st_read("regions.gpkg")
Turn a data frame of coordinates into spatial points
pts <- st_as_sf(df, coords = c("lon", "lat"), crs = 4326)
Check and set the coordinate reference system
st_crs(shp) shp <- st_transform(shp, crs = 3857)
Geometry measurements
st_area(shp); st_length(lines); st_centroid(shp)
Spatial predicates
st_intersects(a, b); st_within(pts, shp) st_contains(shp, pts)
Spatial join, attach polygon attributes to points
st_join(pts, shp, join = st_within)
Buffer, union and clip
st_buffer(pts, dist = 500) st_union(shp) st_intersection(a, b)
sf objects are data frames, so dplyr just works
shp %>% filter(population > 1e6) %>% mutate(density = population / as.numeric(st_area(.)))
Plot a choropleth with ggplot2
ggplot(shp) + geom_sf(aes(fill = density)) + scale_fill_viridis_c() + theme_void()
Interactive map in two lines
library(leaflet) leaflet(pts) %>% addTiles() %>% addCircleMarkers()
Write the result back out
st_write(shp, "out.geojson", delete_dsn = TRUE)
Shiny apps BASEshiny
The smallest complete app
library(shiny) ui <- fluidPage( sliderInput("n", "Sample size", 10, 500, 100), plotOutput("hist") ) server <- function(input, output, session) { output$hist <- renderPlot(hist(rnorm(input$n))) } shinyApp(ui, server)
Run an app from a folder or a file
runApp("app/") runApp("app.R", launch.browser = TRUE)
Common inputs
textInput("name", "Name") numericInput("k", "K", value = 3) selectInput("region", "Region", choices = regions) dateRangeInput("period", "Period") checkboxInput("log", "Log scale") actionButton("go", "Run")
Common outputs and their render functions
plotOutput("p") <- renderPlot() tableOutput("t") <- renderTable() DT::DTOutput("dt") <- DT::renderDT() textOutput("msg") <- renderText() verbatimTextOutput("log") <- renderPrint()
Cache an expensive computation and reuse it in several outputs
filtered <- reactive({ df %>% filter(region == input$region) })
Only recompute when the button is pressed
result <- eventReactive(input$go, { slow_model(input$k) })
Run a side effect, not a value
observeEvent(input$reset, { updateSliderInput(session, "n", value = 100) })
Mutable state shared across observers
state <- reactiveValues(count = 0)
Layout with a sidebar or a dashboard
sidebarLayout(sidebarPanel(...), mainPanel(...)) bslib::page_sidebar(...)
Validate input before the output errors out
req(input$file) validate(need(nrow(df) > 0, "No rows match"))
Show progress during a long task
withProgress(message = "Fitting", { incProgress(0.5) })
Deploy to shinyapps.io or a server
rsconnect::deployApp("app/")

Common R Gotchas and Silent Bugs

The bugs that cost the most time are silent ones. Every entry here returns a plausible wrong answer rather than an error.

Numbers and precision BASEtraps
Floating point equality fails, and this is not an R bug
0.1 + 0.2 == 0.3
Compare with a tolerance instead
isTRUE(all.equal(0.1 + 0.2, 0.3))
Or test the absolute difference
abs((0.1 + 0.2) - 0.3) < 1e-8
R prints seven significant digits by default, the value keeps more
print(1/3, digits = 16)
as.integer truncates towards zero, it does not round
as.integer(-2.7); round(-2.7)
Banker's rounding, halves go to the even digit
round(0.5); round(1.5); round(2.5)
Large numbers switch to scientific notation in output
options(scipen = 999) # turn that off
Integer overflow returns NA with a warning, doubles do not overflow
.Machine$integer.max + 1L
seq with a decimal step can miss the endpoint
seq(0, 1, by = 0.1)[8] == 0.7
Silent type surprises BASEtraps
Single bracket subsetting drops to a vector without warning
class(df[, 1]); class(df[, 1, drop = FALSE])
Factor to numeric returns the level codes, not the labels
as.numeric(factor(c("10", "20")))
c() coerces everything to the most permissive type
c(1, TRUE, "a")
sapply changes its return type depending on the data, use vapply
sapply(list(), length) # returns list() vapply(list(), length, integer(1)) # always integer
Partial matching on list names, a data frame quirk that tibbles remove
l <- list(value = 1) l$val
Comparing against a set with == recycles instead of matching
x == c("a", "b") # wrong x %in% c("a", "b") # right
NA in a condition returns NA, not FALSE, and filtering breaks
c(1, NA, 3)[c(1, NA, 3) > 2]
ifelse drops attributes, including the Date class
class(ifelse(TRUE, Sys.Date(), Sys.Date()))
Empty vectors make sum and prod return their identity element
sum(integer(0)); prod(integer(0))
max on an empty vector warns and returns negative infinity
max(numeric(0))
Structure and scope traps BASETIDYDTtraps
1:0 counts down, so an empty loop runs twice, use seq_len
1:0; seq_len(0)
Growing a vector inside a loop reallocates every iteration
out <- c() # slow out <- numeric(n) # preallocate
Recycling silently pads the shorter vector
c(1, 2, 3, 4) + c(10, 20)
A data frame stays grouped after group_by until you release it
df %>% group_by(a) %>% mutate(...) %>% ungroup()
data.table := changes the original object, even a copy of the name
DT2 <- DT # same object DT2 <- copy(DT) # real copy
Arguments are evaluated lazily, only when first used
f <- function(x) 10 f(stop("never runs"))
<<- writes to the enclosing scope, which is rarely what you want
x <<- 5
stringsAsFactors was TRUE before R 4.0, old scripts assume it
data.frame(x = "a")$x # character since R 4.0
rm(list = ls()) does not reset packages, options or the seed
# restart the session instead: Ctrl + Shift + F10
setwd at the top of a script breaks on every other machine
here::here("data", "raw.csv")
T and F are variables that can be reassigned, TRUE and FALSE cannot
T <- FALSE # legal and dangerous
Trailing commas and missing arguments produce confusing errors
df[, 1] # column df[1, ] # row

Saved R Snippets

What R Is and Who Uses It

R is a language built for data. It was created in 1993 by Ross Ihaka and Robert Gentleman at the University of Auckland as a free implementation of the earlier S language, and it is now maintained by the R Core Team under the GNU General Public License.

Where general purpose languages treat statistics as a library, R treats it as the core. Vectors, missing values, factors, data frames and model formulas are part of the language itself, not bolted on afterwards.

The people who reach for it tend to be statisticians, epidemiologists, econometricians, bioinformaticians, survey researchers, quantitative analysts and data journalists. If you also write scripts in other languages, our Python cheat sheet covers the closest alternative for the same kind of work. If your work ends in a model, a chart or a report rather than a production service, R usually gets you there with less code. So, an R cheat sheet was much needed.

The package ecosystem is the other half of the story. CRAN hosts more than twenty thousand packages, Bioconductor adds thousands more for genomics, and the tidyverse provides a consistent interface for the everyday work of loading, cleaning, reshaping and plotting data.


Installing R and RStudio

Two separate installs, in this order.

First, download R itself from CRAN. Pick the binary for your operating system and accept the defaults. This gives you the language and the plain console.

Second, install an editor. RStudio Desktop from Posit is the standard choice, with a script pane, console, environment browser, plot viewer and package manager in one window. Positron and VS Code with the R extension are reasonable alternatives.

Check the install by opening the console and running a version check.

R.version.string
#> [1] "R version 4.x.y (yyyy-mm-dd)"

Then install the packages you will use most.

install.packages(c("tidyverse", "data.table", "here", "janitor"))

One habit worth adopting on day one: turn off workspace saving. In RStudio, go to Tools, then Global Options, and uncheck restore .RData at startup. Every script should be able to produce its results from a clean session.

R Syntax Basics

R is case sensitive, indexes from 1 rather than 0, and uses an arrow for assignment.

Assignment and objects

x <- 5          # the conventional form
y = 5           # legal, but keep = for function arguments
5 -> z          # right assignment, occasionally useful

Everything in R is an object, and every object has a class. Use class() for the high level type and str() for a compact view of the internals. str() is probably the single most useful function for exploring an unfamiliar object.

Vectors and vectorisation

R has no scalar type. A single number is a vector of length one, which is why arithmetic works on whole columns without a loop.

prices <- c(10, 20, 30)
prices * 1.19
#> [1] 11.9 23.8 35.7

When two vectors have different lengths, the shorter one is recycled. That behaviour is convenient and occasionally dangerous, so pay attention when a warning about object lengths appears.

Missing values

Missing data is a first class citizen. NA propagates through calculations on purpose, which is why mean(x) returns NA when any value is missing. Add na.rm = TRUE once you have decided that dropping those rows is correct.

Note the distinction: NA means a value exists but is unknown, NULL means there is no value at all, NaN is an undefined numeric result, and Inf is the result of dividing by zero.


R Data Structures

Four containers cover almost everything.

StructureDimensionsContentsTypical use
vector1One type onlyA single column or variable
matrix2One type onlyLinear algebra, correlation grids
list1Anything, including other listsModel objects, JSON, mixed results
data.frame2One type per columnAlmost all analysis work

A data frame is really a list of equal length vectors with a shared row index. That is why df$column returns a vector and why every column can hold a different type while each column stays internally consistent.

Tibbles versus data frames

A tibble is a data frame with the sharp edges filed off. It prints ten rows instead of flooding the console, never converts strings to factors, never does partial column name matching, and always returns a tibble when you subset it rather than silently dropping to a vector.

Anything that accepts a data frame accepts a tibble, so the switch costs nothing.

Factors

Factors store categorical data as integer codes with a lookup table of labels. Models need them, and so do ordered axes in plots.

The classic trap is converting a factor of numbers straight to numeric, which returns the internal codes rather than the labels. Always go through character first.

f <- factor(c("10", "20", "30"))
as.numeric(f)                   # 1 2 3, the codes
as.numeric(as.character(f))     # 10 20 30, correct

Importing and Exporting Data

Base R can read most formats, but the modern readers are faster and make fewer assumptions.

  • CSV and delimited text: read.csv() in base R, readr::read_csv() for speed and a tibble, data.table::fread() when the file is large enough that you notice the wait.
  • Excel: readxl::read_excel() for reading, writexl::write_xlsx() for writing. Neither needs Java.
  • Statistical formats: the haven package handles SPSS, Stata and SAS files and keeps the variable labels.
  • Databases: DBI plus a driver package gives you a connection, and dbplyr lets you write dplyr code that is translated into SQL and executed on the server. Keep our SQL cheat sheet open when you need to read the query it generates.
  • R native: saveRDS() for a single object, save() for several. RDS is the better default because you choose the name when you read it back.
  • JSON and other exchange formats: jsonlite handles the parsing, and for one off conversions outside R our CSV to JSON converter and JSON to YAML converter are quicker than writing a script.

Two habits prevent most import pain. Set col_types explicitly once you know the schema, so a column of postal codes is never guessed as a number. And use project relative paths through here::here() so the script runs on a colleague's machine without editing.


Data Wrangling With dplyr

The dplyr grammar is small on purpose. Six verbs handle the bulk of transformation work, and they all take a data frame first, which makes them pipe friendly.

VerbWhat it doesBase R equivalent
filter()Keeps rows matching a conditiondf[df$x > 5, ]
select()Keeps or drops columnsdf[, c("a", "b")]
mutate()Adds or changes columnsdf$new <- ...
arrange()Sorts rowsdf[order(df$x), ]
summarise()Collapses to one row per groupaggregate()
group_by()Splits the frame for the verb that followstapply(), split()

A typical pipeline reads top to bottom like a sentence.

library(dplyr)

report <- sales |>
  filter(year == 2024, !is.na(amount)) |>
  mutate(net = amount - discount) |>
  summarise(
    revenue = sum(net),
    orders  = n(),
    .by     = region
  ) |>
  arrange(desc(revenue))

Two modern additions are worth knowing. The .by argument groups for a single operation and releases the grouping afterwards, which avoids the classic bug of a grouped data frame quietly following you through the rest of the script. And across() applies one function to many columns without repeating yourself.

Joins

Joins follow SQL naming. left_join() keeps every row on the left, inner_join() keeps only matches, full_join() keeps everything, and the filtering joins semi_join() and anti_join() use the second table as a lookup without adding any of its columns.

Check row counts before and after every join. An unexpected increase means the key is not unique on one side, which is the most common silent error in data work.


Reshaping With tidyr

Tidy data has one variable per column and one observation per row. Most spreadsheets arrive in a different shape, with months or years spread across the header row.

pivot_longer() gathers those columns into two: one holding the former column names, one holding the values.

long <- wide |>
  pivot_longer(
    cols      = q1:q4,
    names_to  = "quarter",
    values_to = "revenue"
  )

pivot_wider() does the reverse, which is what you usually want at the very end when producing a table for a report.

Text columns usually need cleaning at the same time. R uses POSIX and Perl compatible expressions, so the patterns in our regex cheat sheet transfer directly, with the caveat that every backslash has to be doubled inside an R string.

The rest of tidyr fills gaps: drop_na() and replace_na() for missing values, fill() for carrying values down a column, complete() for making implicit missing combinations explicit, and separate_wider_delim() for splitting a compound column into parts.


Plotting With ggplot2

ggplot2 implements the grammar of graphics. A plot is data, a mapping from variables to visual properties, and one or more geometric layers. Everything else is optional refinement.

library(ggplot2)

ggplot(mtcars, aes(x = wt, y = mpg, colour = factor(cyl))) +
  geom_point(size = 2.5, alpha = 0.85) +
  geom_smooth(method = "lm", se = FALSE) +
  scale_colour_manual(values = c("#1f6feb", "#0d9488", "#b45c09")) +
  labs(
    title  = "Fuel economy falls as weight rises",
    x      = "Weight (1000 lbs)",
    y      = "Miles per gallon",
    colour = "Cylinders"
  ) +
  theme_minimal(base_size = 13)

The distinction that trips people up early: anything inside aes() is mapped from a column, anything outside it is a fixed value. Writing colour = "blue" inside aes() creates a single category literally named blue rather than a blue plot.

facet_wrap() is the highest value function in the package. One line turns a crowded chart into a small multiple grid, and comparisons that were impossible become obvious.

Save with ggsave() rather than the export button, so the dimensions are recorded in the script and the figure regenerates identically next time.


Statistics and Modelling

This is where R still has no real competition. Formula notation is consistent across the entire modelling ecosystem: the response goes on the left of the tilde, predictors on the right.

model <- lm(mpg ~ wt + hp, data = mtcars)
summary(model)
confint(model)
predict(model, newdata = new_cars, interval = "prediction")

The same syntax drives t.test(score ~ group), aov(), glm() for logistic and Poisson regression, and mixed models through lme4.

Model objects are lists, so you can pull out coefficients, residuals and fitted values directly. When you need results as a data frame instead of printed output, the broom package converts any model into a tidy table with tidy(), glance() and augment(). That is what makes it practical to fit hundreds of models across groups and compare them in a single table.


data.table for Large Datasets

data.table is the other major dialect. It uses a single bracket form, DT[i, j, by], which reads as take these rows, compute this, grouped by that.

library(data.table)

DT <- fread("transactions.csv")
DT[amount > 0, .(total = sum(amount), n = .N), by = .(region, month)]

Its defining feature is modification by reference. The := operator adds or updates a column without copying the table, which is why it stays fast on data measured in gigabytes where a copy would exhaust memory.

The trade off is readability. dplyr code is easier to hand to a colleague, data.table code is faster and more compact. Many teams use dplyr for exploration and data.table where speed matters, and the two coexist in the same script without conflict.


Common R Mistakes

  • Growing objects inside loops. Appending to a vector reallocates memory every iteration. Preallocate with vector("numeric", n) or use an apply function.
  • Forgetting na.rm. A single missing value turns any summary into NA. Decide deliberately whether to exclude those rows.
  • Comparing with == when you mean %in%. The equality operator recycles and gives the wrong answer against a multi value set.
  • Leaving a data frame grouped. Every verb after group_by() stays grouped until you call ungroup(). Use .by where you can.
  • Converting factors to numbers directly. Go through as.character() first or you get the level codes.
  • Floating point equality. 0.1 + 0.2 == 0.3 is FALSE. Use all.equal() or abs(a - b) < 1e-8.
  • setwd() at the top of a script. It breaks on every other machine. Use an RStudio project and here::here().
  • rm(list = ls()) as a reset. It leaves attached packages and options behind. Restart the session instead.

Making an Analysis Reproducible

A script that only runs on your laptop is a draft, not a deliverable. Four habits cover most of the gap.

  • Use a project, not a working directory. An .Rproj file plus here::here() means paths resolve the same way for everyone.
  • Pin your packages. renv::snapshot() writes a lockfile recording every version, and renv::restore() rebuilds that library elsewhere.
  • Version the code. Analyses change constantly and diffs are the only reliable record of why. Our Git cheat sheet covers the commands worth memorising.
  • Freeze the environment when it matters. For long lived pipelines, a container pins the R version and system libraries too. The rocker images are the standard base, and our Docker cheat sheet handles the rest.

Reports belong in the same repository as the code that produces them. R Markdown and Quarto both use Markdown for the prose around your chunks, so the formatting syntax is the same one you already use in README files.


Where to Go Next

R for Data Science is the standard free introduction to the tidyverse workflow. Advanced R covers environments, object systems and metaprogramming once the basics feel comfortable.

For reference while you work, the official R manuals remain the authority on language semantics, and Posit publishes printable one page guides for individual packages.

The fastest way to improve, though, is to take a dataset you actually care about and push it all the way through: import, clean, reshape, plot, model, report. Every function on this page exists because someone hit that exact problem first.