Trying an IMAP client requires a mail server and a populated mailbox — which usually means real credentials, a provider’s rate limits, and results that cannot be reproduced anywhere else. This vignette shows how to run mRpostman against a disposable local IMAP server (Dovecot, in a Docker container) whose mailbox is populated deterministically by the package itself: populate_sandbox() uploads a fixed synthetic corpus with the package’s own APPEND implementation, so any user, on any machine, obtains exactly the same messages — and therefore exactly the same search, fetch, and decoding results shown here.
This is useful for:
mRpostman against a fast, offline, rate-limit-free server.None of the code chunks in this vignette are evaluated when the package is built (there is no Docker server on CRAN’s machines); copy and paste them into your session.
NOTE: the sandbox server is for local experimentation only — it uses plaintext authentication, no TLS, and a static password.
You will need Docker. The Dockerfile and the Dovecot configuration ship with the package; find them with:
system.file("docker", package = "mRpostman")From that folder, build the image and start a container, exposing the IMAP port as 1430 on the host:
docker build -t mrpostman-sandbox .
docker run -d --name mrpostman-sandbox -p 1430:143 mrpostman-sandboxThe server accepts any username with the password sandbox, and each username gets its own empty mailbox.
library(mRpostman)
con <- configure_imap(url = "imap://localhost:1430",
username = "testuser",
password = "sandbox",
use_ssl = FALSE)
con$list_server_capabilities()Note the use_ssl = FALSE and the imap:// (not imaps://) scheme: the sandbox has no TLS. Dovecot advertises, among others, the ESEARCH, SORT, THREAD, MOVE, UNSELECT, and SPECIAL-USE capabilities, so the package’s extension-based methods can be exercised too.
populate_sandbox() generates the synthetic corpus of sandbox_corpus() — 200 RFC 822 messages by default, driven by a fixed RNG seed — and stores it using the package’s own IMAP operations (APPEND, CREATE, STORE):
info <- populate_sandbox(con, n = 200)
#> Appending 200 synthetic messages to INBOX at imap://localhost:1430 ...
#> 50/200
#> 100/200
#> 150/200
#> 200/200
#>
#> Done. INBOX now reports:
#> $EXISTS
#> [1] 200
#> ...The returned (invisible) info data.frame describes what was embedded in each message, so every result below can be checked against it:
head(info)
#> id from subject ...
#> 1 1 felipe.duarte@example.com Meeting minutes ...The corpus is designed so that each feature of the package has matching messages to act upon:
Date: headers spread over the year 2020, for the SENT* date searches;info$attachment_type;In-Reply-To/References), for SORT and THREAD;\Seen, \Flagged, and \Answered flags, adjusted after the upload, for flag searches. (libcurl’s APPEND stores every message with the \Seen flag, so populate_sandbox() removes it from the messages planned as unread — exercising STORE in both directions.)For use from the shell, the wrapper script inst/docker/populate_mailbox.R does the same: Rscript populate_mailbox.R 200.
APPEND stores messages with the current internal date. Searches on the internal date (search_before(), search_since(), search_on(), search_older_than(), search_younger_than()) therefore relate to the moment you populated the mailbox. Searches on the Date: header (search_sent_before(), search_sent_since(), search_sent_on(), search_sent_period()) act on the fixed 2020 dates of the corpus and always return the same results.
The synthetic corpus is not the only way to fill the sandbox. ingest_maildir() uploads any local maildir-style directory (one RFC 5322 message per file) to a server folder via APPEND — mail server backups, exported archives, or public corpora:
manifest <- ingest_maildir(con, path = "~/backup/maildir/archive2020",
folder = "archive2020")
table(manifest$appended)Built on top of it, enron_sandbox() turns the sandbox into a real-data laboratory: it downloads the public Enron e-mail corpus once (~423 MB, with your consent, cached under tools::R_user_dir("mRpostman", "cache")), selects a subset by custodian, folder-name pattern, and Date: header window, and ingests it — one server folder per custodian:
con2 <- configure_imap(url = "imap://localhost:1430",
username = "enron", password = "sandbox",
use_ssl = FALSE)
manifest <- enron_sandbox(con2,
custodians = c("kaminski-v", "lay-k", "skilling-j"),
sent_since = "2000-01-01",
sent_before = "2002-07-01")
table(manifest$custodian, manifest$appended)After the ingestion, everything shown in the guided tour below works on real data too — e.g. con2$search_string(expr = "SEC", where = "TEXT") on a custodian’s folder. The same internal-date caveat applies: query ingested corpora by date through the SENT* searches, which read the Date: header.
con$list_mail_folders()
con$examine_folder("INBOX")
con$select_folder("INBOX")Besides INBOX, populate_sandbox() creates a ProjectAtlas folder with a small subset of the corpus, for folder, copy_msg(), and move_msg() demonstrations.
Single-criterion searches:
# messages "sent" in the first quarter of 2020 (Date: header)
ids_q1 <- con$search_sent_period(since_date_char = "01-Jan-2020",
before_date_char = "01-Apr-2020")
# large messages
ids_large <- con$search_larger_than(size = 8000)
# flagged messages
ids_flagged <- con$search_flag("FLAGGED")Composable custom searches:
ids <- con$search(request = OR(string(expr = "sales", where = "SUBJECT"),
string(expr = "alice.fontes@example.com",
where = "FROM")))The ESEARCH extension can condense results server-side:
con$esearch_count(flag = "SEEN")
con$esearch_min_id(flag = "FLAGGED")
meta <- ids_q1[1:5] %>%
con$fetch_metadata(attribute = c("INTERNALDATE", "RFC822.SIZE"))
texts <- ids_q1[1:5] %>%
con$fetch_text()The corpus includes quoted-printable bodies and MIME encoded-word subjects with accented characters — fetch one of the messages flagged as is_utf8_body in info and decode it with clean_msg_text():
utf8_ids <- info$id[info$is_utf8_body]
raw_text <- con$fetch_text(msg_id = utf8_ids[1])
clean_msg_text(raw_text)
#> [[1]]
#> [1] "Segue em anexo o relatório com os números consolidados. ..."
att_ids <- info$id[info$has_attachment]
# list attachment filenames without downloading
con$fetch_attachments_list(msg_id = att_ids[1:5])
# download them (note the automatic deduplication of repeated filenames)
con$fetch_attachments(msg_id = att_ids[1:5])Since the server accepts any username — and each one gets its own independent mailbox — the quickest way to start over is to simply connect with a new username, with no need to touch the container:
con2 <- configure_imap(url = "imap://localhost:1430",
username = "testuser2", # any new name = pristine mailbox
password = "sandbox",
use_ssl = FALSE)
populate_sandbox(con2, n = 200)This is also handy for keeping several corpora side by side (e.g. one mailbox per experiment or per seed).
Alternatively, the mailboxes live inside the container, so recreating it resets everything at once:
docker rm -f mrpostman-sandbox
docker run -d --name mrpostman-sandbox -p 1430:143 mrpostman-sandboxEither way, a new populate_sandbox() call recreates the exact same corpus.