Main features
- added RFC 2047 quoted-printable and base64 MIME header decoder
NEWS.md
decode_quoted_printable_header(): changed the
conversion engine to iconv and added
useBytes = TRUE to the gsub() call; and added
support to ISO-8859-10 and Windows-1252
(Nordic/latin6), and ISO-8859-9 and
Windows-1254 (Turkish) encodings.
decode_quoted_printable_text(): changed the
conversion engine to iconv and added
useBytes = TRUE to the gsub() call; and added
support to ISO-8859-10 and Windows-1252
(Nordic/latin6), and ISO-8859-9 and
Windows-1254 (Turkish) encodings.
zzz() - removed message from the transition to
R6 structure and added the citation request
instead.
decode_quoted_printable_header()
and decode_quoted_printable_text() (needed to rewind to
v1.1.2 before moving again to v1.1.4)clean_fetch_results(): added argument
useBytes = TRUE to all gsub() calls. According
to Kurt Hornik, “fetching fails using current versions of R for some
contents with non-ASCII characters”.
Added bypass argument as_is to methods/functions
get_attachments(),
execute_fetch_attachments(),
fetch_attachments(); and as_is functionality
to get_attachments() and
execute_fetch_attachments as proposed by Paul Smith to
handle non-base64 files.
list_attachments()
README.Rmd
./allanvcq@gmail.com/INBOX/UID1430/ instead of
“imap.gmail.com/INBOX/UID1430”. This will allow users to better
differentiate between local folders, when using different mailboxes in
the same server.added the clean_msg_text() function. It will be
particularly useful for cleaning and preparing message text retrieved
with fetch_body(..., mime_level = 1). These texts can be
used for sentiment analysis and other text mining tasks;
added internal functions decode_mime_text() and
decode_quoted_printable_text(). The second is called by the
first one;
renamed the original decode_quoted_printable called
from decode_mime_header() to
decode_quoted_printable_text();
included the xml2 and the rvest
packages as dependencies because of the addition of the
clean_msg_text().
mime_level argument to the
fetch_body() method. Now, the user can choose whether the
fetch will retrieve the full body content or an specific one. This is
particularly helpful for retrieving clean text parts without inline and
regular attachments for instance.adapts to the new return pattern of MS Exchange IMAP servers without losing the compatibility with others;
fixed the misbehavior for the attachment fetching of message in
message attachments. Now, either fetch_attachments() and
fetch_text/body() + get_attachments() combo
can handle this specific case;
changed the attachment identification to the “name” field instead of the “filename”. In very rare cases, the “filename” field identification fails because of encoding errors on the sender’s side;
incorrect decoding of ordinary words (such as “ResearchGate”, for
instance) in decode_mime_header().
decode_mime_header() function
get_attachments() and
fetch_attachments() for correctly setting the
filenameslist_mail_folders(): fixed the regex related to the
hierarchy separator to accept the “|” separator. It was causing a
misbehavior in Yandex accounts.
clean_fetch_results() (internal): fixed the regex
responsible for cleaning the attachment content. It was causing a
misbehavior in Yandex accounts. All calls to gsub() in this function now
have ignore.case = TRUE.
reset_*() methods: Except by
reset_password() and reset_xoauth2_bearer(),
the other methods were not reflecting the changes in the
ImapCon$con_params object, although they were succesfully
modifying the ImapCon$con_handle, which was sufficient to
work properly. It could cause the user to thing that the changes
wouldn’t taking any effect.
fetch_attahments() changed
to “\n::mRpostman: the fetch operation is complete.\n”;rename_folder().reset_*() methods now are
x in order to prevent wordy method calls and repetition.
The older ones will be deprecated in version 0.9.1-2;reset_ssl() method had the name changed to
reset_use_ssl() to better reflect the connection parameter
to be reset.metadata argument in fetch_metadata
now is attribute.In this version, in order to drastically enhance the package
usability, we had to adopt several profound changes with no backward
compatibility. We are sorry that these changes will be painful for old
users but it will certainly be strongly beneficial even in the short
term. The primary update is that mRpostman now is built on
an R6 class and its methods, providing a much more
convenient and elegant way of usage. It is structured following an OO
framework that works well either with the tidy style using pipes or
under the base R approach. The syntax now will be
ConnectionObject$method(), where the
ConnectionObject is created with
configure_imap() or ImapCon$new(). This
completely modifies how the connection handle and other configuration
information is passed among the methods/functions in
mRpostman.
This is a summary of the main modifications in the package:
The package title has changed to mRpostman: An IMAP client for R
All the main functions, except by list_attachments
and the custom-search helper functions, now are methods of the R6 class
ImapConf;
The way the connection token is passed between the functions has
changed. The connection handle is created only inside
configure_imap() (or ImapCon$new()) and only
modified with custom requests inside the methods. As a consequence, the
password, username, and other connection parameters are hidden inside
the curl handle C pointer, resulting in a more secure token chain. This
resulted in changes in every request-functions. They do not use
config_handle() anymore, and a call to
curl::set_opt() is made in every request function so that a
custom request is supplied or replaced by a new one in the original
handle.
the argument “by” used in search and fetch functions was replaced
by use_uid, which is a logical with default value set as
FALSE. This is equivalent to the former
by = MSN default configuration.
all functions that returned invisible(0L) now return
invisible(TRUE)
list_mailboxes() ->
list_mail_folders()
select_mailbox() ->
select_folder()
examine_mailbox() ->
examine_folder()
custom_search() -> search()
fetch_full_msg() -> fetch_body()
fetch_msg_header() ->
fetch_header()
fetch_msg_text() -> fetch_text()
fetch_msg_metadata() ->
fetch_metadata()
get_min_id() -> esearch_min_id()
get_max_id() -> esearch_max_id()
count_msg() -> esearch_count()
mbox -> name:
OBS: type has not changed, still a character string
Affected functions: select_folder() (former
select_mailbox), examine_folder() (former select_mailbox),
rename_folder() (former rename_mailbox)
reselect_mbox -> reselect:
OBS: type has not changed, still a logical with TRUE
as default
Affected functions: rename_folder() (former
rename_mailbox), copy_msg(),
move_msg()
by -> use_uid:
OBS: type HAS CHANGED from a character string with default value
MSN to a logical with FALSE as default. Former
by = "UID" now is equivalent to
use_uid = TRUE
Affected functions:
all search operations: search() (former custom
search), search_before(), search_since(),
search_on(), search_period(),
search_sent_before(), search_sent_since(),
search_sent_on(), search_sent_period(),
search_larger_than(), search_smaller_than(),
search_older_than(), search_younger_than(),
search_flag(), search_string();
all fetch operations: fetch_body() (former
fetch_full_msg), fetch_header() (former fetch_msg_text),
fetch_text() (former fetch_msg_text),
fetch_metadata() (former fetch_msg_metadata);
all complementary operations: copy_msg(),
move_msg(), delete_msg(),
esearch_max_id() (former get_max_id),
esearch_min_id() (former get_min_id),
esearch_count() (former count_msg),
add/replace/remove_flags()
flag -> name:
OBS: type has not changed, still a character string containing the flag name
Affected functions: search_flag(),
flag() (custom search helper function)
string -> expr:
OBS: type has not changed, still a character string containing the string or expression to be searched; shifted to the first position of the arguments in the functions/methods.
Affected functions: search_string(),
string() (custom search helper function)
section_or_fields -> where:
OBS: type has not changed, still a character string containing the name of the message section or the header field in which to execute the search for the informed expression;
Affected functions: search_string(),
string() (custom search helper function)
specific_UID -> msg_uid:
OBS: type has not changed, still a numeric vector containing message uids
Affected functions: expunge()
to_mbox -> to_folder:
OBS: type has not changed, still a character vector containing the folder name
Affected functions: copy_msg(),
move_msg()
try_b64decode -> base64_decode:
OBS: type has not changed, still a logical with default value
FALSE
Affected functions: fetch_text() (former
fetch_msg_text)
show_pass -> removed:
OBS: This argument is not available anymore.
Affected functions: configure_imap()
fresh_connect -> removed:
OBS: This curl argument is not accepted by mRpostman
anymore.
Affected functions: configure_imap()
return_imapconf -> removed:
OBS: This argument is not available anymore.
Affected functions: all search methods.
search_string() and string():
expr (former string) now is 1st; where
(former section or fields)` now is 2ndesearch_count() (former count_msgs),
esearch_max_id() (former get_max_id),
esearch_min_id() (former get_min_id):
use_uid (former
by) and flag
returned object:
functions/methods that returned invisible(0L) in the
previous version, now return invisible(TRUE). Applied to:
select_folder(), create_folder(),
rename_folder(), get_attachments(),
fetch_attachments(), expunge()
all search functions now return NA when there is no
match. The previous behavior was to return 0.
add/replace/remove_flags() methods now invisibly
return the msg_ids in case the user intends to chain any further
operation (perhaps expunge) using the pipe.
default value of arguments:
retries = 2 -> retries = 1 in all
functionscreate_folder(): Create a new mail folder (New IMAP
functionality!)
list_flags(): List flags in a selected mail folder
(New IMAP functionality!)
fetch_attachments_list(): Fetch attachments’ list
without the previously need to fetch a message’s text or body
fetch_attachments(): Fetch attachments without the
previously need to fetch a message’s text or body
reset_*(): reset one (*) of the original parameter
that were informed in configure_imap().
mute:
OBS: A logical. If TRUE, mutes the confirmation message when the
command is successfully executed. Default is FALSE. In the case of the
fetch_*() functions, it only has effect when
write_to_disk = TRUE.
applied to methods/functions: select_folder(),
create_folder(), rename_folder(),
fetch_body(), fetch_header(),
fetch_text(), fetch_metadata(),
copy_msg(), move_msg(),
delete_msg(), expunge(),
add/replace/remove_flags(), get_attachments(),
fetch_attachments()
override:
OBS: A logical. If TRUE, overrides existent files containing the same name in the local directory. Default is FALSE.
applied to methods/functions: get_attachments(),
fetch_attachments().
xoauth2_bearer:
OBS: added the xoauth2_bearer parameter for oauth2.0
authentication (libcurl >= 7.70 is required because of bugs in
previous versions). In Linux, if you use Ubuntu 20, you should be fine.
Versions below this require updating libcurl if the user intends to use
oauth2.0 authentication.
applied to methods/functions:
configure_imap()
examine_folder() and rename_folder():
name
argumentflag argument in all search functions:
flag
now accept more than one flag as a filtering parametername argument in
search_flag
flag parameter was added to the
search_string methodget_attachments() and
fetch_attachments():
override argument was added and allows to control over
the file writing processretry bug fixed: it was causing the loss of the search and fetch
customrequest when executing a retry + selection operation
or when there was a considerable period between two requests given that
the second depends on a previous folder selection. When there was a
considerable delay between the executions of two commands, the curl
handle would establish a new connection to execute the last one, but
without the mail folder selection. This was causing an error during the
retry or the next IMAP command since the IMAP session would have lost
the mail folder selection. This bug was happening mainly when the
functions were used under the base R approach
fixed bug that was writing metadata .txt files as textUID*.txt
Better error handling:
An unique internal function called check_args() was
created to check for the validity of the arguments of all methods and
functions, replacing all the cheack_args_* specific functions
added the response_error_handling() function to
catch operation/resolving timeout errors and login error as
well
All methods work as wrappers for internal functions with similar
names and suffix *_int
Search, Fetch and Complementary functions have a central internal
function called execute_*() that is responsible for
configuring and executing the requests towards the IMAP server
removed config_handle()as the connection token chain
has changed
added config_conn_handle() which is called inside
the iniliaze() method of the R6 ImapCon
class.
added adjust_repeated_filenames(),
serialize_filenames() and
extract_MIME_level_and_filenames() as helper functions to
the new attachments fetch operations
added adjust_folder_name() which is called in almost
every request function
some regex adjustments were made to
fix_search_stripping(), has_attachment(), and
clear_fetch_results()
in list_server_capabilities(): changed IMAP command
to “CAPABILITY” instead of establishing a new connection
REGEX structure replacement from stringr to base R
in: list_server_capabalities(),
examine_folder() (former examine_mailbox),
list_mail_folders() (former list_mailboxes),
select_folder() (former select_mailbox)
fetch_attachments_list() and
fetch_attachments() are a faster and smart way to
respectively list and download messages’ attachments. They do not depend
on a former fetching step, unlike list_attachments() and
get_attachments(). the new methods use BODYSTRUCTURE
metadata fetching to identify the attachments, and
fetch_attachments() also issue a FETCH BODY[level.MIME]
command to fetch only the parts of the messages that contain the
attachments. This prevents unnecessary fetching when users are only
interested in attachments. However, get_attachments() and
list_attachments() are still available in the
package.
All fetch_* methods, and
get_attachments() now use a different path for saving the
fetched files. The folder to be created now will have the following
structure: imap_server > mail_folder >
Fetched messages that are saved to disk will have different filename structures:
use_uid = FALSE: bodyuse_uid = TRUE: bodyConfirmation messages were added to methods:
select_folder(), create_folder(),
rename_folder(), copy_msg(),
move_msg(), delete_msg(),
expunge(), fetch_*(),
add/replace/remove_flags(), get_attachments(),
fetch_attachments()
A startup message informing about the breaking changes of the version were added by creating the zzz.R file
Besides the mRpostman Basics vignette, two
more were added: Migrating old code to the new mRpostman’s
syntax, and IMAP OAuth2.0 authentication in
mRpostman
get_attachments() function:
check_args_get_atatchments() helper function:
"content_disposition"
list_attachments() function:
"content_disposition" to outputcheck_args_list_attachments() function:
check_args_list_attachments()
has_attachment() helper function:
loop_fetch_msg_*() functions group:
get_attachments():
check_args_get_attachments():
"content_disposition"
get_attachments() function:
get_attachments()
has_attachment(),
check_args_get_attachments()
list_attachments() function:
list_attachments()
loop_fetch_msg_*() functions:
examineMailbox() to
examine_mailbox()
loop_fetch_full_msg()
[\\/:*?\"<>|],
considering WIN-*NIX-OSX casesfetch_full_msg():
added warning message related to get_attachments()
when keep_in_mem = TRUE
fetch_*() functions:
added
attr(msg_list, which = 'mbox') = new_imapconf$mbox to all –
to be used by get_attachments()
list_server_capabilities() function:
converted regex match/extraction from stringr to base R
examples:
fixed typo - from “configureIMAP” to “configure_imap” in the
following functions examples: list_mailboxes(),
list_server_capabilities()
changed all imaps://imap.gmail.com server examples to a generic one: imaps://your.imap.server.com
devtools to
remotes because the former depends on
curl
configure_imap() exampleconfigure_imap() examplecount_msgs():
loop_fetch_msg_XXXX() functions for fetching msgs:
select_mailbox(imapconf = new_imapconf, mbox = new_imapconf$mbox)
inside the tryCatch in the while loop – it prevents errors especially
when messages have larger attachments taking too much time to fetch. In
some IMAP servers (such as Yandex) it may lose the mbox selection. Other
alternatives: set a larger timeout_ms in
configure_imap().list_mailboxes():
list_mailboxes() was
modified to deal with the return of Yandex IMAP server: from
.*\" \"*(.*?)\\"\r\n' to '.*\" \"*(.*?)[(\\"\r\n)|(\r\n\\*)]
configure_imap():
@param retries description typo: from “retires” to
“retries”conifg_handle():
@family config, so it does not appear as “see
also” in config_IMAP()
flag_options():
@note to be more clear@family check args search:
@param esearch typo: from code{1,2,3,4,5}
to \code{1,2,3,4,5}
changed function name patterns to those specified in the tidyverse style guide
changed return from mailboxes operation functions and some miscellanea functions:
select_mailbox(), rename_mailbox(),
copy_msg(), move_msg(),
delete_msg(), expunge(),
add/remove/replace_flags() outputs are invisible and only
return imapconf or a list (imapconf+msg_ids).changed package logo