Support fetching non-UTF-8 data. #54
Description
The code currently assumes that all data is valid UTF-8, which is not true for a number of services (#11). A past PR (#33) tried to address this by changing all return types to Vec<u8>, but this makes for a much less pleasant API. Some further discussion on this can be found in #33 (comment). Ideally we'd like more structured responses (#28), though that will also eventually have this issue.
Some good points have been brought up in various comments from the aforementioned threads:
- We can't just re-encode data to UTF-8, because it'll cause checksums and signatures to no longer match. Read bytes when fetching or uid_fetching. #33 (comment)
- RFC5738, which adds "UTF-8 compatability" for IMAP servers is often not supported, and also leaves a bunch of data in its original encoding. panic on hotmail/outlook.com IMAP fetch #11 (comment)
I think what we'd want is a way for the user to choose whether they want data to be parsed as UTF-8 or not, and if they choose not, to just get the raw bytes. Crucially though, the interface for when you do have UTF-8 data should not be significantly more complex than today. One way of achieving this is by using a trait that is implemented for Client with an Output associated type that implements From<Vec<u8>> (or something along those lines). @sanmai-NL also discussed a related solution in #11 (comment).