Mathias Bynens

Dear Google, please fix plain text emails in Gmail

· tagged with Email

By default, composing a new email in Gmail results in an HTML email under the hood — even if you don’t use any formatting. Luckily, it’s possible to opt-out of that and use plain text email instead. In the ‘new email’ window, click the downwards arrow and check ‘Plain text mode’.

Gmail remembers your preference, so any new emails you compose after that are automatically in plain text mode, too.

The problem

Unfortunately, Gmail hard-wraps emails that are composed in plain text mode before sending them. If you would send this message, for example:

Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat.

What would actually get sent is the following:

Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam
nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat
volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation
ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat.

Here’s a screenshot of such an email, in Gmail:

And here’s what that same email looks like in another email client (in this case, OS X’s Mail.app):

Instead of filling up the available screen width and letting the text flow naturally, the automatically inserted hard breaks ensure no line is longer than 78 characters. This is especially annoying when viewing such emails on small viewports, e.g. on a smartphone:

Combined with the auto-wrapping due to the small viewport, there are now two lines in the message with just one word on them.

This behavior annoys pretty much anyone who sends plain text emails from the Gmail web client.

Google’s justification

Since I’m a Google Apps for Business customer, I get to file support tickets in hopes of getting issues like these fixed. So I did — I asked whether the hard-wrapping behavior for plain text emails could be disabled. After some painful back-and-forth, this was the response of the Google Enterprise Support team:

The 78 characters per line is not a limitation implemented by Google, but rather an internet standard for plain text messages (RFC 2822) recommended and promoted by the Internet Engineering Task Force (IETF).

It’s true that section 2.1.1 of RFC 2822 recommends a maximum of 78 characters per line (excluding the CRLF newline characters that end each line) in the source of email messages. But if this is what causes the broken behavior in Gmail, how come other mail clients don’t have the same issue? They have to respect the line length limit too, right?

The solution

Sure enough — the problem can be solved without violating any standards and without exceeding the 78-characters-per-line limit. It is perfectly possible to send non-hard-wrapped plain text emails by using the Quoted-Printable encoding as defined by RFC 2045. In fact, Gmail already kind of supports this: it’s capable of decoding and displaying Quoted-Printable-encoded emails flawlessly, and it encodes HTML emails with it before sending them — it just doesn’t do the same thing for plain text emails (yet).

Here’s what the source of a plain text email composed in Gmail looks like. Note that it contains hard line breaks to satisfy the 78-characters-per-line limit:

Content-Type: text/plain; charset=UTF-8	

Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam
nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat
volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation
ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat.

With Quoted-Printable encoding, it could look like this instead:

Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy =
nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Ut wi=
si enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lo=
bortis nisl ut aliquip ex ea commodo consequat.

This way, the lines wrap nicely (no forced line breaks) when viewing the email in Gmail or in another mail client.

Summary

Dear Gmail team, please consider Quoted-Printable-encoding plain text emails composed using the Gmail web interface as shown in the above example, just like you do for HTML emails. That way, no hard-wrapping of the email’s actual contents is necessary. Thanks!

Comments

Alex wrote on :

Notably, Outlook does the same, so that’s probably why Gmail does it as well.

Henri Sivonen wrote on :

You should be asking for format=flowed instead of Quoted-Printable.

Update: Oh, I now see that I should have reloaded to see Thomas Broyer already mentioned format=flowed. It is supported by Apple Mail. Lack thereof in other clients continues to irk me even though I abandoned Apple Mail years ago.

wrote on :

Thomas and Henri: Agreed, format=flowed support would be even better! But like you said a) it doesn’t work in all major email clients, and b) it’s trickier to implement, especially in Gmail’s case: they already have a working Quoted-Printable implementation. Let’s take it one step at a time :)

Aaron Meurer wrote on :

Would that cause it to screw up the formatting of any code in an email?

Some wrote on :

Bijan, please look at the example image from the smartphone. Did you notice how the lines breaks at odd places? The device can only show a limited number of characters, so it has to break the lines, but still breaks it at the place where Google has put the line breaks.

Mikel wrote on :

Bijan, the point of quoted-printable or format=flowed is the receiver’s mail app is free to wrap at any width. The easiest way to do this is to shrink the window to a size you find comfortable, but some mail apps provide more powerful ways.

Without quoted-printable or format=flowed, the mail app has to assume the line breaks have to go where the sender’s mail app put them.

Guillermo wrote on :

I like the way it is, and I really prefer that over emails that have lines of 2000 characters.

If you don’t want the standard text-wrap, just don’t use the plain text mode. HTML mode solves that problem.

wrote on :

Guillermo: Most of the emails I send contain no formatting whatsoever, except for indentation, paragraphs and other formatting that can be represented just fine using plain text. Why should I wastefully send unneeded markup that makes the size of my email several times larger than what’s actually needed?

Scuzzball wrote on :

Guillermo: Many things require plain text emails, such as the Debian mailing lists. This seems to be standard for many mailing lists.

bobbyi_settv wrote on :

This doesn’t even mention the most annoying part of the line wrapping which is how it interacts with GMail’s “Undo Send” feature. If you notice an error in your email and click Undo, they bring you back to the compose mode with the hard newlines still inserted. If you edit the email and send, they now add more hard newlines for where the new line endings are and you end up with a mix of tiny lines and normal lines looking like your email is written in free verse.

foobar wrote on :

Well, your criticism of Google’s defect is perfectly valid, but your suggestion is just as broken as the current state of affairs.

The correct way to display a mail that’s not format=flowed is as-is (after MIME decoding), without any changes in line breaks. Reflowing of long lines in such a mail is essentially a defect in the MUA, and can be quite annoying for some uses (exceptions might apply if the line is too wide for the viewport, but the width of the viewport is something quite different from the optimal reading width, which should be used for paragraph wrapping). QP encoding does solve the “transmission of long lines via email” problem, but what you are transmitting is still a line, while what you want to transmit is a reflowable paragraph, which does not exist as such in “normal” plain text emails.

The only sane solution is format=flowed, as that explicitly allows reflowing for MUA capable of doing so, but still keeps the text readable in MUAs that don’t understand format=flowed by keeping the lines reasonably short, and also keeps the quoting intact for both reflowing and non-reflowing UAs.

Radu Grigore wrote on :

I think a) insertion of \n should be an option the sender has, and b) reflowing lines should be an option the receiver has.

In particular, the smartphone app should give the user the option to reflow plain text. In fact, it should probably be the default.

foobar wrote on :

Radu: No, the point of email is that it is an interoperable system, and encoding or decoding choices for the user to make are a sign of a design flaw and not a solution, as that would mean that users have to manually ensure that their choice matches the choice of the person they are communicating with. The sending MUA should encode the input in a way that conveys the meaning that the user intended according to the standard and the receiving MUA should decode the mail and display it so that it conveys what the mail means according to the standard, then everything would be fine.

There might be an argument for having smartphone MUAs automatically reflow plaintext emails if there isn’t enough space for 80 columns due to the lack of any unambiguous paragraph encoding in “normal” plaintext emails, as that was a use case not anticipated by the old standard. But it should come with an easily accessible toggle in case you get a mail where line breaks are significant.

Radu Grigore wrote on :

foobar: I don’t care about how this is implemented, encoding/decoding mumbo-jumbo.

I like short lines. If Gmail would stop inserting \n, I would type them myself. I understand if other people dislike Gmail being proactive and editing their email, so I think those people should be able to ask Gmail not to do so.

On the receiving end, if you see funny lines on the screen you should be able to ask for a reflow. Whether Gmail inserts \n or not has nothing to do with whether my email looks funny: as I said, I'd be perfectly willing to include the newlines myself. That shouldn’t cause immeasurable pain to receivers with tiny smartphones.

Again: the sender should have control over what message gets sent. The receiver should have control over how the message is displayed. I’m talking about the message. I don’t care about how the message is encoded.

foobar wrote on :

Radu: No, you are probably thinking about it at the wrong level of abstraction if you think about “inserting \ns”. That’s already part of the encoding process. The user enters text and splits it into paragraphs and possibly lines. That’s what needs to be encoded so that after it has been decoded at the other end, the user again sees those paragraphs and lines. What kinds of (control) characters are needed in order to make that happen in a backward compatible way with all kinds of recipients is a matter of the respective email standards and none of the user’s business.

Ultimately, it is somewhat of a confusion to say “The sender should have control over what message gets sent. The receiver should have control over how the message is displayed.”. The structure of the message is part of the message, in the sense that it can be semantically relevant, and as such has to be reproduced by the receiving MUA in order to not lose information. If the receiving user has to instruct the MUA how to decode the mail into paragraphs and/or lines, that’s a workaround for a broken encoding mechanism. The mail itself should indicate what is a praragraph and therefore can be reflowed and what is a line and therefore can not, without any need for user intervention.

I agree that the problem is that Gmail is editing the mail. They split a paragraph(!) that the user entered into a series of lines(!) and send that instead. If the user composes a paragraph, they need to encode a paragraph and not silently convert it to something else.

Evan wrote on :

There’s a really simple solution for this, actually: stop using Gmail.

Radu Grigore wrote on :

foobar: The right level of abstraction, when designing what the user should see, is to ignore all encoding/decoding. I understand that “inserting \n” isn’t quite “inserting \n”, but the result of broken encoding/encoding. That, however, is completely irrelevant to me as a user. What I see is that the message gets edited.

They split a paragraph(!) that the user entered into a series of lines(!) and send that instead.

Actually, they split a line into a series of lines. Plain text has no concept of paragraph. That’s where the problem lies, and that’s why the receiver should be able to choose when reflowing is done.

You seem to think that a paragraph is a long line. I, on the other hand, often use the convention that a paragraph is something separated by two or more newlines.

Eli Barzilay wrote on :

Um, they don’t need to implement QP encoding — they already do that, only for different things. I see it used for Unicode characters and for the space in a quoted -- signature separator. My guess was always that the cost of another configuration option is too high for them.

Radu Grigore wrote on :

A few short points:

  1. I agree with the solution from this post.
  2. Although plain text has no notion of paragraph, it occurs to me while writing this that Markdown has. I would like a mail client with support for Markdown.
  3. Eli, the post already mentions that QP is used in other places in Gmail.

Eli Barzilay wrote on :

Radu: The thing is that it’s phrased as a plea for using QP for plain text, which is supposedly easy since they already use it for HTML. What I’m saying is that it’s even easier than that — they are using QP encoding for text too, only it’s done after the text is reflowed. And since they’re using a conventional QP encoder, the only thing that they need to do is avoid that hard-reflow step, nothing else. IOW, it means that the reply that is listed under “Google’s justification” is bogus — they don’t need to do the under-78-chars thing since the QP step already does it.

(This is easy to verify: just sending out text with a long word shows that it’s QP-encoded fine.)

wrote on :

Radu:

Although plain text has no notion of paragraph, it occurs to me while writing this that Markdown has. I would like a mail client with support for Markdown.

I couldn’t agree more!

Eli: You’re absolutely right. I’ve amended the summary to clarify what I meant.

foobar wrote on :

Radu:

First of all, plain text actually does have a concept of a paragraph, it’s just that the encoding is ambiguous. A paragraph break is encoded as two line breaks in a row, which can not reasonably be distinguished automatically from two genuine line breaks in a row, which happen to have the same encoding, as you essentially have to understand the surrounding natural language text in order to distinguish the two cases. I guess that that’s probably really what you meant, but the distinction is important if one doesn’t want to become confused. BTW, I don’t think of a paragraph as either a long line or something separated by two or more newlines, those are both details of possible encodings — a paragraph is an abstract object of reflowable text that is structurally separate from surrounding text.

Now, I agree that the process by which the editing occurs is somewhat irrelevant to you as a user. But it is really important for you as someone advocating a solution.

Now, when you say “Actually, they split a line into a series of lines”, you are still confusing levels of abstraction, precisely because that actually is not what the user sees, but rather a detail of the encoding process that you happen to know. You know that the text a user enters into a <textarea> in a browser where the browser automatically wraps it into lines for display will be transmitted to the server as a string without any control characters apart from line breaks in those positions where the user explicitly pressed enter. So, when you simply print that string to a (wide) console, it will appear as “a line”. That’s why you think of it as “a line”. But that’s already an encoding of the user’s input: the user doesn’t see that. The user sees a long-ish, auto-wrapped piece of text — and that, from the point of view of the user, is a paragraph. The user interprets it as a “unit of text that will be separated from surrounding units of text and reflowed for optimal viewing” — it walks like a paragraph, it quacks like a paragraph, so it is a paragraph. That the paragraph breaks are encoded as double-linebreaks in the HTTP message is not the level of abstraction that the user is working at (even though the UI may indeed be somewhat leaky there, unfortunately).

A line, by contrast, is an object that is expected to not be reflowed and to be layed out without any structural separation from surrounding lines. The user quite definitely does not expect what is wrapped in the <textarea> to be displayed as one long line without any breaks at the recipient — therefore, it can’t be a line.

And this distinction is not purely academic, but extremely important. A common use for lines is the embedding of code and log excerpts in an email. Code and log excerpts essentially become unreadable if you reflow them, and it’s much better to scroll horizontally than to have to read reflowed code. Any surrounding commentary is likely to be paragraph text, though, so it’s perfectly fine, and for narrow screens very advantageous, to reflow it for display.

And that’s why QP encoding is not a solution. QP encoding transforms long lines so that they can be transported through email. But they stay lines nonetheless. And that’s important. If you put a long line of code or log excerpts into an email, your MUA should encode it into shorter lines using QP encoding, but the recipient should just reverse the encoding and again display one long line. If the receiving MUA were to reflow long lines, that would mean reflowing code and log excerpts, which is terrible.

Also, if you have an extremely wide viewport, paragraph text probably should not be reflowed to fill the full width, but rather some optimal reading width. Code blocks, though, most definitely should use the full width to stay readable.

And not only is the distinction of lines and paragraphs important because both have their use cases, but also, the existing encoding standards for emails define how to encode either of them, and advocating for an encoding or decoding that deviates from those standards just causes more incompatibility and thus pain for the user, rather than solving the problem. You can not just make up what you intuitively consider nice encodings. You have to go by what the standards say, that’s the only way to have a reliably interoperable distributed system, because others might not have the same intuitions as you have, but others can read the same standards. And the standard does not say “a long QP encoded line should be reflowed after QP decoding”.

wrote on :

foobar:

And that’s why QP encoding is not a solution. QP encoding transforms long lines so that they can be transported through email. But they stay lines nonetheless. And that’s important. If you put a long line of code or log excerpts into an email, your MUA should encode it into shorter lines using QP encoding, but the recipient should just reverse the encoding and again display one long line. If the receiving MUA were to reflow long lines, that would mean reflowing code and log excerpts, which is terrible. […] And the standard does not say “a long QP encoded line should be reflowed after QP decoding”.

Aren’t you contradicting yourself here? I’m confused either way. QP encoding makes it possible to encode the original message including the line breaks the author intended, so that it can be decoded into its original form. How is that problematic?

Are you saying what OS X Mail.app is doing (i.e. the proposed QP-based solution) is somehow a Bad Thing™?

foobar wrote on :

Mathias:

Aren’t you contradicting yourself here? I’m confused either way.

I don’t think so?! Even given your following question, I don’t see any contradiction, and also nothing that I’d see how to confuse for a contradiction, but I’ll just try to answer it, maybe that’ll clear things up anyhow ;-)

QP encoding makes it possible to encode the original message including the line breaks the author intended, so that it can be decoded into its original form. How is that problematic?

That’s not problematic at all, and exactly what QP encoding is for. But my point is that if you QP-encode a long line, and then decode it, all you have is a long line, not a paragraph, and you should usually not reflow lines, whether they came QP-encoded or not (and correspondingly, you should not send long lines, just because you can make them fit through SMTP by QP-encoding them, as they still stay long lines at the semantic level). QP encoding is orthogonal to paragraph encoding.

If you want to encode a paragraph rather than a line, you have to use either HTML’s <p> element (bad compatibility with many MUAs) or plain text with format=flowed (much better backward compatibility), or both in a multipart message, if you see any advantage in that. And if you do that right, you might actually not need any QP encoding at all anymore. A format=flowed encoded mail actually represents paragraphs but won’t have any long lines that need QP encoding in order to be transmitted reliably via email.

Are you saying what OS X Mail.app is doing (i.e. the proposed QP-based solution) is somehow a Bad Thing™?

If Mail.app encodes paragraphs (that is, pieces of text that behave like a paragraph when you write them in the program, with automatic wrapping according to viewport size and all that) as one long line which then is QP encoded for transport, that’s a Bad Thing™ ;-), yes.

Is that the same thing as Apple Mail? I’ve seen creative MIME structures from that one before (attachments attached to the HTML alternative in a multipart/alternative mail, so the plain text version apparently didn’t have any attachments at all), so I wouldn’t expect too much from that.

Ultimately, it all comes down to being able to properly support transport of both prose, which is best presented in a paragraph layout and is not sensitive to being reflowed, and fixed-structure text, such as code and log files, which easily become incomprehensible when reflowed and should be displayed as-is, even if that means horizontal scrolling (and maybe even more importantly needs to retain its structure even when it’s quoted in a reply, that’s also something many MUAs get wrong).

Thomas Broyer wrote on :

Just a small note in passing that GMail (both Web and Android app) apparently supports format=flowed (and even DelSp=Yes): I just received a spam that used that and GMail displayed it as expected (for the curious, I was checking the mail headers and stumbled on the Content-Type, which reminded me of this blog post).

David Rush wrote on :

Google does it the way that it does, because it is The Right Thing™. Plain text should mean plain text. I just wish they would use a monospaced font when you are composing a plain text email.

I have no problems with enabling other encodings that are less than full HTML mark-up, but they shouldn’t be called plain text, because they aren’t…

foobar wrote on :

David: There is probably nothing wrong with the mail itself, but if they wanted to produce that output, they should at least make sure it corresponds to the line breaks in the user interface — the problem, IMO, is, that they are silently changing the user input.

If you are referring to format=flowed, that actually is plain text, which is exactly the reason why it is the right thing to use. Or more precisely, it encodes paragraphs in a way that is valid plain text, readable for a human reader just as any other plain text, but with additional information hidden in whitespace that can be used by an appropriately capable MUA to decode it into paragraphs for improved rendering.

Harsh Gupta wrote on :

I didn’t realize that the practice of 78/80 characters can be annoying to people reading mails on phones. Until now I’ve been deliberately doing it. I write my mails in Vim and my editor settings automatically make sure of that. I find long flowing lines annoying and hard to read. Then there are people who write mails in Emacs or Vim — see vmail. I’m pretty sure they prefer their mails to arrive in hard wrapped 78 character format.

The other thing I find particularly annoying is that Gmail doesn’t a monospace font. Not having a monospace font screws up all the ASCII formatting of my mails which looks particularly good in my editor.

Vasil Yonkov wrote on :

Displaying 4-5 words on a line itself makes any text uncomfortable to read. Reading text on such screens is not a ‘normal’ way to read it. The small screen is a limitation, a handicap, and it should be treated as a special case. 78 characters is not an unreasonable length to have as a standard. The ability of the screen to display at least 78 characters a line is a case on its own. This could be a problem even on a 26″ monitor if you overdo the Ctrl + ' inside the browser. Having the paragraph flow on the entire width of a wide screen monitor does not feel “natural” to me. I find the reasonable line length essential when reading the text. Many websites enforce such length, including this one.

What you are arguing here is whether the text should have word-wrap as a property, or if this should be determined by the viewer program. Personally, I support this view from 2 years ago.

Leave a comment

Comment on “Dear Google, please fix plain text emails in Gmail”

Some Markdown is allowed; HTML isn’t. Keyboard shortcuts are available.

It’s possible to add emphasis to text:

_Emphasize_ some terms. Perhaps you’d rather use **strong emphasis** instead?

Select some text and press + I on Mac or Ctrl + I on Windows to make it italic. For bold text, use + B or Ctrl + B.

To create links:

Here’s an inline link to [Google](http://www.google.com/).

If the link itself is not descriptive enough to tell users where they’re going, you might want to create a link with a title attribute, which will show up on hover:

Here’s a [poorly-named link](http://www.google.com/ "Google").

Use backticks (`) to create an inline <code> span:

In HTML, the `p` element represents a paragraph.

Select some inline text and press + K on Mac or Ctrl + K on Windows to make it a <code> span.

Indent four spaces to create an escaped <pre><code> block:

    printf("goodbye world!"); /* his suicide note
was in C */

Alternatively, you could use triple backtick syntax:

```
printf("goodbye world!"); /* his suicide note
was in C */
```

Select a block of text (more than one line) and press + K on Mac or Ctrl + K on Windows to make it a preformatted <code> block.

Quoting text can be done as follows:

> Lorem iPad dolor sit amet, consectetur Apple adipisicing elit,
> sed do eiusmod incididunt ut labore et dolore magna aliqua Shenzhen.
> Ut enim ad minim veniam, quis nostrud no multi-tasking ullamco laboris
> nisi ut aliquip iPad ex ea commodo consequat.

Select a block of text and press + E on Mac or Ctrl + E on Windows to make it a <blockquote>.