How a 5 MB podcast feed became 182 MB

A public podcast feed loaded fine in our Feed view. Then we reloaded the page, and the tab died with "Aw, Snap!" and error code 5.

We reloaded again. It died again. It kept dying, because the app restores the last saved document on load, and the saved document was the thing that killed the tab.

The cause was a few lines of indentation code. They had been wrong since the day we wrote them.

The symptom

The feed was 5.4 MB. It had 2,764 episodes. Each episode had a plain-text description, and most of those descriptions ran over several lines.

Nothing about it looked unusual. It is the kind of feed that podcast hosts publish every day.

The first load worked. The crash came on reload, and then on every reload after that. That pattern told us the problem was not in reading the feed. It was in what we saved, or in what we did with the saved copy.

The wrong lead

We wrote a benchmark of the feed pipeline first. It ran in jsdom, because that is where our tests run.

The benchmark blamed feed parsing. One step took 41 seconds at 20,000 items. That looked like a clear answer, and we started to plan a fix.

Before we wrote any code, we ran the same step in Chrome. It took about 0.5 seconds. jsdom was about 80 times slower than the browser for that step. The 41 seconds was a property of the test environment, not of our code.

The browser did show some real costs. Compressing saved documents with lz-string took about 220 ms per MB on the main thread. That is slow, and we fixed it in the same work. But it did not explain a crash.

The lesson is simple. Measure in the browser you ship to. A benchmark in a fake DOM can point at the wrong code with great confidence.

The real bug

We went back to the real feed and ran each step of the pipeline by hand, in the browser, and printed the size of each result.

One number stood out. The formatted XML was 182,663,639 characters. The input was 5,439,194 characters.

Our formatter had made the document 34 times larger. It had not added any data. It had added whitespace.

This is the shape of the original algorithm:

const lines = serialized.replace(/>\s*</g, '>\n<').split('\n')
for (const line of lines) {
  const trimmed = line.trim()
  if (trimmed.startsWith('</')) indent--
  out += '  '.repeat(indent) + trimmed + '\n'
  if (opensTag(trimmed) && !trimmed.includes('</')) indent++
}

It splits the document into lines, then tracks depth one line at a time. A line that opens a tag and does not close it raises the indent. A line that starts with a close tag lowers it.

That works when every element sits on its own line. It breaks on text that already contains a line break. Here is a small input:

<item><description>line one
line two</description><title>x</title></item>

And here is what the formatter produced:

<item>
  <description>line one
    line two</description>
    <title>x</title>
  </item>

Follow the indent through it. The line <description>line one opens a tag and has no close tag on that line, so the indent goes up.

The close tag is on the next line, after the text line two. That line does not start with </, so the indent never comes down.

Everything after it is now one level too deep. The <title> is too deep. The </item> is too deep.

There is a second problem in the output. The text line two gained leading spaces. The formatter did not only move tags around. It changed the data inside them.

The size math

One misread description costs one extra level of indent for the rest of the document. It never comes back.

Now take a feed with 2,764 episodes and a multi-line description in each. Item 1 sits one level too deep. Item 2 sits two levels too deep. Item k sits about k levels too deep, which is about 2k spaces on every line.

Each item has many lines. So the whitespace added to item k is proportional to k, and the total over all items is proportional to the square of the item count.

That is how 2,764 items turned 5.4 MB into 182 MB. Most of the file was spaces.

The browser could hold 182 MB of string for a short time. It could not do that and also save it, and then parse it again on reload. The saved copy was the same bloated document. Restoring it on load pushed the tab past its memory limit.

That explains the pattern in the symptom. The first load only formatted the feed. The reload had to read the saved 182 MB back in.

The fix

The bug came from using lines as the unit. Lines are the wrong unit for XML, because text can contain line breaks and still be one value.

The fix is to tokenize. We read the document as a stream of tokens, and depth comes from tags only.

Comments, CDATA sections, processing instructions, and a DOCTYPE are each a single token. Each can contain a < or a > that is not a tag boundary, so a simple split on those characters is not safe.

Text that is not only whitespace is copied exactly. The tag after it stays on the same line. Text never changes the indent.

This is the core loop:

for (const [token] of serialized.matchAll(TOKEN)) {
  if (token[0] !== '<') {
    if (!token.trim()) continue
    out.push(token)
    afterText = true
    continue
  }
  const isClose = token.startsWith('</')
  if (isClose) depth = Math.max(0, depth - 1)
  if (!afterText && out.length > 0) out.push('\n' + '  '.repeat(depth))
  out.push(token)
  afterText = false
  if (!isClose && !/^<[?!]/.test(token) && !token.endsWith('/>')) depth++
}

Only a tag can raise or lower depth. A close tag lowers it. An open tag raises it, unless it is a declaration, a comment, a processing instruction, or a self-closing tag, because those have no close tag.

The afterText flag keeps a close tag on the same line as the text before it. That is why <a>x</a> stays on one line, and why the text of an element is the same bytes in and out.

The same feed now formats to 5.47 MB. That is the input plus the indentation, and nothing more.

What else changed

The formatter was the cause. But the investigation found three other weak spots, and we fixed them in the same pass.

Large text skips lz-string. Text of 2 MB or more is now stored without lz-string compression. At about 220 ms per MB, compressing a large document blocked the main thread for a long time, and the saving in size did not justify it.

A reload does not save what it just restored. Before, the app restored the saved document and then saved it again at once. That was wasted work, and for a large document it was also a second chance to run out of memory. A reload now restores and stops.

One large tab no longer fails the whole tab list. The list of open tabs is kept in session storage, which has a quota. If one tab was larger than the quota, the write failed for the whole list, and every tab was lost. Now every tab is saved. A very large tab keeps its name and settings, and its text comes back from History when you open it.

The rule a formatter must keep

A formatter has one rule it must never break. It must not change the text it formats.

Indentation is only for people. Tags can move. Whitespace between tags can change. But the content of an element must come out exactly as it went in.

Our old code broke that rule on every multi-line value, and it did so quietly. The output still parsed. It still looked like XML. It was only larger, and the text was a little different.

Tests now pin that rule. One checks that multi-line text stays exact and does not raise the indent. Another builds fifty items with multi-line text and checks that every <item> line keeps the same indent.

---

Prettify formats XML and RSS or Atom feeds in the browser. Paste a feed into the RSS Feed Viewer to read it, or into the XML formatter to indent it. The same lesson on a different format is in the JSON numbers JavaScript corrupts.