How We Built a Streaming Rich Text Renderer for LLM Responses on iOS

Background

LLM responses usually arrive token by token rather than as a complete document. Waiting for the final response before drawing anything makes the interface feel needlessly slow.

The client should reveal content as it arrives, but LLM output is rarely plain text. It may contain Markdown, HTML-like markup, and formulas; this implementation must handle both HTML and LaTeX while the document is still incomplete.

How it works

Streaming request

AFNetworking-streaming adds callbacks for chunked responses. A request receives one headerCallback, repeated dataCallback invocations as chunks arrive, and a final completion callback when the stream closes.

Each dataCallback may contain one or more application-level records, which the client splits using the delimiter defined by the server protocol.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
NSURL *url = [NSURL URLWithString:@"https://raw.githubusercontent.com"];
self.manager = [[DWHTTPStreamSessionManager alloc] initWithBaseURL:url];
self.manager.itemSerializerProvider = [[DWHTTPJSONItemSerializerProvider alloc] init];
self.manager.responseSerializer = [[DWDummyHTTPResponseSerializer alloc] init];
[self.manager GET:@"deanWombourne/AFNetworking-streaming/master/Example/example.json"
parameters:@{}
data:^(NSURLSessionDataTask *task, NSDictionary *dictionary) {
[self.results addObject:dictionary];
[self.tableView reloadData];
}
success:^(NSURLSessionDataTask *task) {
NSLog(@"Response complete");
}
failure:^(NSURLSessionDataTask *task, NSError *error) {
NSLog(@"Response failed");
}];

Pseudo-stream parsing

The parser must tolerate HTML and LaTeX fragments that end halfway through a construct. Rather than maintain incremental parser state, the client uses a simpler pseudo-streaming strategy:

Whenever new streamed text arrives, the parser appends it to the accumulated text and reparses the full content from the beginning.

Incomplete tag rules

At any point in the stream, the buffer may end in the middle of a tag. That is an expected intermediate state rather than a syntax error, so the parser follows a few recovery rules:

  1. Complete tag openings are parsed, while incomplete openings are withheld.
    1. <div> is complete even if its matching </div> has not arrived yet.
      1. If the text is <div>123, the output is 123.
      2. If the text is <div><span>123</span><b>456, the output is 123**456**.
    2. <div, <di, <d, and <img src="http:// are incomplete openings and remain hidden.
  2. Leaf nodes can be emitted as soon as they are complete.

In this section, “complete” refers to the opening tag itself, not to the presence of a matching closing element.

Incomplete formula rules

The custom tex element needs a second validation step because its contents must also parse as LaTeX.

  1. <tex>\frac{1}{2} has no </tex>, but the formula itself is complete and can be rendered.
  2. <tex>\frac{2}{</tex> closes the element, but the formula is invalid and remains hidden.

Pseudo-stream rendering

The iOS renderer is built on YYText, which expects a complete attributed string and layout rather than an incrementally growing document. Making it truly incremental would require invasive changes to YYText, so we chose a pseudo-streaming design instead.

Core idea

The core of pseudo-stream rendering can be summarized in one word: masking.

For a given NSAttributedString, YYLabel produces a deterministic layout. The renderer can therefore draw the latest complete layout immediately, cover it with an opaque mask, and uncover one character cell at a time. Visually, the response appears to stream even though each layout pass still processes the full buffer.

The first video visualizes the masks for debugging; the second shows the final effect.

The debug view uses a red horizontal mask for the current line and a blue vertical mask for all later lines. Revealing a character shortens the red mask by one cell. At the end of a line, the red mask moves down and the blue mask shrinks by one line height. The loading indicator stays attached to the leading edge of the red mask.

Note: the mask color must match the underlying view color and must not contain transparency.

Streaming rendering flow

How We Built a Streaming Rich Text Renderer for LLM Responses on iOS: Streaming rendering flow (figure 1)
  1. When the client starts the SSE request, it also starts the pseudo-stream renderer and shows the loading state.
  2. The first data callback creates a layout and starts the reveal timer. Later callbacks append text to the buffer and prepare newer layouts. On each timer tick, the renderer reveals one more character cell from the current layout.
  3. The final network callback produces the last layout and marks the source stream as complete. Once the reveal timer reaches the end of that layout, pseudo-streaming stops. If the source is still active, the renderer waits for the next layout instead.
  4. The caller can also interrupt pseudo-stream rendering explicitly. The engine stops the timer and appends ...... at the current position to show that the response ended early.

Card expansion

How We Built a Streaming Rich Text Renderer for LLM Responses on iOS: Card expansion (figure 2)

In production interfaces, streaming content often appears inside a scrolling list. A normal cell receives its size from the surrounding layout, but a streaming cell grows as new content is revealed. The renderer must therefore report updated dimensions to the containing cell throughout the stream.

Updating the cell height alone is not enough. The cell also needs a clipping container so that content outside the revealed region remains hidden.

When a streaming card lives in a list, do not call reloadData or reloadItemsAtIndexPaths: after changing its model height; recreating the cell causes visible flicker. Ask the list to recompute geometry without reloading its content instead.

1
2
3
4
5
6
// UITableView
[self.tableView beginUpdates];
[self.tableView endUpdates];

// UICollectionView
[self.collectionView performBatchUpdates:^{} completion:^(BOOL finished) {}];

Pitfalls

Raw LaTeX flashes before the formula appears

A chunk boundary may split a formula. If the incomplete prefix is treated as ordinary text, raw LaTeX briefly appears and is then replaced by the rendered formula when the closing delimiter arrives.

The preprocessing pass normally rewrites $xxx$ as <tex>xxx</tex>. An incomplete prefix such as $(a-b) does not match that expression, so the HTML parser emits it as plain text. Once the buffer grows to $(a-b)^{2}$, the same range becomes a valid formula and changes representation.

One possible fix is to rewrite an unmatched $xx prefix as <tex>xx as well. The LaTeX parser can then render complete prefixes and suppress invalid ones.

That still left several ambiguous intermediate states, so the iOS implementation uses a conservative rule: an incomplete <tex> element such as <tex>xx produces no visible output until </tex> arrives. During streaming, the formula position simply remains in the loading state.

Text justification

YYLabel supports justified text through view.textAlignment = NSTextAlignmentJustified, but justification can reposition characters that are already visible whenever the line grows. That movement is acceptable for static text and distracting during streaming, so streaming cards avoid justified alignment.

Incorrect mask bounds at line breaks

The masks rely on YYTextLayout for the current character’s rectangle. Around some line breaks, that rectangle is incorrect.

How We Built a Streaming Rich Text Renderer for LLM Responses on iOS: Incorrect mask bounds at line breaks (figure 3)

In the figure, revealing cell N completes the line, so the next cursor position should be cell N+2. YYTextLayout instead returns the oversized N+1 rectangle. That rectangle overlaps content already revealed and content that should still be hidden, making text appear and then disappear behind the mask.

The bug comes from YYTextLayout including zero-width or zero-height rectangles in a union calculation. Ignoring those empty rectangles produces the expected cursor geometry.

How We Built a Streaming Rich Text Renderer for LLM Responses on iOS: Incorrect mask bounds at line breaks (figure 4)

Modify rectForRange: as follows:

1
2
3
4
5
6
7
8
9
10
11
12
- (CGRect)rectForRange:(YYTextRange *)range {
NSArray *rects = [self selectionRectsForRange:range];
if (rects.count == 0) return CGRectNull;
CGRect rectUnion = ((YYTextSelectionRect *)rects.firstObject).rect;
for (NSUInteger i = 1; i < rects.count; i++) {
YYTextSelectionRect *rect = rects[i];
if (fabs(CGRectGetWidth(rect.rect)) > CGFLOAT_MIN && fabs(CGRectGetHeight(rect.rect)) > CGFLOAT_MIN) {
rectUnion = CGRectUnion(rectUnion, rect.rect);
}
}
return rectUnion;
}

Underline spacing

How We Built a Streaming Rich Text Renderer for LLM Responses on iOS: Underline spacing (figure 5)

YYText does not expose the distance between text and its underline, but the product design requires that control. We modified YYTextDrawDecoration to support an underline offset.

1
2
3
4
5
6
7
8
+ (instancetype) decorationWithStyle: (YYTextLineStyle)style width: (nullable NSNumber *)width color: (nullable
UIColor *)color;
@property (nonatomic) YYTextLineStyle style; ///< line style
@property (nullable, nonatomic, strong) NSNumber *width; ///< line width (nil means automatic width)
@property (nullable, nonatomic, strong) NSNumber *offset; ///< line offset(nil means 0, only for underline)
@property (nullable, nonatomic, strong) UIColor *color; ///< line color (nil means automatic color)
@property (nullable, nonatomic, strong) YYTextShadow *shadow; ///< line shadow
@end

YYTextDrawDecoration then adds offset to the underline’s y coordinate.

1
2
3
4
5
6
7
8
9
10
11
12
static void YYTextDrawDecoration(YYTextLayout *layout, CGContextRef context, CGSize size, CGPoint point, YYTextDecorationType type, BOOL (^cancel)(void)) {
TRunGetPositions(run, CFRangeMake (0, 1), &runPosition);
underlineStart.x = strikethroughStart.x = runPosition.x + line position.x;
length = CTRunGetTypographicBounds|run, CFRangeMake(0, 0), NULL, NULL, NULL);

// offset
if (underline.offset) {
underlineStart.y += underline.offset.floatValue;
} else if (underlineOffset) {
underlineStart.y += underline0ffset.floatValue;
}
......

This change can push the final underline beyond the original layout bounds, where it gets clipped. The layout height must therefore include the additional offset.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
+ (YYTextLayout *)layoutWithContainer:(YYTextContainer *)container text:(NSAttributedString *)text range:(NSRange)range {
......
{
YYTextLine *lastLine = lines.lastObject;
CFArrayRef runs = CTLineGetGlyphRuns(lastLine.CTLine);
if (runs != nil) {
CGFloat underlinePositionCustom = 0.f;
YYTextGetRunsMaxMetricOfAttachment(runs, NULL, &underlinePositionCustom, NULL);

CGFloat maxUnderlineOffset = 0.f;
for (NSUInteger r = 0, rMax = CFArrayGetCount(runs); r < rMax; r++) {
CTRunRef run = CFArrayGetValueAtIndex(runs, r);
CFIndex glyphCount = CTRunGetGlyphCount(run);
if (glyphCount == 0) continue;
NSDictionary *attrs = (id)CTRunGetAttributes(run);
YYTextDecoration *underline = attrs[YYTextUnderlineAttributeName];
if (underlinePositionCustom < 0) {
maxUnderlineOffset = MAX(maxUnderlineOffset, fabs(underlinePositionCustom * -1 + underline.width.floatValue));
}
}

if (maxUnderlineOffset > 0) {
CGRect tempRect = textBoundingRect;
tempRect.size.height += maxUnderlineOffset;
textBoundingRect = tempRect;

CGSize tempSize = textBoundingSize;
tempSize.height += maxUnderlineOffset;
textBoundingSize = tempSize;
}
}
}
layout.frameSetter = ctSetter;
layout.frame = ctFrame;
......
if (lineOrigins) free(lineOrigins);
return layout;

Underlining text attachments

Formulas and images enter the NSAttributedString as attachments, often taller than the surrounding text. If the underline still uses the text baseline, it may cut through the attachment.

How We Built a Streaming Rich Text Renderer for LLM Responses on iOS: Underlining text attachments (figure 6)

The line layout now inspects each attachment’s ascent and descent and shifts the decoration using the maximum descent on that line.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
static void YYTextGetRunsMaxMetricOfAttachment(CFArrayRef  _Nonnull runs, CGFloat * _Nullable xHeight, CGFloat * _Nonnull underlinePosition, CGFloat * _Nullable lineThickness) {
CGFloat maxXHeight = 0;
CGFloat maxUnderlinePos = 0;
for (NSUInteger i = 0, max = CFArrayGetCount(runs); i < max; i++) {
CTRunRef run = CFArrayGetValueAtIndex(runs, i);
CFDictionaryRef attrs = CTRunGetAttributes(run);
if (attrs) {
YYTextDecoration *underline = CFDictionaryGetValue(attrs, (__bridge CFStringRef)YYTextUnderlineAttributeName);
YYTextDecoration *attachment = CFDictionaryGetValue(attrs, (__bridge CFStringRef)YYTextAttachmentAttributeName);
YYTextRunDelegate *runDelegate = CTRunDelegateGetRefCon(CFDictionaryGetValue(attrs, kCTRunDelegateAttributeName));
if (underline && attachment && runDelegate) {
if (runDelegate.descent * -1 < maxUnderlinePos) {
maxUnderlinePos = runDelegate.descent * -1;
}
if (runDelegate.ascent < maxXHeight) {
maxXHeight = runDelegate.ascent;
}
}
}
}
if (underlinePosition && *underlinePosition > maxUnderlinePos) {
*underlinePosition = maxUnderlinePos;
}
if (xHeight && *xHeight < maxXHeight) {
*xHeight = maxXHeight;
}
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
static void YYTextDrawDecoration(YYTextLayout *layout, CGContextRef context, CGSize size, CGPoint point, YYTextDecorationType type, BOOL (^cancel)(void)) {
......
for (NSUInteger l = 0, lMax = layout.lines.count; l < lMax; l++) {
if (cancel && cancel()) break;

YYTextLine *line = lines[l];
if (layout.truncatedLine && layout.truncatedLine.index == line.index) line = layout.truncatedLine;
CFArrayRef runs = CTLineGetGlyphRuns(line.CTLine);
for (NSUInteger r = 0, rMax = CFArrayGetCount(runs); r < rMax; r++) {
......

CGFloat xHeight, underlinePosition, lineThickness;
YYTextGetRunsMaxMetric(runs, &xHeight, &underlinePosition, &lineThickness);

// get the runs max metric of attachment
YYTextGetRunsMaxMetricOfAttachment(runs, &xHeight, &underlinePosition, &lineThickness);

......

This fix depends on every attachment reporting accurate ascent and descent values.

Flicker and cursor drift

Another failure appeared when network chunks arrived faster than the reveal timer: the view could adopt a new layout while the masks still referenced the previous one. Repeated YYText refreshes then caused both cursor drift and flicker.

The fix is to stage new layouts instead of applying them immediately. The renderer finishes revealing its current layout, swaps in the newest pending layout, and continues. If no pending layout exists and the network stream has ended, rendering is complete.

Several network updates may replace the pending layout before the current one finishes. Coalescing them this way reduces view refreshes, keeps mask geometry consistent, and improves performance.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
- (void)updateStreamingTextLayout:(YYTextLayout *)textLayout {
if (!self.isStreamingRendering) {
return;
}

// Position animateView the first time a layout is set.
if (![[XXXTimer defaultTimer] hasTimerWithName:self.identify]) {
[self updateTextLayoutAndFrame:textLayout];
[self.animateView locateInRect:CGRectMake(0, 0, 0, CGRectGetHeight(self.loadingView.frame))];
} else {
// Stage the latest layout.
self.tempTextLayout = textLayout;
}

// Start the reveal timer.
[self startRenderingTimer];
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
- (void)startRenderingTimer {
if (![[XXXTimer defaultTimer] hasTimerWithName:self.identify]) {
[[XXXTimer defaultTimer] scheduledTimerWithName:self.identify timeInterval:self.renderingInterval queue:dispatch_get_main_queue() startImmediately:YES repeat:YES action:^{
NSUInteger length = self.textLayout.text.yy_rangeOfAll.length;
if (self.rectIndex < length) {
[self informChangeStreamingRenderingWithInfo];
self.rectIndex++;
} else {
if (self.tempTextLayout != nil && self.textLayout != self.tempTextLayout) {
// update the text layout and frame
[self updateTextLayoutAndFrame:self.tempTextLayout];
self.tempTextLayout = nil;
} else {
if (self.isStreamingRendering) {
// wait
} else {
[self stopAnimation];
[self stopRenderingTimer];
[self informStopStreamingRenderingWithInfo];
}
}
}
}];
}
}

Syncing stream interruption progress to TTS

Our app also converts the rich text into plain speech input for TTS. Under normal conditions this is straightforward, but interruption creates a synchronization problem: the visible response may stop at ... after the full readable text has already entered the speech queue. The result is audio that continues beyond what the user can see.

To keep speech and display in sync, the renderer emits TTS input only as each character becomes visible. TTS maintains its own queue, so interrupting the visual stream naturally stops new speech at the same point. The message model also stores that character index, allowing later playback to reconstruct the same truncated response.

Ambiguous adjacent dollar delimiters

This edge case comes from TeX’s two traditional dollar-delimited math modes:

  • inline math mode, where formulas appear within a paragraph and use $...$.
  • display math mode, where formulas occupy their own line and use $$...$$.

Modern LaTeX documents generally prefer explicit delimiters:

  • For inline math mode, use \(... \) or \begin{math}...\end{math}, instead of $...$.
  • For display math mode, use \[... \] or \begin{displaymath}...\end{displaymath}, instead of $$...$$.

In this content format, LaTeX formulas are embedded inside HTML, so HTML parsing happens first. A custom <tex>...</tex> element marks formula ranges for the LaTeX renderer.

Formula text may itself contain characters that are meaningful to HTML. In <tex>1\+2<5</tex>, for example, < is a mathematical operator, but an HTML parser may interpret it as the start of a tag. The preprocessing pass therefore escapes <, >, &, ", and ' inside every <tex> range before HTML parsing begins.

LLM output may use any of these delimiter styles, so the client accepts all of them. The first implementation handled dollar delimiters with two regular expressions: match $$...$$ first, then $...$, and rewrite each match as <tex>...</tex>.

That approach assumes a literal dollar sign is escaped as \$; an unescaped $ can otherwise shift every later match.

1
2
3
4
5
6
7
8
9
10
- (void)preprocess {
// Replace $$xxx$$ with <tex>xxx</tex>.
[self replaceDoubleDollarWithTexTag];

// Replace $xxx$ with <tex>xxx</tex>.
[self replaceDollarWithTexTag];

// Escape HTML-sensitive characters inside tex tags.
[self processExcapeInTexTag];
}

The pattern $...$$...$ breaks this strategy. It represents two adjacent inline formulas, but the middle $$ also resembles a display-math delimiter. The display regex finds no complete pair; the inline regex then consumes the outer dollar signs and incorrectly sends ...$$... to the LaTeX parser as one formula.

We replaced the regular expressions with a single stateful scan:

  1. If the current $ closes an open $ range, close that inline formula.
  2. Otherwise, inspect the next character.
    1. If it is also $, treat the pair as a $$ delimiter and either open or close display math.
    2. If not, open a new inline formula with $.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
- (void)preprocess {
.....
if (curCh == L'$') {
if ([self.sentinelStack.lastObject isEqualToString:@"$"]) {
[self closeLatexRangeWithSentinel:@"$"];
self.index++;
} else if (self.index + 1 < self.len) {
wchar_t nextCh = *(self.str + self.index + 1);
if (nextCh == L'$') {
if ([self.sentinelStack.lastObject isEqualToString:@"$$"]) {
[self closeLatexRangeWithSentinel:@"$$"];
} else {
[self startLatexRangeWithSentinel:@"$$"];
}
self.index += 2;
} else {
[self startLatexRangeWithSentinel:@"$"];
self.index++;
}
} else {
[self appendWideCharacter:curCh];
self.index++;
}
}
.....
}

The same scanner handles $...$, $$...$$, \(...\), and \[...\], while escaping HTML-sensitive characters inside formula ranges. Keeping those decisions in one pass avoids the overlapping matches that made the regular-expression version fragile.

It is also faster: in our measurements, preprocessing took only 25–50% as long as the regular-expression implementation.

Summary

Building on YYText made it possible to ship pseudo-streaming without replacing the existing text stack. It also exposed several architectural limits:

  1. Several fixes require maintaining a fork of YYText, which increases the cost of upgrades and long-term maintenance.
  2. The host app has limited control over block elements such as tables and lists. YYText is primarily a text layout engine; once content mixes nested blocks, images, and tables, its attachment model makes precise layout control difficult.
  3. HTML and LaTeX converge too late in the pipeline. A formula enters YYText as one attachment view, so it cannot be revealed incrementally like ordinary text.

A longer-term solution would use one rendering engine for several rich-text syntaxes. Markdown, HTML, and LaTeX need different parsers, but they can still share a layout and rendering model. That would make nested blocks and incremental formula rendering easier to support without patching each syntax independently.

The remaining design questions are:

  1. Is it possible to build a truly streaming parser?
  2. Can the number of layout passes be reduced?
  3. Can layout and painting become genuinely incremental rather than merely appear incremental?
-------------本文结束 感谢您的阅读-------------
If this article helped you, your support encourages me to keep writing.

Title:How We Built a Streaming Rich Text Renderer for LLM Responses on iOS

Author:lingyun

Published:2024-08-06 16:25

Updated:2026-10-07 00:00

Original link:https://tsuijunxi.github.io/en/2024/08/06/streaming-rich-text-rendering-in-ios/

License: Attribution-NonCommercial-NoDerivatives 4.0 International Please keep the original link and author when sharing.