GlyphAtlasDisentangling Stroke Detail from Layout for Scene Text Synthesis
Luozheng Qin1Changhai Zhou2Yiyi Zhou1
1 Xiamen University2 Fudan University
↗ View full sizeGlyphAtlas renders stroke-accurate text that blends seamlessly into real scenes. It preserves text accuracy in dense, multi-line layouts, harmonizes typography with scene geometry and lighting, remains legible for small or structurally complex glyphs, and transfers zero-shot to scripts unseen during task-specific training.
Scene text synthesis requires stroke-accurate characters in a specified layout and visual coherence with the surrounding scene. Existing glyph-image conditioning ties each glyph’s resolution to its target region, limiting fine-grained stroke guidance for small, dense or structurally complex text. GlyphAtlas retains the glyph image as a spatial anchor and supplies detailed character or word references through a dedicated glyph atlas, whose cell resolution is independent of target-region size. Atlas-anchored attention lets each text region retrieve detail from its corresponding atlas cells to refine the coarse glyph structure provided by the glyph image. This coarse-to-fine conditioning preserves explicit layout correspondence and supports accurate scene text generation and editing, including zero-shot transfer to scripts unseen during task-specific training. Random resolution augmentation further enables adjustable atlas budgets at inference.
Method
Framework
Built on FLUX.1-Fill-dev, GlyphAtlas retains the scene-aligned glyph image for layout and coarse glyph structure, and introduces a glyph atlas for fine-grained stroke guidance. Each atlas cell contains a character or word rendered at a resolution independent of its target region. The backbone’s frozen VAE encodes these detailed stroke references, which are appended to the transformer’s token sequence alongside prompt, scene and glyph-image tokens.
↗ View full sizeGlyphAtlas combines the glyph image and glyph atlas through atlas-anchored attention for coarse-to-fine conditioning.
Atlas-anchored attention binds the atlas to the scene layout. Scene and glyph-image tokens retrieve stroke detail from the atlas cells assigned to the regions they overlap, while each cell attends to the prompt and its own region. This coarse-to-fine conditioning refines both the scene features and their glyph condition while preserving regional correspondence. Mask-weighted flow matching and a timestep-weighted text perceptual loss jointly supervise text accuracy and scene consistency, while random resolution augmentation makes the atlas token budget adjustable at inference.
Atlas Construction
Each target string is first split into an ordered sequence of text units: individual characters for scripts such as Chinese and Korean, and whole words for Latin and Arabic to preserve word structure and joining forms. Each unit is then rendered in a uniform font into its own cell, square for a character and of natural width for a word. The cell height is set independently of the target region, so even a small scene glyph receives a detailed stroke reference.
↗ View full sizeGlyph atlas construction: target strings are split into text units, rendered into cells, and encoded into atlas tokens.
Finally, the frozen VAE encodes each cell, and patchification converts its latent into atlas tokens. The tokens are projected to the model width and arranged on a separate positional canvas, with one row per region and cells in reading order. Atlas-anchored attention establishes their correspondence with the scene. Adjusting cell height changes both the detail of the reference and its token cost.
Results · 01
Accurate Scene Text Generation and Editing
GlyphAtlas achieves the highest text accuracy among the compared methods on AnyText-benchmark, TextOCR, Total-Text and ReCTS. Detailed local references support both new text generation and editing in real scenes, including small regions, complex Chinese characters and irregular layouts.
Table 1Scene text generation on AnyText-benchmark
Method
English
Chinese
ACC ↑
NED ↑
FID ↓
LPIPS ↓
ACC ↑
NED ↑
FID ↓
LPIPS ↓
DiffSTE
45.23
0.7814
52.74
0.1816
3.63
0.1226
57.49
0.1276
TextDiffuser
51.76
0.7618
29.76
0.1564
5.59
0.1218
34.19
0.1252
DiffUTE
40.54
0.7005
25.35
0.1640
29.78
0.5744
29.08
0.1745
AnyText
68.43
0.8588
21.59
0.1106
64.76
0.8210
20.01
0.0943
TextCtrl
58.53
0.8146
35.73
0.1978
35.80
0.6084
49.79
0.2298
DreamText
64.95
0.8466
28.37
0.1314
59.03
0.8270
58.22
0.0963
AnyText2
66.79
0.8610
32.59
0.1762
63.29
0.8075
29.23
0.1383
TextSSR
58.64
0.7664
24.23
0.1245
50.33
0.7018
29.41
0.1063
TextFlux
74.81
0.8789
13.15
0.0845
54.73
0.7705
13.02
0.0577
FLUX-Text
84.19
0.9400
13.85
0.0729
71.32
0.8510
13.68
0.0541
GlyphAtlas
88.68
0.9555
11.99
0.0759
77.92
0.8806
11.50
0.0534
Table 2Scene text editing on real-scene benchmarks
Method
TextOCR
Total-Text
ReCTS
ACC ↑
NED ↑
FID ↓
ACC ↑
NED ↑
FID ↓
ACC ↑
NED ↑
FID ↓
AnyText
8.12
0.1738
5.48
24.74
0.5139
31.45
34.81
0.5071
22.19
DreamText†
7.47
0.1725
18.92
27.09
0.5217
52.27
48.20
0.7017
54.59
AnyText2
9.54
0.2084
13.20
27.40
0.5422
73.71
35.86
0.5091
39.64
FLUX-Text
24.11
0.3614
2.60
64.06
0.8111
16.87
48.35
0.6477
14.58
TextFlux
23.62
0.3653
2.10
62.29
0.7829
13.58
43.02
0.5930
11.97
TextSSR
27.74
0.4499
2.71
47.06
0.6835
26.60
50.23
0.6876
20.96
GlyphAtlas
33.15
0.5046
2.81
73.17
0.8647
16.05
69.27
0.8259
13.27
ACC is reported in percent. Higher ACC and NED, and lower FID and LPIPS, are better. Bold marks the best result in each column.
Results · 02
Zero-Shot Transfer Across Scripts
GlyphAtlas-Bi is trained on a Chinese–English subset and evaluated on Japanese, Korean and Arabic text editing. At inference, the glyph atlas supplies detailed stroke references for the target script, which the VAE encodes and atlas-anchored attention associates with the corresponding scene regions. The same model achieves the highest ACC among the compared methods without language-specific fine-tuning across all three scripts.
Table 3Zero-shot multilingual editing on STIPLAR
Method
Japanese
Korean
Arabic
ACC ↑
FID ↓
ACC ↑
FID ↓
ACC ↑
FID ↓
AnyText2
47.79
105.62
29.82
63.52
0.10
99.32
FLUX-Text
46.49
80.73
20.83
44.39
2.65
77.30
TextFlux
46.23
67.43
33.86
24.01
6.42
51.96
STELLAR‡
43.38
73.73
80.47
34.47
55.86
62.84
TextSSR
40.26
102.86
19.14
68.11
0.20
92.54
GlyphAtlas-Bi
67.53
84.09
66.96
38.99
39.55
68.10
‡ STELLAR uses a separate model fine-tuned for each language. GlyphAtlas-Bi uses one Chinese–English model for all three scripts.
Results · 03
Qualitative Comparison Results
We compare GlyphAtlas with AnyText2, TextFlux and FLUX-Text on AnyText-benchmark, TextOCR, Total-Text and ReCTS. GlyphAtlas renders the target text accurately across stylized lettering, engraved watch numerals, curved signs and small Chinese shop text, while preserving the appearance of the text and its surroundings.
↗ View full sizeQualitative comparisons on AnyText-benchmark, TextOCR, Total-Text and ReCTS. Gray marks the regions to be filled; GT denotes ground truth.
Results · 04
Component Ablations and Atlas Budgeting
Glyph References and Atlas-Anchored Attention
The glyph image supplies spatial structure; the atlas adds stroke detail. Together with atlas-anchored attention, they improve small-text accuracy from 17.65% to 27.21% and Korean accuracy from 37.90% to 66.96% over the glyph-image-only model under the same training recipe.
Table 4Glyph references and attention · ACC (%)
Glyph reference
Attention
AnyText
Total-Text
STIPLAR
Image
Atlas
English
Chinese
All
Curved
Small
Cluttered
Japanese
Korean
Arabic
✓
—
—
83.90
67.93
51.33
50.86
17.65
48.51
60.26
37.90
19.27
—
✓
Anchored
80.30
74.28
50.53
48.41
22.43
47.79
67.01
58.25
39.96
✓
✓
Full
84.64
75.76
54.25
52.08
21.69
52.12
70.91
65.22
32.93
✓
✓
Anchored
85.89
76.00
56.97
53.91
27.21
56.08
67.53
66.96
39.55
Atlas Budgeting
Randomized cell heights make the model work across atlas resolutions. A square character cell uses 16 tokens at 64 pixels and 9 tokens at 48 pixels, a 43.75% reduction. The allocator instead assigns more detail to small scene glyphs while managing the total sequence length.
Table 5Atlas budget on AnyText-benchmark
Inference
English
Chinese
Atlas length
ACC ↑
Atlas length
ACC ↑
Mean
Max
Fixed
Random
Mean
Max
Fixed
Random
64 px
291
784
88.48
88.68
175
1,200
77.77
77.92
48 px
164
441
87.91
88.10
99
675
72.56
74.81
Alloc.
267
512
88.15
88.22
194
794
75.48
76.86
Fixed and Random refer to training with a fixed atlas height or randomized cell heights. Alloc. assigns cell resolutions according to the token budget and glyph size in the scene.
