: Harnessing Generalizable Visual Generation via Symbolic Policy Learning

Xu Xu1,*, Jinxiu Liu2,*, Zhangbo Qiao1, Jiaxing Lu1
Xiangyu Zhang1, Yubin Gu3, Fangwei Ning1, Yan Shi1
1Beihang University 2The Chinese University of Hong Kong 3National University of Singapore

*Corresponding authors: Xu Xu (xuxu1@buaa.edu.cn), Jinxiu Liu (jinxiuliu0628@foxmail.com)

Abstract

Unified multimodal large language models (MLLMs) and multi-agent systems have advanced visual generation. However, three limitations remain. (1) Existing methods often distill task-specific experience with limited generalizability. (2) Reflection is often deferred until task completion. (3) Knowledge is often acquired only in response to downstream task demands. To address these limitations, we introduce OmniHarness, a framework for generalizable visual generation via symbolic policy learning. OmniHarness abstracts verified executions into symbolic policies for visual generation task families, capturing shared procedures and applicability conditions while removing instance-specific inputs. The harness instantiates, adapts, and composes these policies for new tasks. Intermediate verification guides refinement and failure recovery during execution. Through self-directed inquiry, OmniHarness autonomously generates and executes practice tasks near its capability limits before downstream objectives are specified. Execution feedback continually refines the policies while model parameters remain fixed. Experiments across six benchmarks, three MLLM backbones, and three visual agent frameworks demonstrate strong performance and continual capability expansion. On ComfyBench's Creative tasks, OmniHarness achieves a 95.0% resolve rate, exceeding the strongest baseline by 27.5 percentage points. Frozen policy snapshots improve existing visual agent systems through plug-and-play reuse.

Text-to-Image Generation

Reasoning Generation

Image Editing

Remove the plate together with all the food inside it from the table.

Original dish table image
Original
Original dish table image Edited dish table result
Interactive comparison

Text-to-Video Generation

Image-to-Video Generation

Video-to-Video Generation

Comparison of Visual Generation Paradigms

Comparison of end-to-end unified MLLMs, existing multi-agent systems, and OmniHarness for generalizable visual generation via symbolic policy learning

(a) End-to-end unified MLLMs enable fast multimodal generation but lack deliberate reasoning and self-correction.

(b) Existing multi-agent systems improve collaborative reasoning but lack unified coordination and persistent knowledge accumulation.

(c) OmniHarness integrates self-directed inquiry and feedback-guided execution to learn reusable symbolic policies for generalizable visual generation.

Architecture of OmniHarness

Architecture of OmniHarness for generalizable visual generation via symbolic policy learning

Self-directed inquiry and feedback-guided execution drive symbolic policy learning for generalizable visual generation. The policy library evolves during OmniHarness execution, while frozen snapshots support plug-and-play reuse across visual agents.

Comprehensive Benchmark Evaluation

Differences are relative to the first method in each group. Red indicates an increase, green a decrease, and gray no change; metric arrows indicate whether higher or lower is better.

1. ComfyBench

ComfyBench evaluates autonomous workflow construction, where each agent must produce an executable ComfyUI workflow that satisfies the task requirements.

Quantitative Results

Agent Vanilla Complex Creative Total
Pass↑ Res.↑ Pass↑ Res.↑ Pass↑ Res.↑ Pass↑ Res.↑
GPT-4o + Zero-shot 0.00.00.00.00.00.00.00.0
GPT-4o + Few-shot 32.0↑32.027.0↑27.016.7↑16.78.3↑8.37.5↑7.50.0=0.022.5↑22.516.0↑16.0
GPT-4o + CoT 44.0↑44.029.0↑29.011.7↑11.78.3↑8.312.5↑12.50.0=0.028.0↑28.017.0↑17.0
GPT-4o + CoT-SC 45.0↑45.034.0↑34.011.7↑11.75.0↑5.015.0↑15.00.0=0.029.0↑29.018.5↑18.5
Claude-3.5-Sonnet + RAG 27.0↑27.013.0↑13.023.0↑23.06.7↑6.77.5↑7.50.0=0.022.0↑22.08.5↑8.5
Llama-3.1-70B + RAG 58.0↑58.032.0↑32.023.0↑23.010.0↑10.015.0↑15.05.0↑5.039.0↑39.020.0↑20.0
GPT-4o + RAG 62.0↑62.041.0↑41.045.0↑45.021.7↑21.740.0↑40.07.5↑7.552.0↑52.023.0↑23.0
o1-mini + RAG 32.0↑32.016.0↑16.021.7↑21.78.3↑8.312.5↑12.57.5↑7.525.0↑25.012.0↑12.0
o1-preview + RAG 70.0↑70.046.0↑46.048.3↑48.323.3↑23.330.0↑30.012.5↑12.555.5↑55.532.5↑32.5
Llama-3.1-70B + ComfyAgent 63.035.026.718.320.05.043.524.0
GPT-4o + ComfyAgent 67.0↑4.046.0↑11.048.3↑21.621.7↑3.440.0↑20.015.0↑10.056.0↑12.532.5↑8.5
GPT-4o + ComfyMind 100.0↑37.092.0↑57.0100.0↑73.385.0↑66.7100.0↑80.057.5↑52.5100.0↑56.583.0↑59.0
DeepSeek-V3 + ComfyMind 100.0↑37.090.0↑55.0100.0↑73.371.7↑53.4100.0↑80.060.0↑55.0100.0↑56.578.5↑54.5
Gemini-2.5-Flash + SymbOmni 100.0↑37.095.0↑60.0100.0↑73.383.3↑65.0100.0↑80.067.5↑62.5100.0↑56.586.0↑62.0
GPT-4o + OmniHarness 100.0↑37.095.0↑60.0100.0↑73.376.7↑58.4100.0↑80.095.0↑90.0100.0↑56.589.5↑65.5
Codex GPT-4o + OmniHarness 100.0↑37.097.0↑62.0100.0↑73.383.3↑65.0100.0↑80.095.0↑90.0100.0↑56.592.5↑68.5
Table 1. Quantitative comparison on ComfyBench for autonomous workflow construction. Pass and Resolve are reported as percentages, with higher values indicating better performance.

Qualitative Results

Qualitative comparison on Creative text-to-image tasks including a series poster, handwritten letter, comic strip, and book cover
Figure 1. Qualitative comparison on representative Creative T2I tasks from ComfyBench. These examples evaluate instruction following, text rendering, layout control, multi-panel consistency, and complex visual composition.
Qualitative comparison on image-to-image editing tasks using the same input table image
Figure 2. Qualitative comparison on representative I2I tasks from ComfyBench. These examples evaluate precise target localization, instruction following, and the preservation of unrelated scene content.
Qualitative comparison on challenging ComfyBench image editing and reference-style transfer tasks
Figure 3. Qualitative comparison on representative challenging tasks from the Complex and Creative subsets. These examples evaluate multi-step workflow composition, reference-style transfer, content preservation, restoration quality, and fine-grained instruction following.

2. GenEval

GenEval measures compositional text-to-image fidelity through single-object generation, two-object co-occurrence, counting, color, relative position, and attribute binding.

Quantitative Results

Method Single Obj.↑ Two Obj.↑ Counting↑ Colors↑ Position↑ Attr. Bind.↑ Overall↑
Frozen Text-Encoder Mapping Methods
SDv1.5 0.970.380.350.760.040.060.43
SDv2.1 0.98↑0.010.51↑0.130.44↑0.090.85↑0.090.07↑0.030.17↑0.110.50↑0.07
SD-XL 0.98↑0.010.74↑0.360.39↑0.040.85↑0.090.15↑0.110.23↑0.170.55↑0.12
DALL-E 2 0.94↓0.030.66↑0.280.49↑0.140.77↑0.010.10↑0.060.19↑0.130.52↑0.09
SD3-Medium 0.99↑0.020.94↑0.560.72↑0.370.89↑0.130.33↑0.290.60↑0.540.74↑0.31
Unified Multimodal Models
LlamaGen 0.710.340.210.580.070.040.32
LWM 0.93↑0.220.41↑0.070.46↑0.250.79↑0.210.09↑0.020.15↑0.110.47↑0.15
SEED-X 0.97↑0.260.58↑0.240.26↑0.050.80↑0.220.19↑0.120.14↑0.100.49↑0.17
Emu3-Gen 0.98↑0.270.71↑0.370.34↑0.130.81↑0.230.17↑0.100.21↑0.170.54↑0.22
Janus 0.97↑0.260.68↑0.340.30↑0.090.84↑0.260.46↑0.390.42↑0.380.61↑0.29
JanusFlow 0.97↑0.260.59↑0.250.45↑0.240.83↑0.250.53↑0.460.42↑0.380.63↑0.31
Janus-Pro-7B 0.99↑0.280.89↑0.550.59↑0.380.90↑0.320.79↑0.720.66↑0.620.80↑0.48
GoT 0.99↑0.280.69↑0.350.67↑0.460.85↑0.270.34↑0.270.27↑0.230.64↑0.32
Bagel 0.98↑0.270.94↑0.600.76↑0.550.91↑0.330.69↑0.620.70↑0.660.78↑0.46
GPT-Image-1 0.99↑0.280.92↑0.580.85↑0.640.92↑0.340.75↑0.680.61↑0.570.84↑0.52
Collaborative AI Systems
ComfyAgent 0.690.300.330.500.040.040.32
ComfyMind 1.00↑0.311.00↑0.700.96↑0.630.97↑0.470.63↑0.590.81↑0.770.90↑0.58
SymbOmni 1.00↑0.311.00↑0.700.99↑0.660.98↑0.480.97↑0.930.95↑0.910.98↑0.66
OmniHarness 1.00↑0.311.00↑0.701.00↑0.671.00↑0.501.00↑0.960.98↑0.940.997↑0.677
Table 2. Quantitative comparison on GenEval. Scores measure the fraction of correctly generated images for six compositional tasks, including single-object generation, two-object co-occurrence, counting, color, relative position, and attribute binding.

Qualitative Results

Qualitative comparison on GenEval compositional text-to-image generation tasks
Figure 4. Qualitative comparison on representative GenEval tasks covering single-object generation, two-object co-occurrence, counting, color, spatial relations, and attribute binding.

3. GenEval2

GenEval2 provides a fine-grained evaluation of text-to-image generation across object generation, attribute rendering, counting, spatial relations, and transitive verb relations.

Quantitative Results

Method Object↑ Attribute↑ Count↑ Position↑ Verb↑ Overall↑
Stable Diffusion Model Series
SD 2.1 55.130.422.311.717.827.46
SDXL 74.1↑19.042.4↑12.028.7↑6.416.0↑4.328.9↑11.138.02↑10.56
SD3 87.0↑31.965.5↑35.149.9↑27.641.0↑29.346.7↑28.958.02↑30.56
SD3.5-Large 91.6↑36.570.3↑39.952.2↑29.939.5↑27.855.6↑37.861.84↑34.38
State-of-the-Art Text-to-Image Models
FLUX.1-dev 88.468.355.637.044.458.74
Bagel + CoT 92.9↑4.575.9↑7.655.6=0.050.6↑13.657.8↑13.466.56↑7.82
Qwen-Image 99.1↑10.785.6↑17.370.3↑14.760.2↑23.271.1↑26.777.26↑18.52
Gemini 2.5 Flash Image 99.0↑10.691.4↑23.170.1↑14.570.2↑33.286.7↑42.383.48↑24.74
Collaborative AI Systems
SymbOmni 95.083.674.868.864.577.34
OmniHarness 95.0=0.094.0↑10.494.0↑19.276.9↑8.189.0↑24.589.78↑12.44
Table 3. Quantitative comparison on GenEval2. Overall is the arithmetic mean of all five skill scores.

Qualitative Results

Qualitative comparison on GenEval2 fine-grained compositional text-to-image generation tasks
Figure 5. Qualitative comparison on representative GenEval2 tasks covering object generation, attribute rendering, counting, spatial relations, and transitive verb relations.

4. WISE

WISE evaluates world-knowledge-informed visual synthesis across cultural commonsense, temporal and spatial reasoning, biology, physics, and chemistry.

Quantitative Results

Method Cultural↑ Time↑ Space↑ Biology↑ Physics↑ Chemistry↑ Overall↑
Dedicated T2I Models
SDv1.5 0.340.350.320.280.290.210.32
SDv2.1 0.30↓0.040.38↑0.030.35↑0.030.33↑0.050.34↑0.050.21=0.000.32=0.00
SD-XL 0.43↑0.090.48↑0.130.47↑0.150.44↑0.160.45↑0.160.27↑0.060.43↑0.11
SD3-Medium 0.42↑0.080.44↑0.090.48↑0.160.39↑0.110.47↑0.180.29↑0.080.42↑0.10
SD3.5-Medium 0.43↑0.090.50↑0.150.52↑0.200.41↑0.130.53↑0.240.33↑0.120.45↑0.13
SD3.5-Large 0.44↑0.100.50↑0.150.58↑0.260.44↑0.160.52↑0.230.31↑0.100.46↑0.14
PixArt-Alpha 0.45↑0.110.50↑0.150.48↑0.160.49↑0.210.56↑0.270.34↑0.130.47↑0.15
Playground-v2.5 0.49↑0.150.58↑0.230.55↑0.230.43↑0.150.48↑0.190.33↑0.120.49↑0.17
FLUX.1-schnell 0.39↑0.050.44↑0.090.50↑0.180.31↑0.030.44↑0.150.26↑0.050.40↑0.08
FLUX.1-dev 0.48↑0.140.58↑0.230.62↑0.300.42↑0.140.51↑0.220.35↑0.140.50↑0.18
Unified MLLM Models
Janus-1.3B 0.160.260.350.280.300.140.23
JanusFlow-1.3B 0.13↓0.030.26=0.000.28↓0.070.20↓0.080.19↓0.110.11↓0.030.18↓0.05
Janus-Pro-1B 0.20↑0.040.28↑0.020.45↑0.100.24↓0.040.32↑0.020.16↑0.020.26↑0.03
Janus-Pro-7B 0.30↑0.140.37↑0.110.49↑0.140.36↑0.080.42↑0.120.26↑0.120.35↑0.12
Show-o 0.28↑0.120.36↑0.100.40↑0.050.23↓0.050.33↑0.030.22↑0.080.30↑0.07
Show-o-512 0.28↑0.120.40↑0.140.48↑0.130.30↑0.020.46↑0.160.30↑0.160.35↑0.12
VILA-U-7B 0.26↑0.100.33↑0.070.37↑0.020.35↑0.070.39↑0.090.23↑0.090.31↑0.08
Orthus-7B-base 0.07↓0.090.10↓0.160.12↓0.230.15↓0.130.15↓0.150.10↓0.040.10↓0.13
Orthus-7B-instruct 0.23↑0.070.31↑0.050.38↑0.030.28=0.000.31↑0.010.20↑0.060.27↑0.04
Emu3 0.34↑0.180.45↑0.190.48↑0.130.41↑0.130.45↑0.150.27↑0.130.39↑0.16
BAGEL 0.44↑0.280.55↑0.290.68↑0.330.44↑0.160.60↑0.300.39↑0.250.52↑0.29
BAGEL + CoT 0.76↑0.600.69↑0.430.75↑0.400.65↑0.370.75↑0.450.58↑0.440.70↑0.47
Closed-Source Models
GPT-Image-1 0.810.710.890.830.790.740.80
Collaborative AI Systems
ComfyMind 0.850.660.720.670.700.780.76
SymbOmni 0.90↑0.050.70↑0.040.74↑0.020.75↑0.080.74↑0.040.78=0.000.80↑0.04
OmniHarness 0.88↑0.030.85↑0.190.86↑0.140.84↑0.170.82↑0.120.86↑0.080.86↑0.10
Table 4. Quantitative comparison on WISE. WiScore evaluates world-knowledge-informed semantic synthesis across cultural commonsense, spatiotemporal reasoning, and natural sciences.

Qualitative Results

Qualitative comparison on WISE world-knowledge-informed text-to-image generation tasks
Figure 6. Qualitative comparison on representative WISE tasks spanning cultural, temporal, spatial, biological, physical, and chemical knowledge.

5. Reason-Edit

Reason-Edit evaluates instruction-based image editing through Understanding Scenarios with explicit target cues and Reasoning Scenarios with indirect commonsense descriptions, emphasizing accurate target localization and preservation of unrelated content.

Quantitative Results

Method Understanding Scenarios Reasoning Scenarios
PSNR↑ SSIM↑ LPIPS↓ CLIP↑ PSNR↑ SSIM↑ LPIPS↓ CLIP↑
InstructPix2Pix 21.580.720.0922.7624.230.710.0819.41
MagicBrush 18.12↓3.460.68↓0.040.14↑0.0522.62↓0.1422.10↓2.130.69↓0.010.11↑0.0319.76↑0.34
InstructDiffusion 23.26↑1.680.74↑0.020.07↓0.0223.08↑0.3221.45↓2.780.67↓0.040.12↑0.0319.52↑0.11
SmartEdit-7B 22.05↑0.470.73↑0.010.09≈0.0023.61↑0.8525.26↑1.020.74↑0.040.06↓0.0320.95↑1.54
SmartEdit-13B 23.60↑2.020.75↑0.030.07↓0.0223.54↑0.7725.76↑1.520.75↑0.040.05↓0.0320.78↑1.36
InsightEdit 23.59↑2.010.75↑0.030.07↓0.0223.73↑0.9725.71↑1.480.75↑0.040.05↓0.0320.87↑1.45
OmniHarness 23.89↑2.320.86↑0.140.05↓0.0424.55↑1.7923.87↓0.360.80↑0.090.05↓0.0321.32↑1.91
Table 5. Quantitative comparison on Reason-Edit. PSNR (dB), SSIM, and LPIPS measure background preservation, while CLIP Score measures foreground alignment with the target label. Differences use InstructPix2Pix as the baseline and are computed from the original scores before rounding to two decimals.

Qualitative Results

Qualitative Reason-Edit results for Understanding Scenarios with explicit target cues
Figure 7. Qualitative results of OmniHarness on Reason-Edit Understanding Scenarios. Editing targets are specified through explicit visual attributes, including position, color, mirror relations, relative size, and object addition.
Qualitative Reason-Edit results for Reasoning Scenarios with indirect commonsense descriptions
Figure 8. Qualitative results of OmniHarness on Reason-Edit Reasoning Scenarios. Object removal or replacement is specified through indirect commonsense descriptions, while unrelated content should remain unchanged.

6. KRIS-Bench

KRIS-Bench evaluates factual, conceptual, and procedural knowledge in knowledge-intensive visual generation and editing, spanning reasoning, transformation, and multi-image composition tasks.

Quantitative Results

Method Factual↑ Conceptual↑ Procedural↑ Overall↑
Closed-Source Models
GPT-Image-1 79.8081.3778.3280.09
Gemini 2.0 Flash Experimental 65.26↓14.5459.65↓21.7262.90↓15.4262.41↓17.68
Doubao 63.30↓16.5062.23↓19.1454.17↓24.1560.70↓19.39
Open-Source Models
BAGEL-Think 55.7759.4439.2653.36
BAGEL 47.71↓8.0652.17↓7.2740.23↑0.9747.76↓5.60
Step1X-Edit 45.52↓10.2548.01↓11.4331.82↓7.4443.29↓10.07
Emu2 45.40↓10.3737.54↓21.9034.91↓4.3539.70↓13.66
AnyEdit 39.26↓16.5141.88↓17.5631.74↓7.5238.55↓14.81
MagicBrush 41.84↓13.9339.24↓20.2026.54↓12.7237.15↓16.21
OmniGen 33.11↓22.6628.02↓31.4223.89↓15.3728.85↓24.51
InsPix2Pix 23.33↓32.4425.59↓33.8517.28↓21.9822.82↓30.54
Collaborative AI Systems
SymbOmni 73.3372.2870.2972.18
OmniHarness 74.81↑1.4881.89↑9.6173.22↑2.9377.33↑5.15
Table 6. Quantitative comparison on KRIS-Bench. Scores evaluate factual, conceptual, and procedural knowledge, together with overall performance. Higher values indicate better performance.

Qualitative Results

Qualitative OmniHarness results on knowledge-intensive KRIS-Bench tasks
Figure 9. Qualitative results of OmniHarness on representative KRIS-Bench tasks, covering factual, conceptual, and procedural knowledge through knowledge-guided generation, editing, visual reasoning, and multi-image composition.

BibTeX

@misc{xu2026omniharnessharnessinggeneralizablevisual,
  title={OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning},
  author={Xu Xu and Jinxiu Liu and Zhangbo Qiao and Jiaxing Lu and Xiangyu Zhang and Yubin Gu and Fangwei Ning and Yan Shi},
  year={2026},
  eprint={2609.16057},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
  url={https://arxiv.org/abs/2609.16057},
}