Paper
Compile by Training: Turning Natural-Language Specifications into Local Neural Functions
First arXiv submission: · EMNLP 2026 System Demonstrations
The paper reports public-service measurements from May 2026. Submission history ↗
Compiles a function description into a reusable local program by generating training examples and optimizing an adapter.
Read paper ↗ · ArXiv preprint ↗ · Code ↗ · Demo ↗
What it does
Makes task-specific training a software build step. Raises FuzzyBench-Hard semantic accuracy (LEM) from 22.4% to 83.6% and demonstrates composition in a website helper, avatar control, and English–Claudish translation.
- Conditioning
- The specification guides teacher synthesis. PAW’s fast compiler supplies the initial adapter and runtime prompt.
- Program form
- A LoRA adapter, runtime prompt, specification, and interpreter metadata in a PAW program.
- Execution
- A frozen Qwen3-0.6B interpreter runs the saved function locally without further teacher calls.
Related work
Extends PAW with training at compile time while keeping its program format and runtime interface.
Related terms: compile by training, training-based compilation, specification-specific optimization, synthetic supervision, local neural function, FuzzyBench-Hard, ProgramAsWeights.
Related methods
Fast prediction and training
PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost.
PAW compiler implementations · Compile by Training method · Compile by Training evaluation
Cite this paper
Published version
Yuntian Deng, Pengyu Nie, Stuart Shieber. Compile by Training: Turning Natural-Language Specifications into Local Neural Functions. Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2026.
Publication BibTeX · Publication CSL-JSON
@inproceedings{deng-etal-2026-compile,
title = { {Compile by Training: Turning Natural-Language Specifications into Local Neural Functions} },
author = {Deng, Yuntian and Nie, Pengyu and Shieber, Stuart},
year = {2026},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},
publisher = {Association for Computational Linguistics},
pages = {830--839},
doi = {10.18653/2026.emnlp-demos.78},
month = {October},
address = {Budapest, Hungary},
isbn = {979-8-89176-461-3},
editor = {Agrawal, Sweta and Abad, Alberto},
url = {https://aclanthology.org/2026.emnlp-demos.78/}
}
ArXiv preprint
arXiv:2609.04199, initial submission 2026-09-03.