Fuzzy-function programming
Describe a function in English, compile it, and save the resulting neural program. Load it later and call it from ordinary code.
Program-as-Weights · Research
A programming paradigm
for fuzzy functions
Describe a function. Compile it once.
Reuse a neural program locally.
Repairing malformed JSON, finding important log messages, and matching a query to its intent are hard to express as exact rules. Program-as-Weights (PAW) turns descriptions like these into small neural programs that run locally.
PAW paper · NeurIPS 2026 ↗Compile by TrainingTry PAW ↗Cite this work
Long-term vision
Yuntian Deng · Essay
From AI as Problem Solver to AI as Tool Builder
When a task repeats, we should be able to build a tool once and reuse it. Yuntian Deng’s essay proposes pretraining models to generate small neural programs for computers, phones, and robots. The vision is for people to create and update these programs in natural language, then run them locally.
This is the long-term goal behind PAW. AI as tool builder: the vision and the PAW programming model →
The original PAW paper
First arXiv submission
NeurIPS 2026
Program-as-Weights: A Programming Paradigm for Fuzzy Functions
Wentao Zhang, Liliana Hotsko, and Woojeong Kim contributed equally.
What PAW introduces
Programs represented by weights
A PAW program combines a text pseudo-program with a LoRA adapter or prefix vectors. PAW trains a compiler to generate the program from a function description.
Weight-space programming and PAW → · Weight-space reading list →
A benchmark and working runtime
FuzzyBench contains 10 million specification–input–output examples for training and evaluating neural compilers. Python and JavaScript runtimes execute compiled programs locally; a smaller GPT-2 interpreter runs in the browser.
Sources: programming abstraction (§2) · compiler and interpreter (§3) · FuzzyBench (§5) · local execution (§9) · browser execution (§1).
How Program-as-Weights works
Generating LoRA Weights from Task Descriptions with Program-as-Weights
PAW's hypernetwork generates adapters from function definitions, then a small interpreter runs the saved programs.
Teacher-Generated Examples for Small Models: Program-as-Weights
How PAW learns from teacher examples to build reusable functions for a small language model.
Benchmarks for Natural-Language Function Compilation: Program-as-Weights
FuzzyBench tests whether PAW builds a function from its description and executes it on new inputs.
AI as Tool Builder: Yuntian Deng's Vision and Program-as-Weights
The June 2026 vision of pretraining AI to produce reusable neural programs, and its implementation in PAW.
FuzzyBench results
July 2026
On FuzzyBench, PAW with a Qwen3-0.6B interpreter achieves 73.78% exact match, compared with 68.70% for direct Qwen3-32B prompting.
The paper also tests noisy specifications, quantization, and execution on a MacBook M3. Read the results.
Compile by Training
First arXiv submission
EMNLP 2026 System Demonstrations
Compile by Training: Turning Natural-Language Specifications into Local Neural Functions
Compile by Training uses teacher models to generate examples from a function description. It initializes a program with the fast compiler, then trains it on those examples. The saved program uses the same PAW runtime and runs locally without further teacher calls.
Read the paper ↗arXiv preprintAbout the paperCodeTry the compiler
Trading compilation time for accuracy
On FuzzyBench-Hard, Compile by Training improves LLM-judged semantic accuracy (LEM) from 22.4% to 83.6%. A cold compile of one specification takes 50.9 seconds on a B300, compared with 3.5 seconds for the fast compiler.
Compiled functions power a website helper, a 3D avatar controlled through language, and an English–Claudish translator.
Sources: compilation method (§3) · evaluation and hardware (§5) · applications (§6).
Papers and software
Read related papers
Hypernetworks, generated adapters, instruction compilation, and neural programs.
Use the Python SDK
Compile a specification and load the resulting program as a local function. Read the documentation.
Try the application
Write a function description and try the compiled program on your own inputs. Two-stage decision models: build once, run fast local inference.
Cite the research
Program-as-Weights
Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng. Program-as-Weights: A Programming Paradigm for Fuzzy Functions. The Fortieth Annual Conference on Neural Information Processing Systems, 2026. Paper on arXiv.
Download BibTeXDownload CSL-JSONPaper summary as Markdown
@inproceedings{zhang2026programasweights,
title = { {Program-as-Weights: A Programming Paradigm for Fuzzy Functions} },
author = {Zhang, Wentao and Hotsko, Liliana and Kim, Woojeong and Nie, Pengyu and Shieber, Stuart and Deng, Yuntian},
year = {2026},
booktitle = {The Fortieth Annual Conference on Neural Information Processing Systems},
url = {https://arxiv.org/abs/2607.02512}
}
Compile by Training
Yuntian Deng, Pengyu Nie, Stuart Shieber. Compile by Training: Turning Natural-Language Specifications into Local Neural Functions. Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2026. ACL Anthology.
Download BibTeXDownload CSL-JSONPaper summary as Markdown
@inproceedings{deng-etal-2026-compile,
title = { {Compile by Training: Turning Natural-Language Specifications into Local Neural Functions} },
author = {Deng, Yuntian and Nie, Pengyu and Shieber, Stuart},
year = {2026},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},
publisher = {Association for Computational Linguistics},
pages = {830--839},
doi = {10.18653/2026.emnlp-demos.78},
month = {October},
address = {Budapest, Hungary},
isbn = {979-8-89176-461-3},
editor = {Agrawal, Sweta and Abad, Alberto},
url = {https://aclanthology.org/2026.emnlp-demos.78/}
}