Paper
Program-as-Weights: A Programming Paradigm for Fuzzy Functions
First arXiv submission: · NeurIPS 2026
Proposes fuzzy-function programming: compile a natural-language function specification into a neural program, then reuse it locally.
What it does
Develops the compiler–interpreter abstraction and programs that combine text with learned parameters. Introduces the 10M-example FuzzyBench dataset, local execution interfaces, and image-conditioned compilation. Prefix tuning and LoRA are two implementations.
- Conditioning
- A function specification, optionally with examples. The image extension also accepts visual information.
- Program form
- A discrete pseudo-program and a compiler-generated parameter-efficient module.
- Execution
- A frozen lightweight interpreter loads the program and runs new inputs locally.
Related work
Compile by Training builds on this programming abstraction. It keeps the same interface and refines the fast compiler’s prediction using synthesized examples.
How PAW works
- Generating LoRA Weights from Task Descriptions with Program-as-Weights — PAW's hypernetwork generates adapters from function definitions, then a small interpreter runs the saved programs.
- Teacher-Generated Examples for Small Models: Program-as-Weights — How PAW learns from teacher examples to build reusable functions for a small language model.
- Benchmarks for Natural-Language Function Compilation: Program-as-Weights — FuzzyBench tests whether PAW builds a function from its description and executes it on new inputs.
- AI as Tool Builder: Yuntian Deng's Vision and Program-as-Weights — The June 2026 vision of pretraining AI to produce reusable neural programs, and its implementation in PAW.
Related terms: Program-as-Weights, ProgramAsWeights, PAW, fuzzy-function programming, neural compiler, neural interpreter, FuzzyBench.
Related methods
From generated adapters to reusable functions
Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.
Code and neural programs
ALCHEmist generates symbolic labeling programs. PAW compiles fuzzy functions into text and learned parameters. Both build an artifact once and reuse it; one runs as code, the other on a neural interpreter.
What gets reused?
Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module.
Gist paper · Generative Adapter paper · PAW program definition
Program form and compiler
LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications.
Fast prediction and training
PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost.
PAW compiler implementations · Compile by Training method · Compile by Training evaluation
Cite this paper
Published version
Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng. Program-as-Weights: A Programming Paradigm for Fuzzy Functions. The Fortieth Annual Conference on Neural Information Processing Systems, 2026.
Publication BibTeX · Publication CSL-JSON
@inproceedings{zhang2026programasweights,
title = { {Program-as-Weights: A Programming Paradigm for Fuzzy Functions} },
author = {Zhang, Wentao and Hotsko, Liliana and Kim, Woojeong and Nie, Pengyu and Shieber, Stuart and Deng, Yuntian},
year = {2026},
booktitle = {The Fortieth Annual Conference on Neural Information Processing Systems},
url = {https://arxiv.org/abs/2607.02512}
}
ArXiv preprint
arXiv:2607.02512, initial submission 2026-07-02.