# Neural Compilation and Fuzzy-Function Programming

Reading list by Yuntian Deng. Version 1.3.0.

[Introduction](https://programasweights.com/reading-list/guides/fuzzy-functions/) · [Downloads](https://programasweights.com/reading-list/data/)

[Weight-Space Programming: Methods, Papers, and History](https://programasweights.com/research/weight-space-programming/) — PAW programs through weights. Compare trained neural compilers, direct LLM weight editing, and their publication history.

[Generating LoRA Weights from Task Descriptions with Program-as-Weights](https://programasweights.com/research/hypernetworks-and-lora-generation/) — PAW's hypernetwork generates adapters from function definitions, then a small interpreter runs the saved programs.

[Teacher-Generated Examples for Small Models: Program-as-Weights](https://programasweights.com/research/teacher-generated-training-data/) — How PAW learns from teacher examples to build reusable functions for a small language model.

[Benchmarks for Natural-Language Function Compilation: Program-as-Weights](https://programasweights.com/research/neural-compilation-benchmarks/) — FuzzyBench tests whether PAW builds a function from its description and executes it on new inputs.

[AI as Tool Builder: Yuntian Deng's Vision and Program-as-Weights](https://programasweights.com/research/ai-as-tool-builder/) — The June 2026 vision of pretraining AI to produce reusable neural programs, and its implementation in PAW.

Papers are ordered by first arXiv submission date.

[中文阅读清单](https://programasweights.com/zh/reading-list/) · [中文 Markdown](https://programasweights.com/zh/reading-list/index.md)

# Neural Programmer-Interpreters

Authors: Scott Reed, Nando de Freitas.

Learns neural programs that share a recurrent interpreter and a program memory.

ArXiv initial submission: 2015-11-19.

[Original paper](https://arxiv.org/abs/1511.06279) · [Reading-list entry](https://programasweights.com/reading-list/papers/1511.06279/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/1511.06279/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/1511.06279/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/1511.06279/record.json)

**Contribution:** Learns from execution traces and composes subprograms for addition, sorting, and 3D-object canonicalization.

**Program form:** Program embeddings stored in a key-value memory.

**Conditioning:** A program identifier, arguments, and observations of the environment.

**Execution:** A recurrent interpreter calls subprograms and takes actions in the environment.

**Relationship to PAW:** PAW cites this work on learned program representations and shared interpreters.

**Limitations:** Training requires execution traces and task-specific interfaces.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: neural programmer-interpreter, neural programs, program memory, compositional execution.

## Published version

Neural Programmer-Interpreters. Scott Reed, Nando de Freitas. International Conference on Learning Representations, 2016.
[Publication](https://arxiv.org/abs/1511.06279) · [BibTeX](https://programasweights.com/reading-list/papers/1511.06279/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/1511.06279/publication.csl.json)
[Publication record](https://arxiv.org/pdf/1511.06279).

## Approaches

- [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

学习由共享循环解释器执行的神经程序，并把程序表示存入程序记忆。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# HyperNetworks

Authors: David Ha, Andrew Dai, Quoc V. Le.

Uses one neural network to generate the weights of another.

ArXiv initial submission: 2016-09-27.

[Original paper](https://arxiv.org/abs/1609.09106) · [Reading-list entry](https://programasweights.com/reading-list/papers/1609.09106/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/1609.09106/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/1609.09106/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/1609.09106/record.json)

**Contribution:** Applies weight generation to convolutional and recurrent networks, relaxing weight sharing across layers and generating dynamic weights for sequence models.

**Program form:** Weights for the target network.

**Conditioning:** Layer embeddings or the current recurrent state.

**Execution:** The target network runs with the generated weights.

**Relationship to PAW:** Provides the weight-generation mechanism used by later description-to-adapter methods and PAW.

**Limitations:** The hypernetwork is trained end to end for the target architecture.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: hypernetwork, weight generation, fast weights, conditional parameters.

## Published version

HyperNetworks. David Ha, Andrew M. Dai, Quoc V. Le. International Conference on Learning Representations, 2017.
[Publication](https://openreview.net/forum?id=rkpACe1lx) · [BibTeX](https://programasweights.com/reading-list/papers/1609.09106/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/1609.09106/publication.csl.json)
[Publication record](https://openreview.net/pdf?id=rkpACe1lx).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

用一个神经网络生成另一个神经网络的权重。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# Prefix-Tuning: Optimizing Continuous Prompts for Generation

Authors: Xiang Lisa Li, Percy Liang.

Adapts a frozen language model by learning a short sequence of continuous prefix vectors.

ArXiv initial submission: 2021-01-01.

[Original paper](https://arxiv.org/abs/2101.00190) · [Reading-list entry](https://programasweights.com/reading-list/papers/2101.00190/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2101.00190/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2101.00190/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2101.00190/record.json)

**Contribution:** Trains prefixes for table-to-text generation and summarization while updating far fewer parameters than full fine-tuning.

**Program form:** Continuous prefix activations.

**Conditioning:** Training examples for the target task.

**Execution:** The frozen language model attends to the learned prefix.

**Relationship to PAW:** PAW evaluates compiler-generated prefixes alongside LoRA adapters.

**Limitations:** A separate prefix is optimized for each task.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: prefix-tuning, continuous prompts, virtual tokens, parameter-efficient fine-tuning.

## Method comparisons

**Program form and compiler**

LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications.

Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3)

Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

Prefix-Tuning: Optimizing Continuous Prompts for Generation. Xiang Lisa Li, Percy Liang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021.
[Publication](https://aclanthology.org/2021.acl-long.353/) · [BibTeX](https://programasweights.com/reading-list/papers/2101.00190/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2101.00190/publication.csl.json)
[Publication record](https://aclanthology.org/2021.acl-long.353/).

## Approaches

- [Parameter-efficient building blocks](https://programasweights.com/reading-list/guides/compilation/#peft): Adapt a frozen model with a small set of learned parameters.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

训练一段连续前缀向量，让冻结的语言模型适应目标任务。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# Learning to Generate Task-Specific Adapters from Task Description

Authors: Qinyuan Ye, Xiang Ren.

Introduces Hypter, which generates adapters from natural-language task descriptions.

ArXiv initial submission: 2021-01-02.

[Original paper](https://arxiv.org/abs/2101.00420) · [Reading-list entry](https://programasweights.com/reading-list/papers/2101.00420/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2101.00420/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2101.00420/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2101.00420/record.json)

**Contribution:** Combines a description encoder and adapter generator with BART. Tests generalization to unseen tasks on ZEST and synthetic SQuAD.

**Program form:** Bottleneck adapters inserted into a text-to-text model.

**Conditioning:** A natural-language task description.

**Execution:** BART performs the task with the generated adapters.

**Relationship to PAW:** An early description-to-adapter method cited by PAW. PAW develops the programming interface for saving and calling the resulting neural functions.

**Limitations:** Training requires a collection of tasks with descriptions and labeled examples.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: Hypter, task descriptions, generated adapters, zero-shot learning.

## Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

Learning to Generate Task-Specific Adapters from Task Description. Qinyuan Ye, Xiang Ren. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 2021.
[Publication](https://aclanthology.org/2021.acl-short.82/) · [BibTeX](https://programasweights.com/reading-list/papers/2101.00420/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2101.00420/publication.csl.json)
[Publication record](https://aclanthology.org/2021.acl-short.82/).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

提出 Hypter，根据自然语言任务描述生成适配器，并检验对未见任务的泛化。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# LoRA: Low-Rank Adaptation of Large Language Models

Authors: Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen.

Adapts a pretrained model with low-rank weight updates while keeping the original weights fixed.

ArXiv initial submission: 2021-06-17.

[Original paper](https://arxiv.org/abs/2106.09685) · [Reading-list entry](https://programasweights.com/reading-list/papers/2106.09685/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2106.09685/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2106.09685/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2106.09685/record.json)

**Contribution:** Reduces the parameters trained and stored for each task. Evaluates adaptation across several language-model families.

**Program form:** Low-rank matrices that update selected model weights.

**Conditioning:** Training examples for the target task.

**Execution:** The language model applies the low-rank updates during inference.

**Relationship to PAW:** PAW uses LoRA as a program representation and generates the adapter with a trained compiler.

**Limitations:** A separate adapter is trained for each task.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: LoRA, low-rank adaptation, parameter-efficient fine-tuning, adapters.

## Method comparisons

**Program form and compiler**

LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications.

Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3)

Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

LoRA: Low-Rank Adaptation of Large Language Models. Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen. International Conference on Learning Representations, 2022.
[Publication](https://openreview.net/forum?id=nZeVKeeFYf9) · [BibTeX](https://programasweights.com/reading-list/papers/2106.09685/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2106.09685/publication.csl.json)
[Publication record](https://openreview.net/pdf?id=nZeVKeeFYf9).

## Approaches

- [Parameter-efficient building blocks](https://programasweights.com/reading-list/guides/compilation/#peft): Adapt a frozen model with a small set of learned parameters.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

冻结预训练模型的原始权重，训练低秩权重更新来适应任务。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# HyperTuning: Toward Adapting Large Language Models without Back-propagation

Authors: Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen.

Produces task-specific parameters for a frozen language model from a few demonstrations.

ArXiv initial submission: 2022-11-22.

[Original paper](https://arxiv.org/abs/2211.12485) · [Reading-list entry](https://programasweights.com/reading-list/papers/2211.12485/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2211.12485/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2211.12485/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2211.12485/record.json)

**Contribution:** Introduces HyperT5 models that generate prefixes or LoRA parameters. Hyperpretraining and multitask training prepare the generator for new tasks.

**Program form:** Generated prefix or LoRA parameters.

**Conditioning:** A few examples of the task.

**Execution:** A frozen T5 model runs with the generated parameters.

**Relationship to PAW:** PAW cites HyperTuning as prior work on generating adaptation parameters from examples.

**Limitations:** The hypermodel accepts a limited number of demonstrations. The paper also studies further task-specific tuning.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: HyperTuning, HyperT5, few-shot adaptation, generated LoRA.

## Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

HyperTuning: Toward Adapting Large Language Models without Back-propagation. Jason Phang, Yi Mao, Pengcheng He, Weizhu Chen. Proceedings of the 40th International Conference on Machine Learning, 2023.
[Publication](https://proceedings.mlr.press/v202/phang23a.html) · [BibTeX](https://programasweights.com/reading-list/papers/2211.12485/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2211.12485/publication.csl.json)
[Publication record](https://proceedings.mlr.press/v202/phang23a.html).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

根据几个任务示例，为冻结的语言模型生成前缀或 LoRA 参数。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation

Authors: Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew Peters.

Turns instructions and examples into reusable prefixes, adapters, and encoded instruction representations.

ArXiv initial submission: 2022-12-20.

[Original paper](https://arxiv.org/abs/2212.10315) · [Reading-list entry](https://programasweights.com/reading-list/papers/2212.10315/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2212.10315/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2212.10315/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2212.10315/record.json)

**Contribution:** Combines a hypernetwork with instruction fusion so repeated calls can reuse the instruction computation. Evaluates zero-shot and few-shot generalization.

**Program form:** Prefixes, adapters, and encoded instruction representations.

**Conditioning:** Task instructions and optional examples.

**Execution:** An encoder-decoder model uses the generated modules and attends to cached instruction representations.

**Relationship to PAW:** HINT reuses generated modules and an encoded instruction. PAW reuses a program containing text and adaptation parameters.

**Limitations:** The generated modules and cached instruction representations are tied to the trained encoder-decoder model.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: HINT, instruction tuning, hypernetwork, instruction fusion.

## Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

HINT: Hypernetwork Instruction Tuning for Efficient Zero- and Few-Shot Generalisation. Hamish Ivison, Akshita Bhagia, Yizhong Wang, Hannaneh Hajishirzi, Matthew E. Peters. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023.
[Publication](https://aclanthology.org/2023.acl-long.631/) · [BibTeX](https://programasweights.com/reading-list/papers/2212.10315/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2212.10315/publication.csl.json)
[Publication record](https://aclanthology.org/2023.acl-long.631/).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

把指令和示例转化为前缀、适配器及编码后的指令表示，供多次调用复用。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# Learning to Compress Prompts with Gist Tokens

Authors: Jesse Mu, Xiang Lisa Li, Noah Goodman.

Compresses an instruction into a few reusable token activations.

ArXiv initial submission: 2023-04-17.

[Original paper](https://arxiv.org/abs/2304.08467) · [Reading-list entry](https://programasweights.com/reading-list/papers/2304.08467/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2304.08467/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2304.08467/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2304.08467/record.json)

**Contribution:** Trains with an attention mask that forces later tokens to use gist activations in place of the original prompt.

**Program form:** Cached activations at a few gist-token positions.

**Conditioning:** The instruction or prompt.

**Execution:** The language model generates outputs while attending to the cached gist.

**Relationship to PAW:** PAW cites gisting as another way to reuse instruction computation. Gisting stores activations; PAW stores a program with text and adaptation parameters.

**Limitations:** Compression can reduce output quality. The cached activations are tied to the trained model.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: gist tokens, prompt compression, activation caching, amortized inference.

## Method comparisons

**What gets reused?**

Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module.

Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2)

Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

Learning to Compress Prompts with Gist Tokens. Jesse Mu, Xiang Li, Noah Goodman. Advances in Neural Information Processing Systems, 2023.
[Publication](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3d77c6dcc7f143aa2154e7f4d5e22d68-Abstract-Conference.html) · [BibTeX](https://programasweights.com/reading-list/papers/2304.08467/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2304.08467/publication.csl.json)
[Publication record](https://proceedings.neurips.cc/paper_files/paper/2023/hash/3d77c6dcc7f143aa2154e7f4d5e22d68-Abstract-Conference.html).

## Approaches

- [Context compression](https://programasweights.com/reading-list/guides/compilation/#context-compression): Compress an instruction or context once and reuse the result across calls.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

将一段指令压缩为几个可复用的 gist token 激活值。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators

Authors: Tzu-Heng Huang, Catherine Cao, Vaishnavi Bhargava, Frederic Sala.

Asks a language model to write labeling programs that can be saved and run locally.

ArXiv initial submission: 2024-06-25.

[Original paper](https://arxiv.org/abs/2407.11004) · [Reading-list entry](https://programasweights.com/reading-list/papers/2407.11004/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2407.11004/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2407.11004/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2407.11004/record.json)

**Contribution:** Reuses generated code across many labeling decisions and combines noisy program outputs with weak supervision.

**Program form:** Symbolic labeling programs.

**Conditioning:** A task description, labeling instructions, and a function signature.

**Execution:** Generated programs label examples locally; their outputs can be combined to train a downstream model.

**Relationship to PAW:** PAW evaluates an ALCHEmist-style approach. Both build reusable tools: ALCHEmist produces symbolic code, while PAW produces programs for a neural interpreter.

**Limitations:** Label quality depends on the model’s ability to write useful programs. Complex tasks can produce code that is harder to inspect.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: ALCHEmist, program synthesis, automated labeling, reusable tools.

## Method comparisons

**Code and neural programs**

ALCHEmist generates symbolic labeling programs. PAW compiles fuzzy functions into text and learned parameters. Both build an artifact once and reuse it; one runs as code, the other on a neural interpreter.

Sources: [ALCHEmist paper](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html) · [PAW evaluation](https://arxiv.org/html/2607.02512v1#S6)

Paper details: [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://programasweights.com/reading-list/papers/2407.11004/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

The ALCHEmist: Automated Labeling 500x CHEaper than LLM Data Annotators. Tzu-Heng Huang, Catherine Cao, Vaishnavi Bhargava, Frederic Sala. Advances in Neural Information Processing Systems, 2024.
[Publication](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html) · [BibTeX](https://programasweights.com/reading-list/papers/2407.11004/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2407.11004/publication.csl.json)
[Publication record](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html).

## Approaches

- [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

让语言模型编写可保存、可本地执行的标注程序，并用弱监督组合多个程序的输出。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass

Authors: Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng.

Converts new context into low-rank model updates in one forward pass.

ArXiv initial submission: 2024-11-08.

[Original paper](https://arxiv.org/abs/2411.05877) · [Reading-list entry](https://programasweights.com/reading-list/papers/2411.05877/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2411.05877/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2411.05877/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2411.05877/record.json)

**Contribution:** Trains an adapter generator with self-supervision. Tests learning from documents, demonstrations, and conversation history.

**Program form:** Additive low-rank adapters.

**Conditioning:** Documents, demonstrations, or conversation history, including successive chunks of context.

**Execution:** The adapted model answers new inputs using context stored in its parameter updates.

**Relationship to PAW:** PAW cites this context-to-adapter method. Generative Adapter stores information from context; PAW compiles the behavior of a function.

**Limitations:** The adapter generator is tied to its frozen base model.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: Generative Adapter, contextualization, fast weights, streaming context.

## Method comparisons

**What gets reused?**

Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module.

Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2)

Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


An AFM 2024 OpenReview version appeared before the first arXiv submission. [Source](https://arxiv.org/abs/2411.05877).

## Published version

Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass. Tong Chen, Hao Fang, Patrick Xia, Xiaodong Liu, Benjamin Van Durme, Luke Zettlemoyer, Jianfeng Gao, Hao Cheng. International Conference on Learning Representations, 2025.
[Publication](https://openreview.net/forum?id=bc3sUsS6ck) · [BibTeX](https://programasweights.com/reading-list/papers/2411.05877/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2411.05877/publication.csl.json)
[Publication record](https://openreview.net/pdf?id=bc3sUsS6ck).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.
- [Context compression](https://programasweights.com/reading-list/guides/compilation/#context-compression): Compress an instruction or context once and reuse the result across calls.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

通过一次前向传播，将新上下文转化为低秩参数更新。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# Text-to-LoRA: Instant Transformer Adaption

Authors: Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange.

Uses a hypernetwork to generate LoRA adapters from task descriptions.

ArXiv initial submission: 2025-06-06.

[Original paper](https://arxiv.org/abs/2506.06105) · [Reading-list entry](https://programasweights.com/reading-list/papers/2506.06105/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2506.06105/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2506.06105/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2506.06105/record.json)

**Contribution:** Trains hypernetworks to reconstruct existing adapters or learn from supervised task data. Tests adapter compression and generation for unseen tasks.

**Program form:** Generated LoRA matrices.

**Conditioning:** A natural-language task description.

**Execution:** A base transformer performs the task with the generated LoRA.

**Relationship to PAW:** PAW cites Text-to-LoRA as prior work on generating adapters. PAW develops fuzzy-function programming, FuzzyBench, and an interface for reusable neural programs.

**Limitations:** Reconstruction-trained T2L underperforms the supervised variant on unseen tasks.

[Discussion in the paper](https://arxiv.org/html/2607.02512v1#S10)

Related terms: Text-to-LoRA, T2L, task description, adapter generation.

## Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)


## Published version

Text-to-LoRA: Instant Transformer Adaption. Rujikorn Charakorn, Edoardo Cetin, Yujin Tang, Robert Tjarko Lange. Proceedings of the 42nd International Conference on Machine Learning, 2025.
[Publication](https://proceedings.mlr.press/v267/charakorn25a.html) · [BibTeX](https://programasweights.com/reading-list/papers/2506.06105/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2506.06105/publication.csl.json)
[Publication record](https://proceedings.mlr.press/v267/charakorn25a.html).

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

用超网络根据任务描述生成 LoRA 适配器，研究适配器压缩和未见任务的适配。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution

Authors: Liliana Hotsko, Yinxi Li, Yuntian Deng, Pengyu Nie.

Trains a hypernetwork to generate repository-specific LoRA adapters from source code and update them as commits arrive.

ArXiv initial submission: 2026-06-04.

[Original paper](https://arxiv.org/abs/2606.06492) · [Reading-list entry](https://programasweights.com/reading-list/papers/2606.06492/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2606.06492/citation.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2606.06492/citation.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2606.06492/record.json)

**Contribution:** Code2LoRA-Static maps a repository snapshot to an adapter. Code2LoRA-Evo updates a GRU state with each code diff and generates the adapter from that state.

**Program form:** Generated LoRA matrices for a frozen code language model.

**Conditioning:** A repository snapshot; the evolution variant also receives successive commit diffs.

**Execution:** The code model uses the generated adapter without adding repository context tokens to each request.

**Relationship to PAW:** Code2LoRA and PAW train models to generate reusable weights. Code2LoRA puts repository knowledge into adapters; PAW compiles function specifications into neural programs with a compiler–interpreter interface.

**Limitations:** The paper evaluates Python test-assertion completion with Qwen2.5-Coder-1.5B.

Related terms: Code2LoRA, repository-conditioned adapters, commit-streaming adapters, hypernetworks, weight-space programming.

## Approaches

- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

训练超网络，根据代码仓库生成 LoRA 适配器，并随新的提交更新适配器。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# Program-as-Weights: A Programming Paradigm for Fuzzy Functions

Authors: Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng.

Proposes fuzzy-function programming: compile a natural-language function specification into a neural program, then reuse it locally.

ArXiv initial submission: 2026-07-02.

[Original paper](https://arxiv.org/abs/2607.02512) · [Reading-list entry](https://programasweights.com/reading-list/papers/2607.02512/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2607.02512/preprint.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2607.02512/preprint.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2607.02512/record.json)

Preferred citation: The Fortieth Annual Conference on Neural Information Processing Systems, 2026.
[BibTeX](https://programasweights.com/reading-list/papers/2607.02512/citation.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2607.02512/citation.csl.json)

**Contribution:** Develops the compiler–interpreter abstraction and programs that combine text with learned parameters. Introduces the 10M-example FuzzyBench dataset, local execution interfaces, and image-conditioned compilation. Prefix tuning and LoRA are two implementations.

**Program form:** A discrete pseudo-program and a compiler-generated parameter-efficient module.

**Conditioning:** A function specification, optionally with examples. The image extension also accepts visual information.

**Execution:** A frozen lightweight interpreter loads the program and runs new inputs locally.

**Relationship to PAW:** Compile by Training builds on this programming abstraction. It keeps the same interface and refines the fast compiler’s prediction using synthesized examples.

[Discussion in the paper](https://arxiv.org/html/2609.04199v1#S1)
[Code](https://github.com/programasweights)

Related terms: Program-as-Weights, ProgramAsWeights, PAW, fuzzy-function programming, neural compiler, neural interpreter, FuzzyBench.

## Method comparisons

**From generated adapters to reusable functions**

Hypter, HINT, HyperTuning, and Text-to-LoRA generate adaptations from descriptions or examples. PAW uses generated parameters as part of a program that can be saved, called from ordinary code, and reused by a fixed interpreter.

Sources: [PAW related work](https://arxiv.org/html/2607.02512v1#S10) · [Text-to-LoRA paper](https://proceedings.mlr.press/v267/charakorn25a.html)

Paper details: [Learning to Generate Task-Specific Adapters from Task Description](https://programasweights.com/reading-list/papers/2101.00420/) · [HINT: Hypernetwork Instruction Tuning for Efficient Zero- & Few-Shot Generalisation](https://programasweights.com/reading-list/papers/2212.10315/) · [HyperTuning: Toward Adapting Large Language Models without Back-propagation](https://programasweights.com/reading-list/papers/2211.12485/) · [Text-to-LoRA: Instant Transformer Adaption](https://programasweights.com/reading-list/papers/2506.06105/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

**Code and neural programs**

ALCHEmist generates symbolic labeling programs. PAW compiles fuzzy functions into text and learned parameters. Both build an artifact once and reuse it; one runs as code, the other on a neural interpreter.

Sources: [ALCHEmist paper](https://proceedings.neurips.cc/paper_files/paper/2024/hash/72802bef5cf1a3449e909b20c2ae18d5-Abstract-Conference.html) · [PAW evaluation](https://arxiv.org/html/2607.02512v1#S6)

Paper details: [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://programasweights.com/reading-list/papers/2407.11004/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

**What gets reused?**

Gisting caches a few instruction activations. Generative Adapter stores context in parameter updates. PAW saves a function as a program containing text and a neural module.

Sources: [Gist paper](https://arxiv.org/abs/2304.08467) · [Generative Adapter paper](https://arxiv.org/abs/2411.05877) · [PAW program definition](https://arxiv.org/html/2607.02512v1#S2)

Paper details: [Learning to Compress Prompts with Gist Tokens](https://programasweights.com/reading-list/papers/2304.08467/) · [Generative Adapter: Contextualizing Language Models in Parameters with A Single Forward Pass](https://programasweights.com/reading-list/papers/2411.05877/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

**Program form and compiler**

LoRA represents adaptation as low-rank weight updates; prefix tuning uses continuous activations. Their original methods learn these parameters from examples. PAW generates both forms from specifications.

Sources: [LoRA paper](https://arxiv.org/abs/2106.09685) · [Prefix-tuning paper](https://aclanthology.org/2021.acl-long.353/) · [PAW implementations](https://arxiv.org/html/2607.02512v1#S3)

Paper details: [LoRA: Low-Rank Adaptation of Large Language Models](https://programasweights.com/reading-list/papers/2106.09685/) · [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://programasweights.com/reading-list/papers/2101.00190/) · [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/)

**Fast prediction and training**

PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost.

Sources: [PAW compiler implementations](https://arxiv.org/html/2607.02512v1#S3) · [Compile by Training method](https://arxiv.org/html/2609.04199v1#S3) · [Compile by Training evaluation](https://arxiv.org/html/2609.04199v1#S5)

Paper details: [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) · [Compile by Training: Turning Natural-Language Specifications into Local Neural Functions](https://programasweights.com/reading-list/papers/2609.04199/)


## Published version

Program-as-Weights: A Programming Paradigm for Fuzzy Functions. Wentao Zhang, Liliana Hotsko, Woojeong Kim, Pengyu Nie, Stuart Shieber, Yuntian Deng. The Fortieth Annual Conference on Neural Information Processing Systems, 2026.
[Publication](https://arxiv.org/abs/2607.02512) · [BibTeX](https://programasweights.com/reading-list/papers/2607.02512/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2607.02512/publication.csl.json)
[Publication record](https://arxiv.org/abs/2607.02512).

## Approaches

- [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs.
- [Generated weights and adapters](https://programasweights.com/reading-list/guides/compilation/#generated-adapters): Generate model weights from a task description, examples, or context.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

提出模糊函数编程：将自然语言函数说明编译成神经程序，保存后由小模型在本地反复执行。

[中文阅读清单](https://programasweights.com/zh/reading-list/)

# Compile by Training: Turning Natural-Language Specifications into Local Neural Functions

Authors: Yuntian Deng, Pengyu Nie, Stuart Shieber.

Compiles a function description into a reusable local program by generating training examples and optimizing an adapter.

ArXiv initial submission: 2026-09-03.

[Original paper](https://arxiv.org/abs/2609.04199) · [Reading-list entry](https://programasweights.com/reading-list/papers/2609.04199/)
[Preprint BibTeX](https://programasweights.com/reading-list/papers/2609.04199/preprint.bib) · [Preprint CSL-JSON](https://programasweights.com/reading-list/papers/2609.04199/preprint.csl.json) · [JSON record](https://programasweights.com/reading-list/papers/2609.04199/record.json)

Preferred citation: Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2026.
[BibTeX](https://programasweights.com/reading-list/papers/2609.04199/citation.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2609.04199/citation.csl.json)

**Contribution:** Makes task-specific training a software build step. Raises FuzzyBench-Hard semantic accuracy (LEM) from 22.4% to 83.6% and demonstrates composition in a website helper, avatar control, and English–Claudish translation.

**Program form:** A LoRA adapter, runtime prompt, specification, and interpreter metadata in a PAW program.

**Conditioning:** The specification guides teacher synthesis. PAW’s fast compiler supplies the initial adapter and runtime prompt.

**Execution:** A frozen Qwen3-0.6B interpreter runs the saved function locally without further teacher calls.

**Relationship to PAW:** Extends PAW with training at compile time while keeping its program format and runtime interface.

[Discussion in the paper](https://arxiv.org/html/2609.04199v1#S1)

Versions: [Original version](https://arxiv.org/html/2609.04199v1)
[Code](https://github.com/programasweights/compile-by-training)

Related terms: compile by training, training-based compilation, specification-specific optimization, synthetic supervision, local neural function, FuzzyBench-Hard, ProgramAsWeights.

## Method comparisons

**Fast prediction and training**

PAW’s fast compiler predicts a program. Compile by Training starts from that prediction, generates examples, and trains an adapter. Both use the same runtime interface; the extra training improves semantic accuracy on FuzzyBench-Hard at a higher compilation cost.

Sources: [PAW compiler implementations](https://arxiv.org/html/2607.02512v1#S3) · [Compile by Training method](https://arxiv.org/html/2609.04199v1#S3) · [Compile by Training evaluation](https://arxiv.org/html/2609.04199v1#S5)

Paper details: [Program-as-Weights: A Programming Paradigm for Fuzzy Functions](https://programasweights.com/reading-list/papers/2607.02512/) · [Compile by Training: Turning Natural-Language Specifications into Local Neural Functions](https://programasweights.com/reading-list/papers/2609.04199/)


The paper reports public-service measurements from May 2026. [Source](https://arxiv.org/abs/2609.04199).

## Published version

Compile by Training: Turning Natural-Language Specifications into Local Neural Functions. Yuntian Deng, Pengyu Nie, Stuart Shieber. Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 2026.
[Publication](https://aclanthology.org/2026.emnlp-demos.78/) · [BibTeX](https://programasweights.com/reading-list/papers/2609.04199/publication.bib) · [CSL-JSON](https://programasweights.com/reading-list/papers/2609.04199/publication.csl.json)
[Publication record](https://aclanthology.org/2026.emnlp-demos.78/).

## Approaches

- [Programs and reusable artifacts](https://programasweights.com/reading-list/guides/fuzzy-functions/): Learn programs that can be saved and run on new inputs.
- [Compilation through training](https://programasweights.com/reading-list/guides/compilation/#training-compilation): Generate examples from a function specification and train a reusable neural program.

Reading-list version 1.3.0; updated 2026-11-14.


## 中文简介

根据函数说明生成训练样本，再训练 LoRA 适配器，将一次函数定义构建成可复用的本地神经程序。

[中文阅读清单](https://programasweights.com/zh/reading-list/)