Context Computing: A Process-Theoretic Model of Computation over Learned Semantic State Spaces
Abstract. Context Computing (CC) is a computational framework in which states are distributions over high-dimensional semantic vectors, operations redistribute probability mass over candidate interpretations, and outputs are task-conditioned projections to single meanings. CC is presented as a distinct computational paradigm enabled by modern embedding models, which encode the Zipfian statistical structure of human intent into vector geometry. The theory is situated relative to existing computation paradigms, with connections drawn to categorical quantum mechanics and semantic information theory, and the formal and empirical work needed to establish CC's boundaries is identified.
1. Definition
A context is a pair (V, P) where V = {v₁, …, vₙ} ⊂ ℝd are candidate vectors in a learned embedding space and P = {p₁, …, pₙ} is a probability distribution over them. Three operations define computation under CC:
c_byte— context expansion: grows V by adding candidates, increasing uncertainty.c_gate— context transformation: applies a map f: ℝd → ℝd that redistributes probability mass.c_bit— context collapse: projects (V, P) to a single output v* via argmax, sampling, or thresholding.
The system maintains non-determinism during computation (multiple candidates alive) and resolves to deterministic output at endpoints.
2. Relationship to Existing Paradigms
CC is not claimed to contain digital, analog, and quantum computation as strict projections. Rather, all four paradigms can be modeled as process theories (in the sense of Coecke & Kissinger 2017) over different state spaces and admissible transformations:
| Paradigm | State space | Transformations | Readout |
|---|---|---|---|
| Digital | {0,1}n | Boolean gates | Direct readout |
| Analog | ℝ1 | Continuous dynamics | Threshold |
| Quantum | ℂn | Unitary maps, CP maps | Born measurement |
| Context | ℝd (learned) | Learned transformations | Semantic collapse |
Categorical quantum mechanics (Abramsky & Coecke 2004) and generalized probabilistic theories (Hardy 2001, Chiribella et al. 2011) provide the formal framework for this unification. Hyperdimensional computing (Kanerva; survey by Frady et al. 2021) demonstrates that Boolean logic can be embedded in high-dimensional vector spaces, supporting a representation-level connection. The stricter claim — that CC computationally contains quantum computing — is not asserted and would require proof of complexity-class containment (CC ⊇ BQP), which remains open.
3. The Zipfian Substrate
Human interaction data follows heavy-tailed, approximately Zipfian distributions across multiple domains: word frequencies (Zipf 1949, Piantadosi 2014), command usage (Carroll & Rosson 1986), search queries (Baeza-Yates et al.), and dialogue (ACM IUI 2020).
Modern embedding models trained on this data encode this structure geometrically:
- Contextual representations exhibit anisotropy correlated with frequency (Gao et al. 2019, Ethayarajh 2019).
- Zipfian whitening improves downstream performance (NeurIPS 2024).
- Semantic breadth exhibits Zipfian patterns: frequent words are generic, rare words specific (Zipfian regularities in non-point representations, ScienceDirect 2021).
For a Zipf distribution with s ≈ 1 over N types, the effective vocabulary under the participation-ratio definition scales as Neff ≈ (6/π2)(ln N)2 — polylogarithmic in N. This implies that the navigable structure of intent space is vastly smaller than the theoretical vocabulary, making semantic navigation tractable. This tractability is offered as an explanation for why CC was not practical before large-scale embedding models: prior substrates (one-hot encodings, analog quantities, complex amplitudes) did not encode the statistical structure of human meaning.
4. Internal Non-Determinism and Output Determinism
CC maintains a distribution over candidate interpretations during computation and collapses to a single output at endpoints. This parallels — but is not equivalent to — quantum measurement. The disanalogy is important: classical probability distributions over ℝd are classical mixtures fully described by classical probability theory. They lack interference, contextuality (in the Kochen–Specker sense), and non-classical correlations. The quantum cognition literature (Busemeyer & Bruza 2012, van Rijsbergen 2004, Aerts et al.) uses Hilbert-space formalisms to model cognition, suggesting the analogy is heuristically productive, though no formal equivalence is claimed.
The collapse dynamics — how and when non-determinism resolves — remain the central open question in the theory. Candidate mechanisms include task-conditioned argmax, sampling with temperature, and learned attention-based reranking. Formalizing this requires specifying the readout mechanism and its relationship to measurement theory.
5. Semantic Information and Scaling
Shannon information theory deliberately excludes semantics (Shannon 1948). Semantic information theory (Bar-Hillel & Carnap 1953, Floridi 2004, Kolchinsky & Wolpert 2018) defines information relative to truth, utility, task relevance, or agent viability. CC operates in this semantic regime: the relevant uncertainty is not over symbols but over meanings.
Neural scaling laws (Kaplan et al. 2020, Hoffmann et al. 2022) show that semantic acquisition scales as a power law in parameters and data — distinct from Shannon channel capacity. Emerging work on “quantum semantic communications” (2024) replaces channel capacity with topological distance between meaning clusters in latent space. CC's scaling properties are proposed for analysis within this semantic information-theoretic framework, rather than as a replacement for Shannon theory.
6. Open Problems
- Complexity class. Define CC's complexity class relative to BQP, BPP, and P. Is CC ⊆ BPP? Does it capture problems outside P?
- Born rule. Specify which CC axioms could substitute for Gleason's noncontextuality or Zurek's envariance. Without complex phase structure, it remains unclear why L2 normalization would emerge.
- Contextuality. Test whether CC's collapse exhibits Kochen–Specker contextuality — the defining feature separating quantum from classical probability.
- Intent as latent variable. All Zipfian evidence to date is for behavioral proxies (words, queries, commands), not intent directly. Experiments are needed to test whether latent intent distributions are genuinely Zipfian.
- Falsifiable predictions. Predictions generated from CC should be testable on interaction logs — e.g., that effective vocabulary scaling follows (ln N)2 rather than N, or that collapse dynamics exhibit power-law residence times in attractor regions.