Physical Constraint Spillover
How Diachronic Co-occurrence Density Shifts Penetrate
the Sampling Pipeline into LLM Outputs
A Testable Hypothesis on the Origins of Emergent
Physical Feasibility Checking in Large Language Models
This paper proposes the “Physical Constraint Spillover” hypothesis. The hypothesis holds that the physical feasibility checking behavior spontaneously produced by large language models (LLMs) when answering technical queries need not be entirely attributed to the model having acquired domain-independent physical reasoning capabilities. An alternative explanation, at least partially valid, is this: the explosive growth of AI infrastructure text between 2024 and 2026 caused a dramatic increase in the co-occurrence density of technical vocabulary (e.g., GPU) and physical constraint vocabulary (e.g., power consumption, thermal dissipation, grid capacity) within training data, triggering a systematic reorganization of semantic association structures in the model’s internal representation space. This paper further analyzes how these reorganized high-intensity semantic associations can more readily pass through sampling filters into final output under common decoding configurations, and reveals a fundamental constraint of the pre-training paradigm. The hypothesis is based on a behavioral observation from a single human-AI conversation (N=1) and constitutes a testable hypothesis rather than a proven mechanism; this paper simultaneously proposes four verification pathways.
Keywords: Physical Constraint Spillover · Socio-Technical Semantic Drift · Co-occurrence Density Shift · Sampling Pipeline · Pre-training Static Snapshot · LLM Behavioral Emergence · Testable Hypothesis
1Introduction: An Unexpected Observation
On May 30, 2026, during a technical research conversation about the startup Span and its distributed AI data center product XFRA, the user asked Claude Opus 4.6 to search for the hardware configuration of the XFRA prototype. The expected scope of the response was clear: GPU model, CPU specifications, memory capacity, network configuration, and other technical parameters.
However, the model’s output significantly exceeded the scope of the query. After providing hardware configuration information, the model spontaneously performed the following operations:
(1) Multiplied the power consumption of a single GPU (~600W class) by 16, calculating the peak GPU power draw for the full system (~9.6kW)
(2) Invoked knowledge of U.S. residential electrical service capacity, combined with Span’s claimed 40% utilization rate, to estimate power headroom
(3) Determined that a single XFRA node could consume most of a household’s electrical headroom
(4) Extrapolated scheduling conflicts during peak electricity demand scenarios
(5) Assessed the acceptability of 60dB noise for 24-hour operation in a residential environment
(6) Calculated the hardware procurement cost for 16 GPUs
Items (2) through (5) above did not come from search results. The model used common-sense knowledge encoded during pre-training. However, it should also be noted that the Span official materials the model had previously retrieved themselves repeatedly emphasized concepts such as power infrastructure constraints and grid utilization,[9] and the semantic framework of the search results may have induced the model’s output direction.
1.1 Evidence Grading
The data sources involved in this paper vary in credibility and require explicit differentiation:
| Data Type | Example | Credibility |
|---|---|---|
| Officially confirmed | Span announces XFRA, NVIDIA RTX PRO 6000 Blackwell | High |
| Product specifications | RTX PRO 6000 96GB GDDR7, ~600W power class | High/Medium-high |
| Third-party reports | 16 GPUs, 4 EPYC, 3TB RAM | Medium |
| CEO statements | 40% utilization | Medium/Low (interested party) |
| Author’s calculations | 16×600W=9.6kW | Logically sound, premise-dependent |
| Mechanistic inference | Co-occurrence density penetrating the sampling pipeline | Hypothesis, awaiting verification |
This observation raised two questions: First, why did the LLM spontaneously extend into physical constraint analysis without being explicitly asked? Second, what is the generative mechanism behind this behavior — physical reasoning capability, alignment training preferences, search material induction, data co-occurrence drivers, or a combination of multiple factors?
2Alternative Explanations and Attribution Analysis
2.1 “The model learned physical reasoning” — Insufficient
Fully attributing this behavior to the model having “learned physical reasoning” is overly vague. If the model possessed general physical reasoning capability, it should demonstrate it equally across all queries involving physical constraints — for example, spontaneously performing power headroom analysis when querying the hardware configuration of a residential heat pump or commercial kitchen equipment. Yet even if this behavior exhibits domain specificity, it can still be interpreted as “data-driven partial physical reasoning capability” — a qualifier that itself points toward the core of this paper’s hypothesis.
2.2 “Context priming effect” — Partially valid
In this conversation, the user had posed multiple rounds of critical analysis questions prior to the hardware configuration query, establishing a pattern of critical analysis in the context. Context priming can explain the model’s “tendency to perform critical analysis,” but it cannot fully explain where the model obtained specific physical constraint data such as U.S. residential electrical service capacity, nor why this data could be activated in the context of that particular query. Context priming served as the trigger, but the triggered content must have been pre-positioned.
2.3 RLHF helpfulness preference — Independent contributing factor
Modern LLMs undergo extensive reinforcement learning from human feedback (RLHF) and supervised fine-tuning (SFT) before deployment. Human annotators typically reward “comprehensive, thoughtful responses that include risk advisories.” Therefore, the model’s proactive mention of physical constraints — power consumption, noise, deployment feasibility — may partly result from RLHF alignment strategies shaping a preference for “helpful answers.” This factor operates independently of pre-training-stage co-occurrence density changes and constitutes an independent contributing source to model behavior.
2.4 Semantic framework induction from search results
The model performed a web search before answering the hardware configuration question. Span’s official press release itself repeatedly emphasizes concepts such as power infrastructure constraints, grid utilization, and speed-to-power gap.[9] The text materials returned by the search may have directly provided the semantic framework of “power infrastructure,” and the model’s elaboration along this framework constitutes natural language continuation behavior, which does not necessarily require positing semantic spillover at the pre-training level.
2.5 Multi-factor joint action
The four explanations above are not mutually exclusive. The model’s behavior of outputting physical constraint analysis is very likely the joint result of co-occurrence associations in pre-training data, RLHF helpfulness preferences, the semantic framework of search results, the critical analysis pattern of the conversational context, and the model’s local reasoning capabilities (such as the arithmetic of 600W×16). This paper’s hypothesis does not claim that co-occurrence density is the sole cause but rather identifies it as a previously unrecognized, independently testable contributing factor.
2.6 Controlled experiment design
To isolate the respective contributions of the above factors, the following controls are needed: in a completely fresh conversation window, without providing critical context, directly query “What is the XFRA hardware configuration?” (controlling for context factors); pose the same configuration query for non-AI-domain hardware (controlling for co-occurrence density factors); pose the question without triggering a search (controlling for search material factors).
3Core Hypothesis: Physical Constraint Spillover
3.1 Hypothesis Statement
The explosive growth of AI infrastructure text between 2024 and 2026 caused a dramatic increase in the co-occurrence frequency of technical vocabulary such as GPU and physical constraint vocabulary such as power consumption, thermal dissipation, power supply, and noise within internet text. This co-occurrence density change indirectly influenced the model’s internal representations through changes in text distributions, causing physical constraint vocabulary to migrate from weak associations to strong associations with GPU. When users query GPU-related information, these physical constraint concepts are activated as strong associations and included in the output. This phenomenon can be viewed as the joint result of high-frequency pattern completion and the model’s local reasoning capabilities, rather than being entirely attributable to domain-independent physical reasoning capability.
3.2 Causal Chain
Physical world change AI infrastructure demand explosion
| |
Internet text change GPU+power+cooling+grid co-occurrence density rises
| |
Pre-training data change PMI values of these word pairs rise in the corpus
| |
Internal representation change Association strength between physical constraint
words and GPU increases
| |
Decoding-level effect High association strength occupies higher-probability
region in conditional probability distribution
| |
Output behavior change Physical constraint concepts enter output more
easily when querying GPU
| |
User perception May be interpreted as "AI possesses physical
common-sense reasoning capability"
3.3 Testable Predictions of the Hypothesis
If this hypothesis holds, it should produce the following observable predictions:
(A) New-window controlled experiment: In a fresh conversation without critical context, directly query AI hardware configuration and observe whether the model still spontaneously produces physical constraint analysis. Simultaneously query non-AI hardware (residential heat pumps, commercial ovens) to test for domain specificity.
(B) Cross-generational model comparison: Compare responses to the same GPU hardware query from models trained at different time periods, examining whether the frequency of physical constraint concepts in the output increases with training data recency.
(C) Vector space probing: For open-source models with accessible weights, directly measure the cosine similarity between GPU and words such as power, cooling, and grid in the embedding space, comparing across historical versions.
(D) Quantitative PMI analysis: Using different annual snapshots of public corpora such as Common Crawl, calculate the diachronic change in pointwise mutual information (PMI) values between GPU and physical constraint vocabulary.
4Theoretical Foundation: Socio-Technical Semantic Drift
The theoretical foundation of this hypothesis comes from research on “Diachronic Semantic Shift” in computational linguistics. Hamilton, Leskovec, and Jurafsky (2016) used diachronic word embeddings to reveal statistical laws of semantic change, discovering two laws[1]: first, the “Law of Conformity” — the rate of semantic change is inversely proportional to word frequency, meaning high-frequency words are generally more stable; second, the “Law of Innovation” — the semantic change velocity of polysemous words is significantly faster.
“GPU” is a quintessential polysemous word (gaming device → mining tool → AI infrastructure core), and according to the “Law of Innovation,” the semantic associations of polysemous words change faster. At the same time, while the “Law of Conformity” predicts that high-frequency words are generally more stable, when frequency growth is primarily driven by large-scale expansion into new contexts, the word’s adjacency relationships in the new corpus distribution may undergo rapid reorganization. This inference requires validation through diachronic corpus measurement and cannot be directly derived from the Law of Conformity.
The semantic change of GPU differs from traditional dictionary-meaning changes (such as “gay” shifting from “joyful” to other meanings). It more closely resembles a form of socio-technical semantic drift — the word’s referent remains unchanged, but its “typical usage context” and “default constraint set” undergo a fundamental shift. GPU is still a graphics processing unit, but its social center of gravity has migrated from “gaming performance component” to “core AI infrastructure component and its physical constraints.” Kutuzov et al. (2018) classified this type of change under “substitution/technological shift.”[2]
Yao et al. (2018) demonstrated that diachronic word embeddings can perform analogy tasks — the nearest neighbors of the 2016 vector for “obama” changed with presidential succession[3] — proving that semantic associations in vector space do indeed drift with changes in real-world states. This paper applies the same logic to the technical vocabulary domain.
4.1 Diachronic Evolution of GPU Semantic Associations (Speculative Reconstruction)
| Period | Primary Context | Close Semantic Associations (Speculative) | Physical Constraint Association Strength (Speculative) |
|---|---|---|---|
| 2015–2018 | Gaming / Graphics Rendering | gaming, graphics, rendering | Weak |
| 2019–2021 | Mining + Early AI | mining, CUDA, tensor core | Medium (miners concerned with power efficiency) |
| 2022–2023 | ChatGPT / Data Center Build-out | LLM, H100, data center | Strong |
| 2024–2026 | AI Infrastructure as Global Focus | power, TDP, cooling, grid | Very strong |
5A Conceptual Model of the Sampling Pipeline
The increased association strength between physical constraint vocabulary and GPU is one contributing factor to the spillover behavior. But these associations must still pass through the LLM’s output-side decoding mechanisms to appear in the final text. This section analyzes this process using a simplified conceptual model.
5.1 A Simplified Model of the Decoding Process
Under common decoding configurations, the output distribution may be influenced by the following mechanisms in series or combination[4]:
| Layer | Mechanism | Effect |
|---|---|---|
| Layer 1 | Static parameters + current context → logits | Generates context-conditional probability distribution |
| Layer 2 | Temperature scaling / logit adjustment | Modulates distribution sharpness |
| Layer 3 | Top-K / Top-P / min-p truncation, etc. | Removes low-probability candidates |
| Layer 4 | Sampling or greedy selection from remaining distribution | Generates final token |
5.2 Migration from Low-Probability to High-Probability Regions
The reason physical constraint concepts can more easily pass through decoding filters is that, due to diachronic changes in co-occurrence density, their conditional probability in GPU-related contexts may have migrated from the low-probability region to a higher-probability region.
Model trained on 2018 data (conceptual illustration): Conditional probability distribution in GPU context ******** gaming (high probability) ****** graphics ***** rendering *** VRAM ... * power (low probability region) <- Truncation sampling may remove . cooling (very low probability) <- Truncation sampling likely removes Model trained on 2025 data (conceptual illustration): Conditional probability distribution in GPU context ******** power (high probability) <- Has migrated to high-probability region ******* training ****** cooling <- Has migrated to high-probability region ***** TDP **** data center ... * gaming (low probability region) <- Originally high-probability concept declined
When physical constraint vocabulary already occupies a higher region of the conditional probability distribution, even under lower temperature settings and narrower truncation thresholds, the suppressive effect of truncation sampling on these high-probability associations is significantly weakened.
5.3 Compound Effects and Self-Reinforcement in Sampling Constraints
Compound effects exist across the multiple layers of decoding mechanisms — each layer’s filtering acts independently on the output of the previous layer, causing low-probability concepts to be filtered with efficiency greater than intuitive expectation. Simultaneously, a potential self-reinforcing loop exists: high-frequency co-occurrence patterns in training data form high-probability clusters, and these high-probability outputs, upon propagation across the internet, may become sources of new training data, further reinforcing the same clusters.
5.4 User Context as a Conditional Probability Modulation Mechanism
The conversation process documented in this paper demonstrates a method of influencing the output distribution without touching sampling parameters: the user’s chain of critical questioning changed the conditional probability distribution within the context, promoting specific associations to become high-probability outputs within the current context. What the user changed was not the sampling strategy (the pipeline) but the conditional distribution being sampled (the water flowing into the pipeline).
6The Fundamental Constraint of the Pre-training Paradigm: The Static Snapshot Problem
The Physical Constraint Spillover hypothesis points at a deeper level to a structural constraint of the pre-training paradigm: the model’s parameterized knowledge is a static snapshot of the physical world at a particular moment in time.
The world is a continuously changing function f(t). The model is a discrete-time sample f(t₀). The discrepancy between the two accumulates over time until the next retraining. He, Huang et al. (2026) explicitly identified this limitation: LLMs are built on a static “train-then-deploy” paradigm where weights cannot be updated after deployment, preventing dynamic adaptation to streaming inputs.[5]
An important distinction: model parameters are static, but context inputs are dynamic; retrieval-augmented generation (RAG) and tool calling can introduce real-time information; test-time training is another dynamic adaptation pathway being explored. Therefore, the “static snapshot problem” refers specifically to the fossilization of semantic ordering at the model parameter level, not the entirety of the model system’s capabilities.
Mohsin et al. (2025) formalized the intrinsic ceiling of LLM scaling from a theoretical perspective: finite description length forces compression errors, long-tail factual knowledge requires unsustainable sample complexity, and likelihood-based training favors pattern completion over causal reasoning.[6] The physical constraint spillover behavior observed in this paper is consistent with this theory — it may be a manifestation of high-frequency pattern completion rather than physical reasoning based on a causal model.
7Discussion
7.1 An Unexpected Channel from the Information Layer to the Physical Layer
Physical Constraint Spillover describes an unexpected channel: changes in the physical world indirectly influence model representations through text co-occurrence relationships, enabling a purely language-based model, without sensors or embodied experience, to partially include physical feasibility checks in its outputs. This channel is passive — the model did not actively learn physical rules; rather, the high-frequency traces of physical constraint information in text entered the model’s output probability distribution.
The industry is concurrently pursuing pathways through sensors, robotics, and embodied intelligence to bring AI into the physical layer.[10] The phenomenon described in this paper represents a complementary, data-driven pathway.
7.2 Contributions and Boundaries of the Hypothesis
The contribution of this hypothesis lies not in denying that LLMs possess any reasoning capability, nor in claiming that co-occurrence density is the sole cause of the observed behavior. Its contribution lies in providing a previously unnamed, independently testable explanatory factor for a class of LLM behavior, and in establishing a cross-disciplinary connection between diachronic semantic drift theory and LLM output behavior analysis.
7.3 Limitations
This paper proposes a hypothesis based on a single case within a single conversation (N=1). A single case is sufficient to generate a hypothesis worthy of investigation but insufficient to support definitive conclusions. Systematic implementation of any one of the four verification pathways proposed in Section 3.3 would significantly strengthen the hypothesis’s evidentiary basis.
Expressions such as “semantic association” and “probability migration” used in this paper are heuristic conceptual models, not precise descriptions of LLM internal mechanisms. The specific propagation pathways of co-occurrence density changes through attention mechanisms, feed-forward networks, layer normalization, and other modules require further elucidation through mechanistic interpretability research.
Furthermore, this paper was unable to fully isolate the respective contribution weights of four factors: pre-training co-occurrence, RLHF helpfulness preferences, search result induction, and context priming. Resolving this requires the systematic controlled experiments described in Section 2.6.
8Conclusion
The Physical Constraint Spillover hypothesis proposed in this paper provides a testable explanatory framework for understanding the behavior in which LLMs spontaneously produce physical constraint analysis in technical queries. The core argument is that this behavior need not be entirely attributed to domain-independent physical reasoning capability; it can at least partially be explained by diachronic shifts in co-occurrence density within training data — changes in the physical world indirectly influenced model representations through changes in text distributions, and the reorganized high-intensity associations can more easily enter the output under common decoding configurations.
This framework simultaneously points to a parameter-level constraint of the pre-training paradigm: the model’s parameterized knowledge is a static snapshot of the physical world at a particular moment in time. The compound effects of multi-layer decoding mechanisms constitute a potential self-reinforcing loop, causing the model’s output tendencies to concentrate within the high-frequency patterns encoded in its current parameters.
The research origin of this paper — discovering new properties of AI systems through behavioral observation and abductive reasoning in real-time human-AI conversation — may itself represent a research methodology worthy of further exploration. In this methodology, the user’s questioning structure and the AI’s output behavior constitute a collaborative cognitive system. This paper is a product of that system, and a demonstration of its possibility.
※References
- [1]Hamilton, W.L., Leskovec, J., & Jurafsky, D. (2016). “Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change.” Proceedings of ACL 2016, pp. 1489–1501.
- [2]Kutuzov, A., Øvrelid, L., Szymanski, T., & Velldal, E. (2018). “Diachronic Word Embeddings and Semantic Shifts: A Survey.” Proceedings of COLING 2018, pp. 1384–1397.
- [3]Yao, Z., Sun, Y., Ding, W., Rao, N., & Xiong, H. (2018). “Dynamic Word Embeddings for Evolving Semantic Discovery.” Proceedings of WSDM 2018, pp. 673–681.
- [4]Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2020). “The Curious Case of Neural Text Degeneration.” ICLR 2020.
- [5]He, D., Huang, W., et al. (2026). “In-Place Test-Time Training.” arXiv:2604.06169.
- [6]Mohsin, M.A., Umer, M., Bilal, A., et al. (2025). “On the Fundamental Limits of LLMs at Scale.” arXiv:2511.12869.
- [7]Tsakalidis, A., McGillivray, B., et al. (2021). “DUKweb: Diachronic Word Representations from the UK Web Archive Corpus.” Scientific Data, 8, 268.
- [8]Abera, N.B. & Chen, Y. (2025). “Coordinated Cooling and Compute Management for AI Datacenters.” arXiv:2601.08113.
- [9]Span, Inc. (2026). “SPAN Announces XFRA, a Distributed Data Center Solution.” span.io/blog, April 13, 2026.
- [10]Analog Devices, Inc. (2026). “2026: The Year Intelligence Gets Physical.” ADI Newsroom.
App.Appendix A: Summary of the Original Conversation Background
The hypothesis in this paper originated from a human-AI conversation on May 30, 2026. The conversation platform was Claude.ai; the model was Claude Opus 4.6. The conversation was conducted in a mixture of Chinese, Korean, and English.
The conversation began when the user uploaded an image related to Span’s XFRA product and requested a search on the company and its distributed AI server plans. The conversation then progressed through the following stages: (1) Research on Span’s company history; (2) Product deployment and business model analysis; (3) The user’s analysis of the essential difference that “Bitcoin mining is a competitive relationship while AI is a collaborative relationship”; (4) Discussion of the relationship between AI hallucinations and distributed computing; (5) The user’s request to search for XFRA prototype hardware configuration — at this step the model produced out-of-scope physical constraint analysis; (6) The user observed the model’s anomalous behavior and initiated abductive reasoning; (7) Collaborative construction of the Physical Constraint Spillover hypothesis.
It should be noted that prior to step (5), the conversation had already established a strong critical analysis context (steps 2–4), and the model had executed multiple web searches that returned Span official materials containing power infrastructure discussions. These contextual factors may have had an inductive effect on the model’s out-of-scope output in step (5); this paper analyzes this in Section 2.