<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" ><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://zefancai.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://zefancai.com/" rel="alternate" type="text/html" /><updated>2026-11-16T11:17:04-08:00</updated><id>https://zefancai.com/feed.xml</id><title type="html">Zefan Cai</title><subtitle>Zefan Cai - PhD student at UW-Madison. Test-time training, efficient LLM architecture, and long-context memory.</subtitle><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><entry xml:lang="en"><title type="html">Is the LLM Value Chain the Next Apple Supply Chain? Profits, Capital, and Global Specialization</title><link href="https://zefancai.com/en/blog/ai-vs-apple-chain/" rel="alternate" type="text/html" title="Is the LLM Value Chain the Next Apple Supply Chain? Profits, Capital, and Global Specialization" /><published>2026-10-18T00:00:00-07:00</published><updated>2026-10-18T00:00:00-07:00</updated><id>https://zefancai.com/en/blog/ai-vs-apple-chain-en</id><content type="html" xml:base="https://zefancai.com/en/blog/ai-vs-apple-chain/"><![CDATA[<link rel="stylesheet" href="/assets/css/ar-drift.css" />

<main id="main" class="drift-article" lang="en">
  <article class="drift-prose">

    <p class="drift-kicker"><a href="/blog/">Blog</a> / Industry research</p>

    <h1 id="is-the-llm-value-chain-the-next-apple-supply-chain">Is the LLM Value Chain the Next Apple Supply Chain?</h1>

    <p class="drift-deck">From the global manufacturing of a device to the chips, energy, data, and software behind an intelligent task.</p>
    <p class="drift-meta">Zefan Cai · <time datetime="2026-10-18">October 18, 2026</time> · Global industries and companies</p>

    <p>Apple’s supply chain offers an appealing point of comparison. A company that controls the device, operating system, and customer relationship organizes a complex global manufacturing network. Suppliers invest in processes, equipment, and capacity around new products, turning technological innovation into sustained growth. Today’s emerging supplier networks, from GPUs, high-bandwidth memory, and liquid cooling to expert data, agent training environments, and evaluation, readily bring that earlier supply chain to mind.</p>

    <p>The analogy is useful. But if we use it only to search for “the Apple of AI” or “the next smartphone assembler,” we miss the most consequential changes.</p>

    <p><strong>The LLM value chain is developing several centers of bargaining power: computing platforms, advanced manufacturing, cloud and distribution, models, and applications that control business workflows. It adds a continuing layer of inference costs, along with constraints involving power, financing, and utilization.</strong> A supplier’s ability to earn profits depends on the bottleneck it resolves, how readily customers can replace it, and how much capital its growth consumes.</p>

    <p>This article compares the two chains from the perspective of global industries and companies. Chinese companies are discussed within the global stages in which they operate.</p>

    <div class="drift-callout">
      <p><strong>Data conventions.</strong> Sources were checked through <strong>October 18, 2026</strong>. Annual comparisons primarily use each company’s <strong>2025 annual reporting period</strong>; NVIDIA uses FY2026, ended January 25, 2026. Fiscal year-end dates are shown explicitly, and new disclosures from 2026 are discussed separately. Unless otherwise noted, dollar amounts are in <strong>billions of U.S. dollars</strong>, with B denoting billion. Consolidated revenue, segment revenue, company-reported annualized revenue, industry estimates, and forecasts are labeled separately. Revenue at successive stages is not added together to measure the industry’s size.</p>
    </div>

    <nav class="drift-toc" aria-label="Table of contents">
      <ul>
        <li><a href="#structure">Start with the structure of the two chains</a></li>
        <li><a href="#companies">A global company map: what each stage does</a></li>
        <li><a href="#apple">How Apple retains profits in its supply chain</a></li>
        <li><a href="#profits">Why profits accrue at several points in the AI chain</a></li>
        <li><a href="#capital">The biggest change: who carries the capital spending</a></li>
        <li><a href="#models-data">Cloud, models, and applications: who owns the customer relationship</a></li>
        <li><a href="#data-companies">Data companies in depth: Silicon Valley, Beijing, and UniPat</a></li>
        <li><a href="#overlap">Customer concentration, cash flow, and the overlap between the chains</a></li>
        <li><a href="#updates">What the new data from 2026 changes</a></li>
        <li><a href="#constraints">Power, geography, and trade</a></li>
        <li><a href="#outlook">What to watch next</a></li>
        <li><a href="#references">Sources and calculation notes</a></li>
      </ul>
    </nav>

    <h2 id="structure">Start with the structure of the two chains</h2>

    <p>The demand anchors in Apple’s supply chain are relatively clear: device shipments, replacement purchases, the active installed base, and the services consumed through those devices. IDC’s <strong>preliminary estimates</strong>, published in January 2026, put global smartphone shipments in calendar 2025 at approximately <strong>1.2603 billion units</strong>, up <strong>1.9%</strong> year over year. Apple shipped an estimated <strong>247.8 million units</strong>, for a <strong>19.7%</strong> share. These are estimates of branded device shipments, not unit sales disclosed by Apple or consumer activations. <a href="https://www.idc.com/resource-center/press-releases/q425mobilephonetop5/">Source: IDC</a></p>

    <p>Final demand in the LLM chain may take the form of a subscription, a million tokens, a task completed automatically, or an improvement in advertising conversion or software development productivity. Demand passes through model usage, cloud resource purchases, and data center construction before it becomes an order for chips or equipment. Growth at different stages often arrives at different times.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Dimension</th>
            <th>Apple supply chain</th>
            <th>LLM value chain</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Principal organizers</td>
            <td>Apple determines the main product architecture, system experience, and supplier qualification</td>
            <td>GPU platforms, cloud providers, model developers, and application platforms jointly organize demand</td>
          </tr>
          <tr>
            <td>Basic units of revenue</td>
            <td>Devices, accessories, services, and distribution commissions</td>
            <td>Chips and equipment, compute time, tokens, seats, and business outcomes</td>
          </tr>
          <tr>
            <td>Demand and capacity cycles</td>
            <td>Product launches, inventory builds, and replacement cycles</td>
            <td>Training cluster construction, inference usage, enterprise deployment, and data center commissioning</td>
          </tr>
          <tr>
            <td>After customers receive the product</td>
            <td>Most hardware manufacturing costs have already been incurred; services carry continuing costs</td>
            <td>Each cloud inference request continues to consume compute, memory, networking, and electricity</td>
          </tr>
          <tr>
            <td>Principal switching costs</td>
            <td>The iOS ecosystem, user data, process qualification, yields, and delivery capabilities</td>
            <td>Software ecosystems, model behavior, data permissions, cluster engineering, and business workflows</td>
          </tr>
          <tr>
            <td>Inventory to watch</td>
            <td>Components, work in progress, finished goods, and channel inventory</td>
            <td>Equipment and component inventory, plus computing capacity that is not yet fully utilized</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>These differences are matters of emphasis. Apple also operates cloud and AI infrastructure, and AI suppliers also depend on large buyers. Suppliers such as TSMC can retain considerable strength in both chains. The comparison needs to follow actual transactions and relationships of control.</p>

    <h2 id="companies">A global company map: what each stage does</h2>

    <p>The following map shows representative positions. A company can span several stages, and appearing in the same row does not make two companies interchangeable.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Stage</th>
            <th>Roles and representative companies in the Apple chain</th>
            <th>Roles and representative companies in the LLM chain</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Product and system design</td>
            <td>Apple: device architecture, in-house A- and M-series chips, iOS/macOS, and the end-user experience</td>
            <td>NVIDIA: GPUs and system platforms; AMD: GPUs/CPUs; cloud providers such as Google and AWS: custom accelerators; Broadcom: custom silicon and networking partnerships</td>
          </tr>
          <tr>
            <td>Wafer fabrication and packaging</td>
            <td>Foundries such as TSMC, supported by packaging and testing providers</td>
            <td>TSMC’s advanced processes and packaging; packaging and testing providers such as ASE and Amkor. Compute chips, HBM, substrates, and packaging must be delivered together</td>
          </tr>
          <tr>
            <td>Semiconductor equipment and design tools</td>
            <td>A shared upstream foundation for both chains: ASML, Applied Materials, Lam Research, and KLA; Synopsys and Cadence</td>
            <td>The same manufacturing and design infrastructure serves multiple end markets. Its entire revenue cannot be attributed to AI or Apple</td>
          </tr>
          <tr>
            <td>Memory and storage</td>
            <td>Samsung, SK hynix, and Micron: mobile DRAM, NAND, and related products</td>
            <td>SK hynix, Micron, and Samsung: HBM, server DRAM, and NAND; product mix and industry cycles affect profitability</td>
          </tr>
          <tr>
            <td>Displays, imaging, radio frequency, and structural components</td>
            <td>Samsung Display, LG Display, Sony, Largan, Murata, Broadcom, Qualcomm, Corning, Lens Technology, and others across displays, sensing, lenses, connectivity, and materials</td>
            <td>Not every smartphone component category has an AI server counterpart. Incremental demand is more concentrated in high-speed interconnects, power delivery, cooling, and dense system integration</td>
          </tr>
          <tr>
            <td>Assembly and system integration</td>
            <td>Manufacturing networks including Hon Hai (Foxconn), Luxshare, Pegatron, and Tata Electronics</td>
            <td>Foxconn, Quanta/QCT, Wiwynn, Dell, HPE, Supermicro, and Inspur: servers, racks, system integration, and delivery</td>
          </tr>
          <tr>
            <td>Networking and optical interconnects</td>
            <td>Device connectivity chips, antennas, and wireless communications</td>
            <td>NVIDIA, Broadcom, Arista, and Cisco; optical communications companies such as Coherent, Lumentum, Zhongji Innolight, and Eoptolink, with different roles from components to modules</td>
          </tr>
          <tr>
            <td>Power and thermal management</td>
            <td>Energy for manufacturing facilities, device batteries, and thermal design</td>
            <td>Vertiv, Schneider Electric, Eaton, Delta, Envicool, and grid and energy providers; the focus is power supply, distribution, UPS, cooling, and engineering delivery</td>
          </tr>
          <tr>
            <td>Cloud and compute operations</td>
            <td>Apple’s own and external service infrastructure</td>
            <td>AWS, Microsoft Azure, Google Cloud, Oracle OCI, and CoreWeave. Meta is also a large compute buyer, although selling general-purpose cloud services is not the main way its group monetizes that capacity</td>
          </tr>
          <tr>
            <td>Models and applications</td>
            <td>Apple’s systems, content, and developer ecosystem connect devices to service consumption</td>
            <td>Model ecosystems including OpenAI, Anthropic, Google, Meta, Mistral, Cohere, DeepSeek, and Qwen; application and workflow entry points such as Microsoft/GitHub, Salesforce, ServiceNow, Adobe, and Cursor</td>
          </tr>
          <tr>
            <td>Data, tasks, and evaluation</td>
            <td>Content/IP, engineering specifications, and acceptance testing provide functional comparisons, rather than matching components one for one</td>
            <td>Data and task producers such as Scale, Surge, Mercor, UniPat, Zhineng Zhishi, and TalentsAI; content licensors such as Reddit; enterprise data platforms such as Snowflake and Databricks. See the <a href="#data-companies">data companies section</a></td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>These positions can be read alongside <a href="https://s203.q4cdn.com/367071867/files/doc_downloads/2024/04/Apple-Supplier-List.pdf">Apple’s official supplier list</a>, the company filings cited below, and <a href="https://www.oracle.com/cloud/compute/gpu/">OCI’s GPU infrastructure description</a>. <strong>Supplier lists have their own reporting periods.</strong> The list obtained through Apple’s official 2026 reporting portal still states on its cover that it represents <strong>98% of direct spending on materials, manufacturing, and assembly in FY2023</strong>. It establishes relationships during that coverage period, not suppliers’ shares of every product in 2026. Rows covering equipment, EDA, cloud, and applications also include indirect suppliers and ecosystem participants; they should not all be treated as direct Apple vendors.</p>

    <p>This map exposes the limit of the analogy. Apple’s chain is centered on the product system of one device company. The LLM chain is a network spanning products, computing platforms, clouds, and applications. A data provider and a power equipment manufacturer create value in very different ways.</p>

    <h2 id="apple">How Apple retains profits in its supply chain</h2>

    <p>Apple’s FY2025, the year ended September 27, 2025, provides a clear picture of its profit structure. <a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">Source: Apple 10-K</a></p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Apple FY2025 metric</th>
            <th style="text-align: right">Value</th>
            <th>Interpretation</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Consolidated revenue</td>
            <td style="text-align: right">USD 416.161B</td>
            <td>The whole company, including devices and services</td>
          </tr>
          <tr>
            <td>iPhone revenue</td>
            <td style="text-align: right">USD 209.586B</td>
            <td><strong>50.36%</strong> of consolidated revenue</td>
          </tr>
          <tr>
            <td>Services revenue</td>
            <td style="text-align: right">USD 109.158B</td>
            <td><strong>26.23%</strong> of consolidated revenue</td>
          </tr>
          <tr>
            <td>Products gross margin</td>
            <td style="text-align: right"><strong>36.8%</strong></td>
            <td>All products combined, not the iPhone gross margin alone</td>
          </tr>
          <tr>
            <td>Services gross margin</td>
            <td style="text-align: right"><strong>75.4%</strong></td>
            <td>A mix of advertising, licensing, content, cloud, AppleCare, payments, and other services</td>
          </tr>
          <tr>
            <td>Consolidated operating income</td>
            <td style="text-align: right">USD 133.050B</td>
            <td>Calculated operating margin of <strong>31.97%</strong></td>
          </tr>
          <tr>
            <td>Operating cash flow</td>
            <td style="text-align: right">USD 111.482B</td>
            <td>Distinct from net income and free cash flow</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Services generated <strong>42.17% of consolidated gross profit</strong> from roughly one-quarter of revenue: 82.314 ÷ 195.201. Hardware establishes the customer relationship; the operating system, distribution, and services allow that relationship to generate revenue after the device is sold. Looking only at iPhone shipments understates Apple’s economic structure.</p>

    <p>But a 75.4% gross margin does not mean every app has that profitability. Apple recognizes third-party App Store-related sales on a <strong>net basis</strong>, including only the commission it retains in Services revenue. Hardware companies and manufacturers often recognize revenue on a different, gross basis. Revenue and gross margins reflect economic position, but also product mix and accounting presentation.</p>

    <p>Manufacturing looks very different. In calendar 2025, Foxconn generated <strong>NTD 8.103 trillion</strong> in revenue, with calculated consolidated gross and operating margins of <strong>6.15%</strong> and <strong>3.20%</strong>, respectively. <a href="https://image.honhai.com/upload/202605/financy_by_quarterly/2025Q4_consolidared_report(e)_20260515_5808.pdf">Source: Foxconn consolidated financial statements</a> Scale, engineering expertise, and delivery capabilities can be formidable without producing the profit structure of a company that owns the end-user ecosystem.</p>

    <p>The relatively low ratio of Apple’s own cash purchases of fixed assets to revenue also reflects this division of manufacturing responsibilities. Apple accesses a vast system of factories and equipment through suppliers. That capital does not disappear because it sits outside Apple’s balance sheet. <strong>To understand how little capital a lead company holds itself, we also need to see what its suppliers carry.</strong></p>

    <h2 id="profits">Why profits accrue at several points in the AI chain</h2>

    <p>Start by placing representative companies in one table. These are <strong>unadjusted consolidated financial results</strong>, not margins for AI businesses alone. Their fiscal years are not perfectly aligned, so the table illustrates economic structures rather than ranking profitability over an identical period.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Company / position</th>
            <th>Fiscal year and end date</th>
            <th style="text-align: right">Consolidated gross margin</th>
            <th style="text-align: right">Consolidated operating margin</th>
            <th>Essential qualification</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Apple / devices and ecosystem</td>
            <td>FY2025, 2025-09-27</td>
            <td style="text-align: right">46.9%</td>
            <td style="text-align: right">32.0%</td>
            <td>Includes high-margin Services; not device profitability alone</td>
          </tr>
          <tr>
            <td>NVIDIA / computing platform</td>
            <td>FY2026, 2026-01-25</td>
            <td style="text-align: right">71.1%</td>
            <td style="text-align: right">60.4%</td>
            <td>A mix of GPUs, systems, networking, and other products; not profit on an individual GPU</td>
          </tr>
          <tr>
            <td>TSMC / manufacturing and packaging</td>
            <td>CY2025, 2025-12-31</td>
            <td style="text-align: right">59.9%</td>
            <td style="text-align: right">50.8%</td>
            <td>IFRS as endorsed in Taiwan; serves multiple end markets</td>
          </tr>
          <tr>
            <td>Broadcom / semiconductors and software</td>
            <td>FY2025, 2025-11-02</td>
            <td style="text-align: right">67.8%</td>
            <td style="text-align: right">39.9%</td>
            <td>Infrastructure Software accounts for approximately 42% of revenue</td>
          </tr>
          <tr>
            <td>AMD / CPUs, GPUs, and other products</td>
            <td>FY2025, 2025-12-27</td>
            <td style="text-align: right">49.5%</td>
            <td style="text-align: right">10.7%</td>
            <td>Includes several businesses and acquisition-related accounting costs</td>
          </tr>
          <tr>
            <td>Arista / networking systems</td>
            <td>CY2025, 2025-12-31</td>
            <td style="text-align: right">64.1%</td>
            <td style="text-align: right">42.8%</td>
            <td>Data center, AI, campus, and other markets; not all AI networking</td>
          </tr>
          <tr>
            <td>Micron / memory and storage</td>
            <td>FY2025, 2025-08-28</td>
            <td style="text-align: right">39.8%</td>
            <td style="text-align: right">26.1%</td>
            <td>Sensitive to the memory cycle; changes in 2026 are discussed below</td>
          </tr>
          <tr>
            <td>Vertiv / power and thermal management</td>
            <td>CY2025, 2025-12-31</td>
            <td style="text-align: right">36.3%</td>
            <td style="text-align: right">17.9%</td>
            <td>Includes conventional data centers and other infrastructure demand</td>
          </tr>
          <tr>
            <td>Foxconn / manufacturing and integration</td>
            <td>CY2025, 2025-12-31</td>
            <td style="text-align: right">6.1%</td>
            <td style="text-align: right">3.2%</td>
            <td>IFRS as endorsed in Taiwan; spans consumer electronics and cloud/networking</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: <a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">Apple</a>, <a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-fourth-quarter-and-fiscal-2026">NVIDIA</a>, <a href="https://investor.tsmc.com/english/quarterly-results/2025/q4">TSMC</a>, <a href="https://investors.broadcom.com/news-releases/news-release-details/broadcom-inc-announces-fourth-quarter-and-fiscal-year-2025">Broadcom</a>, <a href="https://data.sec.gov/api/xbrl/companyfacts/CIK0000002488.json">AMD SEC data</a>, <a href="https://data.sec.gov/api/xbrl/companyfacts/CIK0001596532.json">Arista SEC data</a>, <a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-26-Earnings-Deck.pdf">Micron’s FY2025 comparative figures</a>, <a href="https://s205.q4cdn.com/554782763/files/doc_financials/2025/q4/Vertiv_Fourth-Quarter-2025-Earnings-Release.pdf">Vertiv</a>, and <a href="https://image.honhai.com/upload/202605/financy_by_quarterly/2025Q4_consolidared_report(e)_20260515_5808.pdf">Foxconn</a>. Operating margins are calculated as operating income divided by revenue, using GAAP or the applicable IFRS figures without mixing in adjusted or non-GAAP measures.</p>

    <p>The data supports at least three observations.</p>

    <p>First, <strong>upstream suppliers can retain substantial profits too.</strong> NVIDIA’s software, systems, and developer ecosystem; TSMC’s manufacturing processes, yields, and delivery capabilities; and Arista’s networking systems create different forms of switching cost. Apple’s own chain has long included powerful suppliers such as TSMC. The idea that the lead company captures all the profits while every supplier is a low-margin contract manufacturer is incomplete.</p>

    <p>Second, <strong>entering AI does not turn every company into the same kind of business.</strong> Standardization, control over design, the proportion of materials in revenue, customer qualification, scale efficiencies, and supply-and-demand cycles all affect profitability. Delivering AI servers does not entitle a company to the gross margins of a semiconductor platform.</p>

    <p>Third, <strong>business labels are broader than they may appear.</strong> NVIDIA’s FY2026 Data Center revenue was <strong>USD 193.7B</strong>, approximately <strong>89.7%</strong> of total revenue, but it was not all related to LLMs. TSMC’s HPC platform represented <strong>58%</strong> of 2025 revenue, while management described the narrower AI accelerator category as only a <strong>high-teens percentage</strong>. Calling the entire 58% HPC share “AI revenue” substantially overstates the comparable category. <a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-fourth-quarter-and-fiscal-2026">NVIDIA</a>, <a href="https://investor.tsmc.com/english/quarterly-results/2025/q4">TSMC earnings call</a></p>

    <p>Memory cannot be explained by the HBM label alone either. SK hynix reported 2025 revenue of <strong>KRW 97.1467 trillion</strong> and operating profit of <strong>KRW 47.2063 trillion</strong>, and said HBM revenue more than doubled year over year. Other DRAM and NAND products also affected those results. <a href="https://news.skhynix.com/en/sk-hynix-announces-fy25-financial-results/">Source: SK hynix</a> This article does not fill in undisclosed absolute HBM revenue or margins with estimates.</p>

    <h2 id="capital">The biggest change: who carries the capital spending</h2>

    <p>Apple places much of the manufacturing capital within its supplier network. The LLM industry is making cloud and compute operators carry increasingly large investments in equipment and facilities directly.</p>

    <p>To keep the comparison as consistent as possible, the table uses <strong>gross cash purchases of property, plant, and equipment from the cash flow statement</strong>. It excludes noncash lease additions, finance lease principal payments, business acquisitions, and multiyear investment commitments. Microsoft’s and Apple’s FY2025 periods do not fully overlap with calendar 2025 for the other three companies.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Company</th>
            <th>Year-end date</th>
            <th style="text-align: right">Cash PP&amp;E purchases</th>
            <th style="text-align: right">Revenue in the same period</th>
            <th style="text-align: right">Purchases / revenue</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Apple</td>
            <td>2025-09-27</td>
            <td style="text-align: right">USD 12.715B</td>
            <td style="text-align: right">USD 416.161B</td>
            <td style="text-align: right"><strong>3.06%</strong></td>
          </tr>
          <tr>
            <td>Microsoft</td>
            <td>2025-06-30</td>
            <td style="text-align: right">USD 64.551B</td>
            <td style="text-align: right">USD 281.724B</td>
            <td style="text-align: right"><strong>22.91%</strong></td>
          </tr>
          <tr>
            <td>Alphabet</td>
            <td>2025-12-31</td>
            <td style="text-align: right">USD 91.447B</td>
            <td style="text-align: right">USD 402.836B</td>
            <td style="text-align: right"><strong>22.70%</strong></td>
          </tr>
          <tr>
            <td>Amazon</td>
            <td>2025-12-31</td>
            <td style="text-align: right">USD 131.819B</td>
            <td style="text-align: right">USD 716.924B</td>
            <td style="text-align: right"><strong>18.39%</strong></td>
          </tr>
          <tr>
            <td>Meta</td>
            <td>2025-12-31</td>
            <td style="text-align: right">USD 69.691B</td>
            <td style="text-align: right">USD 200.966B</td>
            <td style="text-align: right"><strong>34.68%</strong></td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: <a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">Apple</a>, <a href="https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/msft-20260630.htm">Microsoft</a>, <a href="https://www.sec.gov/Archives/edgar/data/1652044/000165204426000018/goog-20251231.htm">Alphabet</a>, <a href="https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm">Amazon</a>, and <a href="https://www.sec.gov/Archives/edgar/data/1326801/000162828026003942/meta-20251231.htm">Meta</a>. These are consolidated expenditures and include non-AI businesses. Amazon also invests in logistics and other facilities; Meta’s infrastructure also supports advertising and recommendation systems. The table illustrates who bears the capital burden. It is not a ranking of LLM investment.</p>

    <p>Definitions can materially change headline figures. Amazon’s cash capex after deducting asset sales and incentive proceeds was <strong>USD 128.320B</strong> in the same period. Meta reported approximately <strong>USD 72.22B</strong> in capex when finance lease principal payments were added to cash PP&amp;E purchases. This table uses USD 131.819B and USD 69.691B, respectively, to maintain the definition of gross cash purchases. <a href="https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm">Amazon</a>, <a href="https://www.sec.gov/Archives/edgar/data/1326801/000162828026003942/meta-20251231.htm">Meta</a></p>

    <p>By Microsoft’s FY2026, ended June 30, 2026, the same cash purchases line had reached <strong>USD 115.948B</strong>, or <strong>34.94%</strong> of revenue for that period. Its annual report also disclosed that Microsoft Cloud gross margin fell to <strong>66%</strong>, citing factors including continued investment in AI infrastructure and growing AI product usage. Microsoft Cloud should not be equated with Azure alone. <a href="https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/msft-20260630.htm">Source: Microsoft</a></p>

    <p>The capital structure of a specialist GPU cloud is more concentrated. CoreWeave recognized <strong>USD 5.131B</strong> of revenue in 2025, while cash purchases of equipment and capitalized internal-use software reached <strong>USD 10.309B</strong>, approximately <strong>2.01 times</strong> annual revenue. Its largest unnamed customer contributed <strong>67%</strong> of revenue. <a href="https://www.sec.gov/Archives/edgar/data/1769628/000176962826000104/crwv-20251231.htm">Source: CoreWeave 10-K</a> This describes one year of expansion, not a permanent requirement to invest two dollars for every dollar of revenue. It shows that specialist compute operators carry equipment financing, delivery, and major-customer risks at the same time.</p>

    <p>A GPU shipment is therefore not the end of the economic cycle. Equipment must be installed, networked, powered, and supplied with a steady workload. Pricing and utilization must then cover depreciation, energy, operations, maintenance, and financing costs. A supplier recognizing revenue today does not establish that the final computing service has already earned an adequate return.</p>

    <h2 id="models-data">Cloud, models, and applications: who owns the customer relationship</h2>

    <p>Looking only at hardware can turn the LLM chain into another list of electronics manufacturers. At the other end of the commercial cycle, however, the competition is over customer relationships, enterprise data, and workflows.</p>

    <p>Cloud platforms already have established revenue sources. In 2025, AWS generated <strong>USD 128.725B</strong> in revenue and <strong>USD 45.606B</strong> in operating income, for a calculated operating margin of <strong>35.4%</strong>. Google Cloud generated <strong>USD 58.705B</strong> in revenue and <strong>USD 13.910B</strong> in operating income, for a <strong>23.7%</strong> operating margin. <a href="https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm">Amazon</a>, <a href="https://www.sec.gov/Archives/edgar/data/1652044/000165204426000018/goog-20251231.htm">Alphabet</a> Both are entire cloud segments, not AI-only businesses. Google Cloud also includes Workspace and other offerings.</p>

    <p>Model developers are building their own subscriptions, APIs, and application entry points, but their public figures cannot yet be placed alongside those filings without qualification.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Company and disclosure date</th>
            <th>Verifiable company-reported figures</th>
            <th>What they do not represent</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>OpenAI, 2026-01-18</td>
            <td>Annualized revenue in 2025 <strong>exceeded USD 20B</strong></td>
            <td>Not audited revenue recognized over the full year 2025</td>
          </tr>
          <tr>
            <td>OpenAI, 2026-02-27</td>
            <td>ChatGPT had <strong>more than 900 million weekly active users</strong>, <strong>more than 50 million consumer subscribers</strong>, and <strong>more than 9 million paying business users</strong></td>
            <td>These user categories cannot be added as nonoverlapping paying customers; weekly activity is not revenue</td>
          </tr>
          <tr>
            <td>Anthropic, 2026-05-28</td>
            <td>The company said run-rate revenue that month had exceeded <strong>USD 47B</strong></td>
            <td>A dated annualized snapshot; comparison with an OpenAI snapshot from a different date does not establish market share</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: <a href="https://openai.com/index/a-business-that-scales-with-the-value-of-intelligence/">OpenAI CFO disclosure</a>, <a href="https://openai.com/index/scaling-ai-for-everyone/">OpenAI scale announcement</a>, and <a href="https://www.anthropic.com/news/series-h">Anthropic Series H announcement</a>. This article did not obtain complete public income statements for either company on a basis comparable with listed-company annual reports, and therefore does not insert estimated gross or operating margins.</p>

    <p>LLM applications also carry continuing costs: deeper usage can entail more model execution, retrieval, tool calls, retries, and human review. A more useful enterprise metric is therefore <strong>total cost per acceptable completed task = total associated inference, tool, retry, and review costs ÷ the number of successfully completed tasks.</strong> Lower token prices can reduce costs while encouraging more complex usage. A price decline alone does not establish that total compute demand will fall or profits will rise.</p>

    <p>There are consequently at least three possible answers to who resembles Apple. NVIDIA controls a computing developer ecosystem. Model companies such as OpenAI and Anthropic are competing for direct access to users. Microsoft, Google, Amazon, and enterprise software platforms already have extensive customer relationships, distribution, and workflows. An enterprise may use several models while finding it difficult to migrate its existing data permissions and business processes. The eventual division of value will depend on how these different switching costs compete.</p>

    <h2 id="data-companies">Data companies in depth: Silicon Valley, Beijing, and UniPat</h2>

    <p>To place data companies in the value chain, we first need to establish what customers are buying. Some suppliers provide licensable text, images, and video. Others organize experts to produce demonstrations, preferences, and scoring criteria. Some deliver agent task environments that can be run, reset, and tested; others supply data governance and production software. Their roles resemble content/IP suppliers, engineering service providers, research and validation tools, and enterprise software platforms, respectively. Their revenue structures and competitive advantages differ accordingly.</p>

    <p>The following discussion focuses on startups and new entrants. “Silicon Valley” is used broadly to refer to the San Francisco Bay Area’s startup ecosystem, while distinguishing headquarters, offices, legal contact addresses, and remote teams. The Beijing discussion likewise separates registered legal entities, new startups, and established suppliers. Registered experts, total payouts, financing valuations, and revenue are labeled separately.</p>

    <h3 id="silicon-valley-data">Silicon Valley and the Bay Area: from talent networks to post-training engineering</h3>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Company and status</th>
            <th>Location evidence</th>
            <th>Data products and deliverables</th>
            <th>Verifiable commercial or product signal</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td><strong>Scale AI</strong></td>
            <td>Its website explicitly lists San Francisco as headquarters</td>
            <td>Expert demonstrations, preferences, and safety evaluation; SWE Atlas demonstrates executable coding tasks and review in multiple stages</td>
            <td>Reuters reported <strong>USD 870M in 2024 revenue</strong>; not publicly audited financial statements or 2026 revenue</td>
          </tr>
          <tr>
            <td><strong>Surge AI</strong></td>
            <td>Described as San Francisco-based in Reuters’ 2025 reporting</td>
            <td>Difficult expert tasks, ready-made datasets, long-horizon tool tasks, and RL environments; trial training and quality feedback</td>
            <td>The same Reuters report put <strong>2024 revenue above USD 1B</strong>; Surge declined to comment</td>
          </tr>
          <tr>
            <td><strong>Mercor</strong></td>
            <td>Its website identifies San Francisco as HQ, with offices in New York and London</td>
            <td>Expert screening; tasks, rubrics, reference solutions, coding and enterprise work environments; off-the-shelf licenses and custom post-training partnerships</td>
            <td>On 2025-09-09, its CEO told TechCrunch that annualized revenue exceeded <strong>USD 450M</strong>, explicitly including <strong>total customer payments before contractor compensation</strong></td>
          </tr>
          <tr>
            <td><strong>Turing</strong></td>
            <td>Described as Palo Alto-based in TechCrunch’s 2025 reporting</td>
            <td>Extends engineer screening and organization into coding and reasoning training data, while retaining engineering talent and enterprise AI businesses</td>
            <td>A CEO interview in March 2025 reported an <strong>annualized revenue run rate of approximately USD 300M</strong>; a historical snapshot of a mixed business</td>
          </tr>
          <tr>
            <td><strong>Labelbox</strong></td>
            <td>Its website identifies San Francisco as headquarters; founded in 2018</td>
            <td>Expanded from annotation tools into the Alignerr expert network, Horizon/WorldSim environments, trajectories, and verifiers</td>
            <td>On 2026-10-14, announced <strong>more than USD 100M</strong> in combined equity financing and a credit facility; financing, with no absolute revenue figure disclosed from which to fill this table</td>
          </tr>
          <tr>
            <td><strong>micro1</strong></td>
            <td>A legal contact address in San Francisco; the company describes its core team as globally remote</td>
            <td>Expert screening, Realm training/evaluation environments, and Cortex enterprise failure diagnosis and targeted data production</td>
            <td>On 2026-10-12, <strong>the company reported an annual run rate above USD 1B</strong>; it did not provide a complete public policy for gross versus net revenue recognition</td>
          </tr>
          <tr>
            <td><strong>Handshake AI</strong></td>
            <td>Its parent platform has a San Francisco office; founded in 2014 and introduced its AI business in 2025</td>
            <td>Uses university and career networks to verify experts, delivering specialized prompts, response evaluation, and quality management</td>
            <td>A new business within an existing career network, not a newly formed standalone company; payouts to experts are not revenue</td>
          </tr>
          <tr>
            <td><strong>Lightwheel</strong></td>
            <td>Its official careers page lists roles in Santa Clara and Palo Alto</td>
            <td>SimReady simulation assets, EgoSuite egocentric human data, and RoboFinals simulation evaluations</td>
            <td>Adjacent data infrastructure for embodied/physical AI; these locations do not identify the locations of all production teams</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: <a href="https://scale.com/about">Scale’s company profile</a> and <a href="https://scale.com/blog/swe-atlas">SWE Atlas</a>; <a href="https://www.reuters.com/business/scale-ais-bigger-rival-surge-ai-seeks-up-1-billion-capital-raise-sources-say-2025-07-01/">Reuters’ Surge/Scale report</a>; <a href="https://surgehq.ai/ots">Surge’s product catalog</a>; <a href="https://www.mercor.com/mission/">Mercor’s website</a>, <a href="https://www.mercor.com/apex/off-the-shelf-data/">data catalog</a>, and <a href="https://techcrunch.com/2025/09/09/sources-ai-training-startup-mercor-eyes-10b-valuation-on-450m-run-rate/">CEO interview</a>; <a href="https://techcrunch.com/2025/03/06/turing-a-key-coding-provider-for-openai-and-other-llm-producers-raises-111m-at-a-2-2b-valuation/">Turing interview</a>; <a href="https://labelbox.com/company/about/">Labelbox’s company profile</a>, <a href="https://labelbox.com/blog/labelbox-series-e/">financing announcement</a>, and <a href="https://labelbox.com/products/horizon/">Horizon</a>; <a href="https://www.micro1.ai/blog/q3">micro1’s update</a>, <a href="https://www.micro1.ai/legal/privacy-policy">legal contact information</a>, and <a href="https://www.micro1.ai/cortex">Cortex</a>; <a href="https://joinhandshake.com/about/">Handshake’s company profile</a> and <a href="https://joinhandshake.com/blog/our-team/introducing-handshake-ai/">AI business introduction</a>; <a href="https://www.lightwheel.ai/">Lightwheel’s products</a> and <a href="https://www.lightwheel.ai/careers">job locations</a>. These figures have different dates, sources, and recognition bases. They are not a comparable revenue ranking.</p>

    <p>Three developments deserve a closer look.</p>

    <p><strong>Surge brings training results into the sales process.</strong> Its Trusted Program lets qualifying labs train on the full dataset, evaluate the model, and pay when the agreed metrics improve. <a href="https://surgehq.ai/ots">Surge</a> This is a provision of a specific program, but it shows suppliers beginning to demonstrate value through what a dataset actually teaches a model. Delivery capabilities include difficult task design, expert judgment, validation on held-out examples, and iteration speed.</p>

    <p><strong>Mercor, Turing, micro1, and Handshake turn access to talent into data production capabilities.</strong> Model developers need professionals to demonstrate, judge, correct, and design tasks. Mercor’s catalog already includes containerized coding tasks, executable tests, enterprise application environments, and expert rubrics. micro1 connects diagnosis of enterprise agents’ failures to targeted data production and subsequent retesting. <a href="https://www.mercor.com/apex/off-the-shelf-data/">Mercor</a>, <a href="https://www.micro1.ai/cortex">micro1</a> But accounts in a talent network are different from qualified experts completing paid tasks this month. Mercor’s gross revenue example also shows that substantial contractor compensation can flow through a platform; that amount should not be compared directly with net software subscription revenue.</p>

    <p><strong>Labelbox combines tools, experts, and environments.</strong> Horizon delivers enterprise tool simulations, task trajectories, deterministic verifiers, and human preferences. Customer evidence shows a more precise division of work: the appendix of Meta’s GIM paper states that Labelbox was contracted to handle participant recruitment, credential verification, compensation, informed consent, and related work, while the GIM team performed final scoring independently. <a href="https://arxiv.org/html/2605.18663v1">Original paper, §§G.1–G.2</a> The supplier provides production organization and quality support; the customer retains research design and acceptance criteria. This is a more concrete description of the transaction than simply “selling data to model companies.”</p>

    <h3 id="beijing-data">Beijing: new expert teams and established delivery networks</h3>

    <p>Beijing’s data ecosystem is also extending from collection and annotation into expert tasks, environments, and evaluation. Newly founded companies should nevertheless be distinguished from more established delivery networks.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Company / entity and stage</th>
            <th>Publicly described products and forms of delivery</th>
            <th>Verifiable scale or process</th>
            <th>How to interpret the evidence</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td><strong>Zhineng Zhishi (智能知识) / Meetchances / Xpert Studio</strong>; registered in Beijing, with a public corporate profile listing a 2025 founding</td>
            <td>Terminal RL, MCP tasks spanning multiple tools, and coding tasks in Harbor format, including environments, reference solutions, tests, and trajectories; expert evaluation</td>
            <td>Meetchances reports <strong>50,000+ domain experts</strong>; the product catalog documents programmatic verification, quality checks, and rework processes</td>
            <td>The new team’s products resemble external task and evaluation engineering; expert network size is not employee count or revenue</td>
          </tr>
          <tr>
            <td><strong>TalentsAI</strong>; legal entity 原壤（北京）智能科技有限公司, with a public corporate profile listing a 2025 founding</td>
            <td>Experts across disciplines create questions and reasoning, evaluate model responses, and develop rubrics and quality reviews</td>
            <td>Its website reports <strong>100,000+ registered experts, 300+ specialized disciplines, and more than 20,000 active annotation hours per month</strong></td>
            <td>Operational snapshots with different definitions; registered users or hours cannot be converted directly into revenue</td>
          </tr>
          <tr>
            <td><strong>Longmao Data</strong>; Beijing Anjie Zhihe (北京安捷智合), which says it was founded in 2014 and is headquartered in Beijing’s Zhongguancun</td>
            <td>Multimodal data collection and annotation, SFT/RLHF, coding-agent workflows, agent trajectories, and model evaluation</td>
            <td>Its website reports <strong>5,000+ custom projects cumulatively</strong>, and describes the roles of delivery centers in Chongqing and Chengdu</td>
            <td>An established entrepreneurial supplier adapting its delivery capabilities, not a company founded in 2025</td>
          </tr>
          <tr>
            <td><strong>Testin Data</strong>; a data business brand created within Testin in 2017</td>
            <td>Scenario-based data collection, multimodal annotation, privately deployed annotation platforms, and dataset management</td>
            <td>Beijing-headquartered parent Testin was founded in 2011; combines tools, scenarios, and operations</td>
            <td>The brand is not a separate startup; the parent’s historical financing and testing customers are not data-business revenue or customers</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: <a href="https://human-intelligence.cn/products/data/">Zhineng Zhishi’s product catalog</a>, <a href="https://meetchances.com/">Meetchances</a>, and its <a href="https://vc.pedaily.cn/company/818604.html">public corporate profile</a>; <a href="https://www.talents-ai.com/">TalentsAI’s website</a>, <a href="https://www.talents-ai.com/privacy-agreement.html?v=v.3.0.0&amp;date=20260820">entity privacy policy</a>, and <a href="https://vc.pedaily.cn/company/891792.html">public corporate profile</a>; <a href="https://www.longmaodata.com/about">Longmao’s company profile</a>; <a href="https://www.testin.cn/business/about.htm">Testin’s company and brand history</a> and <a href="https://ai.testin.cn/">data products</a>. A Beijing-registered entity does not establish that its entire team works in Beijing. DataBaker provides another example of a company spanning cities: its website says it began in Beijing in 2016 but is now headquartered in Qingdao, so it is not listed here as currently Beijing-headquartered. <a href="https://www.data-baker.com/about.html">DataBaker</a></p>

    <p>Zhineng Zhishi stands out because the coding tasks in its public catalog already include fixed starting environments, reference implementations, tests, and execution trajectories. Such assets can support both training and evaluation; their costs include environment configuration, correctness checks, difficulty calibration, and separation of training and test data. TalentsAI’s public evidence is more concentrated on expert credentials, specialized questions, and review processes. Describing tasks as complex does not establish that it has the same environment infrastructure.</p>

    <p>Public financial information on these companies remains limited. Rather than rank them by unconfirmed valuations, it is more useful to track repeat purchases, acceptance and rework rates, cost per accepted task, and whether environments and quality-control software can be reused across projects.</p>

    <h3 id="unipat-data">UniPat: turning research problems, scoring rules, and execution environments into data assets</h3>

    <p>UniPat AI deserves separate discussion because it publicly demonstrates a relatively complete technical approach: models expand knowledge and construct candidate problems, experts and models jointly validate the problems and scoring rules, and the resulting assets support training and evaluation. This discussion uses its technical releases to examine the form of its capabilities; benchmark or open-source project scale is not treated as commercial orders.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>UniPat public project</th>
            <th>Company-reported scale and date</th>
            <th>Technical and engineering approach demonstrated</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td><strong>UniScientist</strong></td>
            <td>2026-03-04: <strong>4,700+ research-level samples, 50+ disciplines, and 20+ verifiable scoring criteria per problem</strong>; reported average expert handling time of <strong>1–2 hours per sample</strong></td>
            <td>Constructs problems from literature and verified propositions, then co-evolves rubrics; data quality depends on knowledge integration and expert review</td>
          </tr>
          <tr>
            <td><strong>SaaS-Bench</strong></td>
            <td>2026-09-28: <strong>23 deployable SaaS systems and 106 professional workflow tasks</strong></td>
            <td>Constructs evaluation environments with persistent state, dependencies across applications, environment resets, and checks on final outcomes</td>
          </tr>
          <tr>
            <td><strong>PaperBenchX</strong></td>
            <td>Blog index dated 2026-10-08: <strong>93 paper-reproduction tasks, 12 research fields, and 3,168 scorable items</strong></td>
            <td>Requires domain-specific software configuration, completion of reference workflows, and verification through clean-environment replay and evidence-based scoring</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: <a href="https://unipat.ai/blog/UniScientist">UniScientist</a>, <a href="https://unipat.ai/blog/SaaS-Bench">SaaS-Bench</a>, <a href="https://unipat.ai/blog/PaperBenchX">PaperBenchX</a>, and the <a href="https://unipat.ai/blog">release-date index</a>. These are self-reported metrics for public research projects. This article has not independently rerun the reported model evaluations. Nor did this review obtain verifiable full-year revenue or a headquarters address on the company’s website, so neither financial scale nor a headquarters location is inferred from its university collaborations.</p>

    <p>This case shifts the value of a data company beyond the volume of text produced toward three questions: <strong>is the task worth learning, can successful performance be judged reliably, and does the environment allow training and evaluation to be repeated?</strong> A plausible-looking research report does not establish that an experiment was reproduced correctly. An agent clicking the right button does not establish that the final business state is consistent across systems. Encoding these differences in deliverable tasks, tools, trajectories, and scoring rules is itself engineering work.</p>

    <p>An approach such as UniPat’s therefore has a closer counterpart in research-validation tools and engineering services within the Apple chain, combined with the reusability of data assets. Customers need to verify whether those assets produce reliable improvements on new tasks, rather than merely whether the supplier’s public leaderboard results look impressive.</p>

    <h3 id="data-financials">Data production also has financial statements: growth, profits, and customer dependence</h3>

    <p>Private-company disclosures are insufficient to establish reliable profit margins. Listed data service providers offer a comparison of economic structures, provided they are not presented as new startups. The following table uses <strong>USD million (M)</strong>, rather than the billions used in most other tables in this article.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Calendar 2025 consolidated metric</th>
            <th style="text-align: right">Innodata</th>
            <th style="text-align: right">Appen</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Revenue</td>
            <td style="text-align: right"><strong>251.663M</strong></td>
            <td style="text-align: right"><strong>230.774M</strong></td>
          </tr>
          <tr>
            <td>Year-over-year revenue growth, calculated</td>
            <td style="text-align: right"><strong>47.64%</strong></td>
            <td style="text-align: right"><strong>−1.50%</strong></td>
          </tr>
          <tr>
            <td>GAAP / statutory net income attributable to parent shareholders</td>
            <td style="text-align: right"><strong>32.181M</strong></td>
            <td style="text-align: right"><strong>−21.818M</strong></td>
          </tr>
          <tr>
            <td>Net margin, calculated</td>
            <td style="text-align: right"><strong>12.79%</strong></td>
            <td style="text-align: right"><strong>−9.45%</strong></td>
          </tr>
          <tr>
            <td>Operating cash flow</td>
            <td style="text-align: right"><strong>46.752M</strong></td>
            <td style="text-align: right"><strong>22.966M</strong></td>
          </tr>
          <tr>
            <td>Customer concentration, with different numerators</td>
            <td style="text-align: right">Largest unnamed customer: <strong>58%</strong></td>
            <td style="text-align: right">Top five customers: <strong>74.3%</strong></td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: <a href="https://www.sec.gov/Archives/edgar/data/903651/000110465926020655/inod-20251231x10k.htm">Innodata’s 2025 10-K</a> and <a href="https://cdn.prod.website-files.com/656a605609413dfd3feb9d34/699fbc072974a573b2b7a48c_Appen%20Annual%20Report%202025.pdf">Appen’s 2025 annual report</a>. Appen is listed in Australia, but these financial statements are denominated in U.S. dollars. Its highlighted <strong>4.5%</strong> revenue growth excludes the effect of the previous year’s Google contract and does not replace the unadjusted consolidated growth rate in the table. Appen disclosed that GenAI-related work accounted for <strong>33%</strong> of annual revenue; all data services revenue should not be classified as LLM revenue.</p>

    <p>The companies’ costs illustrate the importance of human labor and engineering delivery. Innodata’s direct operating costs increased by approximately <strong>USD 48.8M</strong> in 2025, including a <strong>USD 42.5M</strong> increase in labor-related costs; its contracts include both time-and-materials and fixed-price arrangements. In Appen’s expense table, crowd services and project employee costs totaled <strong>USD 137.773M</strong>, approximately <strong>59.7%</strong> of revenue. The companies classify costs differently, so their reported “gross margins” are not forced into a single ranking here.</p>

    <p>Dependence on major customers also resembles parts of the Apple chain. On January 22, 2024, Appen announced that Google’s relevant global services contract projects would end that March. The announcement put FY2023 Google revenue at <strong>USD 82.8M</strong>, a preliminary, unaudited figure at the time. <a href="https://yourir.info/resources/396cdf0a32f7cb50/announcements/apx.asx/2A1500606/APX_Customer_update_and_FY23_preliminary_results.pdf">Original announcement</a> This does not prove that every customer will bring work in-house or demand lower prices. It does show that a large delivery organization does not automatically secure stable orders.</p>

    <p>Customer payment timing affects cash flow too. Innodata’s operating cash flow reached <strong>USD 164.441M</strong> in the first half of 2026, compared with net income of <strong>USD 29.310M</strong>. Cash flow included increases in customer prepayments and payables. <a href="https://www.sec.gov/Archives/edgar/data/903651/000110465926092021/inod-20260630x10q.htm">2026 interim financial statements</a> Receiving cash before future delivery obligations are fulfilled is different from having already earned an equivalent amount of profit.</p>

    <h3 id="data-rights-platforms">Content licensing and enterprise data software: two other economic structures</h3>

    <p>Reddit, Shutterstock, and Getty Images illustrate the content and licensing chain, but not all of their non-advertising revenue should be labeled “AI training data.” Reddit’s Other revenue was <strong>USD 140.026M</strong> in 2025, or <strong>6.36%</strong> of total revenue, including content licensing and products sold to users. Shutterstock’s Data, Distribution and Services revenue was <strong>USD 203.264M</strong>, or <strong>20.53%</strong> of total revenue, but also included distribution, advertising, and production services. <a href="https://d18rn0p25nwr6d.cloudfront.net/CIK-0001713445/a8e1ae31-e61c-4a09-97be-33e01420abb2.pdf">Reddit annual report</a>, <a href="https://investor.shutterstock.com/static-files/43552bbe-6a3f-4661-a1fa-2e3ae493a012">Shutterstock annual report</a></p>

    <p>The rights these companies sell differ as well. Continuous access and updates, training use, display within answers, and licenses to use generated outputs are not the same contract. For example, Getty’s 2026 OpenAI announcement concerned a <strong>display agreement</strong> for search and discovery within ChatGPT. It does not establish a license to train on Getty’s entire library. <a href="https://investors.gettyimages.com/news-releases/news-release-details/getty-images-announces-display-partnership-openai">Getty’s official announcement</a> Data can be licensed repeatedly, but permitted uses, exclusivity, and the chain of author permissions affect its value.</p>

    <p>Snowflake and Databricks primarily connect models with governed enterprise data and workflows. Snowflake generated <strong>USD 4.684B</strong> in FY2026 revenue, for the year ended January 31, 2026, with a calculated consolidated GAAP gross margin of <strong>67.2%</strong>, but an operating loss of <strong>USD 1.435B</strong>. Its period-end net revenue retention rate of <strong>125%</strong> measured expansion in product revenue from existing customers, not user retention. <a href="https://www.sec.gov/Archives/edgar/data/1640147/000164014726000008/snow-20260131.htm">Snowflake 10-K</a> This illustrates a different software-platform cost structure and reminds us that high gross margins do not guarantee operating profitability.</p>

    <p>On August 13, 2026, Databricks reported a revenue run rate above <strong>USD 7B</strong>. Its product priorities included the Lakebase database for agents, Genie, and governance and cost control across multiple models. <a href="https://www.databricks.com/company/newsroom/press-releases/databricks-grows-80-yoy-surpasses-7b-revenue-run-rate-scales">Databricks update</a> This is not an audited figure directly comparable with Snowflake’s recognized full-year revenue. Neither company automatically owns the copyright to customer data merely by managing it, and neither company’s entire platform revenue should be treated as sales of training corpora to model developers.</p>

    <h3 id="data-bargaining">Bargaining power in data: acceptance, reuse, and customer trust</h3>

    <p>Data suppliers and Apple suppliers both work with specifications, pilots, quality control, acceptance testing, and major customers. But data is not consumed like one component installed in one phone. A nonexclusive data asset can be licensed repeatedly, and a task environment can be run repeatedly; custom expert labor and environment maintenance also create continuing costs. Growth in future inference tokens therefore cannot be translated mechanically into growth in data purchases.</p>

    <p>More useful measures are <strong>the full delivery cost per accepted task, first-pass acceptance rates, rework and update costs, asset reuse rights, repeat orders, and cash collections</strong>. Synthetic models can lower generation costs while shifting bottlenecks toward task selection, factual verification, graders, and control of evaluation contamination. UniPat’s joint human-model rubric development, Zhineng Zhishi’s executable tests, and Surge’s trial training all address such problems, rather than simply increasing the number of records.</p>

    <p>Customer trust is also a commercial asset. Reuters reported in 2025 that Google and OpenAI were then planning to reduce or move away from their relationships with Scale, discussing neutrality concerns following a strategic investment. Scale’s official response pledged not to give Meta access to its internal systems or customers’ confidential information. <a href="https://www.reuters.com/business/scale-ais-bigger-rival-surge-ai-seeks-up-1-billion-capital-raise-sources-say-2025-07-01/">Reuters</a>, <a href="https://scale.com/blog/customer-trust-scale-meta-deal">Scale’s response</a> This is evidence about supplier relationships and trust, not evidence of a data leak, nor proof that every relationship subsequently ended.</p>

    <p>Seen from this layer, the LLM industry’s critical inputs include both expensive chips and professional judgment and validation conditions that are difficult to produce reliably. How much profit a data company can retain depends on whether it can turn one-off deliveries into reusable capabilities while continuing to meet increasingly demanding customer acceptance standards.</p>

    <h2 id="overlap">Customer concentration, cash flow, and the overlap between the chains</h2>

    <p>Many end uses for AI do not mean every supplier has a diversified customer base. CoreWeave’s 67% share from its largest customer is one example; disclosures from electronics manufacturers and optical communications companies show similar patterns.</p>

    <p>The following comparison uses four companies within global manufacturing networks, all on a <strong>calendar 2025 consolidated basis</strong>. Revenue is in <strong>RMB 100 million</strong>, and is not added to dollar-denominated amounts.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Company / principal position</th>
            <th style="text-align: right">Revenue (RMB 100 million)</th>
            <th style="text-align: right">Calculated consolidated gross margin</th>
            <th style="text-align: right">Calculated net margin attributable to parent shareholders</th>
            <th style="text-align: right">Top five customers’ share of revenue</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Luxshare / precision manufacturing and connectivity</td>
            <td style="text-align: right">3,323.44</td>
            <td style="text-align: right">11.91%</td>
            <td style="text-align: right">4.99%</td>
            <td style="text-align: right">65.04%</td>
          </tr>
          <tr>
            <td>Inspur Information / servers and systems</td>
            <td style="text-align: right">1,647.82</td>
            <td style="text-align: right">4.88%</td>
            <td style="text-align: right">1.46%</td>
            <td style="text-align: right">71.29%</td>
          </tr>
          <tr>
            <td>Zhongji Innolight / optical interconnects</td>
            <td style="text-align: right">382.40</td>
            <td style="text-align: right">42.04%</td>
            <td style="text-align: right">28.24%</td>
            <td style="text-align: right">75.98%</td>
          </tr>
          <tr>
            <td>Eoptolink / optical interconnects</td>
            <td style="text-align: right">248.42</td>
            <td style="text-align: right">47.81%</td>
            <td style="text-align: right">38.37%</td>
            <td style="text-align: right">72.34%</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: annual reports from <a href="https://static.cninfo.com.cn/finalpage/2026-04-15/1225104908.PDF">Luxshare</a>, <a href="https://static.cninfo.com.cn/finalpage/2026-04-11/1225096638.PDF">Inspur Information</a>, <a href="https://static.cninfo.com.cn/finalpage/2026-03-31/1225056459.PDF">Zhongji Innolight</a>, and <a href="https://static.cninfo.com.cn/finalpage/2026-04-24/1225172598.PDF">Eoptolink</a>. Net margin attributable to parent shareholders equals net profit attributable to shareholders of the listed company divided by consolidated revenue; it is a different measure from operating margin above. Customer concentration follows each company’s disclosed direct-customer basis. Unnamed customers are not identified as Apple, NVIDIA, or a particular cloud provider.</p>

    <p>The differences are substantial. Participating in AI infrastructure markets can mean supplying relatively profitable optical interconnect components or assembling low-margin systems. At the same time, several AI-related suppliers still have highly concentrated customer bases. A downstream world of many models and applications can still channel procurement through a small number of cloud and equipment buyers.</p>

    <p>Profits also have to turn into cash. In the first half of 2026, Zhongji Innolight reported net profit attributable to parent shareholders of <strong>RMB 13.651 billion</strong>, versus operating cash flow of <strong>RMB 1.800 billion</strong>. <a href="https://static.cninfo.com.cn/finalpage/2026-08-22/1225491753.PDF">Source: interim report</a> Such a gap warrants further examination of receivables, inventory, collection timing, and seasonality. One half-year does not establish that growth is problematic, but rising profits should not automatically be treated as cash collected at the same pace.</p>

    <p>The chains are already overlapping in practice. Foxconn’s cloud and networking products represented approximately <strong>40%</strong> of 2025 revenue, exceeding the <strong>38%</strong> from smart consumer electronics. The category includes servers and communications networks, not AI alone. <a href="https://image.honhai.com/upload/202605/financy_by_year/2025_Annual_Report_20260523_7728.pdf">Source: Foxconn annual report</a> Luxshare’s communications and data center business generated <strong>RMB 24.568 billion</strong> in 2025, or <strong>7.39%</strong> of consolidated revenue. Its segment gross margin of <strong>18.40%</strong> exceeded the consumer electronics segment’s <strong>10.64%</strong>. <a href="https://static.cninfo.com.cn/finalpage/2026-04-15/1225104908.PDF">Source: Luxshare annual report</a> Communications and data center revenue likewise cannot all be labeled LLM revenue.</p>

    <p>Manufacturing capabilities can transfer, but doing so requires new product qualifications, engineering skills, and capital. Connectors, PCBs, optical interconnects, system integration, power delivery, and thermal management enter new order structures in different ways. “Part of the Apple supply chain” and “part of the AI chain” are becoming less useful as mutually exclusive labels.</p>

    <h2 id="updates">What the new data from 2026 changes</h2>

    <p>The annual tables provide a baseline; more recent data tests whether those trends are continuing. The following are published actual results, which must be distinguished from forward guidance.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Company and selected recent period</th>
            <th>New disclosure</th>
            <th>Implication for the industry comparison</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>NVIDIA Q2 FY2027, ended 2026-07-26</td>
            <td>Revenue <strong>USD 96.221B</strong>; Data Center <strong>USD 89.0B</strong>; GAAP gross margin <strong>75.0%</strong></td>
            <td>The computing platform continues to expand in scale and profitability, although all Data Center revenue still cannot be called LLM revenue</td>
          </tr>
          <tr>
            <td>TSMC Q3 2026, announced 2026-10-15</td>
            <td>Quarterly revenue <strong>USD 46.72B</strong>; gross margin <strong>66.4%</strong>; operating margin <strong>58.7%</strong>; HPC revenue share <strong>68%</strong></td>
            <td>Manufacturing also retains strong profitability; HPC remains a broader category than AI accelerators</td>
          </tr>
          <tr>
            <td>Broadcom Q3 FY2026, ended 2026-08-02</td>
            <td>Company revenue <strong>USD 29.591B</strong>; explicitly disclosed AI semiconductor revenue <strong>USD 16.7B</strong>, approximately <strong>56.4%</strong> of total revenue</td>
            <td>Custom accelerators and networking provide a revenue path distinct from general-purpose GPUs</td>
          </tr>
          <tr>
            <td>Micron FY2026, ended 2026-09-03</td>
            <td>Full-year revenue <strong>USD 133.188B</strong>; calculated GAAP gross margin <strong>80.7%</strong></td>
            <td>Memory pricing and supply-demand conditions have changed dramatically; the 2025 profit structure is not a fixed constant</td>
          </tr>
          <tr>
            <td>Apple Q3 FY2026, ended 2026-06-27</td>
            <td>Consolidated gross margin <strong>50.1%</strong>, including approximately <strong>2 percentage points</strong> from tariff refunds</td>
            <td>The improvement includes a nonrecurring effect and should not be extrapolated directly as a normal gross margin</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Sources: <a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-second-quarter-fiscal-2027">NVIDIA</a>, <a href="https://investor.tsmc.com/english/quarterly-results/2026/q3">TSMC</a>, <a href="https://investors.broadcom.com/news-releases/news-release-details/broadcom-inc-announces-third-quarter-fiscal-year-2026-financial">Broadcom</a>, <a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-26-Earnings-Deck.pdf">Micron</a>, and <a href="https://www.apple.com/newsroom/2026/07/apple-reports-third-quarter-results/">Apple</a>. Micron’s 80.7% is calculated from GAAP gross profit of USD 107.504B divided by revenue of USD 133.188B. Its materials also provide non-GAAP figures, which are not mixed into this calculation.</p>

    <p>Micron’s September 2026 disclosure also stated that agreements covered most of its 2027 HBM bit supply. Initial output from its Singapore HBM packaging facility was expected in early 2027, while other new facilities would take longer. <a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-FY26-Prepared-Remarks.pdf">Source: Micron management remarks</a> Agreements are not recognized revenue, and a factory’s first production is not the same as immediately reaching mature capacity. Near-term supply constraints can generate profits, while capacity expansion, product iterations, and price changes can alter the subsequent distribution of value.</p>

    <p>Price increases for conventional DRAM and NAND also contributed to Micron’s growth during the period; the improvement in profitability cannot all be attributed to HBM. The profits in these tables reflect both company structures and the supply-and-demand conditions of the period, and do not justify permanent “high-margin” or “low-margin” labels for entire stages of the industry.</p>

    <h2 id="constraints">Power, geography, and trade</h2>

    <p>Apple’s chain depends on manufacturing clusters across national borders. Its FY2025 annual report still lists mainland China, India, Japan, South Korea, Taiwan, and Vietnam as the principal locations of manufacturing partners, and warns that changing its supply chain can be expensive, time-consuming, and disruptive. <a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">Apple</a> Relocation involves supplier networks, yields, equipment, and engineering organizations. A company’s headquarters country does not establish where it produces.</p>

    <p>The AI chain adds a more prominent constraint: computing capacity must secure electricity, substation equipment, networking, and cooling at a specific location.</p>

    <p>The IEA’s <strong>2025 Energy and AI report</strong> estimates that all data centers globally consumed approximately <strong>415 TWh</strong> of electricity in <strong>2024</strong>, around <strong>1.5%</strong> of global electricity consumption. Its <strong>base-case projection for 2030</strong> is approximately <strong>945 TWh</strong>. <a href="https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai">Source: IEA</a> Both figures include non-AI data centers, and the latter is a forecast, not consumption that has already occurred.</p>

    <p>Local delivery schedules matter more than the global total alone. The same report notes that building new transmission lines in advanced economies often takes <strong>4–8 years</strong>, while a data center may become operational in <strong>2–3 years</strong>. <a href="https://iea.blob.core.windows.net/assets/de9dea13-b07d-42c5-a398-d1b3ae17d866/EnergyandAI.pdf">Full report, pp. 15, 50, and 95</a> There is a long chain of conversions between a data center order, a completed building, power-ready capacity, and actual computing revenue.</p>

    <p>The product positions of Vertiv, Schneider Electric, Eaton, Delta, and others therefore deserve separate analysis. Liquid cooling cannot replace a grid connection, and an equipment supplier’s backlog is not realized revenue. Rising data center power consumption also cannot be translated directly into the revenue growth of a particular cooling company.</p>

    <p>Trade rules change the addressable market as well. A BIS rule published on <strong>January 15, 2026</strong>, moved certain qualifying advanced computing exports to China and Macau to case-by-case license review, covering the H200 and products meeting the relevant parameters and conditions. This did not eliminate controls generally or guarantee license approval. <a href="https://www.federalregister.gov/documents/2026/01/15/2026-00789/revision-to-license-review-policy-for-advanced-computing-commodities">Source: Federal Register</a> Assessing global companies requires attention to technical specifications, destinations, licensing conditions, and subsequent rule changes.</p>

    <p>Capital relationships also merit tracking. OpenAI’s new investment announcement on February 27, 2026, included NVIDIA and Amazon, while Anthropic’s financing and compute partnerships also involve several cloud and hardware companies. <a href="https://openai.com/index/scaling-ai-for-everyone/">OpenAI</a>, <a href="https://www.anthropic.com/news/series-h">Anthropic</a> A company can be an investor, supplier, and distribution partner at the same time. Financing supports expansion, but funding amounts, procurement commitments, and actual payments from final customers are different quantities. Repeatedly adding them together does not establish the size of demand.</p>

    <h2 id="outlook">What to watch next</h2>

    <p>The most useful lesson from Apple’s supply chain is how manufacturing at scale, supplier qualification, and ecosystem control combine to create commercial advantages. The LLM chain requires additional scrutiny: whether sustained, productive demand can absorb heavy compute investment, and whether profits from bottlenecks can persist as capacity expands and alternative technologies emerge.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Stage</th>
            <th>More useful indicators to track</th>
            <th>Potentially misleading substitutes</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Chips and manufacturing</td>
            <td>Deliverable capacity, yields, customer qualification, software migration costs, and clearly defined AI business exposure</td>
            <td>Treating all HPC, semiconductor, or Data Center revenue as AI</td>
          </tr>
          <tr>
            <td>HBM and interconnects</td>
            <td>Qualified supply, product mix, pricing, capacity ramp-up, and customer procurement arrangements</td>
            <td>Treating long-term agreements as current revenue or temporary high margins as permanent</td>
          </tr>
          <tr>
            <td>Servers and racks</td>
            <td>The materials component of revenue, working capital, customer concentration, and system delivery capabilities</td>
            <td>Looking only at revenue scale or securing a leading customer</td>
          </tr>
          <tr>
            <td>Cloud and data centers</td>
            <td>Cash investment, leases and debt, utilization, energized capacity, and contract collections</td>
            <td>Looking only at GPU counts, announced investment, or backlog</td>
          </tr>
          <tr>
            <td>Models and applications</td>
            <td>Paying-customer retention, actual task success rates, total task costs, gross profit, and customer acquisition</td>
            <td>Looking only at token usage, free users, or a single annualized revenue snapshot</td>
          </tr>
          <tr>
            <td>Data and enterprise platforms</td>
            <td>Delivery cost per accepted task, acceptance and rework, asset reuse rights, licensing and permissions, repeat orders, and depth of evaluation and workflow integration</td>
            <td>Looking only at record counts, registered expert counts, or customer reach cited in partnership announcements</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>I see the LLM industry as a division of value that is still taking shape. GPU platforms have established strong positions. Advanced manufacturing and memory retain physical constraints. Cloud providers control capital and distribution, while models and applications are still competing for the final customer. These companies depend on one another while using in-house development, alternative sourcing, and product integration to compete for adjacent roles.</p>

    <p>Whether one company will emerge with Apple’s combination of control over the device, experience, and ecosystem economics remains an open question. A more useful question is: <strong>for each increment of demand, who solves a problem that cannot yet be bypassed, who carries the capital and inventory, and who ultimately retains the customer and the cash flow?</strong></p>

    <p>Following those questions makes the similarities, differences, and existing intersections between the two chains clearer.</p>

    <h2 id="references">Sources and calculation notes</h2>

    <p>This article prioritizes company annual reports, quarterly disclosures, official supplier materials, and original institutional research. For some private companies, it uses reporting and interviews from Reuters and TechCrunch, retaining the reporting dates and the fact that the figures have not been publicly audited. Financial figures are disclosures by the relevant reporting entities; model-company snapshots, product pages, and management plans retain their status as company-reported information. The sources below also provide a way to check reporting periods, financial definitions, and business boundaries.</p>

    <div class="drift-references">

      <ol>
        <li><strong>Apple:</strong> <a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">FY2025 Form 10-K</a>; <a href="https://www.apple.com/newsroom/2026/07/apple-reports-third-quarter-results/">official FY2026 Q3 results</a>; <a href="https://s203.q4cdn.com/367071867/files/doc_downloads/2024/04/Apple-Supplier-List.pdf">official Supplier List, covering FY2023</a>. Product and service revenue, gross profit, cash flow, manufacturing locations, and net revenue recognition.</li>
        <li><strong>IDC:</strong> <a href="https://www.idc.com/resource-center/press-releases/q425mobilephonetop5/">Preliminary global smartphone shipment estimates for 2025</a>, published January 13, 2026. Calendar-year figures, not Apple fiscal-year unit sales.</li>
        <li><strong>Foxconn:</strong> <a href="https://image.honhai.com/upload/202605/financy_by_quarterly/2025Q4_consolidared_report(e)_20260515_5808.pdf">2025 consolidated financial statements</a>; <a href="https://image.honhai.com/upload/202605/financy_by_year/2025_Annual_Report_20260523_7728.pdf">2025 Annual Report</a>. The main financial statements use thousands of New Taiwan dollars; product-line tables separately use millions.</li>
        <li><strong>NVIDIA:</strong> <a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-fourth-quarter-and-fiscal-2026">FY2026 results</a>; <a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-second-quarter-fiscal-2027">Q2 FY2027 results</a>. Consolidated and Data Center figures are used separately.</li>
        <li><strong>TSMC:</strong> <a href="https://investor.tsmc.com/english/quarterly-results/2025/q4">4Q25/2025 financial statements, presentation, and earnings call</a>; <a href="https://investor.tsmc.com/english/quarterly-results/2026/q3">3Q26 financial statements, presentation, and earnings call</a>. HPC and AI accelerator categories differ.</li>
        <li><strong>Broadcom:</strong> <a href="https://investors.broadcom.com/news-releases/news-release-details/broadcom-inc-announces-fourth-quarter-and-fiscal-year-2025">FY2025 results</a>; <a href="https://investors.broadcom.com/news-releases/news-release-details/broadcom-inc-announces-third-quarter-fiscal-year-2026-financial">Q3 FY2026 results</a>. Software, semiconductors, and AI semiconductors are distinguished.</li>
        <li><strong>AMD:</strong> <a href="https://data.sec.gov/api/xbrl/companyfacts/CIK0000002488.json">SEC financial data</a>; <a href="https://ir.amd.com/news-events/press-releases/detail/1295/amd-reports-second-quarter-2026-financial-results">official Q2 2026 results</a>. Annual figures use GAAP amounts; the data center business includes both CPUs and GPUs.</li>
        <li><strong>Memory:</strong> <a href="https://news.skhynix.com/en/sk-hynix-announces-fy25-financial-results/">SK hynix FY2025 results</a>; <a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-26-Earnings-Deck.pdf">Micron FY2026 Q4 earnings deck, including FY2025 comparative figures</a>; <a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-FY26-Prepared-Remarks.pdf">Micron prepared remarks, September 30, 2026</a>.</li>
        <li><strong>Networking and power:</strong> <a href="https://data.sec.gov/api/xbrl/companyfacts/CIK0001596532.json">Arista SEC financial data</a>; <a href="https://www.arista.com/en/company/news/press-release/24401-pr-20260804">Arista Q2 2026 results</a>; <a href="https://s205.q4cdn.com/554782763/files/doc_financials/2025/q4/Vertiv_Fourth-Quarter-2025-Earnings-Release.pdf">Vertiv FY2025 results</a>. Consolidated revenue is not AI-only revenue.</li>
        <li><strong>Diversified cloud providers and capital spending:</strong> <a href="https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/msft-20260630.htm">Microsoft FY2026 10-K, including FY2025 comparatives</a>; <a href="https://www.sec.gov/Archives/edgar/data/1652044/000165204426000018/goog-20251231.htm">Alphabet 2025 10-K</a>; <a href="https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm">Amazon 2025 10-K</a>; <a href="https://www.sec.gov/Archives/edgar/data/1326801/000162828026003942/meta-20251231.htm">Meta 2025 10-K</a>.</li>
        <li><strong>Specialist compute operations:</strong> <a href="https://www.sec.gov/Archives/edgar/data/1769628/000176962826000104/crwv-20251231.htm">CoreWeave 2025 10-K</a>; <a href="https://www.oracle.com/cloud/compute/gpu/">Oracle GPU infrastructure</a>. CoreWeave’s cash equipment purchases include capitalized internal-use software.</li>
        <li><strong>Model companies:</strong> <a href="https://openai.com/index/a-business-that-scales-with-the-value-of-intelligence/">OpenAI CFO, January 18, 2026</a>; <a href="https://openai.com/index/scaling-ai-for-everyone/">OpenAI, February 27, 2026</a>; <a href="https://www.anthropic.com/news/series-h">Anthropic, May 28, 2026</a>. Annualized revenue, users, and financing are different measures.</li>
        <li><strong>Data and enterprise platforms:</strong> <a href="https://scale.com/rlhf">Scale AI Data Engine</a>; <a href="https://redditinc.com/news/reddit-and-google-expand-partnership">Reddit–Google announcement, February 22, 2024</a>; <a href="https://www.databricks.com/company/newsroom/press-releases/databricks-and-openai-launch-groundbreaking-partnership-bring">Databricks–OpenAI announcement, September 25, 2025</a>.</li>
        <li><strong>Manufacturing and optical interconnect examples:</strong> <a href="https://static.cninfo.com.cn/finalpage/2026-04-15/1225104908.PDF">Luxshare 2025 annual report</a>; <a href="https://static.cninfo.com.cn/finalpage/2026-04-11/1225096638.PDF">Inspur Information 2025 annual report</a>; <a href="https://static.cninfo.com.cn/finalpage/2026-03-31/1225056459.PDF">Zhongji Innolight 2025 annual report</a> and <a href="https://static.cninfo.com.cn/finalpage/2026-08-22/1225491753.PDF">2026 interim report</a>; <a href="https://static.cninfo.com.cn/finalpage/2026-04-24/1225172598.PDF">Eoptolink 2025 annual report</a>.</li>
        <li><strong>Power and policy:</strong> <a href="https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai">IEA Energy and AI, 2025 edition</a> and <a href="https://iea.blob.core.windows.net/assets/de9dea13-b07d-42c5-a398-d1b3ae17d866/EnergyandAI.pdf">full PDF</a>; <a href="https://www.federalregister.gov/documents/2026/01/15/2026-00789/revision-to-license-review-policy-for-advanced-computing-commodities">BIS license review rule, January 15, 2026</a>. Historical estimates, scenario projections, and regulatory events are explained separately.</li>
        <li><strong>Data companies section:</strong> Product, legal entity, location, and financial sources follow each company table. <a href="https://unipat.ai/blog">UniPat’s technical releases</a>, <a href="https://www.sec.gov/Archives/edgar/data/903651/000110465926020655/inod-20251231x10k.htm">Innodata’s annual report</a>, <a href="https://cdn.prod.website-files.com/656a605609413dfd3feb9d34/699fbc072974a573b2b7a48c_Appen%20Annual%20Report%202025.pdf">Appen’s annual report</a>, and <a href="https://www.sec.gov/Archives/edgar/data/1640147/000164014726000008/snow-20260131.htm">Snowflake’s annual report</a> provide further verification. Media-reported private-company revenue, annualized figures, registered experts, and financing are presented separately.</li>
      </ol>

    </div>

    <p>Gross margin is calculated as (revenue − cost of sales) ÷ revenue; operating margin as operating income ÷ revenue. Net margin attributable to parent shareholders is used only for the manufacturing examples explicitly labeled as such. Percentages are calculated from original amounts and then rounded. Amounts in different currencies are neither converted nor added together, and companies’ revenues may reflect repeated transactions involving the same input. This article does not infer the identities of unnamed customers or use undisclosed contract prices to fill gaps in “AI revenue.”</p>

    <p class="drift-meta"><a href="/blog/">← All posts</a> · <a href="#structure">Back to the article</a></p>

  </article>
</main>]]></content><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><summary type="html"><![CDATA[Comparing the Apple and LLM value chains through financial filings, official disclosures, and interviews: from chips, manufacturing, and cloud to data startups in Silicon Valley and Beijing and UniPat's approach to tasks and evaluation, with a focus on profits, capital investment, and customer dependence.]]></summary></entry><entry xml:lang="zh-CN"><title type="html">大模型产业链会成为下一条果链吗？利润、资本与全球分工的比较</title><link href="https://zefancai.com/blog/ai-vs-apple-chain/" rel="alternate" type="text/html" title="大模型产业链会成为下一条果链吗？利润、资本与全球分工的比较" /><published>2026-10-18T00:00:00-07:00</published><updated>2026-10-18T00:00:00-07:00</updated><id>https://zefancai.com/blog/ai-vs-apple-chain</id><content type="html" xml:base="https://zefancai.com/blog/ai-vs-apple-chain/"><![CDATA[<link rel="stylesheet" href="/assets/css/ar-drift.css" />

<main id="main" class="drift-article" lang="zh-CN">
  <article class="drift-prose">

    <p class="drift-kicker"><a href="/zh/blog/">中文博客</a> / 产业研究</p>

    <h1 id="section">大模型产业链会成为下一条果链吗？</h1>

    <p class="drift-deck">从一台设备的全球制造，到一次智能任务背后的芯片、能源、数据与软件。</p>
    <p class="drift-meta">Zefan Cai · <time datetime="2026-10-18">2026 年 10 月 18 日</time> · 全球产业与公司比较</p>

    <p>苹果产业链提供了一个很有吸引力的参照：一家掌握终端、操作系统和用户关系的公司，把复杂的全球制造网络组织起来；供应商围绕新产品投入工艺、设备和产能，从技术创新中获得长期增长。今天，从 GPU、HBM 与液冷，到专家数据、Agent 训练环境和评测，新的供应网络很容易让人联想到当年的果链。</p>

    <p>这个类比有用，但只用它来寻找“AI 时代的苹果”或“下一家手机代工厂”，会漏掉最重要的变化。</p>

    <p><strong>大模型产业链正在形成多个拥有议价权的中心：算力平台、先进制造、云与分发、模型，以及掌握业务流程的应用。它比果链多了一层持续发生的推理成本，也多了一组电力、融资和利用率约束。</strong> 供应商能不能赚钱，取决于它解决了什么瓶颈、客户能否替换它，以及增长消耗了多少资本。</p>

    <p>这篇文章从全球产业与公司的角度比较两条链。中国公司的案例放在它们所在的全球环节中讨论。</p>

    <div class="drift-callout">
      <p><strong>数据口径。</strong> 资料核验截至 <strong>2026-10-18</strong>。年度比较以各公司 <strong>2025 年度</strong>报告为主，NVIDIA 使用截至 2026-01-25 的 FY2026；不同财年的截止日期明确列示，2026 年新披露单独讨论。除另注外，美元金额单位均为<strong>十亿美元</strong>，即 1 billion = 10 亿美元。集团收入、分部收入、公司自报年化收入、行业估计和未来预测分别标注；上下游收入不相加计算产业规模。</p>
    </div>

    <nav class="drift-toc" aria-label="文章目录">
      <ul>
        <li><a href="#structure">先看两条链的结构</a></li>
        <li><a href="#companies">全球公司地图：各环节做什么</a></li>
        <li><a href="#apple">果链的利润，怎样留在苹果</a></li>
        <li><a href="#profits">AI 链的利润，为什么分布在多个位置</a></li>
        <li><a href="#capital">最大的变化：谁在承担资本开支</a></li>
        <li><a href="#models-data">云、模型与应用层，谁能掌握客户</a></li>
        <li><a href="#data-companies">数据公司详解：硅谷、北京与 UniPat</a></li>
        <li><a href="#overlap">客户集中、现金流与两条链的交汇</a></li>
        <li><a href="#updates">2026 年的新数据改变了什么</a></li>
        <li><a href="#constraints">电力、地理与贸易环境</a></li>
        <li><a href="#outlook">接下来应该观察什么</a></li>
        <li><a href="#references">资料与复算说明</a></li>
      </ul>
    </nav>

    <h2 id="structure">先看两条链的结构</h2>

    <p>果链的需求锚相对清楚：设备出货、换机、活跃设备，以及这些设备带来的服务消费。IDC 在 2026 年 1 月公布的<strong>初步估计</strong>显示，2025 自然年全球智能手机出货约 <strong>12.603 亿部</strong>，同比增长 <strong>1.9%</strong>；Apple 出货 <strong>2.478 亿部</strong>，占 <strong>19.7%</strong>。这是品牌设备出货估计，不是苹果自己披露的销量，也不是消费者激活量。<a href="https://www.idc.com/resource-center/press-releases/q425mobilephonetop5/">来源：IDC</a></p>

    <p>大模型链的最终需求则可能是一份订阅、一百万个 token、一段自动完成的工作，或者广告转化率和软件开发效率的改善。终端需求要经过模型调用、云资源采购和数据中心建设，才转化为芯片与设备订单。不同环节看到的增长，常常不在同一时间发生。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>比较维度</th>
            <th>苹果产业链</th>
            <th>大模型产业链</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>主要组织者</td>
            <td>Apple 决定主要产品架构、系统体验与供应商认证</td>
            <td>GPU 平台、云厂商、模型公司、应用平台共同组织需求</td>
          </tr>
          <tr>
            <td>收入的基本单位</td>
            <td>设备、配件、服务与渠道佣金</td>
            <td>芯片/设备、算力时长、token、席位、业务结果</td>
          </tr>
          <tr>
            <td>需求与产能的节奏</td>
            <td>新机发布、备货、换机周期</td>
            <td>训练集群建设、推理用量、企业部署与数据中心投产</td>
          </tr>
          <tr>
            <td>客户拿到产品之后</td>
            <td>硬件制造成本主要已发生，服务另有持续成本</td>
            <td>每次云端推理仍占用算力、内存、网络和电力</td>
          </tr>
          <tr>
            <td>核心替换成本</td>
            <td>iOS 生态、用户数据、工艺认证、良率与交付</td>
            <td>软件生态、模型行为、数据权限、集群工程、业务流程</td>
          </tr>
          <tr>
            <td>需要关注的库存</td>
            <td>零部件、在制品、成品与渠道库存</td>
            <td>设备和零部件库存，以及尚未被充分使用的计算容量</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>表中的区别是侧重点。苹果也运营云和 AI 基础设施，AI 厂商也依赖大型采购方；TSMC 等供应商在两条链中都能保持相当强的地位。产业比较要沿着真实的交易和控制关系展开。</p>

    <h2 id="companies">全球公司地图：各环节做什么</h2>

    <p>下面列的是代表性位置。同一家公司可能跨越数层；列在同一行，也不意味着公司之间可以直接替换。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>环节</th>
            <th>苹果链中的位置与代表公司</th>
            <th>大模型链中的位置与代表公司</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>产品与系统设计</td>
            <td>Apple：设备架构、自研 A/M 系列芯片、iOS/macOS、终端体验</td>
            <td>NVIDIA：GPU 与系统平台；AMD：GPU/CPU；Google、AWS 等云厂：定制加速器；Broadcom：定制芯片与网络协作</td>
          </tr>
          <tr>
            <td>晶圆制造与封装</td>
            <td>TSMC 等晶圆制造商，配套封装测试体系</td>
            <td>TSMC 先进制程与封装；ASE、Amkor 等封测体系。算力芯片、HBM、基板与封装需要共同交付</td>
          </tr>
          <tr>
            <td>半导体设备与设计工具</td>
            <td>两条链共同依赖的更上游：ASML、Applied Materials、Lam Research、KLA；Synopsys、Cadence</td>
            <td>同一组制造与设计基础设施服务于多个终端市场，不能把它们的收入全部归入 AI 或 Apple</td>
          </tr>
          <tr>
            <td>内存与存储</td>
            <td>Samsung、SK hynix、Micron：移动 DRAM、NAND 等</td>
            <td>SK hynix、Micron、Samsung：HBM、服务器 DRAM、NAND；产品组合和周期影响盈利</td>
          </tr>
          <tr>
            <td>显示、影像、射频与结构件</td>
            <td>Samsung Display、LG Display、Sony、Largan、Murata、Broadcom、Qualcomm、Corning、蓝思等，分别处于显示、传感、镜头、连接和材料环节</td>
            <td>并非每类手机组件都有 AI 服务器对应物；新增需求更多集中在高速互连、供电、散热与高密度系统集成</td>
          </tr>
          <tr>
            <td>整机与系统集成</td>
            <td>鸿海、立讯、和硕、Tata Electronics 等制造网络</td>
            <td>鸿海、Quanta/QCT、Wiwynn、Dell、HPE、Supermicro、浪潮：服务器、机柜、系统集成与交付</td>
          </tr>
          <tr>
            <td>网络与光互连</td>
            <td>终端连接芯片、天线与无线通信</td>
            <td>NVIDIA、Broadcom、Arista、Cisco；Coherent、Lumentum、中际旭创、新易盛等光通信公司，角色从器件到模块不同</td>
          </tr>
          <tr>
            <td>电力与热管理</td>
            <td>制造设施用能、设备电池及热设计</td>
            <td>Vertiv、Schneider Electric、Eaton、Delta、英维克，以及电网与能源供应商；重点是供电、配电、UPS、冷却和工程交付</td>
          </tr>
          <tr>
            <td>云与计算运营</td>
            <td>Apple 自有与外部服务基础设施</td>
            <td>AWS、Microsoft Azure、Google Cloud、Oracle OCI、CoreWeave；Meta 也是大型算力买方，但其集团变现主要不是对外出售通用云</td>
          </tr>
          <tr>
            <td>模型与应用</td>
            <td>Apple 的系统、内容与开发者生态，把设备连接到服务消费</td>
            <td>OpenAI、Anthropic、Google、Meta、Mistral、Cohere、DeepSeek、Qwen 等模型生态；Microsoft/GitHub、Salesforce、ServiceNow、Adobe、Cursor 等应用与工作入口</td>
          </tr>
          <tr>
            <td>数据、任务与评测</td>
            <td>内容/IP、工程规格与验收体系是功能上的参照，并非一一对应组件</td>
            <td>Scale、Surge、Mercor、UniPat、智能知识、TalentsAI 等数据与任务生产商；Reddit 等内容授权商；Snowflake、Databricks 等企业数据平台，详见<a href="#data-companies">数据公司专章</a></td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>这些位置可结合 <a href="https://s203.q4cdn.com/367071867/files/doc_downloads/2024/04/Apple-Supplier-List.pdf">Apple 官方供应商名单</a>、下文各公司财报以及 <a href="https://www.oracle.com/cloud/compute/gpu/">OCI 的 GPU 基础设施说明</a>阅读。<strong>供应商名单有自己的年份。</strong> 本次从 Apple 2026 官方报告入口取得的名单，封面仍注明覆盖 <strong>FY2023 材料、制造和组装直接采购支出的 98%</strong>。它能证明该覆盖期内的供应关系，不能证明 2026 年每一款产品的份额；设备、EDA、云和应用等行也包含间接或生态参与者，不能一律视为苹果的直接供货商。</p>

    <p>这张地图揭示了类比的边界：果链以一家终端公司的产品体系为中心；大模型链是一张跨产品、跨计算平台、跨云和跨应用的网络。数据服务商与电力设备公司创造价值的方式，相距很远。</p>

    <h2 id="apple">果链的利润，怎样留在苹果</h2>

    <p>Apple FY2025，即截至 2025 年 9 月 27 日的全年，给出了一个清晰的利润结构。<a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">来源：Apple 10-K</a></p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Apple FY2025 指标</th>
            <th style="text-align: right">数值</th>
            <th>含义</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>集团收入</td>
            <td style="text-align: right">USD 416.161B</td>
            <td>公司整体，包含设备与服务</td>
          </tr>
          <tr>
            <td>iPhone 收入</td>
            <td style="text-align: right">USD 209.586B</td>
            <td>占集团收入 <strong>50.36%</strong></td>
          </tr>
          <tr>
            <td>Services 收入</td>
            <td style="text-align: right">USD 109.158B</td>
            <td>占集团收入 <strong>26.23%</strong></td>
          </tr>
          <tr>
            <td>Products 毛利率</td>
            <td style="text-align: right"><strong>36.8%</strong></td>
            <td>所有产品合计，不是单独 iPhone 毛利率</td>
          </tr>
          <tr>
            <td>Services 毛利率</td>
            <td style="text-align: right"><strong>75.4%</strong></td>
            <td>广告、许可、内容、云、AppleCare、支付等组合</td>
          </tr>
          <tr>
            <td>集团营业利润</td>
            <td style="text-align: right">USD 133.050B</td>
            <td>自算营业利润率 <strong>31.97%</strong></td>
          </tr>
          <tr>
            <td>经营现金流</td>
            <td style="text-align: right">USD 111.482B</td>
            <td>与净利润、自由现金流分开</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>Services 用约四分之一的收入贡献了 <strong>42.17% 的集团毛利</strong>：82.314 ÷ 195.201。硬件建立用户关系，系统、分发与服务让这种关系在设备销售之后继续产生收入。只看 iPhone 出货，会低估苹果的经济结构。</p>

    <p>不过，75.4% 不能解释成所有 App 都有同样的利润率。Apple 对第三方 App Store 相关销售按<strong>净额</strong>确认，只把保留的佣金计入 Services；硬件与制造商往往按不同的总额口径确认收入。收入和毛利率既反映经济地位，也受产品组合与会计列报影响。</p>

    <p>制造端的情况很不一样。鸿海 2025 自然年收入为 <strong>8.103 万亿新台币</strong>，自算集团毛利率 <strong>6.15%</strong>、营业利润率 <strong>3.20%</strong>。<a href="https://image.honhai.com/upload/202605/financy_by_quarterly/2025Q4_consolidared_report(e)_20260515_5808.pdf">来源：鸿海合并财报</a> 规模、工程能力和交付能力可以很强，但这并不等同于拥有终端生态的利润结构。</p>

    <p>苹果自身的现金固定资产购置占收入较低，也与制造分工有关。它通过供应商使用了庞大的工厂与设备体系；这些资本并没有因为不在 Apple 的资产负债表上就消失。<strong>观察链主的轻资产程度，必须同时观察供应商承担了什么。</strong></p>

    <h2 id="profits">AI 链的利润，为什么分布在多个位置</h2>

    <p>先把具有代表性的公司放在同一张表里。下表使用<strong>集团口径的未调整财务数据</strong>，不是纯 AI 业务利润率；财年并非严格同期，因此用于观察经济结构，不作为同期利润率排名。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>公司 / 位置</th>
            <th>财年与截止日期</th>
            <th style="text-align: right">集团毛利率</th>
            <th style="text-align: right">集团营业利润率</th>
            <th>解读时必须保留的边界</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Apple / 终端与生态</td>
            <td>FY2025，2025-09-27</td>
            <td style="text-align: right">46.9%</td>
            <td style="text-align: right">32.0%</td>
            <td>含高毛利 Services；不是纯设备利润</td>
          </tr>
          <tr>
            <td>NVIDIA / 计算平台</td>
            <td>FY2026，2026-01-25</td>
            <td style="text-align: right">71.1%</td>
            <td style="text-align: right">60.4%</td>
            <td>GPU、系统、网络等组合；不是单颗 GPU 利润</td>
          </tr>
          <tr>
            <td>TSMC / 制造与封装</td>
            <td>CY2025，2025-12-31</td>
            <td style="text-align: right">59.9%</td>
            <td style="text-align: right">50.8%</td>
            <td>台湾认可 IFRS 口径；服务多个终端市场</td>
          </tr>
          <tr>
            <td>Broadcom / 芯片与软件</td>
            <td>FY2025，2025-11-02</td>
            <td style="text-align: right">67.8%</td>
            <td style="text-align: right">39.9%</td>
            <td>收入约 42% 为 Infrastructure Software</td>
          </tr>
          <tr>
            <td>AMD / CPU、GPU 等</td>
            <td>FY2025，2025-12-27</td>
            <td style="text-align: right">49.5%</td>
            <td style="text-align: right">10.7%</td>
            <td>集团含多个业务及收购相关会计成本</td>
          </tr>
          <tr>
            <td>Arista / 网络系统</td>
            <td>CY2025，2025-12-31</td>
            <td style="text-align: right">64.1%</td>
            <td style="text-align: right">42.8%</td>
            <td>数据中心、AI、园区等，并非全部 AI 网络</td>
          </tr>
          <tr>
            <td>Micron / 内存与存储</td>
            <td>FY2025，2025-08-28</td>
            <td style="text-align: right">39.8%</td>
            <td style="text-align: right">26.1%</td>
            <td>内存周期敏感；2026 新变化见后文</td>
          </tr>
          <tr>
            <td>Vertiv / 供电与热管理</td>
            <td>CY2025，2025-12-31</td>
            <td style="text-align: right">36.3%</td>
            <td style="text-align: right">17.9%</td>
            <td>包含普通数据中心及其他基础设施需求</td>
          </tr>
          <tr>
            <td>鸿海 / 制造与集成</td>
            <td>CY2025，2025-12-31</td>
            <td style="text-align: right">6.1%</td>
            <td style="text-align: right">3.2%</td>
            <td>台湾认可 IFRS 口径；跨消费电子与云网络</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">Apple</a>、<a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-fourth-quarter-and-fiscal-2026">NVIDIA</a>、<a href="https://investor.tsmc.com/english/quarterly-results/2025/q4">TSMC</a>、<a href="https://investors.broadcom.com/news-releases/news-release-details/broadcom-inc-announces-fourth-quarter-and-fiscal-year-2025">Broadcom</a>、<a href="https://data.sec.gov/api/xbrl/companyfacts/CIK0000002488.json">AMD SEC</a>、<a href="https://data.sec.gov/api/xbrl/companyfacts/CIK0001596532.json">Arista SEC</a>、<a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-26-Earnings-Deck.pdf">Micron 财报中的 FY2025 比较列</a>、<a href="https://s205.q4cdn.com/554782763/files/doc_financials/2025/q4/Vertiv_Fourth-Quarter-2025-Earnings-Release.pdf">Vertiv</a>、<a href="https://image.honhai.com/upload/202605/financy_by_quarterly/2025Q4_consolidared_report(e)_20260515_5808.pdf">鸿海</a>。营业利润率按营业利润 ÷ 营收复算，采用 GAAP 或相应 IFRS 数据，不混用 adjusted/non-GAAP 指标。</p>

    <p>这组数据至少支持三个观察。</p>

    <p>第一，<strong>上游供应商也可能保留很强的利润。</strong> NVIDIA 的软件、系统与开发生态，TSMC 的制造工艺、良率和交付能力，Arista 的网络系统能力，分别构成不同的替换成本。果链本身也早已存在 TSMC 这样的强供应商；“链主拿走全部利润、供应商都只是低毛利代工”不是完整事实。</p>

    <p>第二，<strong>并非所有进入 AI 的公司都变成同一种生意。</strong> 标准化程度、设计权、材料在收入中的占比、客户认证、规模效率，以及供需周期，都会影响利润。不能因为一家公司交付 AI 服务器，就把芯片平台的毛利率套给它。</p>

    <p>第三，<strong>业务名字比想象中宽。</strong> NVIDIA FY2026 的 Data Center 收入为 <strong>USD 193.7B</strong>，约占集团收入 <strong>89.7%</strong>，但其中并非只有大模型。TSMC 2025 年的 HPC 平台占收入 <strong>58%</strong>，公司对更窄的 AI accelerator 口径却只说 <strong>high-teens percent</strong>，即百分之十几的较高区间。把 HPC 的 58% 直接写成“AI 收入占比”，会显著夸大可比口径。<a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-fourth-quarter-and-fiscal-2026">NVIDIA</a>、<a href="https://investor.tsmc.com/english/quarterly-results/2025/q4">TSMC 电话会</a></p>

    <p>内存也不能只用一个 HBM 标签解释。SK hynix 披露 2025 年收入 <strong>97.1467 万亿韩元</strong>、营业利润 <strong>47.2063 万亿韩元</strong>，并称 HBM 收入同比超过翻倍；这些结果同时受到其他 DRAM 与 NAND 产品的影响。<a href="https://news.skhynix.com/en/sk-hynix-announces-fy25-financial-results/">来源：SK hynix</a> 本文不把未披露的 HBM 绝对收入或利润率补成估计值。</p>

    <h2 id="capital">最大的变化：谁在承担资本开支</h2>

    <p>苹果把大量制造资本放在供应网络中。大模型产业则让云与计算运营商直接承担越来越多的设备和设施投入。</p>

    <p>为尽量一致，下表取现金流量表中的<strong>现金购置固定资产支出毛额</strong>，不加入非现金租赁新增、融资租赁本金、企业收购或多年投资承诺。Microsoft 和 Apple 的 FY2025 与其他三家的自然年 2025 不完全重合。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>公司</th>
            <th>年度结束日期</th>
            <th style="text-align: right">现金 PP&amp;E 购置</th>
            <th style="text-align: right">同期收入</th>
            <th style="text-align: right">购置 / 收入</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Apple</td>
            <td>2025-09-27</td>
            <td style="text-align: right">USD 12.715B</td>
            <td style="text-align: right">USD 416.161B</td>
            <td style="text-align: right"><strong>3.06%</strong></td>
          </tr>
          <tr>
            <td>Microsoft</td>
            <td>2025-06-30</td>
            <td style="text-align: right">USD 64.551B</td>
            <td style="text-align: right">USD 281.724B</td>
            <td style="text-align: right"><strong>22.91%</strong></td>
          </tr>
          <tr>
            <td>Alphabet</td>
            <td>2025-12-31</td>
            <td style="text-align: right">USD 91.447B</td>
            <td style="text-align: right">USD 402.836B</td>
            <td style="text-align: right"><strong>22.70%</strong></td>
          </tr>
          <tr>
            <td>Amazon</td>
            <td>2025-12-31</td>
            <td style="text-align: right">USD 131.819B</td>
            <td style="text-align: right">USD 716.924B</td>
            <td style="text-align: right"><strong>18.39%</strong></td>
          </tr>
          <tr>
            <td>Meta</td>
            <td>2025-12-31</td>
            <td style="text-align: right">USD 69.691B</td>
            <td style="text-align: right">USD 200.966B</td>
            <td style="text-align: right"><strong>34.68%</strong></td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">Apple</a>、<a href="https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/msft-20260630.htm">Microsoft</a>、<a href="https://www.sec.gov/Archives/edgar/data/1652044/000165204426000018/goog-20251231.htm">Alphabet</a>、<a href="https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm">Amazon</a>、<a href="https://www.sec.gov/Archives/edgar/data/1326801/000162828026003942/meta-20251231.htm">Meta</a>。这些都是集团支出，包含非 AI 业务；Amazon 还有物流等设施，Meta 也服务广告和推荐系统。这张表说明资本承担方式，不能直接视为“大模型投资额”排名。</p>

    <p>口径差异会明显改变标题数字。Amazon 同期扣除资产出售及激励回款后的 cash capex 是 <strong>USD 128.320B</strong>；Meta 在现金 PP&amp;E 之外加入融资租赁本金后，披露的 capex 为约 <strong>USD 72.22B</strong>。本文表格分别使用 USD 131.819B 与 USD 69.691B，是为了保持现金购置毛额的定义。<a href="https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm">Amazon</a>、<a href="https://www.sec.gov/Archives/edgar/data/1326801/000162828026003942/meta-20251231.htm">Meta</a></p>

    <p>到 Microsoft FY2026，截至 2026 年 6 月 30 日，同一项现金购置已达到 <strong>USD 115.948B</strong>，相当于当期收入的 <strong>34.94%</strong>。年报同时披露 Microsoft Cloud 毛利率降至 <strong>66%</strong>，解释中包括持续的 AI 基础设施投资和增长的 AI 产品使用量。这里的 Microsoft Cloud 也不能等同于单独 Azure。<a href="https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/msft-20260630.htm">来源：Microsoft</a></p>

    <p>专业 GPU 云的资本结构更集中。CoreWeave 2025 年确认收入 <strong>USD 5.131B</strong>，购置设备及资本化内部软件的现金支出为 <strong>USD 10.309B</strong>，约为当年收入的 <strong>2.01 倍</strong>；最大匿名客户贡献 <strong>67%</strong> 收入。<a href="https://www.sec.gov/Archives/edgar/data/1769628/000176962826000104/crwv-20251231.htm">来源：CoreWeave 10-K</a> 这描述的是扩张期的一年，不能解释为每一美元收入永久需要两美元投入。它说明专业算力运营商同时承担设备融资、交付与大客户风险。</p>

    <p>因此，GPU 出货不是经济闭环的终点。设备还要安装、联网、通电、获得稳定工作负载，再通过售价与利用率覆盖折旧、能源、运维和资金成本。供应商本期确认了收入，不代表最终计算服务已经获得充分回报。</p>

    <h2 id="models-data">云、模型与应用层，谁能掌握客户</h2>

    <p>如果只看硬件，很容易把大模型链写成另一份电子制造清单。但在商业闭环的另一端，竞争的是用户关系、企业数据和工作流程。</p>

    <p>云平台已经具备成熟收入来源。2025 年 AWS 的收入为 <strong>USD 128.725B</strong>，营业利润 <strong>USD 45.606B</strong>，自算营业利润率 <strong>35.4%</strong>；Google Cloud 收入 <strong>USD 58.705B</strong>，营业利润 <strong>USD 13.910B</strong>，营业利润率 <strong>23.7%</strong>。<a href="https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm">Amazon</a>、<a href="https://www.sec.gov/Archives/edgar/data/1652044/000165204426000018/goog-20251231.htm">Alphabet</a> 这都是整体云分部，并非纯 AI。Google Cloud 还包含 Workspace 等业务。</p>

    <p>模型公司则在建立自己的订阅、API 和应用入口，但其公开数据尚不能与上述财报简单并排。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>披露主体与日期</th>
            <th>可核验的公司自报数字</th>
            <th>它不代表什么</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>OpenAI，2026-01-18</td>
            <td>2025 年年化收入 <strong>超过 USD 20B</strong></td>
            <td>不是已经审计的 2025 全年营业收入</td>
          </tr>
          <tr>
            <td>OpenAI，2026-02-27</td>
            <td>ChatGPT <strong>超过 9 亿周活</strong>、<strong>超过 5,000 万消费者订阅者</strong>、<strong>超过 900 万付费商业用户</strong></td>
            <td>不同用户口径不能相加为互不重叠的付费客户；周活也不是收入</td>
          </tr>
          <tr>
            <td>Anthropic，2026-05-28</td>
            <td>公司称当月 run-rate revenue 已超过 <strong>USD 47B</strong></td>
            <td>是带日期的年化快照；不能和 OpenAI 不同日期快照直接推出市场份额</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://openai.com/index/a-business-that-scales-with-the-value-of-intelligence/">OpenAI CFO 披露</a>、<a href="https://openai.com/index/scaling-ai-for-everyone/">OpenAI 规模公告</a>、<a href="https://www.anthropic.com/news/series-h">Anthropic Series H</a>。本文没有取得两家与上市公司年报同口径的完整公开利润表，因此不填写推测的毛利率和营业利润率。</p>

    <p>大模型应用还有一种持续的成本：用户使用得越深，模型执行、检索、工具调用、重试与人工复核可能越多。因此，企业更值得比较的指标是：<strong>交付一项合格任务的总成本 = 相关推理、工具、重试与复核总成本 ÷ 成功完成的任务数。</strong> 单个 token 降价，既可能降低成本，也可能刺激更复杂的使用；不能由价格下降直接推断总算力需求下降或利润上升。</p>

    <p>谁更像苹果，因而至少有三种答案：NVIDIA 掌握计算开发生态；OpenAI、Anthropic 等模型公司争取用户入口；Microsoft、Google、Amazon 以及企业软件平台已经掌握大量客户、分发和工作流。一个企业可能同时使用多个模型，却难以迁移已有的数据权限和业务流程。最终的价值分配，需要看这些不同的替换成本如何竞争。</p>

    <h2 id="data-companies">数据公司详解：硅谷、北京与 UniPat</h2>

    <p>把数据公司放进产业链，首先要知道客户买的是什么。有人提供可授权的文本、图像与视频；有人组织专家制作示范、偏好和评分规则；有人交付能运行、重置和检验的 Agent 任务环境；还有人提供数据治理与生产软件。它们分别接近内容/IP 供应商、工程服务商、研发验证工具和企业软件平台，收入结构与壁垒并不相同。</p>

    <p>下面重点观察创业公司及新进入者。这里将“硅谷”按旧金山湾区的广义创业生态使用，并区分总部、办公地、法律联系地址和远程团队；北京部分同样区分注册主体、新创公司和成熟供应商。专家注册人数、付款总额、融资估值与营业收入分别标示。</p>

    <h3 id="silicon-valley-data">硅谷与湾区：从人才网络走向后训练工程</h3>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>公司与身份</th>
            <th>地点证据</th>
            <th>数据产品与交付</th>
            <th>可核验的商业或产品信号</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td><strong>Scale AI</strong></td>
            <td>官网明确总部旧金山</td>
            <td>专家示范、偏好、安全评测；SWE Atlas 展示可执行代码任务与多阶段审核</td>
            <td>Reuters 报道 <strong>2024 年收入 USD 870M</strong>；非公开审计财报，也不是 2026 收入</td>
          </tr>
          <tr>
            <td><strong>Surge AI</strong></td>
            <td>Reuters 2025 年报道为旧金山公司</td>
            <td>高难度专家数据、现成数据集、长时工具任务和 RL 环境；提供试训与质量反馈</td>
            <td>同篇 Reuters 报道 <strong>2024 年收入超过 USD 1B</strong>；Surge 未对报道置评</td>
          </tr>
          <tr>
            <td><strong>Mercor</strong></td>
            <td>官网列旧金山 HQ，另有纽约、伦敦办公室</td>
            <td>专家筛选；任务、rubric、参考解、代码/企业工作环境；现成许可与定制后训练合作</td>
            <td>2025-09-09 CEO 向 TechCrunch 表示年化规模已超过 <strong>USD 450M</strong>，明确是<strong>未扣承包者报酬的客户付款总额</strong></td>
          </tr>
          <tr>
            <td><strong>Turing</strong></td>
            <td>TechCrunch 2025 年报道为 Palo Alto 公司</td>
            <td>将工程师筛选和组织能力延伸到代码/推理训练数据，同时保留工程人才和企业 AI 业务</td>
            <td>同年 3 月 CEO 访谈披露约 <strong>USD 300M 年化收入 run rate</strong>；是集团混合业务的历史快照</td>
          </tr>
          <tr>
            <td><strong>Labelbox</strong></td>
            <td>官网明确总部旧金山，2018 年成立</td>
            <td>从标注工具扩展到 Alignerr 专家网络、Horizon/WorldSim 环境、轨迹与验证器</td>
            <td>2026-10-14 宣布超过 <strong>USD 100M</strong> 的股权与授信组合；这是融资，未披露可据此填写的绝对营收</td>
          </tr>
          <tr>
            <td><strong>micro1</strong></td>
            <td>法律联系地址在旧金山；公司称核心团队全球远程办公</td>
            <td>专家筛选、Realm 训练/评测环境、Cortex 企业失败诊断与定向数据生产</td>
            <td>2026-10-12 <strong>公司自报 annual run rate 超过 USD 1B</strong>；未给出完整公开的总额/净额确认政策</td>
          </tr>
          <tr>
            <td><strong>Handshake AI</strong></td>
            <td>母平台有旧金山办公室；2014 年成立、2025 年推出 AI 业务</td>
            <td>利用高校/职业网络核验专家，交付专业 prompt、回答评估与质量管理</td>
            <td>是既有职业网络的新业务，非独立新成立公司；专家付款指标不能当收入</td>
          </tr>
          <tr>
            <td><strong>Lightwheel</strong></td>
            <td>官方招聘页列 Santa Clara 与 Palo Alto 岗位</td>
            <td>SimReady 仿真资产、EgoSuite 第一视角人类数据、RoboFinals 仿真评测</td>
            <td>属于具身/Physical AI 的相邻数据基础设施；上述地点不等于全部生产团队所在地</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://scale.com/about">Scale 公司介绍</a>与<a href="https://scale.com/blog/swe-atlas">SWE Atlas</a>；<a href="https://www.reuters.com/business/scale-ais-bigger-rival-surge-ai-seeks-up-1-billion-capital-raise-sources-say-2025-07-01/">Reuters 对 Surge/Scale 的报道</a>；<a href="https://surgehq.ai/ots">Surge 产品目录</a>；<a href="https://www.mercor.com/mission/">Mercor 官方</a>、<a href="https://www.mercor.com/apex/off-the-shelf-data/">数据目录</a>及<a href="https://techcrunch.com/2025/09/09/sources-ai-training-startup-mercor-eyes-10b-valuation-on-450m-run-rate/">CEO 访谈</a>；<a href="https://techcrunch.com/2025/03/06/turing-a-key-coding-provider-for-openai-and-other-llm-producers-raises-111m-at-a-2-2b-valuation/">Turing 访谈</a>；<a href="https://labelbox.com/company/about/">Labelbox 公司介绍</a>、<a href="https://labelbox.com/blog/labelbox-series-e/">融资公告</a>、<a href="https://labelbox.com/products/horizon/">Horizon</a>；<a href="https://www.micro1.ai/blog/q3">micro1 更新</a>、<a href="https://www.micro1.ai/legal/privacy-policy">法律联系信息</a>、<a href="https://www.micro1.ai/cortex">Cortex</a>；<a href="https://joinhandshake.com/about/">Handshake 公司介绍</a>与<a href="https://joinhandshake.com/blog/our-team/introducing-handshake-ai/">AI 业务介绍</a>；<a href="https://www.lightwheel.ai/">Lightwheel 产品</a>与<a href="https://www.lightwheel.ai/careers">招聘地点</a>。这些数字具有不同年份、来源和确认方式，不构成同口径收入排行榜。</p>

    <p>其中有三种值得深看的变化。</p>

    <p><strong>Surge 把训练效果放进销售过程。</strong> 它的 Trusted Program 允许符合条件的实验室先用完整数据训练、评估模型，再在约定指标改善时付款。<a href="https://surgehq.ai/ots">Surge</a> 这只是特定计划的条款，却说明数据供应商开始围绕“这批数据让模型学会了什么”证明价值。交付能力包括难题设计、专家判断、持出集验证和迭代速度。</p>

    <p><strong>Mercor、Turing、micro1 和 Handshake 把人才入口转化成数据生产能力。</strong> 模型厂商需要专业人士的示范、判断、纠错和任务设计。Mercor 的目录已包含容器化代码任务、可执行测试、企业应用环境和专家 rubric；micro1 则把企业 Agent 的失败诊断连接到定向数据生产与后续复测。<a href="https://www.mercor.com/apex/off-the-shelf-data/">Mercor</a>、<a href="https://www.micro1.ai/cortex">micro1</a> 但人才网络中有多少账户，与本月有多少合格专家完成了多少付费任务，是不同的问题。Mercor 的收入总额案例还说明：流经平台的专家报酬可能很大，不能直接用它与软件订阅净收入比较。</p>

    <p><strong>Labelbox 把工具、专家和环境组合起来。</strong> Horizon 的交付包括企业工具模拟、任务轨迹、确定性验证器和人类偏好。客户证据也能看到更细的分工：Meta 的 GIM 原论文附录说明，Labelbox 按合同负责参与者招募、资格核验、报酬与知情同意等工作，最终评分由 GIM 团队独立执行。<a href="https://arxiv.org/html/2605.18663v1">原论文 §G.1–G.2</a> 供应商提供生产组织与质量支持，客户保留研究设计和验收，这比笼统的“给大模型公司卖数据”更接近实际交易。</p>

    <h3 id="beijing-data">北京：新专家团队与既有交付网络</h3>

    <p>北京的数据生态也出现了从采集标注向专家任务、环境和评测的延伸。不过，新近创立的公司与较成熟的交付网络，需要分开看。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>公司 / 主体与阶段</th>
            <th>公开产品与交付形态</th>
            <th>可核验的规模或流程</th>
            <th>应如何理解</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td><strong>智能知识／一面千识／Xpert Studio</strong>；北京注册，公开工商档案列 2025 年成立</td>
            <td>Terminal RL、MCP 多工具任务、Harbor 格式代码任务，包含环境、参考解、测试与轨迹；专家评测</td>
            <td>一面千识官网自报 <strong>50,000+ 领域专家</strong>；产品目录可见程序验证、质检与返修流程</td>
            <td>新团队的产品已接近外部任务与评测工程；专家网络规模不是员工数或收入</td>
          </tr>
          <tr>
            <td><strong>TalentsAI</strong>；原壤（北京）智能科技有限公司，公开档案列 2025 年成立</td>
            <td>跨学科专家出题、推理、模型回答评估、rubric 与质量审核</td>
            <td>官网自报 <strong>100,000+ 入驻专家、300+ 细分学科、月活跃标注超过 20,000 小时</strong></td>
            <td>是不同口径的运营快照；不能由注册人数或工时推算营业收入</td>
          </tr>
          <tr>
            <td><strong>龙猫数据</strong>；北京安捷智合，官网称 2014 年成立、总部北京中关村</td>
            <td>多模态采集标注、SFT/RLHF、Code Agent 流程、Agent 轨迹和模型评测</td>
            <td>官网自报累计 <strong>5,000+ 定制项目</strong>，并披露重庆、成都交付基地的分工</td>
            <td>较成熟的创业供应商在迁移交付能力，非 2025 年新创公司</td>
          </tr>
          <tr>
            <td><strong>云测数据</strong>；Testin 旗下 2017 年创建的数据业务品牌</td>
            <td>场景采集、多模态标注、私有化标注平台和数据集管理</td>
            <td>北京总部母公司 Testin 成立于 2011 年，工具、场景与运营结合</td>
            <td>品牌不是另一家独立 startup；母公司历史融资和测试客户不等于数据业务收入与客户</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://human-intelligence.cn/products/data/">智能知识产品目录</a>、<a href="https://meetchances.com/">一面千识</a>与<a href="https://vc.pedaily.cn/company/818604.html">公开工商档案</a>；<a href="https://www.talents-ai.com/">TalentsAI 官网</a>、<a href="https://www.talents-ai.com/privacy-agreement.html?v=v.3.0.0&amp;date=20260820">主体隐私政策</a>与<a href="https://vc.pedaily.cn/company/891792.html">公开工商档案</a>；<a href="https://www.longmaodata.com/about">龙猫公司介绍</a>；<a href="https://www.testin.cn/business/about.htm">Testin 公司与品牌历程</a>及<a href="https://ai.testin.cn/">数据产品</a>。北京注册主体不自动等于全部团队在北京办公。标贝科技则是另一个跨城例子：官网写明 2016 年起步于北京、当前总部在青岛，因此不把它列成当前北京总部公司。<a href="https://www.data-baker.com/about.html">标贝</a></p>

    <p>智能知识的区别，在于公开目录中的代码任务已经包含固定起始环境、参考实现、测试和运行轨迹。这类资产可以用于训练与检验；其成本包含环境配置、正确性验证、难度校准和训练/测试隔离。TalentsAI 的公开证据则更集中在专家身份、专业问题和审核流程，不能仅凭“复杂任务”就推定它拥有同样的环境引擎。</p>

    <p>这些公司的公开财务信息仍有限。与其按未确认的估值排列，不如跟踪客户是否反复采购、验收返修率、单个合格任务成本，以及环境和质检软件能否跨项目复用。</p>

    <h3 id="unipat-data">UniPat：把科研问题、评分规则和执行环境做成数据资产</h3>

    <p>UniPat AI 值得单独讨论，因为它公开展示了一条较完整的技术路线：模型负责扩展知识与构造候选问题，专家和模型共同验证问题及评分规则，再将这些资产用于训练和评测。这里依据其技术发布讨论能力形态，不把 benchmark 或开源项目规模当成商业订单。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>UniPat 公开项目</th>
            <th>公司发布的规模与时间</th>
            <th>展示的技术与工程形态</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td><strong>UniScientist</strong></td>
            <td>2026-03-04：<strong>4,700+ 科研级样本、50+ 学科、每题 20+ 可验证评分项</strong>；专家平均处理时间称为 <strong>1–2 小时/样本</strong></td>
            <td>从文献与已验证命题构造问题，再共演化 rubric；数据质量依赖知识整合和专家审核</td>
          </tr>
          <tr>
            <td><strong>SaaS-Bench</strong></td>
            <td>2026-09-28：<strong>23 个可部署 SaaS 系统、106 个专业工作流任务</strong></td>
            <td>构造带持久状态、跨应用依赖、环境重置和最终结果检查的评测环境</td>
          </tr>
          <tr>
            <td><strong>PaperBenchX</strong></td>
            <td>博客目录标 2026-10-08：<strong>93 个论文复现任务、12 个研究领域、3,168 个可评分条目</strong></td>
            <td>需要配置领域软件、完成参考流程，再通过干净环境重放与证据评分检验结果</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://unipat.ai/blog/UniScientist">UniScientist</a>、<a href="https://unipat.ai/blog/SaaS-Bench">SaaS-Bench</a>、<a href="https://unipat.ai/blog/PaperBenchX">PaperBenchX</a>、<a href="https://unipat.ai/blog">发布时间目录</a>。这些是公开研究项目的自报指标，本文未独立复跑其模型成绩；本轮亦未取得可核验的完整年度收入或公司官网总部地址，故不推测财务规模或将合作高校所在地当成总部。</p>

    <p>这个案例把数据公司的价值从“产出多少文本”推进到三个问题：<strong>任务是否值得学，完成得好不好能否可靠判断，环境能否让训练与评测重复执行。</strong> 一份科研报告看起来合理，并不等于实验被正确复现；一个 Agent 点击了正确按钮，也不等于跨系统业务状态最终一致。把这些差别写成可交付的题目、工具、轨迹与评分规则，本身就是工程工作。</p>

    <p>因此，UniPat 这样的路线，与果链中的关系更像研发验证工具和工程服务，兼有数据资产的复用性。客户真正需要验证的，是这些资产能否带来新任务上的稳定能力改善，而不是供应商的公开榜单是否足够漂亮。</p>

    <h3 id="data-financials">数据生产也有财务报表：增长、利润和客户依赖</h3>

    <p>未上市公司的公开数字不足以给出可靠利润率。可以用已上市的数据服务商作经济结构对照，但不把它们冒充新 startup。下表金额单位为 <strong>USD million（百万美元，M）</strong>，并非本文其他表格常用的十亿美元。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>2025 自然年集团指标</th>
            <th style="text-align: right">Innodata</th>
            <th style="text-align: right">Appen</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>收入</td>
            <td style="text-align: right"><strong>251.663M</strong></td>
            <td style="text-align: right"><strong>230.774M</strong></td>
          </tr>
          <tr>
            <td>同比收入增长，自算</td>
            <td style="text-align: right"><strong>47.64%</strong></td>
            <td style="text-align: right"><strong>−1.50%</strong></td>
          </tr>
          <tr>
            <td>GAAP / 法定归母净利润</td>
            <td style="text-align: right"><strong>32.181M</strong></td>
            <td style="text-align: right"><strong>−21.818M</strong></td>
          </tr>
          <tr>
            <td>净利率，自算</td>
            <td style="text-align: right"><strong>12.79%</strong></td>
            <td style="text-align: right"><strong>−9.45%</strong></td>
          </tr>
          <tr>
            <td>经营现金流</td>
            <td style="text-align: right"><strong>46.752M</strong></td>
            <td style="text-align: right"><strong>22.966M</strong></td>
          </tr>
          <tr>
            <td>客户集中度，分子不同</td>
            <td style="text-align: right">最大匿名客户 <strong>58%</strong></td>
            <td style="text-align: right">前五客户 <strong>74.3%</strong></td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://www.sec.gov/Archives/edgar/data/903651/000110465926020655/inod-20251231x10k.htm">Innodata 2025 10-K</a>、<a href="https://cdn.prod.website-files.com/656a605609413dfd3feb9d34/699fbc072974a573b2b7a48c_Appen%20Annual%20Report%202025.pdf">Appen 2025 年报</a>。Appen 在澳大利亚上市，但此处财务报表使用美元；其宣传中的收入增长 <strong>4.5%</strong>剔除了上年 Google 合同影响，不替换表中的集团未调整增速。Appen 披露 GenAI 相关工作占全年收入 <strong>33%</strong>，亦不能把所有数据服务收入当成大模型收入。</p>

    <p>两家的成本说明了人工与工程交付的重要性。Innodata 2025 年直接运营成本增加约 <strong>USD 48.8M</strong>，其中劳动相关成本增加 <strong>USD 42.5M</strong>；合同既有按时间和材料计费，也有固定价格。Appen 的费用表中，crowd 服务与项目员工费用合计 <strong>USD 137.773M</strong>，约占收入 <strong>59.7%</strong>。两家公司对成本的归类不同，本文不把各自宣传的“毛利率”硬拼成排名。</p>

    <p>大客户依赖也很像果链的某些环节。Appen 在 2024-01-22 公告，Google 的相关全球服务合同项目将在当年 3 月终止；公告当时列出的 FY2023 Google 收入为 <strong>USD 82.8M</strong>，属初步未经审计数字。<a href="https://yourir.info/resources/396cdf0a32f7cb50/announcements/apx.asx/2A1500606/APX_Customer_update_and_FY23_preliminary_results.pdf">原公告</a> 这不能证明所有客户都会自建或压价，却能说明：拥有大规模交付组织，并不自动获得稳定订单。</p>

    <p>客户付款节奏也会影响现金流。Innodata 2026 年上半年经营现金流达到 <strong>USD 164.441M</strong>，但同期净利润为 <strong>USD 29.310M</strong>；现金流中包含客户预付款增加和应付款项增长。<a href="https://www.sec.gov/Archives/edgar/data/903651/000110465926092021/inod-20260630x10q.htm">2026 半年财务</a> 现金先到账、后续仍需履约，与当期已经实现同额利润是不同的事。</p>

    <h3 id="data-rights-platforms">内容许可与企业数据软件：另外两种经济结构</h3>

    <p>Reddit、Shutterstock、Getty Images 提供了内容和许可链，但不能将它们所有非广告收入都命名为“AI 训练数据”。Reddit 2025 年 Other revenue 为 <strong>USD 140.026M</strong>、占集团 <strong>6.36%</strong>，包含内容授权及向用户出售的产品；Shutterstock 的 Data, Distribution and Services 为 <strong>USD 203.264M</strong>、占集团 <strong>20.53%</strong>，还包含分发、广告和制作服务。<a href="https://d18rn0p25nwr6d.cloudfront.net/CIK-0001713445/a8e1ae31-e61c-4a09-97be-33e01420abb2.pdf">Reddit 年报</a>、<a href="https://investor.shutterstock.com/static-files/43552bbe-6a3f-4661-a1fa-2e3ae493a012">Shutterstock 年报</a></p>

    <p>这些公司出售的权利也有区别。持续访问和更新、用于训练、在答案中展示、取得生成输出的使用许可，并非一份相同的合同。例如 Getty 与 OpenAI 在 2026 年公布的是 ChatGPT 内搜索与发现的 <strong>display agreement</strong>，不能据此写成整库训练权。<a href="https://investors.gettyimages.com/news-releases/news-release-details/getty-images-announces-display-partnership-openai">Getty 官方公告</a> 数据可以重复许可，但可许可的用途、排他性和作者授权链会影响它的价值。</p>

    <p>Snowflake 与 Databricks 则主要把模型接到受治理的企业数据和工作流程。Snowflake FY2026、截至 2026-01-31 的收入为 <strong>USD 4.684B</strong>，集团 GAAP 毛利率自算 <strong>67.2%</strong>，但营业亏损为 <strong>USD 1.435B</strong>；期末净收入留存率为 <strong>125%</strong>，衡量存量客户的产品收入扩展，并非用户留存率。<a href="https://www.sec.gov/Archives/edgar/data/1640147/000164014726000008/snow-20260131.htm">Snowflake 10-K</a> 这展示了软件平台不同的成本结构，也提醒我们高毛利不等于已经实现营业利润。</p>

    <p>Databricks 在 2026-08-13 自报 revenue run rate 超过 <strong>USD 7B</strong>，产品重点包括面向 Agent 的 Lakebase 数据库、Genie 和多模型治理与成本控制。<a href="https://www.databricks.com/company/newsroom/press-releases/databricks-grows-80-yoy-surpasses-7b-revenue-run-rate-scales">Databricks 更新</a> 它不是可直接与 Snowflake 全年确认收入比较的审计数字。两家公司也不因管理客户数据就自动拥有这些数据的版权，更不能把整个平台收入当作卖给模型厂的训练语料收入。</p>

    <h3 id="data-bargaining">数据公司的议价权：验收、复用和客户信任</h3>

    <p>数据供应商与果链供应商都有规格、试制、质量控制、验收和大客户协作。但数据不按“一部手机用掉一个零件”的方式消耗：同一项非独占数据资产可以重复许可，同一任务环境可以反复运行；定制专家劳动和环境维护又会持续产生费用。因此，未来推理 token 增长多少，并不能机械换算成数据采购增长多少。</p>

    <p>更有用的指标是<strong>每个验收合格任务的完整交付成本、首轮验收率、返修与更新成本、资产复用权、客户续单和回款</strong>。合成模型可以降低生成成本，也会把瓶颈推向任务选择、事实验证、评分器和评测污染控制。UniPat 的人机共建 rubric、智能知识的可执行测试、Surge 的试训，都在处理这类问题，而非简单增加数据条数。</p>

    <p>客户信任也是一项商业资产。2025 年 Reuters 报道 Google、OpenAI 当时计划减少或转离 Scale 的合作，并讨论战略投资带来的中立性担忧；Scale 官方回应承诺，不向 Meta 开放内部系统或客户机密。<a href="https://www.reuters.com/business/scale-ais-bigger-rival-surge-ai-seeks-up-1-billion-capital-raise-sources-say-2025-07-01/">Reuters</a>、<a href="https://scale.com/blog/customer-trust-scale-meta-deal">Scale 回应</a> 这是供应商关系和信任的证据，不是发生数据泄露的证据，也不能推定所有关系此后都已终止。</p>

    <p>从这一层看，大模型产业的关键投入既有昂贵的芯片，也有难以稳定生产的专业判断和验证条件。数据公司最终能留下多少利润，要看它能否把一次性交付变成可复用能力，同时持续满足客户越来越高的验收标准。</p>

    <h2 id="overlap">客户集中、现金流与两条链的交汇</h2>

    <p>AI 终端用途很多，并不意味着每家供应商的客户都很分散。CoreWeave 的 67% 最大客户占比就是一例；电子制造与光通信公司的披露也能看到类似现象。</p>

    <p>下面用四家全球制造网络中的公司作对照，均为 <strong>2025 自然年合并口径</strong>。金额单位是<strong>人民币亿元</strong>，不与美元金额直接相加。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>公司 / 主要位置</th>
            <th style="text-align: right">收入</th>
            <th style="text-align: right">自算集团毛利率</th>
            <th style="text-align: right">自算归母净利率</th>
            <th style="text-align: right">前五客户收入占比</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>立讯 / 精密制造与连接</td>
            <td style="text-align: right">3,323.44</td>
            <td style="text-align: right">11.91%</td>
            <td style="text-align: right">4.99%</td>
            <td style="text-align: right">65.04%</td>
          </tr>
          <tr>
            <td>浪潮 / 服务器与系统</td>
            <td style="text-align: right">1,647.82</td>
            <td style="text-align: right">4.88%</td>
            <td style="text-align: right">1.46%</td>
            <td style="text-align: right">71.29%</td>
          </tr>
          <tr>
            <td>中际旭创 / 光互连</td>
            <td style="text-align: right">382.40</td>
            <td style="text-align: right">42.04%</td>
            <td style="text-align: right">28.24%</td>
            <td style="text-align: right">75.98%</td>
          </tr>
          <tr>
            <td>新易盛 / 光互连</td>
            <td style="text-align: right">248.42</td>
            <td style="text-align: right">47.81%</td>
            <td style="text-align: right">38.37%</td>
            <td style="text-align: right">72.34%</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://static.cninfo.com.cn/finalpage/2026-04-15/1225104908.PDF">立讯</a>、<a href="https://static.cninfo.com.cn/finalpage/2026-04-11/1225096638.PDF">浪潮</a>、<a href="https://static.cninfo.com.cn/finalpage/2026-03-31/1225056459.PDF">中际旭创</a>、<a href="https://static.cninfo.com.cn/finalpage/2026-04-24/1225172598.PDF">新易盛</a>年报。归母净利率 = 归属于上市公司股东净利润 ÷ 合并营业收入，与上文营业利润率是不同指标。客户占比按各公司披露的直接客户口径；不将匿名客户认定为 Apple、NVIDIA 或某家云厂。</p>

    <p>差异非常大：进入 AI 基础设施相关市场，既可能对应较高利润的光互连组件，也可能对应低毛利的整机系统。与此同时，几家 AI 相关供应商的客户集中度依然很高。多模型、多应用的下游世界，仍可能通过少数云与设备买家集中采购。</p>

    <p>利润还要转化为现金。中际旭创 2026 年上半年归母净利润为 <strong>136.51 亿元</strong>，经营现金流净额为 <strong>18.00 亿元</strong>。<a href="https://static.cninfo.com.cn/finalpage/2026-08-22/1225491753.PDF">来源：半年报</a> 这种差距需要继续检查应收、存货、回款节奏与季节性，不能仅凭一个半年断言增长有问题，也不能把利润增长自动当成现金同步回收。</p>

    <p>两条链也已经出现实际重叠。鸿海 2025 年云网络产品收入占比约 <strong>40%</strong>，超过智能消费电子的 <strong>38%</strong>；其产品线包括服务器与通信网络，并非全部 AI。<a href="https://image.honhai.com/upload/202605/financy_by_year/2025_Annual_Report_20260523_7728.pdf">来源：鸿海年报</a> 立讯 2025 年通讯及数据中心业务收入为 <strong>245.68 亿元</strong>，占集团 <strong>7.39%</strong>，该分部毛利率 <strong>18.40%</strong>，高于消费电子分部的 <strong>10.64%</strong>。<a href="https://static.cninfo.com.cn/finalpage/2026-04-15/1225104908.PDF">来源：立讯年报</a> 同样，通讯及数据中心收入不能全部标成大模型收入。</p>

    <p>制造能力能够迁移，但迁移需要新的产品认证、工程能力和资本。连接器、PCB、光互连、系统集成、供电和热管理，会以不同方式进入新的订单结构。“属于果链”与“属于 AI 链”越来越不能作为互斥标签。</p>

    <h2 id="updates">2026 年的新数据改变了什么</h2>

    <p>年度表提供基线，最新数据则检查趋势是否持续。以下都是已公布的实际结果，不能与未来指引混写。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>公司与最新选用期间</th>
            <th>新披露</th>
            <th>对产业判断的影响</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>NVIDIA Q2 FY2027，截至 2026-07-26</td>
            <td>收入 <strong>USD 96.221B</strong>；Data Center <strong>USD 89.0B</strong>；GAAP 毛利率 <strong>75.0%</strong></td>
            <td>计算平台的规模与盈利继续扩张，仍不能把全部 Data Center 收入称为大模型收入</td>
          </tr>
          <tr>
            <td>TSMC Q3 2026，2026-10-15 公布</td>
            <td>季度收入 <strong>USD 46.72B</strong>；毛利率 <strong>66.4%</strong>；营业利润率 <strong>58.7%</strong>；HPC 占比 <strong>68%</strong></td>
            <td>制造端同样保留强盈利；HPC 仍是宽于 AI accelerator 的口径</td>
          </tr>
          <tr>
            <td>Broadcom Q3 FY2026，截至 2026-08-02</td>
            <td>公司收入 <strong>USD 29.591B</strong>；明确披露 AI 半导体收入 <strong>USD 16.7B</strong>，约占 <strong>56.4%</strong></td>
            <td>定制加速器与网络构成不同于通用 GPU 的收入路径</td>
          </tr>
          <tr>
            <td>Micron FY2026，截至 2026-09-03</td>
            <td>全年收入 <strong>USD 133.188B</strong>；GAAP 毛利率自算 <strong>80.7%</strong></td>
            <td>存储价格与供需发生剧烈变化，2025 年利润结构不能当作固定常数</td>
          </tr>
          <tr>
            <td>Apple Q3 FY2026，截至 2026-06-27</td>
            <td>集团毛利率 <strong>50.1%</strong>，其中约 <strong>2 个百分点</strong>来自关税退款</td>
            <td>最新季度的改善含一次性影响，不宜直接外推常态利润率</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>来源：<a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-second-quarter-fiscal-2027">NVIDIA</a>、<a href="https://investor.tsmc.com/english/quarterly-results/2026/q3">TSMC</a>、<a href="https://investors.broadcom.com/news-releases/news-release-details/broadcom-inc-announces-third-quarter-fiscal-year-2026-financial">Broadcom</a>、<a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-26-Earnings-Deck.pdf">Micron</a>、<a href="https://www.apple.com/newsroom/2026/07/apple-reports-third-quarter-results/">Apple</a>。Micron 的 80.7% 来自 GAAP 毛利 USD 107.504B ÷ 收入 USD 133.188B；公司材料另有 non-GAAP 数字，本文没有混用。</p>

    <p>Micron 在 2026 年 9 月的披露还称，大多数 2027 年 HBM bit supply 已有协议安排；其新加坡 HBM 封装设施初始产出预计在 2027 年初，其他新厂也需要更长时间。<a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-FY26-Prepared-Remarks.pdf">来源：Micron 管理层说明</a> 协议不是已经确认的收入，新厂初始投产也不是立即达到成熟产能。短期供应约束能带来利润，扩产、产品迭代和价格变化则会改变后续分配。</p>

    <p>Micron 同期的传统 DRAM 与 NAND 提价也贡献了增长，不能把盈利上行全部归于 HBM。表中的利润是企业结构与当期供需共同作用的结果，不能直接给产业环节贴上永久的“高利润”或“低利润”标签。</p>

    <h2 id="constraints">电力、地理与贸易环境</h2>

    <p>果链依赖跨国制造集群。Apple FY2025 年报仍将中国大陆、印度、日本、韩国、台湾和越南列为制造伙伴的主要所在地，也指出改变供应链可能昂贵、耗时并造成扰动。<a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">Apple</a> 供应链迁移涉及供应商配套、良率、设备与工程组织，不能仅按总部所在国家判断生产位置。</p>

    <p>AI 链增加了一个更突出的问题：计算容量必须在具体地点获得电力、变电设备、网络和冷却。</p>

    <p>IEA 的 <strong>2025 年《Energy and AI》报告</strong>估计，全球所有数据中心在 <strong>2024 年</strong>消耗约 <strong>415 TWh</strong> 电力，约占全球用电 <strong>1.5%</strong>；其 <strong>2030 年基准情景预测</strong>为约 <strong>945 TWh</strong>。<a href="https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai">来源：IEA</a> 两者都包含非 AI 数据中心，后者是预测，不是已经发生的用电量。</p>

    <p>比全球总量更重要的是当地的交付节奏。同份报告指出，先进经济体的新输电线路建设往往需要 <strong>4–8 年</strong>，数据中心则可能在 <strong>2–3 年</strong>内投入运营。<a href="https://iea.blob.core.windows.net/assets/de9dea13-b07d-42c5-a398-d1b3ae17d866/EnergyandAI.pdf">报告原文，页 15、50、95</a> 数据中心订单、建筑竣工、可供电容量与实际算力收入之间，存在很长的转化链条。</p>

    <p>因此，Vertiv、Schneider Electric、Eaton、Delta 等公司的产品位置值得单独研究。液冷并不能替代电网接入；一个设备供应商的 backlog，也不能直接当成已实现营收。数据中心能耗上升同样不能直接换算成某家冷却公司的收入增速。</p>

    <p>贸易规则还会改变可销售市场。BIS 在 <strong>2026-01-15</strong>发布的规则，将部分符合条件的先进计算产品对中国及澳门出口调整为个案许可审查，涉及 H200 及满足相应参数与条件的产品；这并非全面取消管制，也不保证许可证获批。<a href="https://www.federalregister.gov/documents/2026/01/15/2026-00789/revision-to-license-review-policy-for-advanced-computing-commodities">来源：Federal Register</a> 对全球公司的判断，需要同时看技术规格、目的地、许可条件以及随后规则变化。</p>

    <p>资本关系也值得跟踪。OpenAI 在 2026-02-27 宣布的新投资中包括 NVIDIA 与 Amazon；Anthropic 的融资与算力合作也涉及多家云和硬件公司。<a href="https://openai.com/index/scaling-ai-for-everyone/">OpenAI</a>、<a href="https://www.anthropic.com/news/series-h">Anthropic</a> 同一家公司可能同时是投资人、供应商和分发伙伴。融资支持产业扩张，但融资额、采购承诺和最终客户实际付费是不同的量，不能反复相加证明需求规模。</p>

    <h2 id="outlook">接下来应该观察什么</h2>

    <p>果链最值得借鉴的经验，是规模化制造、供应商认证和生态控制如何共同形成商业优势。大模型链则需要额外检查：高额计算投入能否被持续的有效需求吸收，瓶颈利润能否在扩产和替代技术出现后保留。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>环节</th>
            <th>更值得跟踪的指标</th>
            <th>容易误判的替代指标</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>芯片与制造</td>
            <td>可交付产能、良率、客户认证、软件迁移成本、真实 AI 业务口径</td>
            <td>把所有 HPC、半导体或 Data Center 收入当 AI</td>
          </tr>
          <tr>
            <td>HBM 与互连</td>
            <td>合格供应、产品组合、价格、扩产爬坡与客户采购安排</td>
            <td>把长期协议视为当期收入，把阶段性高毛利永久化</td>
          </tr>
          <tr>
            <td>服务器与机柜</td>
            <td>材料价值占比、营运资金、客户集中度、系统交付能力</td>
            <td>只看营收规模或“进入某头部客户”</td>
          </tr>
          <tr>
            <td>云与数据中心</td>
            <td>现金投入、租赁和债务、利用率、已通电容量、合同回款</td>
            <td>只看 GPU 数量、宣布的投资金额或订单积压</td>
          </tr>
          <tr>
            <td>模型与应用</td>
            <td>付费留存、真实任务成功率、任务总成本、毛利与客户获取</td>
            <td>只看 token 用量、免费用户或单点年化收入</td>
          </tr>
          <tr>
            <td>数据与企业平台</td>
            <td>合格任务交付成本、验收与返修、资产复用权、许可与权限、客户续单、评测和工作流深度</td>
            <td>只看数据条数、专家注册人数或合作公告中的客户覆盖数</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>我更倾向于把大模型产业理解为一场仍在进行的价值分配。GPU 平台已经建立了强势地位，先进制造与内存保留着物理约束，云厂商掌握资本与分发，模型和应用还在争取最终客户。它们彼此依赖，也在通过自研、替代采购和产品整合争夺对方的空间。</p>

    <p>未来是否会出现一家像 Apple 那样同时控制终端、体验和生态收益的公司，仍有待观察。更有用的问题是：<strong>在每一笔新增需求中，谁解决了暂时无法绕开的难题，谁承担了资本和库存，谁最终留住了客户与现金流？</strong></p>

    <p>沿着这些问题看，两条链的相似之处、差异和已经发生的交汇，才会变得清楚。</p>

    <h2 id="references">资料与复算说明</h2>

    <p>本文优先使用公司年报、季度披露、官方供应商资料和机构原始报告；部分私营公司采用 Reuters、TechCrunch 的采访报道，并保留报道日期及未经公开审计的性质。财报数字是相应主体的披露；模型公司的业务快照、产品页与管理层计划保留其自报性质。以下来源也提供了核对统计期间、财务定义和业务边界的入口。</p>

    <div class="drift-references">

      <ol>
        <li><strong>Apple：</strong><a href="https://www.sec.gov/Archives/edgar/data/320193/000032019325000079/aapl-20250927.htm">FY2025 Form 10-K</a>；<a href="https://www.apple.com/newsroom/2026/07/apple-reports-third-quarter-results/">FY2026 Q3 官方结果</a>；<a href="https://s203.q4cdn.com/367071867/files/doc_downloads/2024/04/Apple-Supplier-List.pdf">官方 Supplier List，覆盖 FY2023</a>。产品/服务收入、毛利、现金流、制造布局与净额确认。</li>
        <li><strong>IDC：</strong><a href="https://www.idc.com/resource-center/press-releases/q425mobilephonetop5/">2025 全球智能手机初步出货估计</a>，发布于 2026-01-13；自然年数据，不与 Apple 财年销量混同。</li>
        <li><strong>鸿海：</strong><a href="https://image.honhai.com/upload/202605/financy_by_quarterly/2025Q4_consolidared_report(e)_20260515_5808.pdf">2025 合并财报</a>；<a href="https://image.honhai.com/upload/202605/financy_by_year/2025_Annual_Report_20260523_7728.pdf">2025 Annual Report</a>。原财务表以千新台币计，产品线表另以百万新台币计。</li>
        <li><strong>NVIDIA：</strong><a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-fourth-quarter-and-fiscal-2026">FY2026 结果</a>；<a href="https://nvidianews.nvidia.com/news/nvidia-announces-financial-results-for-second-quarter-fiscal-2027">Q2 FY2027 结果</a>。集团与 Data Center 口径分别使用。</li>
        <li><strong>TSMC：</strong><a href="https://investor.tsmc.com/english/quarterly-results/2025/q4">4Q25/2025 财报、演示与电话会</a>；<a href="https://investor.tsmc.com/english/quarterly-results/2026/q3">3Q26 财报、演示与电话会</a>。HPC 与 AI accelerator 定义不同。</li>
        <li><strong>Broadcom：</strong><a href="https://investors.broadcom.com/news-releases/news-release-details/broadcom-inc-announces-fourth-quarter-and-fiscal-year-2025">FY2025 结果</a>；<a href="https://investors.broadcom.com/news-releases/news-release-details/broadcom-inc-announces-third-quarter-fiscal-year-2026-financial">Q3 FY2026 结果</a>。软件、半导体与 AI 半导体分别列示。</li>
        <li><strong>AMD：</strong><a href="https://data.sec.gov/api/xbrl/companyfacts/CIK0000002488.json">SEC 财务数据</a>；<a href="https://ir.amd.com/news-events/press-releases/detail/1295/amd-reports-second-quarter-2026-financial-results">Q2 2026 官方结果</a>。年度采用 GAAP 金额，数据中心业务包含 CPU 与 GPU。</li>
        <li><strong>内存：</strong><a href="https://news.skhynix.com/en/sk-hynix-announces-fy25-financial-results/">SK hynix FY2025</a>；<a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-26-Earnings-Deck.pdf">Micron FY2026 Q4 earnings deck，含 FY2025 比较列</a>；<a href="https://s25.q4cdn.com/621799436/files/doc_financials/2026/q4/Q4-FY26-Prepared-Remarks.pdf">Micron 2026-09-30 prepared remarks</a>。</li>
        <li><strong>网络与电力：</strong><a href="https://data.sec.gov/api/xbrl/companyfacts/CIK0001596532.json">Arista SEC 财务数据</a>；<a href="https://www.arista.com/en/company/news/press-release/24401-pr-20260804">Arista Q2 2026</a>；<a href="https://s205.q4cdn.com/554782763/files/doc_financials/2025/q4/Vertiv_Fourth-Quarter-2025-Earnings-Release.pdf">Vertiv FY2025</a>。集团收入不等于纯 AI 收入。</li>
        <li><strong>综合云与资本开支：</strong><a href="https://www.sec.gov/Archives/edgar/data/789019/000119312526323660/msft-20260630.htm">Microsoft FY2026 10-K，含 FY2025 比较</a>；<a href="https://www.sec.gov/Archives/edgar/data/1652044/000165204426000018/goog-20251231.htm">Alphabet 2025 10-K</a>；<a href="https://www.sec.gov/Archives/edgar/data/1018724/000101872426000004/amzn-20251231.htm">Amazon 2025 10-K</a>；<a href="https://www.sec.gov/Archives/edgar/data/1326801/000162828026003942/meta-20251231.htm">Meta 2025 10-K</a>。</li>
        <li><strong>专业计算运营：</strong><a href="https://www.sec.gov/Archives/edgar/data/1769628/000176962826000104/crwv-20251231.htm">CoreWeave 2025 10-K</a>；<a href="https://www.oracle.com/cloud/compute/gpu/">Oracle GPU infrastructure</a>。CoreWeave 的现金设备投入包含资本化内部软件。</li>
        <li><strong>模型公司：</strong><a href="https://openai.com/index/a-business-that-scales-with-the-value-of-intelligence/">OpenAI CFO，2026-01-18</a>；<a href="https://openai.com/index/scaling-ai-for-everyone/">OpenAI，2026-02-27</a>；<a href="https://www.anthropic.com/news/series-h">Anthropic，2026-05-28</a>。年化收入、用户和融资不是相同指标。</li>
        <li><strong>数据与企业平台：</strong><a href="https://scale.com/rlhf">Scale AI Data Engine</a>；<a href="https://redditinc.com/news/reddit-and-google-expand-partnership">Reddit–Google 公告，2024-02-22</a>；<a href="https://www.databricks.com/company/newsroom/press-releases/databricks-and-openai-launch-groundbreaking-partnership-bring">Databricks–OpenAI 公告，2025-09-25</a>。</li>
        <li><strong>制造与光互连案例：</strong><a href="https://static.cninfo.com.cn/finalpage/2026-04-15/1225104908.PDF">立讯 2025 年报</a>；<a href="https://static.cninfo.com.cn/finalpage/2026-04-11/1225096638.PDF">浪潮 2025 年报</a>；<a href="https://static.cninfo.com.cn/finalpage/2026-03-31/1225056459.PDF">中际旭创 2025 年报</a>及<a href="https://static.cninfo.com.cn/finalpage/2026-08-22/1225491753.PDF">2026 半年报</a>；<a href="https://static.cninfo.com.cn/finalpage/2026-04-24/1225172598.PDF">新易盛 2025 年报</a>。</li>
        <li><strong>电力与政策：</strong><a href="https://www.iea.org/reports/energy-and-ai/energy-demand-from-ai">IEA《Energy and AI》，2025 版</a>及<a href="https://iea.blob.core.windows.net/assets/de9dea13-b07d-42c5-a398-d1b3ae17d866/EnergyandAI.pdf">完整 PDF</a>；<a href="https://www.federalregister.gov/documents/2026/01/15/2026-00789/revision-to-license-review-policy-for-advanced-computing-commodities">BIS 2026-01-15 许可审查规则</a>。历史估计、情景预测和法规事件分别解释。</li>
        <li><strong>数据公司专章：</strong>各公司表后附产品、主体、地域及财务来源；<a href="https://unipat.ai/blog">UniPat 技术发布</a>、<a href="https://www.sec.gov/Archives/edgar/data/903651/000110465926020655/inod-20251231x10k.htm">Innodata 年报</a>、<a href="https://cdn.prod.website-files.com/656a605609413dfd3feb9d34/699fbc072974a573b2b7a48c_Appen%20Annual%20Report%202025.pdf">Appen 年报</a>和<a href="https://www.sec.gov/Archives/edgar/data/1640147/000164014726000008/snow-20260131.htm">Snowflake 年报</a>提供进一步复核入口。私营公司的媒体收入、年化规模、注册专家和融资分开呈现。</li>
      </ol>

    </div>

    <p>毛利率按（收入 − 营业成本）÷ 收入计算；营业利润率按营业利润 ÷ 收入计算；归母净利率仅用于明确标示的制造案例。百分比由原始金额计算后四舍五入。跨币种金额不换算、不相加；公司收入之间也可能包含同一项投入的多次流转。本文未根据未具名客户推测身份，也未用未公开的合同价格补足“AI 收入”。</p>

    <p class="drift-meta"><a href="/zh/blog/">← 所有博客</a> · <a href="#structure">回到正文</a></p>

  </article>
</main>]]></content><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><summary type="html"><![CDATA[用财报、官方披露与采访报道比较苹果和大模型产业链：从芯片、制造与云，到硅谷及北京的数据创业公司、UniPat 的任务与评测路线，分析利润分配、资本投入和客户依赖。]]></summary></entry><entry xml:lang="en"><title type="html">When AI Reaches the Limits of Human Understanding: Echoes of Frontier Physics</title><link href="https://zefancai.com/en/blog/frontier-ai-physics/" rel="alternate" type="text/html" title="When AI Reaches the Limits of Human Understanding: Echoes of Frontier Physics" /><published>2026-10-12T00:00:00-07:00</published><updated>2026-10-12T00:00:00-07:00</updated><id>https://zefancai.com/en/blog/frontier-ai-physics-en</id><content type="html" xml:base="https://zefancai.com/en/blog/frontier-ai-physics/"><![CDATA[<link rel="stylesheet" href="/assets/css/ar-drift.css" />

<main id="main" class="drift-article" lang="en">
  <article class="drift-prose">

    <p class="drift-kicker"><a href="/blog/">Blog</a> / Essay</p>

    <h1 id="when-ai-reaches-the-limits-of-human-understanding">When AI Reaches the Limits of Human Understanding</h1>

    <p class="drift-deck">Echoes of frontier physics: are the capabilities we can create beginning to exceed the capabilities we can understand?</p>
    <p class="drift-meta">Zefan Cai · <time datetime="2026-10-12">October 12, 2026</time> · Essay</p>

    <p>I increasingly feel that frontier AI is going through a transition that physics went through long ago.</p>

    <p>As physics moved from tabletop experiments to large colliders, it gradually accustomed us to a fact: one person can rarely master the theory, the experimental apparatus, and all the evidence at once. Explanation requires more knowledge, verification demands more resources, and work at the frontier becomes harder for any individual to hold in their head in its entirety.</p>

    <p>Physics has taken this situation to an extraordinary degree. Now AI is beginning to evoke a similar feeling: faced with a complex reasoning trace, we may recognize every word and be able to inspect every tool call, yet still be unable to judge why the whole chain holds together.</p>

    <p><strong>Capabilities may be growing faster than our ability to understand them.</strong> I think this will become an increasingly important feature of frontier AI.</p>

    <nav class="drift-toc" aria-label="Table of contents">
      <ul>
        <li><a href="#physics">How physics arrived here</a></li>
        <li><a href="#traces">Why AI traces evoke a similar feeling</a></li>
        <li><a href="#understanding">Seeing, understanding, and verifying are different things</a></li>
        <li><a href="#knowledge">The deeper parallel is how knowledge is produced</a></li>
        <li><a href="#frontier">How the limits of human understanding may shift</a></li>
        <li><a href="#references">References</a></li>
      </ul>
    </nav>

    <h2 id="physics">How physics arrived here</h2>

    <p>As physics moved from classical mechanics to relativity and quantum theory, it came to depend increasingly on mathematical understanding developed through training. Everyday experience helps us picture how a ball moves, but offers little direct intuition for quantum fields. Learning to calculate, understanding a theory’s structure, and forming an everyday mental picture gradually became different things.</p>

    <p>At the same time, the experimental requirements for some questions at the frontier of fundamental physics have risen. Higher energies, more precise measurements, and rarer signals demand larger instruments, longer observations, and more people working together. The theory of the Higgs field was proposed in 1964; the Higgs boson was not discovered until 2012. That discovery depended on a large collider, detectors, and statistical evidence identifying decay products across many collisions. <a href="https://home.cern/science/physics/higgs-boson">CERN’s account of this process</a> illustrates how theory, engineering, and verification can have to wait for one another.</p>

    <p>Today, proposed facilities such as the <a href="https://home.cern/science/accelerators/future-circular-collider">Future Circular Collider</a> also show why some questions cannot be settled by an individual researcher simply designing an experiment and trying it. Whether an experiment can be done has become closely tied to infrastructure and commitments spanning generations.</p>

    <p>If we say that physics has reached “the pinnacle of human intelligence,” I prefer to read that as an expression of how it feels: <strong>physics places extraordinary demands on the abstract reasoning an individual can develop through years of training.</strong> It does not mean that we have measured the upper limit of human intelligence, or that physics as a whole can no longer be tested. Much of physics still has active experimental feedback; some profoundly counterintuitive theories have extraordinarily precise experimental support.</p>

    <p>But the change is real: at the frontier, understanding and verification increasingly depend on a whole system of tools and organizations that extends beyond any one person.</p>

    <h2 id="traces">Why AI traces evoke a similar feeling</h2>

    <p>In AI, a similar difficulty can appear in something that seems unusually transparent: a trace.</p>

    <p>By trace, I mean the process records available to us: reasoning text produced by a model, code execution, retrieval results, tool calls, and an agent’s task breakdowns and revisions. These records capture different things, and none amounts to a complete explanation of the model’s internal computation.</p>

    <p>Imagine an agent working on a research problem. It reads several papers, develops candidate approaches, writes code to run experiments, revises its hypotheses after failures, and eventually produces an answer. We can save those steps, and even replay most of the operations.</p>

    <p>To truly understand the trajectory, however, we need to know more. Why did it choose those hypotheses? Which experiments ruled out which possibilities? Was an apparently unrelated observation in the middle actually crucial to the later conclusion? Would the answer still hold if we removed a step?</p>

    <p>This requires more than reading. We have to reconstruct part of the research process in our own minds. When a trajectory is long enough and draws on enough different fields, reviewing it can approach the effort of doing the task again. A model may also produce process records much faster than a human can review them.</p>

    <p>A once-familiar reassurance begins to weaken: “As long as it shows me the process, I can understand what it is doing.”</p>

    <p>Natural language makes this gap especially striking. When we encounter a page of quantum field theory equations, we readily recognize that we lack the background. When we encounter a page of fluent AI reasoning, we can mistake readable language for an understanding of the logic and mechanisms behind it.</p>

    <p>Of course, difficulty of understanding does not itself demonstrate greater intelligence. Verbosity, errors, repetition, and confusion can all make a trace hard to read. The claim that “its capabilities exceed our direct understanding” becomes meaningful only when a system consistently produces good, independently checkable results on well-defined tasks, while its methods remain difficult for us to grasp.</p>

    <h2 id="understanding">Seeing, understanding, and verifying are different things</h2>

    <p>To discuss these limits, we need to separate at least three questions.</p>

    <p><strong>Seeing</strong> means having a record of the process. We know which tool it called, what text it generated, and what code it ran.</p>

    <p><strong>Understanding</strong> means knowing how those steps relate, which factors actually affected the outcome, and what might happen if the conditions changed.</p>

    <p><strong>Verifying</strong> means testing a conclusion against sufficiently independent evidence and establishing the conditions under which it is reliable.</p>

    <p>None automatically guarantees the others. Even when every sentence in a trace is clear, the reasons a model writes down may not faithfully reflect the factors that determined its answer. In the controlled experiments reported in <a href="https://arxiv.org/abs/2505.05410">Reasoning Models Don’t Always Say What They Think</a>, Anthropic researchers found that models sometimes used information in a prompt without disclosing its influence in their chain of thought. Under those experimental conditions, reasoning text could not simply be treated as a complete causal record.</p>

    <p>Mechanistic interpretability research tries to investigate the internal computation more directly. For example, <a href="https://transformer-circuits.pub/2025/attribution-graphs/biology.html">On the Biology of a Large Language Model</a> uses internal features and interventions to analyze specific behaviors, including cases in which a textual explanation differs from the actual computation. But this work still faces limits of scope and method. It remains far from providing complete explanations for arbitrary complex tasks.</p>

    <p>Conversely, we may be able to verify some results reliably without fully understanding the process. Once the axioms and formal statement are fixed, a formal proof can be checked by a proof checker; a program can be run and subjected to independent tests. Passing tests provides evidence that the program behaves correctly on the inputs and under the conditions tested; formal verification guarantees the proposition as formalized. Both provide evidence beyond “this explanation sounds plausible.”</p>

    <p>The hardest cases are questions whose answers are themselves difficult to check: whether a new scientific hypothesis is correct, whether a long-term decision is effective, or whether a complex system will remain reliable in a different environment. In these tasks, the difficulties of understanding the trace and verifying the result can arise together.</p>

    <h2 id="knowledge">The deeper parallel is how knowledge is produced</h2>

    <p>The experimental conditions in the two fields differ substantially. Many experiments on AI behavior can still be repeated at low cost, and software systems let us save inputs, change conditions, and run them again. That is very different from building a new particle collider.</p>

    <p>I think the deeper similarity is that <strong>it is becoming harder for one person to follow the production of knowledge independently from beginning to end.</strong> We need tools, a division of work, and mutual checks to assemble the parts that each person can understand into a reliable whole.</p>

    <p>AI makes this problem more acute. It is both an object of study and a potential participant in proposing theories, designing experiments, and interpreting results. When we ask one model to explain another model’s trace, the explanation itself becomes something we need to verify. Agreement between two models can provide a clue. If they share similar blind spots, however, agreement alone cannot establish that the conclusion is valid.</p>

    <p>This means that tools for evaluation, explanation, and verification need to develop alongside model capabilities. Where a task can be executed, evidence should come from execution. Where a mechanism permits intervention, we should check whether changing the key conditions actually changes the behavior. Where a judgment can only accumulate empirical support, we should preserve its uncertainty and limits of applicability.</p>

    <h2 id="frontier">How the limits of human understanding may shift</h2>

    <p>The “limits of human understanding” are not a fixed line.</p>

    <p>There is a considerable difference between what one person can understand unaided, what a trained team can understand, and what humanity can understand with mathematics, instruments, and computers. The history of progress in physics is a history of expanding the latter two.</p>

    <p>Frontier AI raises an urgent question: <strong>the range of tasks a system can complete keeps growing, while the range of processes we can grasp and results we can check at a reasonable cost may be expanding more slowly.</strong></p>

    <p>I suspect that the following situation will become increasingly common: a system produces a result, we can confirm important parts of it through independent checks, yet we struggle to grasp the entire path by which it found the answer within a reasonable amount of time.</p>

    <p>That would feel much like physics does: the world can still be studied, and knowledge can still advance, but understanding it requires far more than everyday intuition.</p>

    <p>The intelligence we create with our own hands may bring us to a similar place. The challenge ahead is to help our ability to understand and verify intelligence keep pace, as far as possible, with our ability to create it.</p>

    <h2 id="references">References</h2>

    <div class="drift-references">

      <ol>
        <li><strong>CERN: The Higgs boson.</strong> <a href="https://home.cern/science/physics/higgs-boson">Official overview</a>. The theoretical background of the Higgs field, the 2012 discovery, and the search for signals through decay products.</li>
        <li><strong>CERN: Future Circular Collider.</strong> <a href="https://home.cern/science/accelerators/future-circular-collider">Overview of the proposed facility</a>. An example of the infrastructure required for some frontier physics experiments; the project remains under planning and evaluation.</li>
        <li><strong>Reasoning Models Don’t Always Say What They Think.</strong> <a href="https://arxiv.org/abs/2505.05410">arXiv:2505.05410</a>. Controlled prompting experiments on chain-of-thought faithfulness, with conclusions bounded by their experimental scope.</li>
        <li><strong>On the Biology of a Large Language Model.</strong> <a href="https://transformer-circuits.pub/2025/attribution-graphs/biology.html">Transformer Circuits, 2025</a>. Analysis of internal model computation through feature attribution and interventions, including discussion of the methods’ limitations.</li>
      </ol>

    </div>

    <p class="drift-meta"><a href="/blog/">← All posts</a> · <a href="#physics">Back to the essay</a></p>

  </article>
</main>]]></content><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><summary type="html"><![CDATA[Frontier physics has long required mathematics, instruments, and large collaborations to make sense of the world. Frontier AI is raising a similar question: when capabilities grow faster than our understanding, how do we check reasoning we can no longer follow from beginning to end?]]></summary></entry><entry xml:lang="zh-CN"><title type="html">当 AI 走到人类理解的边界：前沿物理学的回声</title><link href="https://zefancai.com/blog/frontier-ai-physics/" rel="alternate" type="text/html" title="当 AI 走到人类理解的边界：前沿物理学的回声" /><published>2026-10-12T00:00:00-07:00</published><updated>2026-10-12T00:00:00-07:00</updated><id>https://zefancai.com/blog/frontier-ai-physics</id><content type="html" xml:base="https://zefancai.com/blog/frontier-ai-physics/"><![CDATA[<link rel="stylesheet" href="/assets/css/ar-drift.css" />

<main id="main" class="drift-article" lang="zh-CN">
  <article class="drift-prose">

    <p class="drift-kicker"><a href="/zh/blog/">中文博客</a> / 随笔</p>

    <h1 id="ai-">当 AI 走到人类理解的边界</h1>

    <p class="drift-deck">前沿物理学的回声：我们能创造出的能力，是否正在超过我们能理解的能力？</p>
    <p class="drift-meta">Zefan Cai · <time datetime="2026-10-12">2026 年 10 月 12 日</time> · 随笔</p>

    <p>我越来越觉得，frontier AI 正在经历一个物理学早已经历过的转变。</p>

    <p>从桌面实验走到大型对撞机，物理学逐渐让我们习惯了一件事：一个人很难同时掌握理论、实验装置和全部证据。解释需要的知识越来越多，验证需要的资源越来越大，最前沿的成果也越来越难被一个人完整地装进脑子里。</p>

    <p>物理学已经把这种处境推到了极高的位置。现在，AI 也开始让人产生类似的感受：面对一条复杂的 reasoning trace，即使每个词都认识、每个工具调用都能打开，我们仍然可能无法判断，整条路径为什么成立。</p>

    <p><strong>能力的增长，可能正在快于人类理解它的能力。</strong> 我觉得这会成为 frontier AI 越来越重要的特征。</p>

    <nav class="drift-toc" aria-label="文章目录">
      <ul>
        <li><a href="#physics">物理学如何走到这一步</a></li>
        <li><a href="#traces">AI 的 trace 为什么带来类似的感觉</a></li>
        <li><a href="#understanding">看见、理解和验证，是三件事</a></li>
        <li><a href="#knowledge">真正相似的是知识生产的方式</a></li>
        <li><a href="#frontier">人类理解的边界，会怎样移动</a></li>
        <li><a href="#references">参考资料</a></li>
      </ul>
    </nav>

    <h2 id="physics">物理学如何走到这一步</h2>

    <p>从经典力学走向相对论和量子理论，物理学越来越依赖经过训练的数学理解。日常经验可以帮助我们想象一个球怎样运动，却很难直接提供理解量子场的直觉。学会计算、理解理论结构、形成日常意义上的画面，逐渐成了不同的事情。</p>

    <p>与此同时，某些前沿基础物理问题的实验门槛也在升高。更高的能量、更精密的测量、更稀少的信号，要求更大的仪器、更长的观测时间和更多人的协作。关于希格斯场的理论在 1964 年提出，希格斯玻色子直到 2012 年才被发现；发现依靠的是大型对撞机、探测器，以及从大量碰撞中识别衰变产物的统计证据。<a href="https://home.cern/science/physics/higgs-boson">CERN 对这段过程的介绍</a>，本身就是理论、工程和验证如何相互等待的例子。</p>

    <p>今天，像 <a href="https://home.cern/science/accelerators/future-circular-collider">Future Circular Collider</a> 这样的拟议设施，也说明了某些问题为什么无法仅靠一个研究者“设计一个实验、试一下”来解决。实验能不能做，已经与基础设施和跨代际投入密切相关。</p>

    <p>如果说物理学已经触及“人类智商的巅峰”，我更愿意把这句话理解为一种感受：<strong>它已经把个体经过长期训练所能掌握的抽象推理，推到了极高的强度。</strong> 这并不意味着我们测出了人类智力的上限，也不意味着物理学整体已经无法验证。大量物理研究仍然有活跃的实验反馈；一些极其反直觉的理论，恰恰拥有极其精确的实验支持。</p>

    <p>但那个变化是真实的：在最前沿，理解与验证越来越需要一整套超出单个人的工具和组织。</p>

    <h2 id="traces">AI 的 trace 为什么带来类似的感觉</h2>

    <p>在 AI 中，类似的困难会出现在一种看起来非常透明的东西上：trace。</p>

    <p>这里的 trace，是我们能拿到的过程记录，例如模型输出的推理文本、代码执行、检索结果、工具调用，以及 agent 对任务的拆分和修订。它们记录的内容不同，也都不等于模型内部计算的完整解释。</p>

    <p>设想一个 agent 为了解决一个研究问题，阅读多篇论文，构造几个候选方案，写代码做实验，根据失败结果修改假设，最后给出一个答案。我们可以保存这些步骤，甚至重放其中的大部分操作。</p>

    <p>然而，要真正理解这条轨迹，人需要知道更多：它为什么选了这些假设？哪些实验排除了哪些可能性？中间那个看似无关的观察，是否恰好是后面结论的关键？如果删掉某一步，答案还会成立吗？</p>

    <p>这要求的不只是阅读，而是在自己的脑中重建一部分研究过程。当轨迹足够长、依赖的领域足够多时，复核它可能已经接近重新做一遍这个任务。模型生成过程记录的速度，也可能远高于人类审查它的速度。</p>

    <p>于是，一种曾经熟悉的安全感开始减弱：“只要把过程展示出来，我就能看懂它在做什么。”</p>

    <p>自然语言让这种落差尤其明显。读到一页量子场论公式时，我们很容易意识到自己缺少背景；读到一页流畅的 AI 推理时，我们却容易把语言上的可读性，当成逻辑和机制上的可理解性。</p>

    <p>当然，难懂本身不能证明智能更高。冗长、错误、重复和混乱，都能制造难以阅读的 trace。只有当一个系统在明确任务上持续给出可独立检验的好结果，而我们仍难以掌握它的方法时，“能力超过了我们的直接理解”才成为一个有意义的判断。</p>

    <h2 id="understanding">看见、理解和验证，是三件事</h2>

    <p>讨论这里的边界，至少需要分开三个问题。</p>

    <p><strong>看见</strong>，是有过程记录。我们知道它调用了哪个工具，生成了哪段文字，运行了哪份代码。</p>

    <p><strong>理解</strong>，是知道这些步骤之间的关系，知道哪些因素真正影响了结果，并且能说明条件改变后可能发生什么。</p>

    <p><strong>验证</strong>，是用足够独立的证据检验结论，明确它在哪些条件下可靠。</p>

    <p>三者并不自动相互保证。即使 trace 中的每句话都很清楚，模型写出来的理由也未必忠实反映决定答案的因素。Anthropic 等研究者在 <a href="https://arxiv.org/abs/2505.05410">Reasoning Models Don’t Always Say What They Think</a> 的受控实验中发现，模型有时利用了提示中的信息，却没有在 chain of thought 中披露这种影响。这说明，在那些实验条件下，不能把推理文本直接当成完整的因果记录。</p>

    <p>机制解释研究尝试进一步追问内部计算。例如 <a href="https://transformer-circuits.pub/2025/attribution-graphs/biology.html">On the Biology of a Large Language Model</a> 通过内部特征和干预分析具体行为，也展示了文字解释与实际计算不一致的案例。但这种工作仍有覆盖范围和方法上的限制，距离为任意复杂任务提供完整解释还很远。</p>

    <p>反过来，我们也可能在没有完全理解过程的情况下，可靠地验证某些结果。一份形式化证明，在公理和形式化陈述确定之后，可以由证明检查器检验；一个程序可以运行，并接受独立测试。测试通过，提供的是程序在被测输入和条件下表现正确的证据；形式验证所保证的是形式化的命题。它们都提供了超出“这段解释听起来合理”的证据。</p>

    <p>真正棘手的是那些答案本身也难以检验的问题：一条新的科学假说是否正确，一项长期决策是否有效，一个复杂系统换了环境后是否仍然可靠。在这些任务里，读懂 trace 和验证结果的困难可能同时出现。</p>

    <h2 id="knowledge">真正相似的是知识生产的方式</h2>

    <p>两者的实验条件有明显区别。许多 AI 行为实验仍然可以低成本重复，软件系统也允许我们保存输入、修改条件并重跑；这与建造一台新的粒子对撞机很不一样。</p>

    <p>我认为更深的共同点，是<strong>知识越来越难由一个人独立地从头走到尾。</strong> 我们需要借助工具、分工和相互校验，把每个人只能理解的一部分组织成可靠的整体。</p>

    <p>AI 还会让这个问题变得更尖锐：它既是我们研究的对象，也可能参与提出理论、设计实验和解释结果。当我们开始让一个模型解释另一个模型的 trace，解释本身也进入了需要验证的范围。两个模型给出一致答案，可以提供线索；若它们共享相似的盲点，一致性就不能单独构成结论成立的证据。</p>

    <p>这意味着，评估、解释和验证工具需要与模型能力一起发展。对于能执行的任务，就让证据落到执行结果上；对于能做干预的机制，就检查改变关键条件是否真的改变行为；对于只能积累经验支持的判断，就把不确定性和适用范围保留下来。</p>

    <h2 id="frontier">人类理解的边界，会怎样移动</h2>

    <p>所谓“人类理解的边界”，也并不是一条固定的线。</p>

    <p>一个人不借助工具能理解的事情，一个受过训练的团队能理解的事情，以及人类借助数学、仪器和计算机能理解的事情，范围差别很大。物理学过去的进步，正是不断扩展后两种范围的历史。</p>

    <p>Frontier AI 正在提出一个紧迫的问题：<strong>系统能完成的任务范围不断扩大，而我们能以合理成本掌握其过程、检查其结果的范围，可能扩张得更慢。</strong></p>

    <p>我想，这种场景会越来越常见：它交出一个结果，我们能通过独立检查确认其中重要的部分，却很难在合理时间内掌握它找到答案的整条路径。</p>

    <p>那会很像物理学给我们的感受：世界依然能够被研究，知识依然能够向前推进，但理解它所需要的东西，已经远远超过日常直觉。</p>

    <p>我们亲手创造的智能，也可能把我们带到类似的位置。接下来的挑战，是让人类理解和验证智能的能力，尽可能跟上创造智能的能力。</p>

    <h2 id="references">参考资料</h2>

    <div class="drift-references">

      <ol>
        <li><strong>CERN: The Higgs boson.</strong> <a href="https://home.cern/science/physics/higgs-boson">官方介绍</a>。希格斯场的理论背景、2012 年的发现，以及通过衰变产物寻找信号的方法。</li>
        <li><strong>CERN: Future Circular Collider.</strong> <a href="https://home.cern/science/accelerators/future-circular-collider">拟议设施介绍</a>。用于说明部分前沿物理实验的基础设施需求；该项目仍处于规划与评估过程。</li>
        <li><strong>Reasoning Models Don’t Always Say What They Think.</strong> <a href="https://arxiv.org/abs/2505.05410">arXiv:2505.05410</a>。通过受控提示实验研究 chain-of-thought 的忠实性；结论有其具体实验范围。</li>
        <li><strong>On the Biology of a Large Language Model.</strong> <a href="https://transformer-circuits.pub/2025/attribution-graphs/biology.html">Transformer Circuits, 2025</a>。通过特征归因和干预分析模型内部计算，并讨论解释方法的局限。</li>
      </ol>

    </div>

    <p class="drift-meta"><a href="/zh/blog/">← 所有博客</a> · <a href="#physics">回到正文</a></p>

  </article>
</main>]]></content><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><summary type="html"><![CDATA[前沿物理学早已迫使我们借助数学、仪器和大型协作理解世界。Frontier AI 也正在提出类似的问题：当能力的增长快于可理解性的增长，我们该如何检验自己已经无法从头掌握的推理？]]></summary></entry><entry xml:lang="en"><title type="html">Policy Consistency for Recurrent Models in RL: Why LastKV Places Feedback at Chunk Boundaries</title><link href="https://zefancai.com/en/blog/lastkv-rl/" rel="alternate" type="text/html" title="Policy Consistency for Recurrent Models in RL: Why LastKV Places Feedback at Chunk Boundaries" /><published>2026-10-01T00:00:00-07:00</published><updated>2026-10-01T00:00:00-07:00</updated><id>https://zefancai.com/en/blog/lastkv-rl-en</id><content type="html" xml:base="https://zefancai.com/en/blog/lastkv-rl/"><![CDATA[<link rel="stylesheet" href="/assets/css/ar-drift.css" />

<link rel="stylesheet" href="/assets/css/lastkv-rl.css" />

<main id="main" class="drift-article" lang="en">
  <article class="drift-prose">

    <p class="drift-kicker"><a href="/blog/">Blog</a> / Research notes</p>

    <h1 id="in-rl-does-the-trainer-evaluate-the-policy-that-generated-the-rollout">In RL, Does the Trainer Evaluate the Policy That Generated the Rollout?</h1>

    <p class="drift-deck">From higher-layer feedback at every token to LastKV's fixed chunk recurrence.</p>
    <p class="drift-meta">Zefan Cai · <time datetime="2026-10-01">October 1, 2026</time> · Research notes</p>

    <p>Consider a practical RL implementation for a recurrent language model. To control latency, the prompt uses parallel prefill. During response generation, each token reads the higher-layer state produced by the completed computation of the previous token, recurring one token at a time. During actor updates, the trainer preserves throughput by feeding the entire recorded sequence through a finite number of fully parallel refinement passes to recompute log-probabilities.</p>

    <p><strong>These three paths may not execute the same policy.</strong> Identical token sequences and parameters do not imply identical feedback states. The rollout uses states produced by the actual recurrence; the trainer uses states produced by a finite parallel approximation. They may assign different next-token probabilities to the same prefix.</p>

    <p>In the previous <a href="/en/blog/ar-drift/">article on drift in RL and autoregressive video</a>, we distinguished conditional distribution mismatch, gradient bias, and optimization objectives. This article focuses on one specific issue in RL: <strong>whether sampling and gradient recomputation follow the same state update rules, and why LastKV fixes cross-layer feedback at chunk boundaries.</strong></p>

    <div class="drift-callout">
      <p>The central claim: <strong>finite parallel replay creates a structural policy mismatch if it cannot reproduce the feedback states of token-by-token rollout. LastKV defines chunk recurrence as part of the model’s behavior, allowing parallel recomputation within a chunk to correspond exactly to token-by-token decoding. The cost is that computation across chunks remains sequential.</strong></p>
    </div>

    <nav class="drift-toc" aria-label="Table of contents">
      <ul>
        <li><a href="#policy-identity">Defining the consistency RL needs</a></li>
        <li><a href="#feedback">The issue is feedback state, not teacher forcing</a></li>
        <li><a href="#papers">How LRT, FBT, and T²MLR discuss this issue</a></li>
        <li><a href="#policy-gradient">Why it affects policy gradients</a></li>
        <li><a href="#lastkv">LastKV: making recurrence boundaries part of the policy</a></li>
        <li><a href="#prompt-boundary">The end of a prompt is not a chunk boundary</a></li>
        <li><a href="#gradients">Matching probabilities does not guarantee complete gradients</a></li>
        <li><a href="#verification">How to verify consistency, and what this design costs</a></li>
        <li><a href="#references">References</a></li>
      </ul>
    </nav>

    <h2 id="policy-identity">Defining the consistency RL needs</h2>

    <p>Fix a prompt \(x\) and write the recorded response as \(a_{1:T}\). Let \(\mu_\theta\) denote the policy defined by the actual rollout path, and \(p_\theta\) the policy recomputed by the actor replay path. At identical parameters, we want each step to satisfy:</p>

\[p_\theta(a_t\mid x,a_{&lt;t})
=\mu_\theta(a_t\mid x,a_{&lt;t}).\]

    <p>A stronger check compares the entire next-token distribution, rather than only the probability of the token that happened to be sampled. This does not require updated parameters to remain equal to the old parameters. It requires <strong>both execution paths to express the same policy for any given set of parameters.</strong></p>

    <p>An ordinary causal Transformer provides a familiar example. Once the full trajectory is known, all positions can be computed in parallel under a causal mask. Token-by-token decoding instead caches historical KV at each layer. Under ideal numerical conditions, these are two execution orders of the same causal computation graph. Parallel training and sequential generation are therefore not inherently a problem.</p>

    <p>The rollout considered here is a <strong>hybrid policy</strong>: the prompt passes through a specified parallel prefill, then the response continues with sequential recurrence from the state left by that prefill. Exact recomputation must reproduce the prefill, the state at the transition, and the subsequent recurrence. Replacing the whole sequence with a different parallel forward changes response states. Recomputing the whole sequence sequentially from the start may also change the initial state left by the prompt.</p>

    <h2 id="feedback">The issue is feedback state, not teacher forcing</h2>

    <p>Teacher forcing simply feeds known tokens to the model. In RL replay, those tokens come from a trajectory the model has already generated, rather than a separate labeled answer. <strong>Knowing the tokens does not mean knowing the recurrent states that the current parameters should produce.</strong> LastKV replay can also use teacher forcing.</p>

    <p>A simple recurrence illustrates the distinction. Suppose the sequential model computes:</p>

\[h_t=x_t+\rho h_{t-1},\qquad h_0=0.\]

    <p>Parallel computation initializes \(h_t^{(0)}=x_t\) and performs the following update in each Jacobi refinement round:</p>

\[h_t^{(k)}=x_t+\rho h_{t-1}^{(k-1)},
\qquad h_0^{(k)}=0.\]

    <p>With \(x_t=1\) and \(\rho=0.5\), the difference between one refinement round and the actual sequential states is already visible:</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Position</th>
            <th>Actual sequential state</th>
            <th>One parallel refinement round after initialization</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>1</td>
            <td>1</td>
            <td>1</td>
          </tr>
          <tr>
            <td>2</td>
            <td>1.5</td>
            <td>1.5</td>
          </tr>
          <tr>
            <td>3</td>
            <td>1.75</td>
            <td>1.5</td>
          </tr>
          <tr>
            <td>4</td>
            <td>1.875</td>
            <td>1.5</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>This is an example of the dependency structure, not a model experiment from any paper. Finite refinement propagates feedback only a finite number of times, whereas sequential computation continues propagating it through the entire generated history. If logits depend on these states, the conditional probabilities may differ. Enough rounds can recover this strictly causal recurrence, but that changes the compute budget of a fixed small number of parallel passes.</p>

    <p>The relevant comparison is therefore <strong>whether reading the previous token’s completed recurrent state and reading a state from an earlier refinement round follow the same dependency chain.</strong></p>

    <h2 id="papers">How LRT, FBT, and T²MLR discuss this issue</h2>

    <p>These papers already study the relevant tradeoffs. It would be inaccurate to say that they overlook mismatch, or to treat the rules of a local adaptation as the rules of every original method.</p>

    <ul>
      <li><strong><a href="https://arxiv.org/html/2605.26797v2#S4.SS6">Latent Recurrent Transformer (LRT)</a></strong> states directly in §4.6 that rollout recurrent states are generated autoregressively, while policy probabilities are recomputed through parallel refinement, introducing mismatch. The paper initializes the state buffer for recomputation with cached rollout states to reduce the discrepancy. Its claim is mitigation, rather than general exact equivalence.</li>
      <li><strong><a href="https://arxiv.org/html/2608.08888v2#S5">Full-bandwidth Transformer (FBT)</a></strong> discusses replay of latent-feedback trajectories in §5, distinguishing sequential recomputation, hidden-state replay, and multi-round refinement. It describes finite-round replay as a finite-horizon approximation to the full autoregressive distribution and notes that reusing behavior-policy hidden states may introduce bias. The RL proposal in §5.2 is explicitly marked as not yet experimentally validated.</li>
      <li><strong><a href="https://arxiv.org/html/2607.15178v2#A2.SS2">T²MLR</a></strong> discusses caching exact rollout latents and constructing a BPTT graph in Appendix B.2. In particular, <a href="https://arxiv.org/html/2607.15178v2#A2.SS5">Appendix B.5</a> states that the main experiments use <strong>exact sequential prompt prefill</strong>. Jacobi prefill is an optional method that trades approximation error for latency.</li>
    </ul>

    <p>The premise of this article is thus the execution combination specified at the start: <strong>parallel prefill, token-by-token recurrent generation, and a finite number of fully parallel actor replay passes.</strong> It is not a literature claim that all three papers always use parallel prefill. An approximation may work well on measured tasks, but similar average losses or benchmark scores do not establish equal probabilities at each prefix.</p>

    <h2 id="policy-gradient">Why it affects policy gradients</h2>

    <p>To optimize the expected reward of the actual rollout policy, the objective and its score-function gradient are:</p>

\[\begin{aligned}
J(\theta)&amp;=\mathbb E_{\tau\sim\mu_\theta}[R(\tau)],\\
\nabla_\theta J
&amp;=\mathbb E_{\tau\sim\mu_\theta}\!\left[\right.\\
&amp;\qquad\left.R(\tau)\nabla_\theta\log\mu_\theta(\tau)\right].
\end{aligned}\]

    <p>Here we omit environment terms that do not depend on the parameters and assume the reward has no additional explicit parameter dependence. If the update instead uses \(\nabla_\theta\log p_\theta(\tau)\), it cannot generally be identified with this gradient. <strong>The recurrent policy generates the data, while differentiation passes through the parallel approximation policy.</strong></p>

    <p>This discrepancy can occur even before the first optimizer update. If a PPO-style probability ratio stores the actual rollout old log-probability in its denominator and uses actor replay for its numerator, then at \(\theta=\theta_{\rm old}\):</p>

\[r_t=
\frac{p_{\theta_{\rm old}}(a_t\mid x,a_{&lt;t})}
{\mu_{\theta_{\rm old}}(a_t\mid x,a_{&lt;t})}\]

    <p>may already differ from 1. This can still be a valid density ratio between two distributions, but it includes <strong>a change of execution mode</strong>, rather than the intended ratio before and after updating the same deployed policy. Recomputing the denominator with the parallel engine as well can make the initial ratio equal to 1 while concealing the actual sampling discrepancy.</p>

    <p>This connects to the previous discussion of <a href="https://arxiv.org/html/2609.20807v1">Score Centering</a>. At a fixed prefix, let \(g=\nabla_\theta\log p_\theta\). An uncentered score update decomposes as:</p>

\[\mathbb E_\mu[Rg]
=\mathbb E_\mu[R]\,\mathbb E_\mu[g]
+\operatorname{Cov}_\mu(R,g).\]

    <p>When the sampler and trainer differ, \(\mathbb E_\mu[g]\) is generally no longer zero. Centering can subtract this mean-score term, but it does not automatically turn the remaining term into the exact policy gradient of \(\mu_\theta\). This explains one source of bias; it does not imply that all RL implementations using baselines, clipping, or within-group normalization will drift in the same way.</p>

    <h2 id="lastkv">LastKV: making recurrence boundaries part of the policy</h2>

    <p><a href="/research/lastkv/">LastKV</a> chooses a different set of state update rules. It does not require every token’s last-layer state to feed back immediately to all lower layers of the next token. It defers this cross-layer feedback to fixed chunk boundaries.</p>

    <p>Start with the core rule without blending. Within chunk \(c\), layer \(\ell\) reads last-layer KV from completed chunks as history and uses its own per-layer KV for the current chunk:</p>

\[A_c^\ell=
\operatorname{Attn}\left(
Q_c^\ell,
[K_{&lt;c}^{L};K_c^\ell],
[V_{&lt;c}^{L};V_c^\ell]
\right).\]

    <p>All history precedes the current chunk, and a causal mask applies to the current portion. <strong>The rules for reading cross-chunk history remain fixed while this chunk is processed.</strong> A token within the current chunk is not promoted to historical feedback that every layer immediately reads merely because it has completed the last layer.</p>

    <p>Prefill and actor replay therefore process chunks sequentially, with parallel computation inside each chunk. Decoding samples one token at a time, but grows the local KV of each layer within the chunk. For identical input tokens, causal attention ensures that earlier positions are unaffected by later positions that have not yet been generated. The chunk’s role in subsequent history changes only once the chunk is complete.</p>

    <div class="drift-loop-grid">
  <div class="drift-loop">
    <p class="drift-loop-label">Actor replay</p>
    <p class="drift-loop-path">Fixed history → parallel causal forward within the chunk → complete chunk → update history → next chunk</p>
  </div>
  <div class="drift-loop">
    <p class="drift-loop-label">Rollout decode</p>
    <p class="drift-loop-path">Same history → generate tokens and grow per-layer KV within the chunk → update history at the same boundary → next chunk</p>
  </div>
</div>

    <p>Both paths can execute the same causal graph. Exactness here means mathematical forward equivalence for given parameters and a given trajectory. Different kernels, precision, dropout, and cache implementations still require separate checks.</p>

    <p>With <strong>previous-chunk blending</strong>, the immediately preceding completed chunk also retains its per-layer KV, mixing it with last-layer KV through gates per layer and per KV head:</p>

\[\widetilde K_{c-1}^\ell
=(1-\alpha_\ell)K_{c-1}^\ell+\alpha_\ell K_{c-1}^{L},
\qquad
\widetilde V_{c-1}^\ell
=(1-\alpha_\ell)V_{c-1}^\ell+\alpha_\ell V_{c-1}^{L}.\]

    <p>A newly completed chunk therefore does not immediately discard all per-layer KV. It participates in the blend for the next chunk; only when it becomes older history does it use last-layer KV alone. This remains part of the same state transition as long as rollout and replay use identical blending and transition timing.</p>

    <h2 id="prompt-boundary">The end of a prompt is not a chunk boundary</h2>

    <p>Suppose the chunk size is \(C=512\) and the prompt contains 600 tokens. The first chunk is complete, while the second contains only 88 tokens. Response generation should continue from this partial chunk’s per-layer KV, positions, and offset. The second chunk completes after 424 more tokens are processed, when the total input length reaches 1024.</p>

    <p>The count here is the <strong>number of tokens fed into and fully processed by the model</strong>. The last prompt token’s logits can already be used to sample the first response token. Sampling alone has not yet written that response token into the KV cache.</p>

    <figure class="lastkv-lab" data-lastkv-lab="" data-lang="en">
  <label>Prompt length (tokens)<output data-lastkv-prompt-value="">600</output>
    <input type="range" min="1" max="1536" step="1" value="600" data-lastkv-prompt="" disabled="" />
  </label>
  <div data-lastkv-summary="" aria-live="polite">
    <dl class="lastkv-stats">
      <div><dt>Fully filled chunks</dt><dd>1</dd></div>
      <div><dt>Tokens in current chunk</dt><dd>88 / 512</dd></div>
      <div><dt>Tokens to next boundary</dt><dd>424</dd></div>
      <div><dt>Total tokens at next boundary</dt><dd>1024</dd></div>
    </dl>
    <p class="lastkv-blend-note">The most recent full chunk keeps its per-layer KV for t−1 blending in the next chunk; it does not immediately become pure last-layer KV.</p>
  </div>
  <div data-lastkv-timeline="">
    <div class="lastkv-legend"><span class="lastkv-prompt-key">Prompt</span><span class="lastkv-response-key">Response continuation (schematic)</span></div>
    <div class="lastkv-chunk">
      <div class="lastkv-chunk-label">Chunk 1 · 1–512</div>
      <div class="lastkv-bar" aria-hidden="true"><span class="lastkv-prompt-part" style="width:100%"></span></div>
      <div class="lastkv-chunk-split">Prompt 512 + response continuation 0</div>
    </div>
    <div class="lastkv-chunk lastkv-chunk-active">
      <div class="lastkv-chunk-label">Chunk 2 · 513–1024 · current</div>
      <div class="lastkv-bar" aria-hidden="true"><span class="lastkv-prompt-part" style="width:17.1875%"></span><span class="lastkv-response-part" style="width:82.8125%"></span></div>
      <div class="lastkv-chunk-split">Prompt 88 + response continuation 424</div>
    </div>
    <div class="lastkv-chunk">
      <div class="lastkv-chunk-label">Chunk 3 · 1025–1536</div>
      <div class="lastkv-bar" aria-hidden="true"><span class="lastkv-response-part" style="width:100%"></span></div>
      <div class="lastkv-chunk-split">Prompt 0 + response continuation 512</div>
    </div>
    <div class="lastkv-chunk">
      <div class="lastkv-chunk-label">Chunk 4 · 1537–2048</div>
      <div class="lastkv-bar" aria-hidden="true"><span class="lastkv-response-part" style="width:100%"></span></div>
      <div class="lastkv-chunk-split">Prompt 0 + response continuation 512</div>
    </div>
  </div>
  <figcaption>With fixed C = 512, these bars show positions in the first 2048 tokens. Response color marks future positions, not model outputs. Move the prompt-length slider: chunk boundaries remain fixed at multiples of 512. The default example remains visible with JavaScript disabled.</figcaption>
</figure>

    <p>If an inference service commits the partial chunk early at the end of the prompt, or the trainer starts the response in a new chunk, it changes when feedback becomes visible. This behavior could be defined as another explicit policy, but replay would have to reproduce it. It cannot be treated as a free implementation adjustment under fixed chunk rules.</p>

    <p>Likewise, padding, packed documents, truncation, and minibatch slicing must not silently reset the chunk phase. <strong>Matching the chunk size is only the first step. Absolute boundaries, document resets, RoPE positions, and cache transition rules must also match.</strong></p>

    <h2 id="gradients">Matching probabilities does not guarantee complete gradients</h2>

    <p>Caching rollout latents and reusing them during training may bring states closer. But two questions must be answered separately: whether the current forward uses the correct states, and whether gradients pass through the computation that produced them.</p>

    <p>If \(h_t=F_\theta(h_{t-1},a_t)\), the log-probability derivative generally includes both a direct parameter term and a historical state term:</p>

\[\begin{aligned}
\frac{d\log\pi_\theta}{d\theta}
&amp;=\frac{\partial\log\pi_\theta}{\partial\theta}\\
&amp;\quad+\frac{\partial\log\pi_\theta}{\partial h}
\frac{dh}{d\theta}.
\end{aligned}\]

    <p>Before parameters are updated, reusing exact rollout states may align the forward computation. Detaching those states as constants, however, omits the corresponding recurrent derivatives. Continuing to reuse old latents or KV after a parameter update may also introduce forward discrepancies from stale states.</p>

    <p>LastKV is subject to the same distinction. Detaching the historical KV of completed chunks can leave logits unchanged while removing gradient paths from future losses through the producers of historical KV. Complete recurrent gradients require preserving or recomputing the corresponding computation graph; TBPTT is another approximation that can be chosen explicitly. <strong>Forward policy mismatch and backward gradient truncation are separate problems. A single claim of consistency cannot cover both.</strong></p>

    <h2 id="verification">How to verify consistency, and what this design costs</h2>

    <p>The most direct check does not require a full RL experiment. Fix a checkpoint, record a rollout, and recompute the same trajectory with the actor’s training forward. Disable dropout, align masks, positions, and precision, and use the same comparison convention for temperature and logits processing. Then compare distributions step by step.</p>

    <ol>
      <li><strong>Check logits and full distributions first.</strong> Report per-position log-probability differences or KL. Matching argmax is insufficient to establish consistency. If logits differ only by a common additive constant, softmax probabilities can still be identical.</li>
      <li><strong>Check the ratio before any update.</strong> Use the actual rollout old log-probabilities in the denominator. A denominator recomputed by replay must not conceal execution differences.</li>
      <li><strong>Cover boundaries explicitly.</strong> For prompts of length \(C-1\), \(C\), \(C+1\), and nonintegral numbers of chunks, check the prompt/response transition, consecutive chunk commits, and t−1 blending.</li>
      <li><strong>Check gradients independently.</strong> Use a small complete autograd reference to verify paths across chunks. If gradients are truncated, report the truncation scope explicitly.</li>
    </ol>

    <p>LastKV’s benefits and costs can be compared in the same table:</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Execution rule</th>
            <th>Actor replay</th>
            <th>Relationship to rollout</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Ordinary causal Transformer</td>
            <td>Parallel causal forward over the full sequence</td>
            <td>Can correspond exactly to token-by-token cached decoding</td>
          </tr>
          <tr>
            <td>Higher-layer feedback at every token + finite parallel approximation</td>
            <td>Multi-round refinement over the full sequence</td>
            <td>Generally approximates actual sequential feedback; state differences need checking</td>
          </tr>
          <tr>
            <td>Exact reproduction of the hybrid recurrent trajectory</td>
            <td>Match prefill, then recompute response states sequentially</td>
            <td>Can preserve forward consistency while retaining temporal dependencies</td>
          </tr>
          <tr>
            <td>LastKV with fixed chunk recurrence</td>
            <td>Sequential across chunks, parallel within each chunk</td>
            <td>Can preserve forward consistency under the same chunk rules</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p><strong>LastKV does not obtain full-sequence parallelism and token-by-token deep feedback for free.</strong> An input of length \(N\) still has approximately \(\lceil N/C\rceil\) sequential chunk stages. Larger chunks offer more parallelism while deferring higher-layer feedback longer. Token-by-token sampling itself remains sequential, and inference KV storage does not directly determine training memory when the recurrent graph is retained.</p>

    <p>This article analyzes architecture and execution rules; it reports no new LastKV RL experiments or serving benchmarks. Actual implementations still need to verify cached decoding, replay, and backward paths. The design advantage is <strong>to first define recurrence at chunk granularity, share those rules between rollout and gradient recomputation, and then exploit parallel causal computation within each chunk.</strong></p>

    <h2 id="references">References</h2>

    <div class="drift-references">

      <ol>
        <li><strong>Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior.</strong> <a href="https://arxiv.org/abs/2605.26797v2">arXiv:2605.26797v2</a>. §4.4: prefill and recurrent decoding; §4.6: RL mismatch and rollout state caching.</li>
        <li><strong>Full-bandwidth Transformer.</strong> <a href="https://arxiv.org/abs/2608.08888v2">arXiv:2608.08888v2</a>. §3.2: plain prompt and fused response; §5: latent-feedback replay, approximations, and recurrent gradients.</li>
        <li><strong>T²MLR: Transformer with Temporal Middle-Layer Recurrence.</strong> <a href="https://arxiv.org/abs/2607.15178v2">arXiv:2607.15178v2</a>. §2.4: parallel approximation; Appendices B.2–B.5: RL, Jacobi causal convergence, and prefill.</li>
        <li><strong>Score Centering Stabilizes Off-policy Reinforcement Learning.</strong> <a href="https://arxiv.org/html/2609.20807v1">arXiv:2609.20807v1</a>. Mean score and drift decomposition under a fixed sampling distribution.</li>
        <li><strong>LastKV: Last Layer Key-Value Cache is All You Need.</strong> <a href="/research/lastkv/">Research overview on this site</a>. Chunk and previous-chunk blending rules here follow the LastKV design under discussion.</li>
        <li><a href="/en/blog/ar-drift/">Why Models Drift: From LLM Reinforcement Learning to Autoregressive Video</a>. Background on state distributions, gradient bias, and optimization objectives.</li>
      </ol>

    </div>

    <p class="drift-meta"><a href="/blog/">← All posts</a> · <a href="#policy-identity">Back to the article</a></p>

  </article>
</main>

<script src="/assets/js/lastkv-rl.js" defer=""></script>]]></content><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><summary type="html"><![CDATA[Why can parallel prefill, token-by-token recurrent rollout, and fully parallel policy replay define different policies? A look at LastKV's fixed chunk recurrence through probability and gradient consistency in RL.]]></summary></entry><entry xml:lang="zh-CN"><title type="html">RL 中递归模型的训推一致性：为什么 LastKV 把反馈放在 chunk 边界</title><link href="https://zefancai.com/blog/lastkv-rl/" rel="alternate" type="text/html" title="RL 中递归模型的训推一致性：为什么 LastKV 把反馈放在 chunk 边界" /><published>2026-10-01T00:00:00-07:00</published><updated>2026-10-01T00:00:00-07:00</updated><id>https://zefancai.com/blog/lastkv-rl</id><content type="html" xml:base="https://zefancai.com/blog/lastkv-rl/"><![CDATA[<link rel="stylesheet" href="/assets/css/ar-drift.css" />

<link rel="stylesheet" href="/assets/css/lastkv-rl.css" />

<main id="main" class="drift-article" lang="zh-CN">
  <article class="drift-prose">

    <p class="drift-kicker"><a href="/zh/blog/">中文博客</a> / 研究笔记</p>

    <h1 id="rl--rollout--policy-">RL 中，训练器算的是 rollout 的那个 policy 吗？</h1>

    <p class="drift-deck">从逐 token 的高层反馈，到 LastKV 的固定 chunk recurrence。</p>
    <p class="drift-meta">Zefan Cai · <time datetime="2026-10-01">2026 年 10 月 1 日</time> · 中文研究笔记</p>

    <p>考虑一种很实际的递归语言模型 RL 实现：为了控制延迟，prompt 使用并行 prefill；生成 response 时，当前 token 读取前一个 token 完成计算后的高层状态，逐 token 递归；更新 actor 时，为了保留训练吞吐，又把记录下来的整条序列送进有限轮的全并行 refinement，重算 log-prob。</p>

    <p><strong>这三条路径可能没有在执行同一个 policy。</strong> token 序列和参数相同，不代表反馈状态也相同。rollout 用真实递归得到的状态，训练器用有限轮并行近似得到的状态，二者就可能给同一个 prefix 分配不同的下一 token 概率。</p>

    <p>在上一篇<a href="/blog/ar-drift/">关于 RL 与自回归视频 drift 的文章</a>里，我们区分了条件分布偏移、梯度偏置和优化目标。这篇只讨论 RL 中更具体的一处：<strong>采样与梯度重算是否遵循同一套状态更新规则，以及 LastKV 为什么把跨层反馈固定在 chunk 边界。</strong></p>

    <div class="drift-callout">
      <p>核心判断：<strong>有限轮并行 replay 若不能复现逐 token rollout 的反馈状态，就存在结构性的 policy mismatch。LastKV 把 chunk recurrence 定义为模型本身的行为，使 chunk 内并行重算与逐 token decode 可以精确对应；代价是跨 chunk 仍需顺序执行。</strong></p>
    </div>

    <nav class="drift-toc" aria-label="文章目录">
      <ul>
        <li><a href="#policy-identity">先定义 RL 需要的一致性</a></li>
        <li><a href="#feedback">问题来自反馈状态，不是 teacher forcing</a></li>
        <li><a href="#papers">LRT、FBT、T²MLR 如何讨论这个问题</a></li>
        <li><a href="#policy-gradient">为什么它影响 policy gradient</a></li>
        <li><a href="#lastkv">LastKV：把递归边界写进 policy</a></li>
        <li><a href="#prompt-boundary">Prompt 结束不是 chunk 边界</a></li>
        <li><a href="#gradients">概率一致，还不等于梯度完整</a></li>
        <li><a href="#verification">如何验证，以及这项设计的代价</a></li>
        <li><a href="#references">参考资料</a></li>
      </ul>
    </nav>

    <h2 id="policy-identity">先定义 RL 需要的一致性</h2>

    <p>固定 prompt \(x\)，把记录下来的 response 写成 \(a_{1:T}\)。记 \(\mu_\theta\) 为实际 rollout 路径定义的策略，\(p_\theta\) 为 actor replay 路径重算出的策略。在相同参数下，我们希望每一步满足：</p>

\[p_\theta(a_t\mid x,a_{&lt;t})
=\mu_\theta(a_t\mid x,a_{&lt;t}).\]

    <p>更强的检查是整个下一 token 分布相同，而不仅是碰巧采样到的那个 token 概率相同。这也不要求更新后的参数仍等于旧参数；它要求的是，<strong>对于任意给定的一组参数，两条执行路径表达同一个策略。</strong></p>

    <p>普通 causal Transformer 就是熟悉的例子：已知整条轨迹后，可以在 causal mask 下并行计算所有位置；逐 token decode 则缓存各层历史 KV。理想数值条件下，它们是同一个因果计算图的两种执行顺序。因此，“并行训练、串行生成”本身并不构成问题。</p>

    <p>这里的 rollout 是一个 <strong>hybrid policy</strong>：prompt 经过指定的并行 prefill，response 从该 prefill 留下的状态继续串行递归。要精确重算，就必须复现这段 prefill、切换时的状态，以及后续递归。把整段改成另一个并行 forward 会改变 response 状态；把整段从头改成串行计算，也可能改变 prompt 留下的初始状态。</p>

    <h2 id="feedback">问题来自反馈状态，不是 teacher forcing</h2>

    <p>teacher forcing 只是把已知 token 喂给模型。RL replay 中这些 token 来自模型已经生成的轨迹，而不是另一份标注答案。<strong>知道 token，并不等于知道当前参数下应该产生的递归状态。</strong> LastKV 的 replay 同样可以使用 teacher forcing。</p>

    <p>用一个简化递推说明区别。设串行模型执行：</p>

\[h_t=x_t+\rho h_{t-1},\qquad h_0=0.\]

    <p>并行初始化为 \(h_t^{(0)}=x_t\)，每轮 Jacobi refinement 执行：</p>

\[h_t^{(k)}=x_t+\rho h_{t-1}^{(k-1)},
\qquad h_0^{(k)}=0.\]

    <p>取 \(x_t=1\)、\(\rho=0.5\)，一轮 refinement 与真实串行状态的区别已经可见：</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>位置</th>
            <th>真实串行状态</th>
            <th>初始化后的一轮并行 refinement</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>1</td>
            <td>1</td>
            <td>1</td>
          </tr>
          <tr>
            <td>2</td>
            <td>1.5</td>
            <td>1.5</td>
          </tr>
          <tr>
            <td>3</td>
            <td>1.75</td>
            <td>1.5</td>
          </tr>
          <tr>
            <td>4</td>
            <td>1.875</td>
            <td>1.5</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>这只是说明依赖关系的算例，不是任何论文的模型实验。有限轮 refinement 只传播有限次反馈，串行计算则沿已生成的全部历史持续传播。若 logits 使用这些状态，条件概率就可能不同。足够多轮可以恢复这个严格因果递推，但那已经改变了固定少量并行 passes 的计算预算。</p>

    <p>因此，真正的比较是：<strong>读取前一 token 已完成的递归状态，与读取某个较早 refinement 轮次的状态，是否是同一条依赖链。</strong></p>

    <h2 id="papers">LRT、FBT、T²MLR 如何讨论这个问题</h2>

    <p>这些论文已经研究了相关代价，不能简单地说它们没有意识到 mismatch，也不能把某个本地 adaptation 的规则当成所有原方法的规则。</p>

    <ul>
      <li><strong><a href="https://arxiv.org/html/2605.26797v2#S4.SS6">Latent Recurrent Transformer（LRT）</a></strong> 在 §4.6 直接指出：rollout 的递归状态以自回归方式生成，而策略概率由并行 refinement 重算，这会引入 mismatch。论文用缓存的 rollout 状态初始化重算时的状态 buffer，以减小差异；它宣称的是缓解，而非普遍的精确等价。</li>
      <li><strong><a href="https://arxiv.org/html/2608.08888v2#S5">Full-bandwidth Transformer（FBT）</a></strong> 在 §5 讨论 latent-feedback 轨迹的 replay，区分串行重算、hidden-state replay 与多轮 refinement。它把有限轮 replay 描述为对完整自回归分布的有限视野近似，也指出复用 behavior policy 的 hidden state 可能产生偏差。§5.2 的 RL 方案明确标注为尚未经过实验验证。</li>
      <li><strong><a href="https://arxiv.org/html/2607.15178v2#A2.SS2">T²MLR</a></strong> 在附录 B.2 讨论缓存 rollout 的精确 latent 与构建 BPTT 图。尤其需要注意，<a href="https://arxiv.org/html/2607.15178v2#A2.SS5">附录 B.5</a> 说明主实验使用 <strong>exact sequential prompt prefill</strong>；Jacobi prefill 是以近似误差换取延迟的可选方案。</li>
    </ul>

    <p>所以本文的前提是开头指定的执行组合：<strong>并行 prefill、逐 token 递归生成、有限轮全并行 actor replay。</strong> 它不是“三篇论文都固定使用并行 prefill”的文献结论。近似可能在实测任务上很好，但相近的平均 loss 或 benchmark 分数不证明逐 prefix 的概率相同。</p>

    <h2 id="policy-gradient">为什么它影响 policy gradient</h2>

    <p>如果想优化实际 rollout 策略的期望奖励，目标与相应的 score-function 梯度是：</p>

\[\begin{aligned}
J(\theta)&amp;=\mathbb E_{\tau\sim\mu_\theta}[R(\tau)],\\
\nabla_\theta J
&amp;=\mathbb E_{\tau\sim\mu_\theta}\!\left[\right.\\
&amp;\qquad\left.R(\tau)\nabla_\theta\log\mu_\theta(\tau)\right].
\end{aligned}\]

    <p>这里省略与参数无关的环境项，并假设奖励没有额外的显式参数依赖。若实际更新改用 \(\nabla_\theta\log p_\theta(\tau)\)，一般不能把它等同于上面的梯度。<strong>数据由递归策略生成，求导却经过并行近似策略。</strong></p>

    <p>这个差异甚至可以发生在第一次 optimizer update 之前。若 PPO 风格的概率比分母保存了真正的 rollout old log-prob，分子用 actor replay，那么在 \(\theta=\theta_{\rm old}\) 时：</p>

\[r_t=
\frac{p_{\theta_{\rm old}}(a_t\mid x,a_{&lt;t})}
{\mu_{\theta_{\rm old}}(a_t\mid x,a_{&lt;t})}\]

    <p>也可能偏离 1。这个比值仍然可以是两个分布的合法密度比，但它包含了<strong>执行模式的切换</strong>，不是预期的“同一部署策略更新前后”的比值。若分母也由并行引擎重算，初始 ratio 等于 1 又可能把真正的采样差异藏起来。</p>

    <p>这与前文的 <a href="https://arxiv.org/html/2609.20807v1">Score Centering</a> 讨论相接。在固定 prefix 下，令 \(g=\nabla_\theta\log p_\theta\)，未经中心化的 score 更新可分解为：</p>

\[\mathbb E_\mu[Rg]
=\mathbb E_\mu[R]\,\mathbb E_\mu[g]
+\operatorname{Cov}_\mu(R,g).\]

    <p>当 sampler 与 trainer 不一致时，\(\mathbb E_\mu[g]\) 通常不再为零。中心化可以扣掉这里的平均 score 项，但不自动把剩余项变成 \(\mu_\theta\) 的精确策略梯度。这解释一种偏置来源，并不意味着所有带 baseline、clipping 或组内归一化的 RL 实现都会以同样方式漂移。</p>

    <h2 id="lastkv">LastKV：把递归边界写进 policy</h2>

    <p><a href="/zh/research/lastkv/">LastKV</a> 选择另一套状态更新规则。它不要求每一个 token 的最后层状态立即反馈给下一个 token 的所有低层，而是把这次跨层反馈推迟到固定的 chunk 边界。</p>

    <p>先看不带 blending 的核心规则。对 chunk \(c\) 中的第 \(\ell\) 层，历史由完整 chunk 的最后层 KV 构成，当前 chunk 则使用本层自己的 KV：</p>

\[A_c^\ell=
\operatorname{Attn}\left(
Q_c^\ell,
[K_{&lt;c}^{L};K_c^\ell],
[V_{&lt;c}^{L};V_c^\ell]
\right).\]

    <p>历史完整地位于当前 chunk 之前，当前部分施加 causal mask。<strong>在处理这个 chunk 的过程中，跨 chunk 的历史读取规则保持固定。</strong> 当前 chunk 内也不会因为某个 token 已经过完最后层，就把它升级成所有层立即读取的历史反馈。</p>

    <p>这样，prefill 与 actor replay 都按 chunk 顺序执行，每个 chunk 内并行计算；decode 虽然逐 token 采样，但在 chunk 内增长的是各层自己的局部 KV。对相同的输入 token，causal attention 保证早期位置不受尚未生成的后续位置影响。直到 chunk 完成，才更新它在后续计算中的历史角色。</p>

    <div class="drift-loop-grid">
  <div class="drift-loop">
    <p class="drift-loop-label">Actor replay</p>
    <p class="drift-loop-path">固定历史 → chunk 内并行 causal forward → 完成 chunk → 更新历史 → 下一 chunk</p>
  </div>
  <div class="drift-loop">
    <p class="drift-loop-label">Rollout decode</p>
    <p class="drift-loop-path">同一历史 → chunk 内逐 token 生成并增长本层 KV → 同一边界更新历史 → 下一 chunk</p>
  </div>
</div>

    <p>两条路径可以执行同一个因果图。这里的精确性指给定参数和轨迹时的数学前向等价；不同 kernel、精度、dropout 或 cache 实现仍需单独核验。</p>

    <p>实际使用 <strong>previous-chunk blending</strong> 时，紧邻的完整 chunk 还保留本层 KV，并按每层、每 KV head 的门控与最后层 KV 混合：</p>

\[\widetilde K_{c-1}^\ell
=(1-\alpha_\ell)K_{c-1}^\ell+\alpha_\ell K_{c-1}^{L},
\qquad
\widetilde V_{c-1}^\ell
=(1-\alpha_\ell)V_{c-1}^\ell+\alpha_\ell V_{c-1}^{L}.\]

    <p>所以刚完成的 chunk 不会立即丢掉所有本层 KV：它在下一 chunk 中参与 blend；再成为更老的历史时，才只使用最后层 KV。只要 rollout 与 replay 保持相同的 blending 和切换时机，这也属于同一套状态转移。</p>

    <h2 id="prompt-boundary">Prompt 结束不是 chunk 边界</h2>

    <p>设 chunk size 为 \(C=512\)，prompt 长度为 600。第一个 chunk 已完成，第二个 chunk 只填了 88 个 token。生成 response 时，应继续使用这个 partial chunk 的本层 KV、位置与 offset；再处理 424 个 token，总输入长度达到 1024，才完成第二个 chunk。</p>

    <p>这里按<strong>已送入模型并完成处理的 token 数</strong>计数。最后一个 prompt token 的 logits 已可用于采样第一个 response token；采样动作本身还没有把该 response token 写进 KV。</p>

    <figure class="lastkv-lab" data-lastkv-lab="" data-lang="zh-CN">
  <label>Prompt 长度（token）<output data-lastkv-prompt-value="">600</output>
    <input type="range" min="1" max="1536" step="1" value="600" data-lastkv-prompt="" disabled="" />
  </label>
  <div data-lastkv-summary="" aria-live="polite">
    <dl class="lastkv-stats">
      <div><dt>已填满的 chunk</dt><dd>1</dd></div>
      <div><dt>当前 chunk 已填入</dt><dd>88 / 512</dd></div>
      <div><dt>距下一边界还需</dt><dd>424</dd></div>
      <div><dt>下一边界的总 token 数</dt><dd>1024</dd></div>
    </dl>
    <p class="lastkv-blend-note">刚完成的 chunk 在下一 chunk 中仍保留本层 KV 参与 t−1 blending；它不会立即变成纯最后层 KV。</p>
  </div>
  <div data-lastkv-timeline="">
    <div class="lastkv-legend"><span class="lastkv-prompt-key">Prompt</span><span class="lastkv-response-key">后续 response（示意）</span></div>
    <div class="lastkv-chunk">
      <div class="lastkv-chunk-label">Chunk 1 · 1–512</div>
      <div class="lastkv-bar" aria-hidden="true"><span class="lastkv-prompt-part" style="width:100%"></span></div>
      <div class="lastkv-chunk-split">Prompt 512 + 后续 response 0</div>
    </div>
    <div class="lastkv-chunk lastkv-chunk-active">
      <div class="lastkv-chunk-label">Chunk 2 · 513–1024 · 当前</div>
      <div class="lastkv-bar" aria-hidden="true"><span class="lastkv-prompt-part" style="width:17.1875%"></span><span class="lastkv-response-part" style="width:82.8125%"></span></div>
      <div class="lastkv-chunk-split">Prompt 88 + 后续 response 424</div>
    </div>
    <div class="lastkv-chunk">
      <div class="lastkv-chunk-label">Chunk 3 · 1025–1536</div>
      <div class="lastkv-bar" aria-hidden="true"><span class="lastkv-response-part" style="width:100%"></span></div>
      <div class="lastkv-chunk-split">Prompt 0 + 后续 response 512</div>
    </div>
    <div class="lastkv-chunk">
      <div class="lastkv-chunk-label">Chunk 4 · 1537–2048</div>
      <div class="lastkv-bar" aria-hidden="true"><span class="lastkv-response-part" style="width:100%"></span></div>
      <div class="lastkv-chunk-split">Prompt 0 + 后续 response 512</div>
    </div>
  </div>
  <figcaption>固定 C = 512，展示前 2048 个 token 的位置划分。response 颜色只表示后续位置，不是模型生成结果。拖动 prompt 长度，chunk 边界仍固定在 512 的倍数；关闭 JavaScript 时保留默认示例。</figcaption>
</figure>

    <p>如果推理服务在 prompt 结束处提前提交 partial chunk，或者训练器把 response 从一个新 chunk 开始，便改变了反馈的可见时机。这样的行为可以被定义成另一套明确的 policy，但 replay 必须复现它；不能把它当成固定 chunk 规则下免费的实现调整。</p>

    <p>同理，padding、packed documents、截断和 minibatch 切片也不能悄悄重设 chunk 的相位。<strong>匹配 chunk size 只是第一步；绝对边界、文档 reset、RoPE 位置与缓存切换规则也要匹配。</strong></p>

    <h2 id="gradients">概率一致，还不等于梯度完整</h2>

    <p>缓存 rollout 的 latent，再在训练时复用，看起来可以让状态更接近。但需要分开回答两个问题：当前 forward 使用的状态是否正确，以及梯度是否经过产生这些状态的计算。</p>

    <p>若 \(h_t=F_\theta(h_{t-1},a_t)\)，log-prob 的导数一般包括直接参数项与历史状态项：</p>

\[\begin{aligned}
\frac{d\log\pi_\theta}{d\theta}
&amp;=\frac{\partial\log\pi_\theta}{\partial\theta}\\
&amp;\quad+\frac{\partial\log\pi_\theta}{\partial h}
\frac{dh}{d\theta}.
\end{aligned}\]

    <p>在尚未更新参数时，复用精确 rollout 状态可能让 forward 对上；把状态当常量 detach，却会省去相应的递归导数。更新参数后继续复用旧 latent 或 KV，还可能引入状态陈旧导致的前向差异。</p>

    <p>LastKV 也不例外。detach 完整 chunk 的历史 KV 可以保持 logits 不变，同时删掉未来损失经过历史 KV producer 的梯度路径。要获得完整的递归梯度，需要保留或重算相应计算图；TBPTT 是另一个可以明确选择的近似。<strong>前向 policy mismatch 与反向梯度截断是两类问题，不能用同一个“已经一致”概括。</strong></p>

    <h2 id="verification">如何验证，以及这项设计的代价</h2>

    <p>最直接的检查不需要先跑完整 RL 实验：固定 checkpoint，记录一条 rollout，然后用 actor 的训练 forward 重算同一轨迹。关闭 dropout，统一 mask、位置、精度及 temperature / logits processing 的比较口径，逐步比较分布。</p>

    <ol>
      <li><strong>先查 logits 与完整分布。</strong> 报告逐位置的 log-prob 差异或 KL；相同 argmax 不足以证明一致。若 logits 只差一个公共常数，softmax 概率仍可以完全相同。</li>
      <li><strong>再查未更新时的 ratio。</strong> 使用真实 rollout old log-prob 作分母；不要让 replay 重算的分母掩盖执行差异。</li>
      <li><strong>专门覆盖边界。</strong> 对 \(C-1\)、\(C\)、\(C+1\) 和非整数 chunk 的 prompt，检查 prompt/response 切换、连续多次 chunk 提交及 t−1 blending。</li>
      <li><strong>独立检查梯度。</strong> 用小规模完整 autograd reference，验证跨 chunk 路径；若使用截断，就明确报告截断范围。</li>
    </ol>

    <p>LastKV 的收益与代价可以放在同一张表里：</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>执行规则</th>
            <th>Actor replay</th>
            <th>与 rollout 的关系</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>普通 causal Transformer</td>
            <td>整段并行 causal forward</td>
            <td>可精确对应逐 token cached decode</td>
          </tr>
          <tr>
            <td>逐 token 高层反馈 + 有限轮并行近似</td>
            <td>全序列、多轮 refinement</td>
            <td>一般近似真实串行反馈；需检查状态差异</td>
          </tr>
          <tr>
            <td>精确复现 hybrid 递归轨迹</td>
            <td>匹配 prefill，再串行重算 response 状态</td>
            <td>可保持前向一致，但保留时间依赖</td>
          </tr>
          <tr>
            <td>LastKV 固定 chunk recurrence</td>
            <td>跨 chunk 顺序、chunk 内并行</td>
            <td>同一 chunk 规则下可保持前向一致</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p><strong>LastKV 没有获得“整段全并行，又保留逐 token 深层反馈”的免费组合。</strong> 长度为 \(N\) 的输入仍有约 \(\lceil N/C\rceil\) 个顺序 chunk 阶段；更大的 chunk 提供更多并行性，也把高层反馈推迟得更久。逐 token 采样本身仍然串行，保留递归图的训练内存也不能由推理 KV 存储量直接推断。</p>

    <p>本文给出的是架构与执行规则的分析，不报告新的 LastKV RL 实验或 serving benchmark。实际实现仍需验证 cached decode、replay 与反向路径。它的设计优势是：<strong>先把递归粒度定义成 chunk，让 rollout 与梯度重算共享这套规则，再利用 chunk 内因果计算的并行性。</strong></p>

    <h2 id="references">参考资料</h2>

    <div class="drift-references">

      <ol>
        <li><strong>Latent Recurrent Transformer: Architecture Exploration, Training Strategies, and Scaling Behavior.</strong> <a href="https://arxiv.org/abs/2605.26797v2">arXiv:2605.26797v2</a>。§4.4：prefill 与递归 decode；§4.6：RL mismatch 与 rollout state cache。</li>
        <li><strong>Full-bandwidth Transformer.</strong> <a href="https://arxiv.org/abs/2608.08888v2">arXiv:2608.08888v2</a>。§3.2：plain prompt 与 fused response；§5：latent-feedback replay、近似与递归梯度。</li>
        <li><strong>T²MLR: Transformer with Temporal Middle-Layer Recurrence.</strong> <a href="https://arxiv.org/abs/2607.15178v2">arXiv:2607.15178v2</a>。§2.4：并行近似；附录 B.2–B.5：RL、Jacobi 因果收敛与 prefill。</li>
        <li><strong>Score Centering Stabilizes Off-policy Reinforcement Learning.</strong> <a href="https://arxiv.org/html/2609.20807v1">arXiv:2609.20807v1</a>。固定采样分布下的 score 均值与 drift 分解。</li>
        <li><strong>LastKV: Last Layer Key-Value Cache is All You Need.</strong> <a href="/zh/research/lastkv/">本站研究介绍</a>。本文的 chunk 与 previous-chunk blending 规则按所讨论的 LastKV 设计定义。</li>
        <li><a href="/blog/ar-drift/">模型为什么越走越偏：从 LLM 强化学习到自回归视频的 Drift</a>。关于状态分布、梯度偏置与优化目标的背景讨论。</li>
      </ol>

    </div>

    <p class="drift-meta"><a href="/zh/blog/">← 所有博客</a> · <a href="#policy-identity">回到正文</a></p>

  </article>
</main>

<script src="/assets/js/lastkv-rl.js" defer=""></script>]]></content><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><summary type="html"><![CDATA[并行 prefill、逐 token 递归 rollout、全并行 policy replay 为什么可能对应不同的 policy？从 RL 的概率与梯度一致性出发，解释 LastKV 固定 chunk recurrence 的优势与代价。]]></summary></entry><entry xml:lang="en"><title type="html">Why Models Drift: From LLM Reinforcement Learning to Autoregressive Video</title><link href="https://zefancai.com/en/blog/ar-drift/" rel="alternate" type="text/html" title="Why Models Drift: From LLM Reinforcement Learning to Autoregressive Video" /><published>2026-09-26T00:00:00-07:00</published><updated>2026-09-26T00:00:00-07:00</updated><id>https://zefancai.com/en/blog/ar-drift-en</id><content type="html" xml:base="https://zefancai.com/en/blog/ar-drift/"><![CDATA[<link rel="stylesheet" href="/assets/css/ar-drift.css" />

<main id="main" class="drift-article" lang="en">
  <article class="drift-prose">

    <p class="drift-kicker"><a href="/blog/">Blog</a> / Research notes</p>

    <h1 id="why-models-drift">Why Models Drift</h1>

    <p class="drift-deck">A shared perspective on drift, from LLM reinforcement learning to autoregressive video.</p>
    <p class="drift-meta">Zefan Cai · <time datetime="2026-09-26">September 26, 2026</time> · Research notes</p>

    <p>A reasoning model suddenly collapses during reinforcement learning. An autoregressive video looks less and less like its opening frames. A quantized KV cache lets historical frames “steal” attention from the current frame. These seem like separate problems: optimization instability, temporal consistency, and numerical precision. Yet they share a structure worth studying: <strong>the model helps construct the distribution it will encounter next, while the training or update rule does not always make correct assumptions about that distribution.</strong></p>

    <p>This article brings together <a href="https://arxiv.org/html/2609.20807v1">Score Centering</a>, <a href="https://arxiv.org/html/2609.15987v1">Bellman Policy Optimization (BPO)</a>, and <a href="https://arxiv.org/html/2602.02710v3">Maximum Likelihood Reinforcement Learning (MaxRL)</a> with several approaches to video generation. The aim is not to give them all the same label, but to answer a more specific question: where does bias enter the feedback loop, and where should we intervene?</p>

    <div class="drift-callout">
      <p>The central distinction: <strong>Score Centering corrects systematic bias in updates; Self Forcing changes the histories encountered during training; BPO rewrites the policy optimization objective; MaxRL changes the learning weight assigned to successful trajectories.</strong> These methods can complement one another, but they are not interchangeable.</p>
    </div>

    <nav class="drift-toc" aria-label="Table of contents">
      <ul>
        <li><a href="#two-clocks">Two kinds of drift, two clocks</a></li>
        <li><a href="#distribution">Locating mismatch at different levels of the distribution</a></li>
        <li><a href="#score-centering">Score Centering: removing an update that should not exist</a></li>
        <li><a href="#bpo">BPO: deriving a local loss from trajectory relations</a></li>
        <li><a href="#maxrl">MaxRL: how successful samples should be counted</a></li>
        <li><a href="#video-loop">Back to AR video: bringing the actual feedback loop into training</a></li>
        <li><a href="#diagnosis">Diagnosing a new system through this lens</a></li>
        <li><a href="#references">References</a></li>
      </ul>
    </nav>

    <h2 id="two-clocks">Two kinds of drift, two clocks</h2>

    <p>First, we need to clarify what advances when a model “drifts further.”</p>

    <p>In LLM reinforcement learning, the trainer produces parameters, an inference engine uses those parameters to generate rollouts, and the trainer updates the parameters from those rollouts. Quantization, different numerical kernels, asynchronous sampling, and stale checkpoints can all cause the inference engine and trainer to assign different probabilities to the same token. Once bias enters the update, the new parameters are sent back to the sampler, creating feedback across training steps.</p>

    <p>AR video has another feedback chain: the model generates the next chunk from its history, adds that chunk to the history, and uses it to generate what follows. Even with completely fixed weights, a small error in a person’s shape or a slight shift in geometry can enter the conditioning history and continue to propagate.</p>

    <div class="drift-loop-grid">
  <div class="drift-loop">
    <p class="drift-loop-label">LLM RL · Training step k</p>
    <p class="drift-loop-path">Parameters θ<sub>k</sub> → sampler → rollouts → gradient → parameters θ<sub>k+1</sub></p>
    <p>The evolving state includes model parameters; biased updates are repeatedly fed back.</p>
  </div>
  <div class="drift-loop">
    <p class="drift-loop-label">AR video · Generation step t</p>
    <p class="drift-loop-path">History h<sub>t</sub> → next video chunk → append to history → h<sub>t+1</sub></p>
    <p>The parameters can remain fixed; what changes is the history the model reads next.</p>
  </div>
</div>

    <p>Both systems have feedback loops, but different things accumulate. In Score Centering, drift is a precisely defined term in a gradient decomposition. In video, drift usually refers to the degradation of identity, geometry, motion, or scene structure over time. Reducing the latter entirely to the former would obscure where changes are actually needed.</p>

    <p>A useful local analytical perspective is:</p>

\[e_{n+1}\approx J_n e_n+b_n+\xi_n.\]

    <p>Here, \(e_n\) is a perturbation in a chosen representation, \(J_n\) describes how it propagates, \(b_n\) is systematic bias, and \(\xi_n\) is random disturbance. This is a local approximation for understanding mechanisms, <strong>not a drift theorem shared by all generative models</strong>. Bias does not necessarily make a system diverge; whether the propagation operator amplifies errors matters too.</p>

    <p>The simple model below isolates this distinction. It has just one scalar state, starting at \(e_0=0.2\). The solid line follows \(e_{n+1}=a e_n+b\), while the dashed line removes only \(b\) and retains exactly the same propagation coefficient \(a\).</p>

    <figure class="drift-lab" id="drift-lab">
  <div class="drift-controls">
    <label for="drift-gain">Error propagation coefficient a <output id="drift-gain-value" for="drift-gain">1.00</output>
      <input type="range" id="drift-gain" min="0.8" max="1.1" step="0.01" value="1" />
    </label>
    <label for="drift-bias">Systematic bias per step b <output id="drift-bias-value" for="drift-bias">0.025</output>
      <input type="range" id="drift-bias" min="0" max="0.1" step="0.005" value="0.025" />
    </label>
  </div>
  <svg id="drift-chart" viewBox="0 0 720 280" role="img" aria-labelledby="drift-chart-title drift-chart-desc">
    <title id="drift-chart-title">Recurrence with and without systematic bias</title>
    <desc id="drift-chart-desc">The initial error is 0.2. With the defaults a = 1 and b = 0.025, after 40 steps the biased recurrence reaches 1.2 and the bias-corrected recurrence stays at 0.2.</desc>
    <g data-drift-fallback="">
      <path d="M60 40V230H690" fill="none" stroke="#a5adae" />
      <path d="M60 202L690 63" fill="none" stroke="#ad5436" stroke-width="3" />
      <path d="M60 202H690" fill="none" stroke="#147a75" stroke-width="3" stroke-dasharray="8 5" />
      <text x="70" y="30" fill="#ad5436" font-size="15">With bias: 0.2 → 1.2</text>
      <text x="390" y="30" fill="#147a75" font-size="15">Bias removed: 0.2 → 0.2</text>
      <text x="58" y="255" font-size="14">0</text><text x="645" y="255" font-size="14">40 steps</text>
    </g>
  </svg>
  <p id="drift-lab-summary" aria-live="polite">With the default parameters, the errors after 40 steps are 1.2 and 0.2, respectively.</p>
  <figcaption>An interactive illustration, not a paper experiment. Removing systematic bias does not automatically remove the initial error or guarantee stability when the propagation coefficient exceeds 1. The default curves remain readable with JavaScript disabled.</figcaption>
</figure>

    <p>This example explains why “correcting bias” and “improving robustness in a feedback loop” are different tasks. The former addresses the newly injected \(b_n\); the latter must also consider how the system continues operating when errors already exist.</p>

    <h2 id="distribution">Locating mismatch at different levels of the distribution</h2>

    <p>For an input \(x\), a generated history \(h_t\), and the next generated unit \(a_t\), we can write the actual sampling process as:</p>

\[x\sim\rho,\qquad h_t\sim d_t^q(\cdot\mid x),\qquad
a_t\sim q(\cdot\mid h_t).\]

    <p>\(\rho\) is the prompt distribution, \(d_t^q\) is the distribution of histories visited by the sampler at step \(t\), and \(q\) is the conditional sampling distribution given a history. Let \(p_\theta\) denote the probabilities the trainer uses to compute gradients. “Mismatch between training and actual operation” can therefore occur in at least three places.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Level</th>
            <th>What should we ask?</th>
            <th>Representative interventions</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Conditional distribution at a fixed history</td>
            <td>At the same \(h\), do the sampler’s \(q\) and the trainer’s \(p_\theta\) agree?</td>
            <td>Score Centering; bias correction for computational operators</td>
          </tr>
          <tr>
            <td>Distribution of visited histories</td>
            <td>Does the training distribution \(d_t\) cover the histories the model actually generates?</td>
            <td>Self Forcing; training on actual rollouts</td>
          </tr>
          <tr>
            <td>Weights on inputs and successful trajectories</td>
            <td>Which prompts and complete outputs receive more learning signal?</td>
            <td>MaxRL; success-conditioned or reward-weighted fitting</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>BPO asks another question that cuts across this table: <strong>which overall policy objective does the local loss actually approximate?</strong> Sometimes the difficulty comes from a distribution mismatch, sometimes from the design of a surrogate objective, and sometimes from both.</p>

    <p>One easily confused point is that the sum of teacher-forced token NLLs is already the exact sequence NLL:</p>

\[-\log p_\theta(a_{1:T}\mid x)
=-\sum_t\log p_\theta(a_t\mid x,a_{&lt;t}).\]

    <p>So the problem is not that “token losses inherently cannot see a whole trajectory.” It is that, when a model has finite error, learning conditional behavior on real prefixes does not mean it can also recover from the off-distribution prefixes it generates itself. The video distribution-matching surrogate objectives discussed here also cannot simply be equated with maximum likelihood on real data.</p>

    <h2 id="score-centering">Score Centering: removing an update that should not exist</h2>

    <p><a href="https://arxiv.org/html/2609.20807v1">Score Centering</a> starts from a powerful test: if every output receives the same positive reward, should the model keep learning? Under standard on-policy policy gradients, the answer is no, because the environment supplies no signal distinguishing better outputs from worse ones.</p>

    <p>Fix a history \(h\) and define the parameter score:</p>

\[g_\theta(a,h)=\nabla_\theta\log p_\theta(a\mid h).\]

    <p>Under the usual differentiability and normalization conditions, \(\mathbb E_{p_\theta}[g_\theta\mid h]=0\). But when tokens are actually sampled from \(q\), \(\mathbb E_q[g_\theta\mid h]\) can generally be nonzero. Taking the conditional expectation over the current action and its subsequent rollout, the update decomposes into:</p>

\[\mathbb E_q[Rg_\theta\mid h]
=\underbrace{\mathbb E_q[R\mid h]\,
\mathbb E_q[g_\theta\mid h]}_{\text{drift}}
+\underbrace{\operatorname{Cov}_q(R,g_\theta\mid h)}_{\text{reward-related signal}}.\]

    <p>The first term depends on reward only through its mean; it does not care which action leads to a better outcome. For a positive conditional mean reward, it pushes the trainer toward the sampler. If the sampler is a fixed teacher, this resembles ordinary distillation. If it is a quantized or stale copy of the trainer, the trainer follows it each round and sends its updated parameters back, allowing bias to accumulate around the loop.</p>

    <p>Score Centering directly replaces:</p>

\[g_\theta(a,h)\longrightarrow
g_\theta(a,h)-\mathbb E_q[g_\theta\mid h].\]

    <p>With the full conditional expectation, the drift term is removed exactly, leaving \(\operatorname{Cov}_q(R,g_\theta\mid h)\). <strong>But this is still a covariance under \(q\), not a full restoration of the on-policy update under \(p_\theta\).</strong> The paper’s practical implementation further approximates this expectation using the sampler’s top-\(k\) probabilities and the trainer’s tail distribution.</p>

    <p>This also explains why centering rewards or advantages within a prompt is insufficient. Having advantages sum to zero across a whole rollout group does not mean the conditional mean advantage is zero at every intermediate prefix. A prefix already close to a correct answer and a prefix that has already gone wrong generally have different conditional means.</p>

    <h3 id="video-counterpart-attention-bias-after-kv-quantization">Video counterpart: attention bias after KV quantization</h3>

    <p>In <a href="https://arxiv.org/html/2605.26266v1">Quantized Keys Steal Attention</a>, historical keys in AR video are compressed to low precision, while keys in the current chunk retain full precision. Even if quantization error is approximately zero-mean in attention logits, the exponential inside softmax changes the mean:</p>

\[\hat\ell_i=\ell_i+\delta_i,\quad \mathbb E[\delta_i]=0,
\qquad
\mathbb E[e^{\hat\ell_i}]
=e^{\ell_i}\mathbb E[e^{\delta_i}]
\ge e^{\ell_i}.\]

    <p>The unnormalized contribution of historical keys is systematically inflated, shifting attention toward cached tokens. The paper derives an additive correction for each attention score:</p>

\[b_i=\log\mathbb E[e^{\delta_i}],\qquad
\tilde\ell_i=\hat\ell_i-b_i,
\qquad b_i\approx\tfrac12\operatorname{Var}(\delta_i).\]

    <p>When the noise model holds and the exact \(b_i\) is used, \(\mathbb E[e^{\tilde\ell_i}]=e^{\ell_i}\). The practical method uses a second-order approximation and requires no retraining. The paper validates it on MAGI-1, SkyReels-V2, and HY-WorldPlay.</p>

    <p>The closest connection to Score Centering is this causal path: <strong>efficient computation introduces bias → analyze the expected bias → compensate with an additive term, without requiring efficient computation to be abandoned.</strong> Here, however, the restored quantity is the expectation of the unnormalized attention contribution. This does not directly imply that normalized softmax is fully unbiased, much less that arbitrarily long videos will no longer drift.</p>

    <p class="drift-note">Three meanings of “score” are easy to confuse here: in RL, the parameter score is ∇<sub>θ</sub> log p; an attention score is a logit before softmax; in diffusion models, the score is usually ∇<sub>x</sub> log p. Sharing a name does not make their formulas interchangeable.</p>

    <h2 id="bpo">BPO: deriving a local loss from trajectory relations</h2>

    <p><a href="https://arxiv.org/html/2609.15987v1">BPO</a> begins with Policy Mirror Descent (PMD), rather than replacing a coefficient in an existing GRPO loss. For a fixed rollout policy \(\mu\), PMD balances advantage against the KL cost of deviating from the old policy at each state:</p>

\[\max_\pi\;
\mathbb E_{a\sim\pi(\cdot\mid h)}[A^\mu(h,a)]
-\frac{1}{\eta}D_{\mathrm{KL}}(\pi\Vert\mu).\]

    <p>Its ideal solution satisfies \(\pi^+(a\mid h)\propto\mu(a\mid h)e^{\eta A^\mu(h,a)}\). The difficulty is that directly using the advantage of every prefix in language generation usually requires estimating values for many intermediate states.</p>

    <p>BPO uses the fact that appending a token is deterministic and that reward is given only at the endpoint. Following the paper’s convention, write the terminal reward as a terminal value, \(V^\mu(h_{T+1})=R\). Then:</p>

\[A^\mu(h_t,a_t)=V^\mu(h_{t+1})-V^\mu(h_t),
\qquad
\sum_tA^\mu(h_t,a_t)=R-V^\mu(x).\]

    <p>The intermediate values cancel along the path, leaving only the terminal reward and the expected reward of the initial prompt. The latter still needs to be estimated, but grouped rollouts for the same prompt can do this without separately training a critic for intermediate states.</p>

    <h3 id="a-direct-connection-between-bpo-and-score-centering">A direct connection between BPO and Score Centering</h3>

    <p>Define the per-token term in BPO’s full objective:</p>

\[u_\theta(a,h)=
\log\frac{p_\theta(a\mid h)}{\mu(a\mid h)}
+D_{\mathrm{KL}}\!\left(\mu(\cdot\mid h)\Vert p_\theta(\cdot\mid h)\right).\]

    <p>When \(p_\theta=\pi^+\), PMD’s optimality condition gives the trajectory relation \(\sum_tu_\theta=\eta(R-V^\mu(x))\). BPO therefore constructs a squared-residual objective:</p>

\[\delta=\eta(R-V^\mu(x))-\sum_tu_\theta(a_t,h_t),
\qquad
L=\mathbb E_{x\sim\rho,\,\tau\sim P_\mu(\cdot\mid x)}
\left[\frac{\phi(x)}{2\eta}\delta^2\right],\quad\phi(x)&gt;0.\]

    <p>Within the policy space and reachable states assumed in the paper, the full objective has the same optimum as PMD. This is a statement about the objective, not a guarantee that any neural network and optimizer will reach that solution.</p>

    <p>The equations above also yield an interesting connection directly. Differentiating while holding \(\mu\) fixed:</p>

\[\nabla_\theta u_\theta(a,h)
=g_\theta(a,h)-\mathbb E_{b\sim\mu}[g_\theta(b,h)].\]

    <p><strong>The right-hand side is exactly score centering with respect to the actual sampling distribution.</strong> This is an algebraic observation obtained from the equations in the two papers: the full KL term that BPO derives from an overall policy objective contains SC’s centering structure in its gradient. It does not mean the two algorithms have identical objectives and updates in every respect.</p>

    <p>Practical BPO additionally applies residual linearization, group estimation, a binary KL approximation, smoothing, and clipping, ultimately using a complementary-probability weight:</p>

\[\omega_t=
\frac{1+\epsilon-\mu(a_t\mid h_t)}
{1+\epsilon-p_\theta(a_t\mid h_t)}.\]

    <p>The practical token loss therefore does not automatically inherit the exact zero-mean property of full-KL centering. This step is a particularly useful reminder: <strong>knowing the final loss is not enough; we must also know which conditions were discarded on the way from the overall objective.</strong></p>

    <h3 id="video-counterpart-analytical-elimination-in-flow-dpo">Video counterpart: analytical elimination in Flow-DPO</h3>

    <p>Flow-DPO in <a href="https://arxiv.org/html/2501.13918v2">Improving Video Generation with Human Feedback</a> likewise starts from an objective over the complete generation distribution. For a video \(v\) and condition \(c\), the ideal solution to KL-regularized reward maximization has the form:</p>

\[p^*(v\mid c)=\frac{p_{\rm ref}(v\mid c)e^{R(v,c)/\beta}}{Z(c)}.\]

    <p>Rearranging this relation expresses reward in terms of a ratio of generation probabilities. For preferred and rejected videos under the same prompt, the reward difference cancels \(\log Z(c)\), yielding a preference optimization formulation that does not require an explicit reward model in that training objective. A diffusion/flow likelihood surrogate then turns this into training on denoising or velocity prediction errors.</p>

    <p>The methodological parallel is clear:</p>

    <div class="drift-loop-grid">
  <div class="drift-loop">
    <p class="drift-loop-label">BPO</p>
    <p class="drift-loop-path">PMD → Bellman elimination of intermediate values → trajectory residual → token loss</p>
  </div>
  <div class="drift-loop">
    <p class="drift-loop-label">Flow-DPO</p>
    <p class="drift-loop-path">KL-regularized reward objective → eliminate reward and normalizer → video preferences → velocity loss</p>
  </div>
</div>

    <p>This is a correspondence between derivation patterns. Flow-DPO does not use BPO’s Bellman elimination or its complementary-probability weights. Practical Flow-DPO also replaces the time-dependent \(\beta_t\) from its derivation with a constant to improve training. Here, time means <strong>denoising time</strong>, not the video frame index.</p>

    <p>To actually transfer BPO to video, one cannot simply copy \((1-\mu)/(1-p)\): it comes from binary KL on discrete token probabilities. The probability density of a continuous latent can exceed 1, so “one minus the density” is no longer the probability of a complementary event. A new derivation is needed for the specific video states, actions, and sampler.</p>

    <h2 id="maxrl">MaxRL: how successful samples should be counted</h2>

    <p><a href="https://arxiv.org/html/2602.02710v3">MaxRL</a> addresses another kind of mismatch: average success probability and the log-likelihood of success are different objectives. Let \(s_\theta(x)\) be the success probability for prompt \(x\). Then:</p>

\[J_{\rm RL}=\mathbb E_{x\sim\rho}[s_\theta(x)],
\qquad
J_{\rm ML}=\mathbb E_{x\sim\rho}[\log s_\theta(x)].\]

    <p>Both encourage success, but allocate gradients differently. The latter weights by \(1/s_\theta(x)\): in the gradient expression, problems with rare successes receive greater relative weight. This changes the allocation of learning signal; it does not correct probability differences between sampler and trainer.</p>

    <p>Let \(\tau\) be a complete trajectory and let success be determined by a binary criterion. Under the required differentiability and support conditions:</p>

\[\nabla_\theta\log s_\theta(x)
=\mathbb E_{\tau\sim P_\theta(\cdot\mid x,\mathrm{success})}
[\nabla_\theta\log P_\theta(\tau\mid x)].\]

    <p>This gives maximum likelihood a simple interpretation: average the scores of successful trajectories. For a prompt, draw independent samples from the current \(P_\theta(\cdot\mid x)\): \(N\) trajectories in total, of which \(K\) succeed. We can construct an estimator that averages only successful trajectories:</p>

\[\hat g_N(x)=
\begin{cases}
\frac{1}{K}\sum_{i=1}^{N}r_i\nabla_\theta\log P_\theta(\tau_i\mid x),&amp;K&gt;0,\\
0,&amp;K=0.
\end{cases}\]

    <p>With finite sampling, success will not be observed for every prompt, so this is not an unbiased estimator of the exact ML gradient. MaxRL’s key result is that its expectation corresponds exactly to a truncated objective:</p>

\[J_N(x)=-\sum_{j=1}^{N}\frac{(1-s_\theta(x))^j}{j},
\qquad
\nabla J_N(x)=\sum_{j=1}^{N}\frac1j\nabla\operatorname{pass@}j(x).\]

    <p>At \(N=1\), this reduces to the standard expected-reward objective, up to a constant; as \(N\to\infty\), it approaches log-likelihood. Increasing the rollout count therefore does more than reduce estimation variance for a fixed objective: it changes the objective being approximated. This discussion concerns the estimator that averages only successes; practical implementations with a baseline must be analyzed according to their specific form.</p>

    <h3 id="video-counterpart-lifts-success-filtering-variant">Video counterpart: LiFT’s success-filtering variant</h3>

    <p>The main method in <a href="https://arxiv.org/html/2412.04814v3">LiFT</a> is reward-weighted learning: generated videos are scored by a learned evaluation model, higher-reward videos receive larger training weights, and a training loss on real videos is mixed in. It also studies a rejection sampling variant: keep only videos rated Good on all three dimensions—semantic consistency, motion smoothness, and video fidelity—and fine-tune on those samples.</p>

    <p>“Fitting the generation distribution conditioned on success” is close to MaxRL, but the way samples are normalized changes the result. Suppose two prompts each produce 10 videos: A has 8 successes and B has 1.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>How are these 9 successful videos used?</th>
            <th>Prompt A</th>
            <th>Prompt B</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Pool all successes and weight every video equally</td>
            <td>Contributes 8 training samples</td>
            <td>Contributes 1 training sample</td>
          </tr>
          <tr>
            <td>Average successful trajectories within each prompt, then average across prompts</td>
            <td>First average its 8 successes</td>
            <td>First average its 1 success</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>The second option is the key structure of MaxRL’s estimator: a prompt with more successes does not automatically receive a larger total coefficient simply because it contributes more retained samples. This does not guarantee equal gradient norms for the two prompts; a prompt with \(K=0\) still provides no successful learning signal in this estimator.</p>

    <p>Thus, “filter good videos and train again” is not automatically a video version of MaxRL. A closer implementation requires online sampling, success normalization within each prompt, and an explicit terminal success criterion. If a diffusion video model uses a denoising loss instead of exact generation log-likelihood, that introduces another surrogate objective. LiFT’s existing results do not establish this equivalence to MaxRL.</p>

    <h2 id="video-loop">Back to AR video: bringing the actual feedback loop into training</h2>

    <p>The three pairings above do not yet directly solve the most common history-distribution problem in AR video. During training, history comes from real videos; during generation, it comes from the model itself. The same parameters can behave very differently under those two conditions.</p>

    <p><a href="https://arxiv.org/html/2506.08009v2">Self Forcing</a> intervenes most directly: training itself performs autoregressive self-rollouts, making each later chunk depend on previously generated content, and then trains with video-level distribution-matching objectives such as DMD, SiD, or GAN losses. The training objective thus sees the video distribution produced by the actual generation process, rather than only the outputs of local predictions conditioned on real histories.</p>

    <p>The related <a href="https://arxiv.org/html/2407.01392v1">Diffusion Forcing</a> trains with independent noise levels for each token or frame, helping the model adapt to histories of varying reliability. This provides robustness, but adding noise to real histories does not automatically sample the erroneous histories that the model will actually produce.</p>

    <p>Self Forcing also does not guarantee that arbitrarily long generation will avoid degradation. Its practical implementation controls cost through gradient truncation, randomly ending denoising early, and detaching gradients through the KV cache. Quality may still decline beyond the training context length. It directly narrows exposure bias without eliminating every numerical bias and every long-horizon error.</p>

    <h3 id="stability-must-also-serve-the-right-objective">Stability must also serve the right objective</h3>

    <p><a href="https://arxiv.org/html/2512.04678v2">Reward Forcing</a> provides a more concrete example: retaining initial frames as a long-term attention sink helps maintain a historical reference, but can also make the model repeatedly copy the initial content, weakening motion or even causing visual flashbacks. It uses EMA-Sink to update this part of the history representation and Re-DMD to give self-rollouts with better motion quality greater weight in distribution matching.</p>

    <p>This shows why “more stable” and “better video” must also be distinguished. A nearly motionless video may find it easy to maintain a consistent appearance. Reward Forcing’s reward-weighting mechanism is a broad neighbor of MaxRL, but it does not optimize the same \(\log P(\mathrm{success}\mid c)\) objective or use the same success normalization within each prompt.</p>

    <p><a href="https://arxiv.org/html/2603.17461v2">AR-CoPO</a> instead focuses on the relationship between exploration and the actual sampler. For few-step AR video, switching to an SDE with intermediate noise injection to make an RL loss easier to compute may move training exploration away from the actual ODE generation process. AR-CoPO perturbs initial noise at a selected pivot chunk, branches into candidate videos, and then makes local updates through a contrastive surrogate policy. It shares BPO’s motivation to reconsider how a policy loss should be constructed, but does not include its Bellman elimination.</p>

    <h2 id="diagnosis">Diagnosing a new system through this lens</h2>

    <p>The most useful outcome of bringing these methods together is not a unified algorithm name, but a map of where to intervene.</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>Observed problem</th>
            <th>What to locate first</th>
            <th>Related methods</th>
            <th>What this does not establish</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>Sampling and training probabilities differ on the same prefix; updates keep drifting</td>
            <td>The conditional distribution at a fixed history and its mean score</td>
            <td>Score Centering; BPO’s full KL structure</td>
            <td>Full recovery of the on-policy gradient</td>
          </tr>
          <tr>
            <td>Quantizing historical KV shifts attention toward the past</td>
            <td>How nonlinear computation turns noise into bias</td>
            <td>Quantized Keys Steal Attention</td>
            <td>Removal of all history errors or long-video drift</td>
          </tr>
          <tr>
            <td>The first chunks look normal, but quality degrades when relying on generated history</td>
            <td>Visitation distributions of real and generated histories</td>
            <td>Self Forcing; Diffusion Forcing</td>
            <td>Indefinite stability beyond the training length</td>
          </tr>
          <tr>
            <td>Overall reward improves little, and the local policy loss is difficult to explain</td>
            <td>The derivation from the overall objective to the surrogate loss</td>
            <td>BPO; Flow-DPO; AR-CoPO</td>
            <td>Mathematical equivalence among the three</td>
          </tr>
          <tr>
            <td>Easy scenarios succeed repeatedly, while hard scenarios lack usable candidates</td>
            <td>Learning weights on prompts and successful trajectories</td>
            <td>MaxRL; LiFT’s filtering variant; Reward Forcing</td>
            <td>Correction of sampler mismatch or temporal errors</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>For a new AR video system, I would begin with three independent checks. First, fix the history and parameters, change only sampling precision, the KV representation, or the compute engine, and look for a systematic shift in conditional outputs. Second, keep the sampler fixed, compare performance under real histories and self-rollout histories, and report degradation curves over generation length. Third, inspect success rates and successful sample counts by prompt to determine whether training repeatedly reinforces only the scenarios that are already easy.</p>

    <p>Validation should be separated accordingly: bias correction should be evaluated on the expectation it claims to correct; history-distribution training on the long-term quality of actual rollouts; and success reweighting on coverage of hard prompts and gains under different sampling budgets. A single average reward or overall video score can rarely establish all these mechanisms individually.</p>

    <p>A reasonable research plan for combining these ideas would be to first train on actual AR self-rollouts, bringing the history distribution closer to deployment; add targeted corrections for computational biases that have been verified; and then use an objective with a clear trajectory-level meaning to decide how much learning signal each generation receives. This is a design proposal informed by the literature, not a combined system already validated by these papers together.</p>

    <p><strong>The shared problem across domains is how a model interacts with the states, data, and learning signals it produces itself.</strong> Identifying the common feedback structure helps us transfer ideas; retaining each method’s intervention point and mathematical limits lets us determine what that transfer has actually fixed.</p>

    <h2 id="references">References</h2>

    <div class="drift-references">

      <p>This article analyzes and compares mechanisms from public papers; it reports no new model experiments. The gradient connection between BPO and Score Centering is an algebraic observation derived from the original equations. The other cross-domain pairings are analogies between mechanisms or derivation patterns, as specified in the text.</p>

      <ol>
        <li><strong>Score Centering Stabilizes Off-policy Reinforcement Learning.</strong> Martin Marek, Max Ryabinin. 2026. <a href="https://arxiv.org/html/2609.20807v1">arXiv:2609.20807</a>. Main sources: §4, drift decomposition and score centering; Appendix A, the top-\(k\) implementation.</li>
        <li><strong>Bellman Policy Optimization.</strong> Zhuoqing Song et al. 2026. <a href="https://arxiv.org/html/2609.15987v1">arXiv:2609.15987</a>. Main sources: §3.1–3.2, the full trajectory objective, PMD equivalence, and practical approximations.</li>
        <li><strong>Maximum Likelihood Reinforcement Learning.</strong> Fahim Tajwar et al. 2026. <a href="https://arxiv.org/html/2602.02710v3">arXiv:2602.02710, v3</a>. Main sources: §3–4, the truncated objective and success-conditioned estimator.</li>
        <li><strong>Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion.</strong> Tuna Tuncer et al. 2026. <a href="https://arxiv.org/html/2605.26266v1">arXiv:2605.26266</a>. Main sources: §4.1–4.2, Jensen bias and additive correction.</li>
        <li><strong>Improving Video Generation with Human Feedback.</strong> 2025. <a href="https://arxiv.org/html/2501.13918v2">arXiv:2501.13918, v2</a>. Main source: §4, Flow-DPO, Flow-RWR, and the design of time weights.</li>
        <li><strong>LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment.</strong> 2024 / 2025. <a href="https://arxiv.org/html/2412.04814v3">arXiv:2412.04814, v3</a>. Main sources: §3.4 and §4.4, reward-weighted learning and the rejection sampling variant.</li>
        <li><strong>Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion.</strong> 2025. <a href="https://arxiv.org/html/2506.08009v2">arXiv:2506.08009, v2</a>. Main sources: §3, self-rollouts and video distribution matching; §5, limitations.</li>
        <li><strong>Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion.</strong> 2024. <a href="https://arxiv.org/html/2407.01392v1">arXiv:2407.01392</a>. Per-token noise training and stability of long rollouts.</li>
        <li><strong>Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation.</strong> 2025. <a href="https://arxiv.org/html/2512.04678v2">arXiv:2512.04678, v2</a>. Main source: §3, EMA-Sink and Re-DMD.</li>
        <li><strong>AR-CoPO: Align Autoregressive Video Generation with Contrastive Policy Optimization.</strong> 2026. <a href="https://arxiv.org/html/2603.17461v2">arXiv:2603.17461, v2</a>. Main source: §3, pivot-chunk exploration and the contrastive surrogate policy.</li>
      </ol>

    </div>

    <p class="drift-meta"><a href="/blog/">← All posts</a> · <a href="#two-clocks">Back to the article</a></p>

  </article>
</main>

<script src="/assets/js/ar-drift.js" defer=""></script>]]></content><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><summary type="html"><![CDATA[From Score Centering, BPO, and MaxRL to Self Forcing, video preference optimization, and KV quantization correction: understanding drift across domains through distributions, feedback loops, and points of intervention.]]></summary></entry><entry xml:lang="zh-CN"><title type="html">模型为什么越走越偏：从 LLM 强化学习到自回归视频的 Drift</title><link href="https://zefancai.com/blog/ar-drift/" rel="alternate" type="text/html" title="模型为什么越走越偏：从 LLM 强化学习到自回归视频的 Drift" /><published>2026-09-26T00:00:00-07:00</published><updated>2026-09-26T00:00:00-07:00</updated><id>https://zefancai.com/blog/ar-drift</id><content type="html" xml:base="https://zefancai.com/blog/ar-drift/"><![CDATA[<link rel="stylesheet" href="/assets/css/ar-drift.css" />

<main id="main" class="drift-article" lang="zh-CN">
  <article class="drift-prose">

    <p class="drift-kicker"><a href="/zh/blog/">中文博客</a> / 研究笔记</p>

    <h1 id="section">模型为什么越走越偏</h1>

    <p class="drift-deck">从 LLM 强化学习到自回归视频，用同一视角理解 Drift。</p>
    <p class="drift-meta">Zefan Cai · <time datetime="2026-09-26">2026 年 9 月 26 日</time> · 中文研究笔记</p>

    <p>一个推理模型在强化学习中突然崩溃，一段自回归视频越生成越不像开头，一个量化后的 KV cache 让历史帧“抢走”当前帧的注意力。它们看起来属于不同问题：优化不稳定、时间一致性、数值精度。但它们有一个值得共同研究的结构：<strong>模型参与构造自己下一步所处的分布，而训练或更新规则对这个分布的假设并不总是正确。</strong></p>

    <p>这篇文章把 <a href="https://arxiv.org/html/2609.20807v1">Score Centering</a>、<a href="https://arxiv.org/html/2609.15987v1">Bellman Policy Optimization（BPO）</a>、<a href="https://arxiv.org/html/2602.02710v3">Maximum Likelihood Reinforcement Learning（MaxRL）</a> 与视频生成中的几类方法放在一起。目的不是给它们贴上同一个标签，而是回答一个更具体的问题：偏差从闭环的哪一处进入，又应当在哪一处修正？</p>

    <div class="drift-callout">
      <p>本文的核心区分：<strong>Score Centering 修正更新的系统性偏置；Self Forcing 调整训练所见的历史；BPO 重写策略优化目标；MaxRL 改变成功轨迹获得的学习权重。</strong> 它们可以互补，但不能互相替代。</p>
    </div>

    <nav class="drift-toc" aria-label="文章目录">
      <ul>
        <li><a href="#two-clocks">两种 drift，两条时间轴</a></li>
        <li><a href="#distribution">把不一致定位到分布的不同层级</a></li>
        <li><a href="#score-centering">Score Centering：扣掉不该存在的更新</a></li>
        <li><a href="#bpo">BPO：从轨迹关系推导局部 loss</a></li>
        <li><a href="#maxrl">MaxRL：成功样本应当怎样被计数</a></li>
        <li><a href="#video-loop">回到 AR 视频：让训练进入真实闭环</a></li>
        <li><a href="#diagnosis">用这个视角诊断一个新系统</a></li>
        <li><a href="#references">参考论文</a></li>
      </ul>
    </nav>

    <h2 id="two-clocks">两种 drift，两条时间轴</h2>

    <p>先把“越走越偏”中的“走”说清楚。</p>

    <p>在 LLM 强化学习中，训练器产生参数，推理引擎用这些参数生成 rollout，训练器再依据 rollout 更新参数。量化、不同数值 kernel、异步采样和旧 checkpoint，都可能让推理引擎与训练器给同一个 token 的概率不同。偏置一旦进入更新，新的参数又会被送回采样器，形成跨训练步的反馈。</p>

    <p>AR 视频还有另一条反馈链：模型根据历史生成下一段，把这一段写入历史，再据此生成后续内容。哪怕权重完全不变，人物形状的一点误差、几何关系的一点偏移，也可能进入后续条件并继续传播。</p>

    <div class="drift-loop-grid">
  <div class="drift-loop">
    <p class="drift-loop-label">LLM RL · 训练步 k</p>
    <p class="drift-loop-path">参数 θ<sub>k</sub> → sampler → rollouts → 梯度 → 参数 θ<sub>k+1</sub></p>
    <p>变化的状态包括模型参数；有偏更新被反复反馈。</p>
  </div>
  <div class="drift-loop">
    <p class="drift-loop-label">AR 视频 · 生成步 t</p>
    <p class="drift-loop-path">历史 h<sub>t</sub> → 下一段视频 → 写入历史 → h<sub>t+1</sub></p>
    <p>参数可以固定；变化的是模型随后读取的历史。</p>
  </div>
</div>

    <p>两者共享的是闭环结构，累积的对象却不同。Score Centering 中的 drift 是一个有明确定义的梯度分解项；视频中常说的 drift 通常是身份、几何、动作或场景随时间退化。把后者全部归结为前者，会错过真正需要修改的部位。</p>

    <p>一个有用的局部分析视角是：</p>

\[e_{n+1}\approx J_n e_n+b_n+\xi_n.\]

    <p>这里，\(e_n\) 是选定表示中的扰动，\(J_n\) 描述扰动如何传播，\(b_n\) 是系统性偏差，\(\xi_n\) 是随机扰动。这个式子是理解机制的局部近似，<strong>不是所有生成模型共享的 drift 定理</strong>。即使存在偏差，系统也不一定发散；传播算子是否放大误差同样重要。</p>

    <p>下面的简单模型把这种区别单独拿出来。它只有一个标量状态，起点为 \(e_0=0.2\)；实线执行 \(e_{n+1}=a e_n+b\)，虚线只去掉 \(b\)，保留完全相同的传播系数 \(a\)。</p>

    <figure class="drift-lab" id="drift-lab">
  <div class="drift-controls">
    <label for="drift-gain">误差传播系数 a <output id="drift-gain-value" for="drift-gain">1.00</output>
      <input type="range" id="drift-gain" min="0.8" max="1.1" step="0.01" value="1" />
    </label>
    <label for="drift-bias">每步系统偏差 b <output id="drift-bias-value" for="drift-bias">0.025</output>
      <input type="range" id="drift-bias" min="0" max="0.1" step="0.005" value="0.025" />
    </label>
  </div>
  <svg id="drift-chart" viewBox="0 0 720 280" role="img" aria-labelledby="drift-chart-title drift-chart-desc">
    <title id="drift-chart-title">有偏递推与去掉系统偏差后的递推</title>
    <desc id="drift-chart-desc">初始误差为 0.2。默认 a 等于 1、b 等于 0.025，40 步后有偏递推为 1.2，去偏递推为 0.2。</desc>
    <g data-drift-fallback="">
      <path d="M60 40V230H690" fill="none" stroke="#a5adae" />
      <path d="M60 202L690 63" fill="none" stroke="#ad5436" stroke-width="3" />
      <path d="M60 202H690" fill="none" stroke="#147a75" stroke-width="3" stroke-dasharray="8 5" />
      <text x="70" y="30" fill="#ad5436" font-size="15">有系统偏差：0.2 → 1.2</text>
      <text x="390" y="30" fill="#147a75" font-size="15">去掉偏差：0.2 → 0.2</text>
      <text x="58" y="255" font-size="14">0</text><text x="672" y="255" font-size="14">40 步</text>
    </g>
  </svg>
  <p id="drift-lab-summary" aria-live="polite">默认参数下，40 步后的误差分别为 1.2 和 0.2。</p>
  <figcaption>交互示意，不是论文实验。去掉系统偏差，并不自动消除初始误差，也不保证传播系数大于 1 时系统稳定。关闭 JavaScript 时仍可阅读默认曲线。</figcaption>
</figure>

    <p>这个例子解释了为什么“校正偏差”与“提高闭环鲁棒性”是两个不同任务。前者处理新注入的 \(b_n\)，后者还要关心系统面对已经存在的误差时如何继续运行。</p>

    <h2 id="distribution">把不一致定位到分布的不同层级</h2>

    <p>对输入 \(x\)、生成历史 \(h_t\) 和下一生成单元 \(a_t\)，可以把实际采样过程写成：</p>

\[x\sim\rho,\qquad h_t\sim d_t^q(\cdot\mid x),\qquad
a_t\sim q(\cdot\mid h_t).\]

    <p>\(\rho\) 是 prompt 分布，\(d_t^q\) 是采样器运行到第 \(t\) 步时访问的历史分布，\(q\) 是给定历史后的条件采样分布。训练器用于计算梯度的概率记作 \(p_\theta\)。于是，“训练与实际运行不一致”至少有三个不同位置。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>层级</th>
            <th>应当问的问题</th>
            <th>代表性干预</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>固定历史下的条件分布</td>
            <td>同一个 \(h\) 上，sampler 的 \(q\) 与 trainer 的 \(p_\theta\) 是否一致？</td>
            <td>Score Centering；计算算子的偏置校正</td>
          </tr>
          <tr>
            <td>访问的历史分布</td>
            <td>训练见到的 \(d_t\)，是否覆盖模型实际生成的历史？</td>
            <td>Self Forcing；在实际 rollout 上训练</td>
          </tr>
          <tr>
            <td>输入与成功轨迹的权重</td>
            <td>哪些 prompt、哪些完整输出获得了更多学习信号？</td>
            <td>MaxRL；成功条件化或奖励加权拟合</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>BPO 则追问贯穿这张表的另一个问题：<strong>实际采用的局部 loss，究竟是哪个整体策略目标的近似？</strong> 有时困难来自分布错位，有时来自代理目标的设计，有时两者同时存在。</p>

    <p>一个容易混淆的点是：teacher-forced token NLL 的求和，本来就是精确的序列 NLL，</p>

\[-\log p_\theta(a_{1:T}\mid x)
=-\sum_t\log p_\theta(a_t\mid x,a_{&lt;t}).\]

    <p>所以问题不是“token loss 天生看不到整条轨迹”。问题在于：模型有有限误差时，在真实前缀上学到的条件行为，并不等于它在自己产生的偏离前缀上也能恢复；而本文涉及的视频分布匹配代理目标，也不能直接等同于真实数据上的最大似然目标。</p>

    <h2 id="score-centering">Score Centering：扣掉不该存在的更新</h2>

    <p><a href="https://arxiv.org/html/2609.20807v1">Score Centering</a> 从一个很有力的检验出发：如果所有输出都得到相同的正奖励，模型应不应该继续学习？在标准 on-policy policy gradient 中，答案是不应该，因为环境没有提供区分优劣的信号。</p>

    <p>固定历史 \(h\)，定义参数 score：</p>

\[g_\theta(a,h)=\nabla_\theta\log p_\theta(a\mid h).\]

    <p>在通常的可微与归一化条件下，\(\mathbb E_{p_\theta}[g_\theta\mid h]=0\)。但 token 实际从 \(q\) 采样时，\(\mathbb E_q[g_\theta\mid h]\) 一般可以非零。对当前 action 及其后续 rollout 取条件期望，更新可以拆成：</p>

\[\mathbb E_q[Rg_\theta\mid h]
=\underbrace{\mathbb E_q[R\mid h]\,
\mathbb E_q[g_\theta\mid h]}_{\text{drift}}
+\underbrace{\operatorname{Cov}_q(R,g_\theta\mid h)}_{\text{reward-related signal}}.\]

    <p>第一项只通过奖励的均值依赖奖励，不关心哪个 action 导致更好的结果。对于正的条件平均奖励，它会把训练器推向采样器。若采样器是一个固定教师，这类似普通蒸馏；若采样器是训练器的量化或滞后副本，训练器每轮追随它，再把更新后的参数送回去，偏差便可能循环累积。</p>

    <p>Score Centering 直接替换：</p>

\[g_\theta(a,h)\longrightarrow
g_\theta(a,h)-\mathbb E_q[g_\theta\mid h].\]

    <p>完整条件期望下，drift 项被精确消去，留下 \(\operatorname{Cov}_q(R,g_\theta\mid h)\)。<strong>但这仍然是 \(q\) 下的 covariance，不是完整恢复了 \(p_\theta\) 下的 on-policy 更新。</strong> 论文实际实现还用 sampler 的 top-\(k\) 概率与 trainer 的尾部分布近似这个期望。</p>

    <p>这也解释了为什么 prompt 内的 reward/advantage centering 不够。整组 rollout 的 advantage 总和为零，不代表每个中间 prefix 的条件平均 advantage 为零。一个已接近正确答案的前缀与一个已经出错的前缀，条件均值通常不同。</p>

    <h3 id="kv--attention-">视频对应：KV 量化之后的 attention 偏置</h3>

    <p>在 <a href="https://arxiv.org/html/2605.26266v1">Quantized Keys Steal Attention</a> 中，AR 视频的历史 keys 被压缩到低比特，当前 chunk 的 keys 则保持完整精度。即使量化误差在 attention logit 上近似零均值，softmax 中的指数函数也会改变均值：</p>

\[\hat\ell_i=\ell_i+\delta_i,\quad \mathbb E[\delta_i]=0,
\qquad
\mathbb E[e^{\hat\ell_i}]
=e^{\ell_i}\mathbb E[e^{\delta_i}]
\ge e^{\ell_i}.\]

    <p>历史 keys 的未归一化贡献被系统性放大，注意力因而向 cached tokens 偏移。论文推导出一个逐 attention score 的加性补偿：</p>

\[b_i=\log\mathbb E[e^{\delta_i}],\qquad
\tilde\ell_i=\hat\ell_i-b_i,
\qquad b_i\approx\tfrac12\operatorname{Var}(\delta_i).\]

    <p>在噪声模型成立且使用精确 \(b_i\) 时，\(\mathbb E[e^{\tilde\ell_i}]=e^{\ell_i}\)；实际采用二阶近似，且不需要重新训练。论文在 MAGI-1、SkyReels-V2 和 HY-WorldPlay 上验证这一方法。</p>

    <p>这与 Score Centering 最像的地方，是同一条因果路线：<strong>高效计算引入偏差 → 分析偏差的期望 → 以加性项补偿，而非要求彻底放弃高效计算。</strong> 但这里恢复的是未归一化 attention contribution 的期望，不能直接推出归一化后的 softmax 完全无偏，更不能推出任意长视频都不再漂移。</p>

    <p class="drift-note">这里有三种容易混淆的 “score”：RL 的参数 score 是 ∇<sub>θ</sub> log p；attention score 是 softmax 前的 logit；扩散模型的 score 通常是 ∇<sub>x</sub> log p。名字相同，不意味着可以直接替换公式。</p>

    <h2 id="bpo">BPO：从轨迹关系推导局部 loss</h2>

    <p><a href="https://arxiv.org/html/2609.15987v1">BPO</a> 的起点不是给现成 GRPO loss 换一个系数，而是 Policy Mirror Descent（PMD）。对于固定的 rollout policy \(\mu\)，PMD 在每个状态上权衡 advantage 与偏离旧策略的 KL 代价：</p>

\[\max_\pi\;
\mathbb E_{a\sim\pi(\cdot\mid h)}[A^\mu(h,a)]
-\frac{1}{\eta}D_{\mathrm{KL}}(\pi\Vert\mu).\]

    <p>其理想解满足 \(\pi^+(a\mid h)\propto\mu(a\mid h)e^{\eta A^\mu(h,a)}\)。难点是：语言生成中若要直接使用每个 prefix 的 advantage，通常需要估计大量中间状态的 value。</p>

    <p>BPO 利用 token append 的确定性与只有终点奖励的设定。按论文的约定，把终点 reward 写成终点 value，\(V^\mu(h_{T+1})=R\)，于是：</p>

\[A^\mu(h_t,a_t)=V^\mu(h_{t+1})-V^\mu(h_t),
\qquad
\sum_tA^\mu(h_t,a_t)=R-V^\mu(x).\]

    <p>中间 value 沿路径抵消，只剩终点奖励和初始 prompt 的期望奖励。后者仍要估计，但可以通过同一 prompt 的 grouped rollouts 完成，不需要单独训练中间状态 critic。</p>

    <h3 id="bpo--score-centering-">BPO 与 Score Centering 的直接连接</h3>

    <p>定义 BPO 完整目标中的逐 token 项：</p>

\[u_\theta(a,h)=
\log\frac{p_\theta(a\mid h)}{\mu(a\mid h)}
+D_{\mathrm{KL}}\!\left(\mu(\cdot\mid h)\Vert p_\theta(\cdot\mid h)\right).\]

    <p>在 \(p_\theta=\pi^+\) 时，PMD 的最优性条件让轨迹满足 \(\sum_tu_\theta=\eta(R-V^\mu(x))\)。BPO 因而构造平方残差目标：</p>

\[\delta=\eta(R-V^\mu(x))-\sum_tu_\theta(a_t,h_t),
\qquad
L=\mathbb E_{x\sim\rho,\,\tau\sim P_\mu(\cdot\mid x)}
\left[\frac{\phi(x)}{2\eta}\delta^2\right],\quad\phi(x)&gt;0.\]

    <p>在论文设定的策略空间与可达状态上，完整目标与 PMD 具有相同的最优解。这是目标层面的结论，不是任意神经网络与优化器都会达到该解的保证。</p>

    <p>从上面的式子还能直接得到一个有意思的联系。固定 \(\mu\) 求导：</p>

\[\nabla_\theta u_\theta(a,h)
=g_\theta(a,h)-\mathbb E_{b\sim\mu}[g_\theta(b,h)].\]

    <p><strong>右侧正是以实际采样分布为参照的 score centering。</strong> 这是从两篇论文公式得到的代数观察：BPO 通过整体策略目标推导出的完整 KL 项，在梯度上包含 SC 的中心化结构。它不意味着两种算法的全部目标与更新完全相同。</p>

    <p>实际 BPO 还要做残差线性化、group estimation、binary KL 近似、平滑与 clipping，最后使用互补概率权重：</p>

\[\omega_t=
\frac{1+\epsilon-\mu(a_t\mid h_t)}
{1+\epsilon-p_\theta(a_t\mid h_t)}.\]

    <p>因此，实际 token loss 不能自动继承完整 KL 中心化的精确零均值性质。这一步尤其提醒我们：<strong>知道最终 loss 的形式，还不够；也要知道从整体目标走到它时，舍弃了哪些条件。</strong></p>

    <h3 id="flow-dpo-">视频对应：Flow-DPO 的解析消元</h3>

    <p><a href="https://arxiv.org/html/2501.13918v2">Improving Video Generation with Human Feedback</a> 中的 Flow-DPO，也从完整生成分布上的目标出发。对于视频 \(v\) 和条件 \(c\)，KL 正则奖励最大化的理想解具有形式：</p>

\[p^*(v\mid c)=\frac{p_{\rm ref}(v\mid c)e^{R(v,c)/\beta}}{Z(c)}.\]

    <p>把这个关系反过来，就能用生成概率比表示 reward。同一 prompt 下，preferred 与 rejected 视频的 reward 差会消去 \(\log Z(c)\)，从而得到不需要显式 reward 模型参与该训练目标的偏好优化形式。再通过 diffusion/flow 的似然代理，把训练落实到去噪或 velocity prediction 误差上。</p>

    <p>两条路线的共同方法论很清楚：</p>

    <div class="drift-loop-grid">
  <div class="drift-loop">
    <p class="drift-loop-label">BPO</p>
    <p class="drift-loop-path">PMD → Bellman 消去中间 value → 轨迹残差 → token loss</p>
  </div>
  <div class="drift-loop">
    <p class="drift-loop-label">Flow-DPO</p>
    <p class="drift-loop-path">KL 正则奖励目标 → 消去 reward 与归一化常数 → 视频偏好 → velocity loss</p>
  </div>
</div>

    <p>这是一种推导范式的对应。Flow-DPO 并未使用 BPO 的 Bellman 消元，也没有它的互补概率权重。实际 Flow-DPO 还把推导中的时间相关 \(\beta_t\) 改为常数以改善训练；这里的时间是<strong>去噪时间</strong>，不是视频帧序号。</p>

    <p>如果真要把 BPO 移植到视频，不能直接复制 \((1-\mu)/(1-p)\)：它来自离散 token 概率的 binary KL；连续 latent 的概率密度可以大于 1，“一减密度”不再是补事件的概率。需要在具体的视频状态、动作和采样器定义下重新推导。</p>

    <h2 id="maxrl">MaxRL：成功样本应当怎样被计数</h2>

    <p><a href="https://arxiv.org/html/2602.02710v3">MaxRL</a> 讨论的是另一类不一致：平均成功率与成功事件的 log-likelihood 不是同一个目标。令 \(s_\theta(x)\) 表示 prompt \(x\) 的成功概率，便有：</p>

\[J_{\rm RL}=\mathbb E_{x\sim\rho}[s_\theta(x)],
\qquad
J_{\rm ML}=\mathbb E_{x\sim\rho}[\log s_\theta(x)].\]

    <p>两者都鼓励成功，但梯度分配不同。后者的权重是 \(1/s_\theta(x)\)：在梯度表达式中，成功稀少的问题获得更大的相对权重。它改变的是怎样分配学习信号，不是校正 sampler 与 trainer 的概率差异。</p>

    <p>设 \(\tau\) 是完整轨迹，成功由二值判据决定。在可微与支持条件成立时：</p>

\[\nabla_\theta\log s_\theta(x)
=\mathbb E_{\tau\sim P_\theta(\cdot\mid x,\mathrm{success})}
[\nabla_\theta\log P_\theta(\tau\mid x)].\]

    <p>这让最大似然有了一个简单解释：平均成功轨迹的 score。对一个 prompt，独立地从当前 \(P_\theta(\cdot\mid x)\) 采样 \(N\) 条轨迹，其中 \(K\) 条成功，可以构造纯成功均值估计器：</p>

\[\hat g_N(x)=
\begin{cases}
\frac{1}{K}\sum_{i=1}^{N}r_i\nabla_\theta\log P_\theta(\tau_i\mid x),&amp;K&gt;0,\\
0,&amp;K=0.
\end{cases}\]

    <p>有限采样下，并非每个 prompt 都能观察到成功，所以它不是精确 ML 梯度的无偏估计。MaxRL 的关键结果是：它的期望恰好对应一个截断目标，</p>

\[J_N(x)=-\sum_{j=1}^{N}\frac{(1-s_\theta(x))^j}{j},
\qquad
\nabla J_N(x)=\sum_{j=1}^{N}\frac1j\nabla\operatorname{pass@}j(x).\]

    <p>\(N=1\) 时退化为标准期望奖励目标（相差常数），\(N\to\infty\) 时趋近 log-likelihood。因此增加 rollout 数，不只降低固定目标的估计方差，还改变了逼近的目标。这里讨论的是纯成功均值估计器；加入 baseline 的实际实现需按其具体形式分析。</p>

    <h3 id="lift-">视频对应：LiFT 的成功筛选变体</h3>

    <p><a href="https://arxiv.org/html/2412.04814v3">LiFT</a> 的主方法是 reward-weighted learning：生成视频后，用学习到的评价模型打分，让高奖励视频具有更大训练权重，同时混入真实视频的训练损失。它还研究了一个 rejection sampling 变体：仅保留语义一致性、运动流畅度、视频保真度三个维度都被评为 Good 的视频，再对这些样本微调。</p>

    <p>“拟合成功条件下的生成分布”与 MaxRL 很接近，但样本怎样归一化会改变结果。设两个 prompt 各生成 10 条视频，A 成功 8 条，B 成功 1 条：</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>怎样使用这 9 条成功视频？</th>
            <th>prompt A</th>
            <th>prompt B</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>全部混合，每条成功视频等权</td>
            <td>贡献 8 条训练样本</td>
            <td>贡献 1 条训练样本</td>
          </tr>
          <tr>
            <td>先在各 prompt 内平均成功轨迹，再平均 prompt</td>
            <td>先平均 8 条</td>
            <td>先平均 1 条</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>第二种正是 MaxRL 估计器的关键结构：成功更多的 prompt 不会仅因留下更多样本而自动占据更大的总系数。这并不保证两个 prompt 的梯度范数相同；\(K=0\) 的 prompt 在该估计器中仍然没有成功学习信号。</p>

    <p>因此，“筛好视频再训”不自动等于视频版 MaxRL。更接近它的方案需要在线采样、逐 prompt 的成功归一化，以及明确的终点成功判据。若扩散视频用去噪 loss 代替精确生成 log-likelihood，还增加了一层代理目标。LiFT 的现有结果没有提供这套 MaxRL 等价保证。</p>

    <h2 id="video-loop">回到 AR 视频：让训练进入真实闭环</h2>

    <p>上述三组对应还没有直接解决 AR 视频最常见的历史分布问题。训练时历史来自真实视频，生成时历史来自模型自己；同一组参数可能在这两种条件下表现截然不同。</p>

    <p><a href="https://arxiv.org/html/2506.08009v2">Self Forcing</a> 的干预位置最直接：训练时就进行自回归 self-rollout，让后一段依赖先前生成的内容，再用 DMD、SiD 或 GAN 等视频级分布匹配目标训练。这样，训练目标看到的是实际生成过程所产生的视频分布，而非仅仅在真实历史上完成局部预测得到的输出。</p>

    <p>相邻的 <a href="https://arxiv.org/html/2407.01392v1">Diffusion Forcing</a> 通过逐 token/帧的独立噪声水平训练，让模型适应不同可靠程度的历史。这提供了鲁棒性，但对真实历史加噪，并不自动等于采样到了模型实际会产生的错误历史。</p>

    <p>Self Forcing 也不保证无限长生成不再退化。它的实用实现使用梯度截断、随机提前结束去噪以及 KV cache 的梯度分离来控制成本；超出训练上下文长度后，仍然可能出现质量下降。它直接缩小了 exposure bias，却没有把所有数值偏置和所有长时误差一起消除。</p>

    <h3 id="section-1">稳定，还要朝正确的目标稳定</h3>

    <p><a href="https://arxiv.org/html/2512.04678v2">Reward Forcing</a> 给出了一个更具体的例子：长期保留初始帧作为 attention sink，有助于维持历史参照，也可能让模型反复复制初始内容，导致运动减弱甚至视觉回闪。它用 EMA-Sink 更新这部分历史表示，并用 Re-DMD 对运动质量更好的 self-rollout 赋予更大的分布匹配权重。</p>

    <p>这说明“更稳定”与“更好的视频”也需要分开。一段几乎不运动的视频可能很容易保持外观一致。Reward Forcing 的奖励加权机制是 MaxRL 的宽泛近邻，但它并没有优化同一个 \(\log P(\mathrm{success}\mid c)\) 目标，也没有相同的逐 prompt 成功归一化。</p>

    <p><a href="https://arxiv.org/html/2603.17461v2">AR-CoPO</a> 则把注意力放在探索与真实采样器的关系上。对于 few-step AR 视频，改用中间注噪的 SDE 来方便计算 RL loss，可能让训练探索偏离实际 ODE 生成过程。AR-CoPO 在选定的 pivot chunk 扰动初始噪声、分叉候选视频，再通过对比代理策略进行局部更新。这与 BPO 共享“重新审视 policy loss 应怎样构造”的动机，但不包含它的 Bellman 消元。</p>

    <h2 id="diagnosis">用这个视角诊断一个新系统</h2>

    <p>把这些方法放在一起，最有用的结果不是一个统一算法名称，而是一张干预位置图。</p>

    <div class="drift-table-wrap">

      <table>
        <thead>
          <tr>
            <th>观察到的问题</th>
            <th>先定位什么</th>
            <th>相关方法</th>
            <th>不能由此推出什么</th>
          </tr>
        </thead>
        <tbody>
          <tr>
            <td>同样的前缀，采样与训练概率不一致；更新持续偏移</td>
            <td>固定历史下的条件分布与平均 score</td>
            <td>Score Centering；BPO 的完整 KL 结构</td>
            <td>已恢复全部 on-policy 梯度</td>
          </tr>
          <tr>
            <td>历史 KV 量化后，注意力向过去倾斜</td>
            <td>非线性计算把噪声变成了怎样的偏置</td>
            <td>Quantized Keys Steal Attention</td>
            <td>所有历史误差或长视频漂移都消失</td>
          </tr>
          <tr>
            <td>前几段正常，依赖自生成历史后退化</td>
            <td>真实与生成历史的访问分布</td>
            <td>Self Forcing；Diffusion Forcing</td>
            <td>超出训练长度仍无限稳定</td>
          </tr>
          <tr>
            <td>整体奖励提升有限，局部 policy loss 难以解释</td>
            <td>从整体目标到代理 loss 的推导</td>
            <td>BPO；Flow-DPO；AR-CoPO</td>
            <td>三者在数学上等价</td>
          </tr>
          <tr>
            <td>容易场景反复成功，困难场景缺少可用候选</td>
            <td>prompt 与成功轨迹的学习权重</td>
            <td>MaxRL；LiFT 的筛选变体；Reward Forcing</td>
            <td>sampler mismatch 或时间误差已被修复</td>
          </tr>
        </tbody>
      </table>

    </div>

    <p>对一个新的 AR 视频系统，我会先做三个相互独立的检查。第一，固定历史与参数，仅改变采样精度、KV 表示或计算引擎，观察条件输出是否发生系统偏移。第二，保持采样器不变，比较真实历史与 self-rollout 历史下的表现，并沿生成长度报告退化曲线。第三，按 prompt 查看成功率与成功样本数，判断训练是否只在已经容易成功的场景上反复强化。</p>

    <p>对应的验证也应当分开：偏置校正看它宣称修正的期望量；历史分布训练看真实 rollout 的长期质量；成功重加权看困难 prompt 的覆盖及不同采样预算下的收益。单一的平均 reward 或总视频分数，很难替这些机制逐一作证。</p>

    <p>如果要组合这些思想，一个合理的研究方案是：先在实际 AR self-rollout 上训练，使历史分布接近部署；对已经确认的计算偏差增加有针对性的校正；再用有清晰轨迹含义的目标，决定哪些生成获得多少学习信号。这是从文献出发的设计建议，而不是这些论文已经共同验证过的组合系统。</p>

    <p><strong>跨 domain 的共同问题，是模型如何与自己产生的状态、数据和学习信号相处。</strong> 找到共享的反馈结构，可以帮助我们迁移思路；保留每种方法的干预位置与数学边界，才能判断这种迁移到底修好了什么。</p>

    <h2 id="references">参考论文</h2>

    <div class="drift-references">

      <p>本文是公开论文的机制分析与对照，不报告新的模型实验。BPO 与 Score Centering 的梯度联系是由原文公式得到的代数观察；其余跨领域配对按文中说明属于机制或推导范式的类比。</p>

      <ol>
        <li><strong>Score Centering Stabilizes Off-policy Reinforcement Learning.</strong> Martin Marek, Max Ryabinin. 2026. <a href="https://arxiv.org/html/2609.20807v1">arXiv:2609.20807</a>。主要依据 §4：drift 分解与 score centering；附录 A：top-\(k\) 实现。</li>
        <li><strong>Bellman Policy Optimization.</strong> Zhuoqing Song et al. 2026. <a href="https://arxiv.org/html/2609.15987v1">arXiv:2609.15987</a>。主要依据 §3.1–3.2：完整轨迹目标、PMD 等价性与实用近似。</li>
        <li><strong>Maximum Likelihood Reinforcement Learning.</strong> Fahim Tajwar et al. 2026. <a href="https://arxiv.org/html/2602.02710v3">arXiv:2602.02710, v3</a>。主要依据 §3–4：截断目标与成功条件化估计器。</li>
        <li><strong>Quantized Keys Steal Attention: Bias Correction for KV-Cache Compression in Video Diffusion.</strong> Tuna Tuncer et al. 2026. <a href="https://arxiv.org/html/2605.26266v1">arXiv:2605.26266</a>。主要依据 §4.1–4.2：Jensen bias 与加性补偿。</li>
        <li><strong>Improving Video Generation with Human Feedback.</strong> 2025. <a href="https://arxiv.org/html/2501.13918v2">arXiv:2501.13918, v2</a>。主要依据 §4：Flow-DPO、Flow-RWR 与时间权重设计。</li>
        <li><strong>LiFT: Leveraging Human Feedback for Text-to-Video Model Alignment.</strong> 2024 / 2025. <a href="https://arxiv.org/html/2412.04814v3">arXiv:2412.04814, v3</a>。主要依据 §3.4 与 §4.4：reward-weighted learning 和 rejection sampling 变体。</li>
        <li><strong>Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion.</strong> 2025. <a href="https://arxiv.org/html/2506.08009v2">arXiv:2506.08009, v2</a>。主要依据 §3：self-rollout 与视频分布匹配；§5：局限。</li>
        <li><strong>Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion.</strong> 2024. <a href="https://arxiv.org/html/2407.01392v1">arXiv:2407.01392</a>。逐 token 噪声训练与长 rollout 的稳定性。</li>
        <li><strong>Reward Forcing: Efficient Streaming Video Generation with Rewarded Distribution Matching Distillation.</strong> 2025. <a href="https://arxiv.org/html/2512.04678v2">arXiv:2512.04678, v2</a>。主要依据 §3：EMA-Sink 与 Re-DMD。</li>
        <li><strong>AR-CoPO: Align Autoregressive Video Generation with Contrastive Policy Optimization.</strong> 2026. <a href="https://arxiv.org/html/2603.17461v2">arXiv:2603.17461, v2</a>。主要依据 §3：pivot-chunk 探索与对比代理策略。</li>
      </ol>

    </div>

    <p class="drift-meta"><a href="/zh/blog/">← 所有博客</a> · <a href="#two-clocks">回到正文</a></p>

  </article>
</main>

<script src="/assets/js/ar-drift.js" defer=""></script>]]></content><author><name>Zefan Cai</name><email>zefncai@gmail.com</email></author><summary type="html"><![CDATA[从 Score Centering、BPO、MaxRL 到 Self Forcing、视频偏好优化与 KV 量化校正：用分布、反馈循环和干预位置，理解不同 domain 的 drift。]]></summary></entry></feed>