Sean from Sequoia introduces Dylan Patel and praises SemiAnalysis as the leading independent research firm in semiconductors, covering technical details, supply chains, and the big picture after semis had lost sex appeal in the West. He notes rumors of the firm recently passing $100 million in revenue and possible plans for a venture fund, while highlighting the trusted brand Dylan has built. The conversation quickly turns to Dylan’s personal background and how he reached his current position. Dylan begins by describing his early life in a family-run motel and gas station business and figuring out the technical details, economics and margins of semiconductors.

TLDR – KEY INSIGHTS
– annual gains of roughly 40–60× in intelligence per watt and per dollar
– 100X Improvements from Full Stack Co-design
– Most revenue and usage at leading labs still comes from their best (largest) models, where fast inference modes deliver clear ROI. Dylan emphasizes rigorous tracking of token spend and ROI on every task to decide when fast mode is worth the cost. Cerebras occupies a valuable but specialized niche rather than replacing general-purpose GPU or TPU clusters for all workloads
– WRONG claims (of others) – WRONG AI has no ROI or that model progress has plateaued, RIGHT AI has ROI, Models continue to improve quickly
– co-packaged optics to arrive toward the end of the decade
– Specialized chips will carve out profitable niches even as Nvidia and a few hyperscaler ASICs dominate the majority of the market
– compute crunch persists because demand for useful AI tasks is expanding faster than new gigawatts of capacity are coming online, despite quarterly increases in deployed power. High gross margins at frontier labs allow them to pay substantial premiums for additional compute without destroying profitability

Dylan received an Xbox 360 as a Christmas gift around age 8 after the console was announced on his birthday. When it suffered the red ring of death hardware failure, he opened it up and successfully fixed it by shorting the temperature sensor after other methods failed. This incident “opened Pandora’s box” and sparked his deep interest in hardware tinkering. By age 12 he was already active in hardware communities and forums. The event marked the beginning of his lifelong fascination with electronics and problem-solving.

Dylan earned degrees unrelated to semiconductors and worked for two years as a quant at a small risk firm. A series of personal setbacks in early 2020—including workplace issues, his grandmother’s death from dementia, and COVID lockdowns—led him to move in with his brother in Nashville. While there he posted more frequently online, traded stocks profitably around COVID and semiconductor shortages, and was eventually doxed. On his 24th birthday he launched SemiAnalysis with two detailed public blog posts under his real name, which gained significant traction. These posts quickly led to consulting work and marked the formal beginning of his independent research career.

After personal struggles, Dylan spent six months living out of a truck and tent while visiting national parks across America, negotiating cheap motel rooms during the week and reading textbooks on semiconductors and AI on weekends. He continued posting detailed blogs throughout the trip and later traveled extensively in Latin America. He began attending over 40 conferences per year worldwide, from massive AI events like NeurIPS to highly technical niche shows such as SPIE lithography conferences. At these events he built relationships with experts, learning arcane supply-chain details through direct conversations that were rarely published. This period dramatically deepened his expertise while he remained effectively homeless from mid-2020 onward.

InferenceX and Benchmarking
Dylan explains that InferenceX was created because traditional point-in-time benchmarks quickly become outdated due to rapid model releases and constant software optimizations. The platform runs automated daily benchmarks across the latest models on donated hardware worth over $50 million from providers including Nvidia, AMD, Google, Amazon, and others. It focuses on the critical throughput-versus-interactivity (latency) curve and publicly shares optimal configurations so anyone can achieve near-peak performance. The project also tracks cost and power efficiency, revealing annual gains of roughly 40–60× in intelligence per watt and per dollar. Dylan believes inference will become one of the largest markets on Earth, far exceeding oil in economic impact over time.

Sparse vs Dense Models
OpenAI’s models tend to be more sparse while Anthropic’s are relatively more dense, creating fundamentally different optimization requirements. These architectural differences affect how models map onto specific hardware such as GPUs versus TPUs. The choice of sparsity influences matrix-multiply shapes, attention mechanisms, and expert routing. Hardware interconnect and network topology further reinforce these divergences. Co-design between model architecture and underlying hardware is therefore essential for peak performance.

Interconnect Shapes Architecture
Nvidia’s NVLink connects up to 72 GPUs through dedicated switches, while Google’s ICI allows up to 8,000 chips to communicate at high bandwidth without switches by routing through other chips. These contrasting interconnect designs create different latency, bandwidth, and scaling characteristics. The physical shape and connectivity of the hardware directly influence which model architectures perform best on each platform. As a result, model companies optimize their architectures for the specific interconnect they primarily use. This hardware-model co-design creates strong path dependence for each ecosystem.

CUDA Moat Is Eroding and Shifting – AI Coding Tools Are Helping Build Custom Kernels for AMD and Other Chips
The traditional CUDA moat is partially eroding because frontier model labs can now use AI coding tools to write custom kernels for alternative chips. With only a small number of major model developers, the need for broad programmability across thousands of customers has diminished. However, the real advantage often comes from downstream ecosystem effects as models that are heavily co-optimized for Nvidia hardware run sub-optimally on other platforms. Big labs frequently fork open-source frameworks or build their own stacks, reducing reliance on standard CUDA tooling. The moat is shifting from raw programmability toward full-stack co-design advantages.

Ecosystems and Co-Design Enable 100X Improvements from Full Stack Co-design
Chinese labs have produced models explicitly co-designed for Nvidia GPUs, making them less efficient on TPUs and other architectures. Major Western labs similarly co-optimize across model architecture, infrastructure software, and target hardware to achieve multiplicative gains. When optimization spans all layers simultaneously, improvements can reach 100× rather than simple additive or multiplicative gains from individual layers. Smaller teams still rely heavily on open-source tools like vLLM and SGLang, while frontier labs have the resources to customize everything. This full-stack co-design is becoming the primary source of competitive advantage.

Cerebras Speed and Limits
Cerebras excels at very fast inference, which SemiAnalysis itself uses extensively for high-value tasks where speed justifies premium pricing. The company’s SRAM-based architecture faces challenges scaling to extremely large models with long context lengths. Most revenue and usage at leading labs still comes from their best (largest) models, where fast inference modes deliver clear ROI. Dylan emphasizes rigorous tracking of token spend and ROI on every task to decide when fast mode is worth the cost. Cerebras occupies a valuable but specialized niche rather than replacing general-purpose GPU or TPU clusters for all workloads.

ROI Debates and Hot Takes
Dylan becomes particularly frustrated by claims that AI has no ROI or that model progress has plateaued. He points out that capabilities have consistently moved up and to the right, with old benchmarks saturating and new, harder benchmarks showing rapid gains. Semiconductors involve thousands of complex layers, and even experts learn new details daily about chemicals, processes, and supply chains. People often possess accurate facts yet reach completely incorrect conclusions due to missing context across abstraction layers. He views ongoing model improvement and economic value creation as undeniable based on both data and direct observation.

Ten Year Tech Bets
Dylan is highly excited about space-based data centers and related SpaceX-enabled opportunities over the next decade. He expects co-packaged optics to arrive toward the end of the decade, with debate mainly around exact timing. Long-term bets such as analog compute and energy-based models represent high-risk, high-reward explorations that could take many years to materialize. In the end state he predicts greater hardware diversity, with most major players running their own ASIC programs while still maintaining general-purpose capacity. Specialized chips will carve out profitable niches even as Nvidia and a few hyperscaler ASICs dominate the majority of the market.

Compute Crunch and NeoClouds
A significant compute crunch persists because demand for useful AI tasks is expanding faster than new gigawatts of capacity are coming online, despite quarterly increases in deployed power. High gross margins at frontier labs allow them to pay substantial premiums for additional compute without destroying profitability. Data-center and power-management quality varies widely. sophisticated operators extract more value per gigawatt through better utilization and workload orchestration. NeoClouds emerged because hyperscalers’ traditional CPU-era expertise was sometimes counterproductive for high-performance GPU clusters, while motivated new entrants moved faster. Nvidia actively supports the NeoCloud ecosystem to prevent hyperscalers from consolidating too much power through their own ASICs.