Bristol and Silicon Gorge

Bristol: maritime city, creative centre and engineering powerhouse

Bristol is a self-governing city in South West England, built around the River Avon and its historic floating harbour. Its identity comes from an unusual combination of maritime history, engineering, independent culture, universities and advanced technology.

The city is compact enough to feel coherent, but its neighbourhoods are distinct:

• Harbourside and the Old City reflect Bristol’s trading and shipbuilding past.
• Clifton is associated with Georgian architecture, the Avon Gorge and Brunel’s Clifton Suspension Bridge.
• Stokes Croft, Montpelier and Bedminster are strongly associated with street art, music and independent culture.
• Temple Quarter is becoming a major regeneration and business district around Temple Meads station.
• North Bristol, Filton and Aztec West contain much of the aerospace, defence, semiconductor and corporate technology economy.

Bristol’s population is now around half a million within the city authority area. It is highly diverse: Bristol City Council records residents originating from more than 185 countries, speaking over 90 languages, while the University of Bristol and UWE together account for about 68,000 registered students.

A city shaped by trade and engineering

Bristol became wealthy as an Atlantic port. It traded in wine, tobacco, sugar and manufactured goods and was deeply implicated in colonialism and the transatlantic slave economy. The modern city increasingly confronts that history through institutions such as M Shed, public debate and reinterpretation of its maritime heritage.

Engineering is equally central to Bristol’s story. Isambard Kingdom Brunel’s work—including the Clifton Suspension Bridge, the Great Western Railway and SS Great Britain—gave the city a lasting association with ambitious infrastructure and technological experimentation.

The modern economy continues that engineering lineage through:

• aerospace and defence;
• semiconductor and processor design;
• telecommunications;
• robotics and autonomous systems;
• creative media, film and animation;
• financial and professional services;
• environmental and low-carbon technology.

Bristol also has one of the UK’s most distinctive creative clusters, with roughly 6,000 creative businesses and an estimated 25,000–30,000 people working in film, television, animation, design, publishing, festivals and related fields.

What is Silicon Gorge?

Silicon Gorge is the informal name for the high-technology cluster centred on Bristol and extending through the wider West of England and parts of South West England.

The name is a deliberate play on:

• Silicon Valley, because of the area’s concentration of electronics and computing companies; and
• the Avon Gorge, the dramatic limestone gorge beneath Clifton Suspension Bridge.

It is not a formally bounded administrative area. Depending on who is using the term, Silicon Gorge can mean:

1. Bristol and North Bristol;
2. the Bristol–Bath city region;
3. a broader corridor including South Gloucestershire, Swindon, Gloucestershire, Exeter and links into the South Wales semiconductor cluster.

UK Research and Innovation has described Silicon Gorge as one of Europe’s significant high-technology regions, supported by academic centres at Bristol, Bath, Exeter and UWE alongside British and international businesses.

Why the cluster formed around Bristol

1. Aerospace created the engineering base

The Bristol area already possessed a deep reservoir of engineering expertise because of companies such as the Bristol Aeroplane Company and the later aerospace organisations concentrated around Filton.

Aircraft development required many of the capabilities that later became important in electronics:

• control systems;
• radio and communications;
• signal processing;
• real-time computing;
• safety-critical software;
• advanced materials;
• precision manufacturing.

That made the region fertile ground for electronics and computer companies during the 1960s and 1970s.

2. INMOS became the foundational semiconductor company

The pivotal event was the establishment of INMOS in Bristol in 1978. Backed by the UK government, INMOS was intended to create a nationally important microelectronics company.

Its best-known product was the transputer, a processor designed for parallel computing and high-speed communication between processors. The company’s design activity was centred in Bristol, while manufacturing was established across the Severn in Newport.

Although INMOS was later sold and ultimately absorbed into STMicroelectronics, its most important regional legacy was human capital. Former INMOS engineers founded or staffed a succession of businesses, including:

• Meiko;
• Division;
• Motion Media;
• PixelFusion;
• picoChip;
• XMOS;
• Blu Wireless;
• Gnodal;
• Icera;
• Graphcore.

This continuing movement of engineers, founders and intellectual property between companies is sometimes described as the INMOS diaspora. It plays a role in Bristol similar to the “Fairchildren”—companies created by former Fairchild Semiconductor employees—in Silicon Valley.

3. Hewlett-Packard and international firms added critical mass

Hewlett-Packard chose Bristol for one of its important research operations, helping establish the region as more than a collection of start-ups.

Over time, companies including Broadcom, NVIDIA, Toshiba, Infineon, IBM, Sony, Imagination Technologies and others established engineering or research activities in the wider region. A West of England strategic plan identified a large local silicon-design cluster and listed international technology businesses alongside indigenous start-ups.

These inward investors benefited from the local labour market, while also training engineers who later moved into start-ups or founded companies themselves.

4. Universities continuously replenish the ecosystem

The cluster is reinforced by four major academic institutions:

• University of Bristol — communications, electrical engineering, computer science, AI, quantum technology and high-performance computing;
• University of the West of England — robotics, applied engineering, digital technology and industry-facing research;
• University of Bath — electronics, wireless systems, computer science and advanced engineering;
• University of Exeter, within the broader regional definition — materials, photonics and computing.

The Bristol Robotics Laboratory, jointly operated by UWE and the University of Bristol, is a particularly visible example of institutional cooperation. The region also benefits from SETsquared, university incubators, Engine Shed, Future Space and the Bristol & Bath Science Park.

What Silicon Gorge specialises in

Silicon Gorge is broader than ordinary software development. Its strongest identity is in deep technology, particularly fields where hardware, physics and software meet.

Semiconductor and processor design

Bristol has specialised heavily in fabless semiconductor design. That means companies design processors, accelerators, communications chips and intellectual property locally but generally use overseas foundries for fabrication.

Important regional traditions include:

• parallel processor architectures;
• graphics and multimedia processors;
• cellular modem silicon;
• wireless baseband processing;
• network processors;
• AI accelerators;
• automotive control electronics;
• configurable and embedded processors.

Graphcore’s Intelligence Processing Unit is a recent expression of this lineage: a specialised processor architecture for machine-learning workloads. Graphcore itself emerged from founders and engineers with links to earlier Bristol semiconductor companies, particularly XMOS and Icera.

Wireless and telecommunications

The cluster has long-standing expertise in:

• cellular modem design;
• radio-frequency engineering;
• 5G and emerging 6G systems;
• millimetre-wave communications;
• network optimisation;
• satellite communications.

Companies such as Icera, picoChip and Blu Wireless illustrate this strand.

Aerospace, robotics and autonomous systems

Filton and North Bristol form one of Britain’s most important aerospace concentrations, with activities connected to Airbus, Rolls-Royce, defence, composites and advanced manufacturing.

This intersects with:

• robotics;
• drones;
• autonomous vehicles;
• machine vision;
• safety-critical control;
• simulation and digital twins.

The National Composites Centre and Bristol Robotics Laboratory give the region substantial research infrastructure in these areas. The West of England strategy describes the regional aerospace and defence concentration as the largest in the UK and among the largest in Europe.

AI and high-performance computing

The region’s background in parallel processors naturally feeds into AI computing and supercomputing.

Bristol now hosts Isambard-AI, a major national AI research-computing facility associated with the University of Bristol and the National Composites Centre. This reinforces a regional chain stretching from semiconductor architecture through systems software to large-scale AI infrastructure.

Creative technology

Bristol’s unusual advantage is that its engineering community overlaps with a major creative sector.

The city has strengths in:

• natural-history television;
• animation;
• visual effects;
• games;
• immersive media;
• virtual production;
• digital sound;
• motion capture.

This creates demand for graphics, storage, networking, rendering, AI and media-processing technology. Bristol is therefore not only a silicon cluster; it is also a place where advanced computing is applied to cultural production.

Geography of the cluster

Silicon Gorge is distributed rather than concentrated in one downtown district.

Central Bristol and Temple Quarter contain start-ups, incubators, investors, university activity and digital businesses.

Clifton and the University of Bristol precinct contain research groups, spin-outs and academic laboratories.

Frenchay contains UWE and the Bristol Robotics Laboratory.

Filton and Abbey Wood form the aerospace, defence and advanced-engineering zone.

Aztec West became particularly associated with electronics, semiconductor design and corporate research facilities.

Emersons Green and the Bristol & Bath Science Park contain the National Composites Centre, science businesses and research infrastructure.

Bath adds university research, software, publishing, chips, wireless technology and creative businesses.

This dispersed geography is one reason “Silicon Gorge” feels more like a regional knowledge network than a single technology park.

What makes Bristol attractive to technology companies?

Bristol offers a combination that is difficult to replicate:

• a large pool of experienced hardware and software engineers;
• universities producing highly qualified graduates;
• a lineage of semiconductor companies going back several decades;
• aerospace, defence and telecommunications customers nearby;
• proximity to the South Wales compound-semiconductor ecosystem;
• relatively strong access to London, the Midlands and South Wales;
• a cultural environment that attracts technical and creative workers;
• repeated circulation of staff between multinationals, scale-ups and start-ups.

Its most valuable asset is arguably not any single company. It is the dense professional network through which experienced engineers repeatedly form new teams.

Limitations and pressures

Silicon Gorge should not be romanticised as a British equivalent of Silicon Valley at the same scale.

The region faces significant constraints:

Capital: British deep-tech companies frequently struggle to raise the very large, patient investment rounds needed to compete with US semiconductor and AI companies.

Acquisition leakage: Many successful local companies have been purchased by larger foreign businesses. These acquisitions reward founders and investors but can eventually move ownership, strategy or intellectual property outside the region.

Manufacturing: Bristol is predominantly a design and R&D cluster. It does not possess leading-edge semiconductor fabrication capacity comparable with Taiwan, South Korea or the United States.

Housing and transport: High property costs, congestion and fragmented public transport make recruitment and commuting harder. The West of England’s own economic planning has identified high housing costs and congestion as constraints on growth.

Scale-up gap: The ecosystem is good at producing technically sophisticated start-ups, but fewer companies grow into enduring, independently controlled global corporations.

Unequal prosperity: The high-income technology economy exists alongside substantial deprivation and inequality in other parts of Bristol. The benefits of the cluster are not evenly distributed across the city.

Bristol’s wider significance

Bristol’s importance is not simply that it contains a collection of technology offices. It represents a long, continuous evolution:

maritime engineering → aviation → electronics → semiconductors → telecommunications → parallel computing → AI, robotics and advanced systems.

That continuity explains why the Silicon Gorge label is credible. Bristol has repeatedly adapted its engineering culture to successive technological eras.

The term therefore describes both a location and an industrial lineage: a regional ecosystem in which universities, aerospace, chip design, communications, software and creative industries continually reinforce one another.

Graphcore

Graphcore is a Bristol-founded semiconductor company that created the Intelligence Processing Unit, or IPU—a massively parallel processor designed specifically for artificial intelligence rather than adapted from graphics or general-purpose computing.

It became one of the most prominent companies associated with Bristol’s Silicon Gorge, but struggled to convert strong technology into enough commercial scale to compete with Nvidia. In July 2024, Graphcore was acquired by Japan’s SoftBank Group and became a wholly owned subsidiary. It continues to operate under the Graphcore name, with its headquarters in central Bristol.

Origins in Bristol

Graphcore emerged from Bristol’s long semiconductor lineage.

The original idea was developed by Nigel Toon and Simon Knowles, experienced British chip entrepreneurs whose previous companies included Element 14, picoChip, Icera and XMOS. Graphcore says the concept began during a meeting in Bath in 2012, with a development team subsequently assembled in Bristol. The company was formally incorporated in 2016.

That history matters because Graphcore was not an isolated start-up. It drew on a regional network of engineers with experience in:

• parallel processor architecture;
• mobile communications silicon;
• digital signal processing;
• compiler design;
• embedded systems;
• high-speed interconnects;
• semiconductor verification and physical design.

Graphcore is therefore one of the clearest modern descendants of the INMOS–transputer tradition in Silicon Gorge: use many independently programmable processors, connect them efficiently and let software expose large-scale parallelism.

What is an IPU?

Graphcore named its processor the Intelligence Processing Unit.

A conventional CPU contains a relatively small number of sophisticated cores designed to execute many different kinds of sequential software. A GPU contains many simpler execution units optimised primarily for dense, regular numerical operations. Graphcore’s IPU takes a different approach: it uses a very large number of independently programmable processing tiles, each with local memory, connected by a high-bandwidth on-chip communication fabric.

The objective is to keep both computation and model state distributed across the processor rather than repeatedly moving data between a distant external memory system and a central compute array.

Conceptually, an IPU looks like this:

Many processing tiles
↕
Local SRAM attached directly to each tile
↕
Fast on-chip exchange fabric
↕
High-speed links to other IPUs

This architecture is especially suited to workloads containing:

• sparse computation;
• irregular execution graphs;
• many small operations;
• graph neural networks;
• dynamic or branching models;
• models requiring extensive communication between parallel tasks;
• workloads where low latency matters as much as raw matrix throughput.

The tile architecture

Graphcore’s second-generation GC200 processor contains 1,472 independent processor tiles capable of running thousands of parallel threads. Each tile has dedicated local memory, while the processor provides approximately 900 MB of distributed on-chip memory.

This is fundamentally different from treating memory as one large external pool.

On a conventional accelerator, a kernel often fetches tensors from external high-bandwidth memory, performs a calculation and writes the result back. On an IPU, the compiler attempts to place program code and tensor data close to the tile that will operate on them. Intermediate values can remain on-chip and move directly between tiles.

That gives the IPU three architectural characteristics:

Fine-grained parallelism

Individual tiles can execute independent instruction streams. The processor is therefore closer to a distributed computer condensed onto one chip than to a single large matrix engine.

Distributed local memory

Memory is physically distributed across the tiles. This gives very high aggregate on-chip bandwidth, but it also means that data placement is crucial.

Explicit communication

Data movement between tiles is scheduled through the IPU’s exchange fabric. This can make communication predictable and efficient, although it gives the compiler and programming tools more work to do.

Poplar: the software stack

The IPU cannot be separated from Poplar, Graphcore’s software platform.

Poplar maps a machine-learning computation graph onto IPU tiles. It decides:

• which tile executes each operation;
• where tensors and program state reside;
• when data moves between tiles;
• how computation and communication overlap;
• how a model is partitioned across multiple IPUs;
• how pipeline and data parallelism are organised.

Graphcore also developed libraries, framework integrations, debugging tools and profiling software around Poplar. IPU systems support the Poplar SDK as the complete programming and runtime environment for Graphcore hardware.

This was both Graphcore’s technical strength and one of its commercial difficulties.

By controlling the complete stack, Graphcore could optimise software specifically for its architecture. However, customers had to adopt a platform outside the dominant Nvidia CUDA ecosystem. That meant retraining engineers, porting workloads and accepting a much smaller library and community base.

Graphcore’s processor generations

Colossus MK1

Graphcore’s first commercial IPU established the basic tiled architecture and demonstrated that a processor designed specifically for machine intelligence could be manufactured and used in production systems.

Colossus MK2—GC200

The second-generation GC200 significantly increased processing capacity, local memory and system scalability. Graphcore used multiple GC200 processors inside its IPU-Machine systems and larger IPU-Pod configurations.

Bow IPU

Bow was an enhanced version of the GC200 architecture using wafer-on-wafer three-dimensional silicon integration.

Instead of obtaining power through conventional package connections alone, Graphcore stacked a specialised power-delivery wafer beneath the processor wafer. Graphcore reported that Bow systems delivered as much as 40% higher performance and improved energy efficiency over the earlier generation.

The important point is that Bow was not simply a smaller manufacturing node. It was an innovative packaging and power-delivery design intended to let the existing compute silicon operate more effectively.

IPU-Machines and IPU-Pods

Graphcore sold complete accelerator systems rather than only bare processor chips.

A Bow-2000 IPU-Machine contains four Bow IPUs and can be connected to host servers. Multiple machines can then be assembled into IPU-Pods:

• Bow Pod16: four IPU-Machines and 16 IPUs;
• Bow Pod64: 16 IPU-Machines and 64 IPUs;
• Bow Pod256: 64 IPU-Machines and 256 IPUs.

Graphcore’s reference design for Bow Pod64 delivers more than 22 petaFLOPS of quoted AI compute, while Pod systems can be combined into still larger installations.

Graphcore also developed its IPU-Link and IPU-Fabric technologies to connect processors within machines, racks and larger scale-out systems.

Where the IPU could outperform GPUs

Graphcore’s architecture can be particularly effective when the workload does not map neatly onto large, dense matrix multiplications.

Potentially favourable workloads include:

• graph neural networks;
• recommendation models;
• probabilistic models;
• sparse neural networks;
• molecular and scientific simulations;
• small-batch, latency-sensitive inference;
• models with complex control flow;
• applications requiring frequent communication between many independent operations.

Independent research has found the IPU technically interesting for fields including particle physics, confidential machine learning, battery-material modelling and compiler research. Results vary by application, but they demonstrate that the IPU is a credible alternative architecture rather than merely a branding exercise.

Why Graphcore did not displace Nvidia

Graphcore was repeatedly described as a potential “British Nvidia,” but that comparison underestimated how much of Nvidia’s advantage comes from its ecosystem.

CUDA’s software moat

Nvidia offers more than GPUs. CUDA is surrounded by:

• mature compilers;
• numerical libraries;
• distributed-training frameworks;
• profilers;
• inference servers;
• container images;
• cloud services;
• vendor-qualified systems;
• a vast developer community.

Graphcore had to persuade customers not only that its processor was competitive but that migrating away from CUDA was worthwhile.

AI workloads became increasingly GPU-friendly

Graphcore expected future AI to involve sparse, irregular and graph-like computation. Some workloads did evolve that way, but the commercial AI boom became dominated by transformers and large language models.

Transformers rely heavily on dense matrix multiplication. GPUs—and subsequently specialised tensor cores—are extremely effective at this. Nvidia was therefore able to exploit its existing architecture, ecosystem and manufacturing scale.

Customers wanted standard infrastructure

Cloud operators and major AI laboratories generally preferred hardware that could support a broad portfolio of existing workloads with minimal porting. Nvidia systems became the de facto standard, making alternative accelerators harder to introduce.

Scale and capital

Developing an advanced processor is only part of the cost. A successful AI-compute company must also fund:

• new silicon generations;
• compiler and library development;
• board and server engineering;
• networking technology;
• cloud deployment;
• manufacturing commitments;
• customer support;
• global sales;
• developer relations.

Graphcore raised substantial investment and reached a reported valuation of $2.77 billion in 2020, but remained much smaller than Nvidia, AMD, Intel, Google or the largest cloud providers. By 2024 it had reduced its workforce and withdrawn from several markets while seeking further capital.

Loss of the Chinese market

China had been a potentially important market for Graphcore. US export restrictions affecting advanced AI technology and systems containing US-origin components made that business increasingly difficult, and Graphcore ultimately closed its Chinese operation.

The SoftBank acquisition

On 11 July 2024, Graphcore announced that it had been acquired by SoftBank Group.

The price was not officially disclosed. Contemporary reporting placed it in the region of $500–600 million—far below Graphcore’s previous $2.77 billion valuation and reportedly below the total amount it had raised from investors. The acquisition nevertheless removed the immediate financing threat and gave Graphcore access to a parent company prepared to invest heavily in AI infrastructure.

Graphcore retained:

• its company name;
• its Bristol headquarters;
• its engineering organisation;
• its focus on AI silicon, systems and software.

SoftBank already controlled Arm and was assembling a broader portfolio around AI infrastructure. In its 2025 strategy presentation, SoftBank explicitly grouped Graphcore with Arm and Ampere as part of its plan to supply computing capacity for AI data centres.

Graphcore co-founder Simon Knowles left the company in August 2025, according to Companies House records and Graphcore’s confirmation to Sifted.

Graphcore’s current direction

As of July 2026, Graphcore appears to be transitioning from an independent accelerator vendor into a well-funded AI-compute engineering organisation inside SoftBank.

Its public messaging now refers to developing an entire next-generation AI stack, including:

• silicon;
• accelerator systems;
• system software;
• compilers;
• large-scale AI infrastructure.

Graphcore has also opened an AI engineering campus in Bengaluru. It announced plans to invest approximately £1 billion in India over ten years and create around 500 semiconductor engineering jobs. The company says the Indian team will work on semiconductor products for advanced AI users.

This suggests a strategic change. The original Graphcore proposition was:

Buy an IPU instead of, or alongside, an Nvidia GPU.

The emerging proposition appears closer to:

Use Graphcore’s architecture, compiler, silicon and systems expertise to help SoftBank construct a broader AI-compute platform.

Precisely how much of that future platform will preserve the existing IPU architecture has not been publicly detailed. Graphcore’s public product pages still describe IPUs and Bow systems, but the company’s newer communications emphasise next-generation end-to-end AI systems more than direct promotion of current-generation Bow hardware. That is a reasonable indication of a technological and commercial reset, although the architecture of the resulting products remains undisclosed.

Why Graphcore matters to Bristol

Graphcore is arguably the most ambitious semiconductor company created in Bristol in the modern era.

It demonstrates that the region can still produce:

• original processor architectures;
• world-class compiler technology;
• complex multi-die systems;
• advanced silicon packaging;
• globally recognised deep-technology companies.

It also connects several generations of Bristol chip engineering:

INMOS and the transputer
→ Element 14, picoChip and Icera
→ XMOS
→ Graphcore and the IPU

Its Bristol headquarters remains on Wine Street in the city centre, rather than in a remote science park. Graphcore also lists engineering locations including Cambridge, London, Gdańsk, Bengaluru, Taiwan and several US offices.

Overall assessment

Graphcore was technically bold but commercially mistimed.

Its central insight—that machine intelligence would require a different kind of highly parallel computer—was correct. Its tiled processors, distributed memory model, Poplar compiler and wafer-on-wafer Bow design were serious innovations.

The difficulty was that Graphcore was not competing against a single Nvidia chip. It was competing against:

• CUDA;
• Nvidia’s software libraries;
• the major cloud platforms;
• the global server supply chain;
• Nvidia’s networking business;
• customer familiarity;
• an accelerating cadence of GPU development.

As an independent company, Graphcore did not have the capital, market access or ecosystem required to overcome those barriers. Under SoftBank, it now has a second opportunity—but likely with a different mission: not simply to sell IPUs as an alternative accelerator, but to contribute to a vertically integrated, next-generation AI-compute platform.

Vertically Integrated Next-generation AI Compute

A vertically integrated AI-compute platform is not merely an accelerator card or a faster processor. It is a coordinated system in which one organisation—or a tightly controlled group of companies—designs and optimises most of the critical layers:

models and workloads
↓
AI frameworks, compilers and runtime
↓
distributed execution and orchestration
↓
accelerator, CPU and network architecture
↓
servers, racks, cooling and power
↓
data-centre infrastructure and cloud services

The purpose of vertical integration is to eliminate mismatches between these layers. Instead of designing a chip and hoping existing software uses it efficiently, the platform co-designs the silicon, memory, interconnect, compiler, runtime, rack and operating model around the expected AI workloads.

Graphcore publicly describes its current direction as building a “next-generation AI stack” spanning silicon, hardware and software. SoftBank’s ownership of Graphcore, Arm and Ampere gives it access to complementary accelerator, CPU and semiconductor-IP capabilities, although the precise architecture of any future combined platform has not been publicly disclosed.

---

1. The fundamental design unit becomes the rack

Traditional computing treats the processor or server as the main product.

Next-generation AI infrastructure increasingly treats the entire rack as one computer.

A rack-scale system might contain:

• 32–72 accelerator devices;
• 16–36 host or control CPUs;
• terabytes of high-bandwidth accelerator memory;
• several layers of high-speed switching;
• dedicated data-processing units, or DPUs;
• local NVMe storage;
• liquid-cooling distribution;
• rack-level power conversion;
• hardware management controllers;
• integrated telemetry and failure recovery.

The accelerators are joined by a very high-bandwidth, low-latency scale-up fabric, allowing the whole rack to behave more like one large parallel machine than dozens of independent servers.

Nvidia’s GB300 NVL72 is a current example of this architectural direction: it integrates 72 Blackwell Ultra GPUs and 36 Grace CPUs, using NVLink to create a rack-scale compute domain. It is not the only possible design, but it illustrates what “platform” now means in AI infrastructure.

A future Graphcore-oriented platform would likely apply the same principle using a successor to its tiled IPU architecture:

AI rack
├── Accelerator trays
│   ├── AI compute chiplets
│   ├── High-bandwidth memory
│   ├── On-package fabric
│   └── Scale-up links
├── CPU / control trays
├── Fabric switches
├── Local storage
├── Management network
├── Power shelves
└── Liquid-cooling manifold

The rack would be sold and operated as a tested unit with known:

• power consumption;
• cooling requirements;
• communication topology;
• collective-operation performance;
• model capacity;
• failure domains;
• training and inference throughput.

---

2. AI accelerator silicon

The centre of the platform is a specialised AI processor.

A next-generation accelerator would not simply provide more floating-point operations. It would need to optimise several different classes of computation.

Dense tensor computation

Transformers depend heavily on matrix multiplication:$$C = A \times B$$C=A×B

The accelerator therefore needs highly efficient units for:

• matrix multiply-accumulate;
• attention operations;
• convolution;
• tensor transposition;
• reductions;
• normalisation;
• activation functions.

It must support several numerical formats, such as:

• FP32 for high-precision accumulation;
• BF16 and FP16 for training;
• FP8 for high-throughput training and inference;
• INT8 and INT4 for quantised inference;
• potentially still lower or custom formats.

The important metric is not peak operations per second, but sustained useful operations per watt while running actual models.

Sparse and irregular computation

Next-generation workloads increasingly include:

• mixture-of-experts routing;
• sparse attention;
• graph neural networks;
• retrieval;
• agentic workflows;
• dynamic batching;
• variable-length sequences;
• reinforcement learning;
• model-based scientific simulation.

These workloads require more than large matrix units. The processor also needs:

• efficient local control flow;
• independent processing cores;
• fast scatter/gather;
• dynamic task scheduling;
• sparse tensor support;
• low-overhead synchronisation;
• rapid communication between processing elements.

This is where Graphcore’s tiled architecture may remain relevant. Its design philosophy distributes many programmable processing tiles and local memories across the chip, rather than treating the accelerator mainly as a collection of large tensor engines.

A future architecture could combine both approaches:

Next-generation AI die
├── Large tensor engines
├── Programmable processing tiles
├── Sparse-compute engines
├── Attention accelerators
├── Data-movement engines
├── Collective-communication engines
├── Security processors
└── On-chip network

That would make the processor less specialised around one current model architecture and more capable of adapting to future workloads.

---

3. Chiplets and advanced packaging

Building one enormous monolithic processor is increasingly difficult and expensive. A next-generation platform is therefore likely to use chiplets.

Instead of manufacturing one large die containing every function, the package may combine:

• one or more compute chiplets;
• memory-controller chiplets;
• scale-up fabric chiplets;
• PCIe or CXL I/O dies;
• security and management dies;
• high-bandwidth memory stacks;
• power-delivery components.

                    AI accelerator package

 ┌────────────┬────────────┬────────────┬────────────┐
 │ Compute 0  │ Compute 1  │ Compute 2  │ Compute 3  │
 └─────┬──────┴─────┬──────┴─────┬──────┴─────┬──────┘
       │         High-speed die fabric        │
 ┌─────┴──────┬────────────┬────────────┬─────┴────────┐
 │ HBM stack  │ Fabric I/O │ CPU/CXL I/O│ HBM stack    │
 └────────────┴────────────┴────────────┴──────────────┘

Chiplets offer several advantages:

• better manufacturing yield;
• reusable functional blocks;
• mixing different semiconductor processes;
• faster product iteration;
• configurable products for different markets;
• easier scaling of compute and I/O;
• potential integration of third-party IP.

Arm is actively developing chiplet-oriented compute subsystems and has contributed a vendor-neutral Chiplet System Architecture specification to the Open Compute Project. Arm describes this as part of an effort to make interoperable AI-data-centre chiplets easier to create.

Graphcore already demonstrated advanced packaging with its Bow IPU, which placed a power-delivery wafer beneath the processor wafer. A successor could go further through:

• 2.5D silicon interposers;
• wafer-on-wafer stacking;
• backside power delivery;
• vertically stacked SRAM;
• optical I/O;
• chiplet-to-chiplet interfaces.

The next performance gains will increasingly come from packaging, memory placement and power delivery—not only smaller transistors. Arm similarly identifies chiplets, 3D stacking and advanced packaging as important directions beyond conventional process scaling.

---

4. A heterogeneous CPU–accelerator architecture

The accelerator cannot efficiently perform every task. A complete platform also requires general-purpose CPUs.

The CPU handles:

• operating-system services;
• workload launch and control;
• data preprocessing;
• tokenisation;
• storage coordination;
• networking;
• model orchestration;
• checkpoint management;
• CPU-side inference stages;
• control-plane agents;
• security and monitoring.

A vertically integrated SoftBank architecture could plausibly combine:

• Arm intellectual property;
• Ampere server-CPU engineering;
• Graphcore accelerator and compiler expertise.

This is an architectural possibility, not a publicly confirmed product design.

The CPU and accelerator need a much tighter relationship than the traditional model of a PCIe-connected graphics card. Possible configurations include:

Discrete attachment

Arm CPU ── PCIe/CXL ── AI accelerator

This is relatively straightforward but may create a bandwidth and latency bottleneck.

Coherent attachment

CPU memory  ⇄  coherent interconnect  ⇄  accelerator memory

The CPU and accelerator can access parts of a shared address space. This simplifies programming and reduces explicit copying.

Integrated superchip

┌───────────────────────────────────────┐
│ Arm CPU chiplets                      │
│        coherent die-to-die fabric     │
│ AI accelerator chiplets               │
│ Shared memory and I/O controllers     │
└───────────────────────────────────────┘

This provides lower latency and potentially lower energy consumption, but it is more expensive and complex to design.

---

5. Memory hierarchy

AI performance is increasingly constrained by moving data rather than performing arithmetic.

The platform therefore needs a deliberate memory hierarchy.

Level 1: registers and tile-local memory

Memory immediately adjacent to the execution units provides:

• very low latency;
• high aggregate bandwidth;
• deterministic access;
• efficient storage of temporary values.

Graphcore’s IPUs emphasised distributed local SRAM. A future design could retain this while adding larger tensor engines and high-bandwidth external memory.

Level 2: shared on-chip cache or SRAM

This supports:

• communication between nearby cores;
• reuse of attention blocks;
• shared expert weights;
• collective-operation buffers;
• local key-value cache.

Level 3: high-bandwidth memory

HBM provides the main accelerator memory for:

• model parameters;
• activations;
• gradients;
• optimiser state;
• inference key-value caches.

A modern AI processor may require hundreds of gigabytes of HBM per package and multiple terabytes per rack.

Level 4: coherent pooled memory

CXL or a proprietary fabric may expose additional memory connected to CPUs, memory appliances or other accelerators.

This lower-cost capacity could hold:

• cold model weights;
• embeddings;
• large retrieval indexes;
• inactive mixture-of-experts parameters;
• checkpoint staging buffers.

Level 5: local NVMe flash

NVMe storage provides:

• fast checkpoint writes;
• dataset caching;
• model loading;
• temporary training data;
• inference model repositories.

Level 6: distributed storage

The rack connects to an object or parallel filesystem containing:

• training datasets;
• checkpoints;
• model versions;
• vector indexes;
• evaluation data;
• operational logs.

The compiler and runtime should understand this hierarchy and decide what belongs where.

Fastest / smallest

Registers
Tile-local SRAM
Shared on-chip cache
HBM
Pooled CXL memory
Local NVMe
Distributed flash storage
Object storage

Slowest / largest

---

6. Scale-up interconnect inside the rack

The scale-up network joins accelerators into a single compute domain.

This is different from an ordinary Ethernet network. It needs:

• extremely high bandwidth;
• very low latency;
• remote memory access;
• hardware collective operations;
• congestion control;
• topology awareness;
• link-level retry;
• predictable synchronisation.

Important collective operations include:

• all-reduce;
• all-gather;
• reduce-scatter;
• broadcast;
• all-to-all.

For example, distributed training frequently uses an all-reduce operation to combine gradients:$$G = \sum_{i=1}^{N} G_i$$G=i=1∑N​Gi​

If this communication is inefficient, accelerators spend much of their time idle.

Mixture-of-experts models place especially high pressure on all-to-all communication, because tokens must be routed to different expert processors and their results returned.

A next-generation fabric may contain dedicated collective engines so that reductions and data movement occur without occupying the main compute cores.

Accelerator 0 ─┬──────────┬─ Accelerator 1
Accelerator 2 ─┤ Fabric   ├─ Accelerator 3
Accelerator 4 ─┤ Switches ├─ Accelerator 5
Accelerator 6 ─┴──────────┴─ Accelerator 7

At rack level, the architecture may provide a uniform communication domain across dozens of accelerators, similar in principle to Nvidia’s use of NVLink within its NVL72 rack systems.

---

7. Scale-out networking between racks

A single rack is insufficient for the largest training runs. Hundreds or thousands of racks must communicate.

The scale-out network typically uses:

• high-speed Ethernet or InfiniBand;
• leaf-spine topology;
• rail-optimised connectivity;
• remote direct memory access;
• adaptive routing;
• congestion telemetry;
• lossless or near-lossless transport;
• in-network collective acceleration.

                         Spine switches
                  ┌────────┬────────┬────────┐
                  │        │        │        │
              ┌───┴──┐ ┌───┴──┐ ┌───┴──┐ ┌───┴──┐
              │Leaf 0│ │Leaf 1│ │Leaf 2│ │Leaf 3│
              └─┬─┬──┘ └─┬─┬──┘ └─┬─┬──┘ └─┬─┬──┘
                │ │      │ │      │ │      │ │
                Rack     Rack     Rack     Rack

Multiple network planes are often separated:

1. Compute fabric — accelerator-to-accelerator traffic.
2. Storage fabric — dataset and checkpoint traffic.
3. Management network — provisioning, telemetry and control.
4. Service network — inference and customer API traffic.

Keeping these traffic classes separate prevents checkpoint transfers or user requests from disrupting distributed training.

Modern reference architectures commonly use scalable leaf-spine designs for this reason.

---

8. Data-processing units and infrastructure processors

A vertically integrated system should avoid wasting expensive accelerator cycles on infrastructure work.

A DPU or infrastructure processing unit can offload:

• virtual networking;
• storage protocols;
• encryption;
• firewalling;
• software-defined networking;
• remote direct memory access;
• telemetry;
• multi-tenant isolation;
• storage compression;
• checkpoint transport.

External network
      │
      ▼
┌──────────────┐
│ DPU / IPU    │
│ Networking   │
│ Storage      │
│ Security     │
└──────┬───────┘
       │
       ▼
CPU and AI accelerators

This creates a secure boundary between the customer workload and the physical host.

A DPU can also permit the cloud operator to manage and inspect the machine even when the customer controls the host CPU.

---

9. Compiler-led hardware operation

The compiler is one of the most important layers in the entire platform.

A vertically integrated compiler does much more than convert source code into machine instructions. It must map an enormous distributed computation onto thousands of processor cores and multiple memory domains.

Its responsibilities include:

• graph capture;
• operator fusion;
• tensor layout;
• numerical precision selection;
• kernel generation;
• memory allocation;
• activation rematerialisation;
• communication scheduling;
• pipeline partitioning;
• tensor parallelism;
• data parallelism;
• expert parallelism;
• checkpoint placement;
• topology-aware routing.

For example, a transformer block may be partitioned across devices using several simultaneous dimensions:

Global model
├── Pipeline parallelism
│   ├── Layers 0–19  → Rack group A
│   ├── Layers 20–39 → Rack group B
│   └── Layers 40–59 → Rack group C
│
├── Tensor parallelism
│   └── Each matrix split across 8 accelerators
│
├── Data parallelism
│   └── Model replica repeated 16 times
│
└── Expert parallelism
    └── MoE experts distributed across 64 devices

The compiler must choose a strategy that minimises communication while respecting:

• device memory;
• interconnect topology;
• power limits;
• numerical requirements;
• batch size;
• target latency.

Graphcore’s Poplar stack already follows a compiler-centric approach, mapping computation and data placement onto IPU tiles. A successor platform could extend this concept from one processor to an entire rack or data centre.

---

10. Kernel and library ecosystem

The platform requires highly optimised libraries for standard AI operations.

These include equivalents of:

• BLAS and matrix multiplication libraries;
• attention kernels;
• convolution libraries;
• collective communication libraries;
• quantisation kernels;
• embedding lookup;
• sparse operations;
• graph operations;
• random-number generation;
• scientific-computing primitives.

The difference between theoretical and actual performance often lies in these libraries.

A processor may advertise excellent peak throughput but perform poorly if its software cannot efficiently execute:

• grouped-query attention;
• mixture-of-experts routing;
• paged key-value caches;
• speculative decoding;
• low-rank adapters;
• dynamic sequence lengths;
• quantised matrix multiplication.

The software team must continually optimise for changing model architectures.

This is one reason vertical integration matters: the model-runtime team can identify a bottleneck, the compiler team can alter scheduling, and the silicon team can add a hardware instruction or data path in the next generation.

---

11. Framework compatibility

Most users should not need to program the processor directly.

The platform needs support for mainstream frameworks and model representations:

• PyTorch;
• JAX;
• TensorFlow where required;
• ONNX;
• MLIR-based intermediate representations;
• Hugging Face models;
• common distributed-training interfaces.

A practical flow might be:

PyTorch model
     │
     ▼
Graph capture / torch.compile
     │
     ▼
Platform intermediate representation
     │
     ├── graph optimisation
     ├── operator fusion
     ├── placement
     └── communication planning
     ▼
Device kernels and distributed executable

Compatibility is strategically critical. Requiring customers to rewrite complete applications for proprietary APIs creates a major adoption barrier.

The ideal platform provides two modes:

• a portable compatibility layer for easy migration;
• a native programming model for maximum performance.

---

12. Distributed runtime

The runtime executes the compiler’s plan and manages the live system.

It controls:

• device discovery;
• process launch;
• memory registration;
• distributed barriers;
• communication groups;
• collective operations;
• error propagation;
• fault recovery;
• dynamic workload resizing;
• profiling;
• performance counters.

At scale, the runtime must assume hardware failures are normal.

A training job using 10,000 accelerators over several weeks will probably encounter:

• link errors;
• memory errors;
• device resets;
• power incidents;
• network switch failures;
• storage timeouts;
• operating-system crashes.

The runtime must detect and respond without discarding days of work.

Possible strategies include:

• redundant communication paths;
• spare accelerators;
• process restart;
• failed-rank replacement;
• incremental checkpoints;
• asynchronous checkpoints;
• elastic training;
• degraded-topology operation.

---

13. Training platform

The training environment combines the compute runtime with a control plane.

A user submits a job describing:

• model;
• dataset;
• accelerator count;
• parallelisation policy;
• container image;
• checkpoint location;
• priority;
• expected duration;
• fault-recovery policy.

The platform then performs:

1. capacity allocation;
2. topology-aware placement;
3. environment construction;
4. dataset staging;
5. process launch;
6. telemetry collection;
7. checkpoint management;
8. failure recovery;
9. result publication.

User / MLOps pipeline
        │
        ▼
Training control plane
        │
        ├── Scheduler
        ├── Topology manager
        ├── Dataset service
        ├── Checkpoint service
        ├── Secrets manager
        └── Observability
        │
        ▼
Rack-scale compute cluster

Schedulers may build on Kubernetes, Slurm or a proprietary system, but they need AI-specific extensions.

Ordinary CPU scheduling is insufficient because AI jobs require:

• gang scheduling;
• simultaneous allocation of hundreds of devices;
• network-topology awareness;
• homogeneous accelerator groups;
• predictable communication performance;
• long-running reservation;
• rapid recovery.

---

14. Inference platform

Inference has different requirements from training.

Training prioritises:

• aggregate throughput;
• large batches;
• collective communication;
• checkpoint resilience.

Inference prioritises:

• time to first token;
• inter-token latency;
• requests per second;
• predictable tail latency;
• model-loading speed;
• energy per token;
• multi-tenancy.

A next-generation inference platform needs a model-serving layer capable of:

• continuous batching;
• dynamic batching;
• paged key-value cache management;
• prompt caching;
• speculative decoding;
• model parallelism;
• expert routing;
• quantisation;
• adapter loading;
• autoscaling;
• admission control.

User requests
      │
      ▼
Global request router
      │
      ├── Model selection
      ├── Tenant policy
      ├── Prompt-cache lookup
      └── Load balancing
      │
      ▼
Inference workers
      │
      ├── Prefill accelerators
      ├── Decode accelerators
      ├── KV-cache tier
      └── Retrieval services

Disaggregated inference

A sophisticated platform may separate prefill and decode.

Prefill processes the input prompt and is highly compute-intensive.

Decode generates tokens one at a time and is often constrained by memory bandwidth and key-value-cache access.

Different processor configurations can therefore be assigned to each stage:

Prompt
  │
  ▼
Prefill cluster — high tensor throughput
  │
  ▼
Distributed KV cache
  │
  ▼
Decode cluster — low latency, high memory bandwidth
  │
  ▼
Generated tokens

Vertical integration allows the silicon, memory system and scheduler to be optimised separately for both phases.

---

15. AI-native storage

AI storage cannot be treated as an afterthought.

The platform needs to support:

• multi-petabyte datasets;
• billions of small objects;
• parallel sequential reads;
• random sample access;
• checkpoint bursts;
• model registries;
• vector databases;
• key-value-cache spill;
• dataset versioning.

A typical hierarchy is:

Object-storage data lake
          │
          ▼
High-performance shared filesystem
          │
          ▼
Cluster flash cache
          │
          ▼
Rack-local NVMe
          │
          ▼
Accelerator HBM

The storage system should understand AI workloads and perform:

• prefetching;
• sharding;
• caching;
• decompression;
• format conversion;
• data augmentation;
• checkpoint deduplication;
• erasure coding;
• replication.

The accelerator should receive data in the format it requires without expensive CPU-side transformation.

---

16. Data pipelines

Training accelerators are extremely expensive. They must not sit idle while CPUs decode files or fetch remote objects.

A vertically integrated data pipeline may use specialised services or hardware engines for:

• image decoding;
• video decoding;
• tokenisation;
• compression;
• encryption;
• dataset filtering;
• shuffle generation;
• augmentation;
• batch assembly.

Object storage
      │
      ▼
Dataset reader
      │
      ▼
Decode / tokenisation workers
      │
      ▼
Shuffle and batch service
      │
      ▼
Local cache
      │
      ▼
Accelerator

The platform scheduler should allocate data-pipeline capacity together with accelerator capacity.

---

17. Rack power and liquid cooling

Next-generation AI racks may consume roughly an order of magnitude more power than traditional enterprise racks.

This changes the design of the entire data centre.

The rack requires:

• high-voltage input;
• power shelves;
• busbars;
• high-efficiency voltage conversion;
• battery or ride-through support;
• rack-level power monitoring;
• liquid cooling;
• leak detection;
• coolant distribution units;
• thermal control software.

Facility chilled-water loop
          │
          ▼
Coolant distribution unit
          │
          ▼
Rack manifold
          │
          ├── Accelerator cold plates
          ├── CPU cold plates
          ├── Switch cold plates
          └── Power-conversion cooling

The platform must coordinate compute scheduling with physical constraints.

For example, the scheduler may avoid launching several maximum-power jobs in the same electrical zone simultaneously.

Dynamic power management can trade performance against:

• site power limits;
• electricity price;
• cooling capacity;
• carbon intensity;
• component temperature;
• reliability targets.

Rack-scale systems such as NVL72 already use tightly integrated liquid-cooled designs, demonstrating that thermal engineering is now part of the compute architecture itself.

---

18. Firmware and hardware management

Every component needs remotely manageable firmware.

The management plane controls:

• device boot;
• firmware updates;
• secure boot;
• hardware inventory;
• power cycling;
• link diagnostics;
• memory diagnostics;
• thermal telemetry;
• component replacement;
• fault isolation.

A rack-level management controller should expose a unified model rather than requiring operators to manage dozens of individual devices.

Fleet control plane
        │
        ▼
Rack management controller
        │
        ├── Compute trays
        ├── Fabric switches
        ├── Power shelves
        ├── Cooling system
        └── Sensors

Firmware compatibility is part of platform release management. A validated release might specify an exact combination of:

• accelerator firmware;
• CPU firmware;
• switch firmware;
• compiler version;
• runtime version;
• kernel version;
• model-server version.

---

19. Security architecture

A shared AI platform must assume that customer models and datasets are highly sensitive.

Security should include:

• hardware root of trust;
• measured and secure boot;
• signed firmware;
• encrypted memory where practical;
• encrypted links;
• workload identity;
• isolated device memory;
• tenant-specific network segmentation;
• confidential-computing support;
• key management;
• supply-chain attestation.

A user should be able to verify that a workload is running on:

• approved hardware;
• approved firmware;
• an approved system image;
• a correctly isolated accelerator allocation.

The platform may generate a cryptographic attestation report before releasing model weights or encryption keys.

---

20. Observability and performance engineering

AI systems need observability at several layers.

Hardware telemetry

• accelerator utilisation;
• tensor-unit utilisation;
• memory bandwidth;
• HBM errors;
• link utilisation;
• temperature;
• voltage;
• power;
• throttling;
• corrected and uncorrected errors.

Runtime telemetry

• collective-operation duration;
• communication stalls;
• kernel occupancy;
• memory fragmentation;
• tensor rematerialisation;
• data-loader delay;
• checkpoint time.

Model telemetry

• tokens per second;
• time to first token;
• inter-token latency;
• batch size;
• sequence length;
• cache-hit rate;
• expert load balance;
• training loss;
• gradient norms.

Service telemetry

• request rate;
• queue depth;
• tail latency;
• failed requests;
• model availability;
• tenant consumption;
• cost per request.

A vertically integrated platform can correlate these layers:

Model slowdown
     │
     ▼
Runtime identifies slow all-to-all
     │
     ▼
Fabric telemetry identifies congested link
     │
     ▼
Rack manager identifies degraded switch port
     │
     ▼
Scheduler evacuates affected jobs

That is much more powerful than separately monitoring processors, servers and applications.

---

21. Digital twins and fleet optimisation

At hyperscale, the operator may maintain a digital model of every rack and cluster.

The digital twin records:

• topology;
• component age;
• firmware;
• failure history;
• thermal behaviour;
• power profile;
• performance characteristics;
• maintenance state.

The scheduler can use this information to:

• avoid degraded components;
• place tightly coupled jobs on the best fabric;
• predict failures;
• adjust clocks and power;
• schedule maintenance;
• compare real performance with expected performance.

Machine learning may then be used to operate the AI infrastructure itself.

---

22. Cloud control plane

A full platform must expose consumable services.

Possible interfaces include:

• bare-metal rack reservations;
• accelerator virtual machines;
• Kubernetes device pools;
• managed training jobs;
• managed inference endpoints;
• serverless inference;
• model APIs;
• fine-tuning services;
• private AI clusters.

The control plane handles:

• identity;
• quotas;
• billing;
• capacity reservation;
• scheduling;
• network isolation;
• storage attachment;
• model deployment;
• autoscaling;
• service-level objectives.

Customer API
     │
     ▼
Cloud control plane
     ├── Identity
     ├── Billing
     ├── Capacity
     ├── Scheduler
     ├── Network
     ├── Storage
     └── Model lifecycle
     │
     ▼
AI infrastructure fleet

Without this layer, the company is still largely selling hardware. With it, the company sells AI compute as an operating platform.

---

23. Model-level optimisation

The highest form of vertical integration reaches into the model architecture itself.

The platform provider may co-design models with the hardware by controlling:

• attention layout;
• expert count;
• sparsity pattern;
• quantisation;
• sequence-length strategy;
• activation checkpointing;
• cache format;
• tensor dimensions;
• parallelisation strategy.

For example, a hardware-aware mixture-of-experts model could select:

• expert sizes aligned with local memory;
• routing groups aligned with rack topology;
• tensor dimensions aligned with matrix engines;
• quantisation aligned with hardware formats;
• expert placement aligned with interconnect bandwidth.

This can deliver considerably greater benefit than optimising hardware or software independently.

The danger is lock-in: models may become difficult to run efficiently on another platform.

---

24. Continuous hardware–software co-design

Vertical integration creates a feedback loop.

Production model telemetry
          │
          ▼
Identify dominant bottlenecks
          │
          ▼
Improve kernels and compiler
          │
          ▼
Modify runtime and rack topology
          │
          ▼
Add capabilities to next silicon
          │
          ▼
Deploy next platform generation

Examples:

• If inference is limited by KV-cache bandwidth, add larger local memory or compression engines.
• If MoE traffic dominates, improve all-to-all fabric operations.
• If accelerators stall during checkpointing, add DPU-based checkpoint offload.
• If model loading is slow, integrate flash closer to the compute rack.
• If low-precision inference causes accuracy loss, add better mixed-precision instructions.
• If sparse models dominate, increase programmable sparse-compute capacity.

This feedback loop is the main strategic advantage of controlling the full stack.

---

25. What the SoftBank–Arm–Graphcore form could look like

No detailed unified product architecture has yet been publicly announced. However, the assets could theoretically support the following structure:

Arm

Potential contribution:

• CPU instruction-set architecture;
• server CPU subsystems;
• coherent interconnect;
• memory architecture;
• chiplet standards;
• software ecosystem;
• security architecture.

Ampere

Potential contribution:

• high-core-count Arm server CPUs;
• server-platform expertise;
• data-centre deployment experience;
• CPU power-efficiency engineering.

Graphcore

Potential contribution:

• AI accelerator architecture;
• massively parallel processing;
• compiler and graph placement;
• accelerator system design;
• scale-up fabrics;
• AI software tools.

SoftBank

Potential contribution:

• capital;
• strategic coordination;
• data-centre investment;
• market access;
• portfolio integration;
• long investment horizon.

Graphcore explicitly says it now has access to the resources and complementary skills of the wider SoftBank ecosystem. SoftBank has also presented Arm, Graphcore and Ampere as parts of its wider AI-infrastructure strategy.

A hypothetical integrated system could therefore be:

SoftBank AI cloud and data centres
              │
              ▼
Managed training and inference platform
              │
              ▼
Graphcore compiler, runtime and AI system software
              │
              ▼
Rack-scale accelerator fabric
              │
      ┌───────┴────────┐
      ▼                ▼
Graphcore-derived    Arm/Ampere-derived
AI accelerators      host CPUs
      │                │
      └───────┬────────┘
              ▼
Coherent memory, networking and storage
              │
              ▼
Custom liquid-cooled AI racks

This would be analogous in scope—not necessarily architecture or maturity—to Nvidia’s integrated combination of:

• GPUs;
• Grace CPUs;
• NVLink;
• ConnectX networking;
• BlueField DPUs;
• DGX systems;
• CUDA;
• AI libraries;
• model-serving software;
• reference data-centre architectures.

Nvidia’s current platform demonstrates why integration across CPU, accelerator, networking and rack design has become strategically important.

---

26. What would make it genuinely “next generation”?

Merely combining a new CPU and accelerator would not be enough. A genuinely next-generation platform would need several distinguishing capabilities.

One rack-level memory and execution domain

The rack should behave as one logical machine, with efficient access to distributed model state.

First-class support for reasoning workloads

It should optimise for:

• long contexts;
• test-time computation;
• agentic loops;
• reinforcement learning;
• tool use;
• dynamic branching;
• large KV caches.

Native mixture-of-experts execution

The hardware and fabric should support fine-grained expert routing without network bottlenecks.

Unified training and inference

The same platform should efficiently support:

• pre-training;
• post-training;
• reinforcement learning;
• fine-tuning;
• batch inference;
• interactive inference.

Memory-centric design

The architecture should optimise bytes moved per generated token, not merely headline floating-point throughput.

Open framework compatibility

It should run standard PyTorch and common open models without extensive porting.

Automated topology-aware compilation

Users should describe the model and performance target; the compiler should determine the device mapping.

Resilient operation

A failed chip or link should not terminate a multi-week training job.

Energy-aware scheduling

The system should optimise useful model output per watt and per pound—not only raw speed.

Rapid generational iteration

Chiplets and modular rack design should allow compute, memory and networking to evolve independently.

---

27. The central engineering objective

The platform is ultimately trying to minimise the cost of producing useful intelligence.

For training, useful metrics include:$$\text{Cost per trained model}$$Cost per trained model $$\text{Time to target accuracy}$$Time to target accuracy $$\text{Energy per training token}$$Energy per training token

For inference:$$\text{Cost per million tokens}$$Cost per million tokens $$\text{Joules per generated token}$$Joules per generated token $$\text{Requests per second within latency SLO}$$Requests per second within latency SLO

A platform with lower theoretical compute could outperform a nominally faster competitor if it provides:

• better memory utilisation;
• faster communication;
• higher reliability;
• more effective compiler scheduling;
• lower power consumption;
• better model-serving software.

---

Overall architecture

┌──────────────────────────────────────────────────────────────┐
│                    AI applications and agents                │
├──────────────────────────────────────────────────────────────┤
│ Models, training recipes, inference and retrieval services   │
├──────────────────────────────────────────────────────────────┤
│ PyTorch/JAX integration, compiler, kernels and libraries     │
├──────────────────────────────────────────────────────────────┤
│ Distributed runtime, scheduler and model-serving platform    │
├──────────────────────────────────────────────────────────────┤
│ Cloud control plane, security, billing and observability     │
├──────────────────────────────────────────────────────────────┤
│ Scale-out network, storage and data pipeline                 │
├──────────────────────────────────────────────────────────────┤
│ Rack-scale accelerator fabric and coherent memory            │
├──────────────────────────────────────────────────────────────┤
│ AI accelerators, Arm CPUs, DPUs and fabric switches          │
├──────────────────────────────────────────────────────────────┤
│ Chiplets, HBM, advanced packaging and power delivery         │
├──────────────────────────────────────────────────────────────┤
│ Racks, liquid cooling, electrical plant and data centres     │
└──────────────────────────────────────────────────────────────┘

The essential point is that none of these layers is designed independently.

The model influences the compiler.
The compiler influences the processor.
The processor influences the fabric.
The fabric influences the rack.
The rack influences the cooling system.
The infrastructure telemetry feeds back into workload placement and future silicon.

That is what makes it a vertically integrated AI-compute platform rather than simply an AI chip installed in a conventional server.

Delivery and Competition

Graphcore can probably deliver a credible new AI accelerator and rack-scale system, but delivering the complete vertically integrated platform shown in the infographic is much harder. It will only be plausible as a SoftBank group programme—not as Graphcore acting alone.

Can Graphcore deliver it?

What Graphcore is genuinely capable of

Graphcore already has several of the hardest technical components:

• original accelerator architecture;
• advanced silicon-design capability;
• the Poplar compiler and programming stack;
• multi-accelerator systems and scale-up interconnect experience;
• advanced packaging experience from the Bow IPU;
• engineers spanning silicon, hardware, systems software and AI research.

The company is now explicitly recruiting to build a “next-generation AI stack” covering silicon, hardware and software. It has expanded hiring in Britain and Europe and is establishing a Bengaluru engineering centre intended to contribute to its next-generation AI-compute platform.

That makes the following deliverables credible:

1. a new AI accelerator;
2. compiler and runtime software;
3. accelerator boards or trays;
4. rack-scale interconnected systems;
5. reference server and data-centre designs;
6. software integration with PyTorch and mainstream model ecosystems.

What Graphcore cannot realistically deliver alone

The infographic describes substantially more than an accelerator product. It includes:

• general-purpose CPUs;
• DPUs or infrastructure processors;
• rack networking;
• cloud control planes;
• hyperscale scheduling;
• storage and data pipelines;
• global data-centre deployment;
• managed training and inference services;
• model APIs and potentially model co-design.

Graphcore has not historically operated hyperscale cloud services, built global data-centre fleets or developed every component of that stack. Its previous generation also failed to create sufficient commercial adoption against Nvidia, despite having credible technology. Before its acquisition, it cut staff, closed several international operations and needed additional capital to continue competing.

The realistic structure is therefore:

Layer	Likely provider
AI accelerator architecture	Graphcore
Compiler, runtime and system software	Graphcore
CPU architecture and coherent fabric IP	Arm
Server CPU products	Ampere and/or Arm
Data-centre capital and strategic coordination	SoftBank
Cloud deployment and customers	SoftBank partners, hyperscalers and operators
Models and AI services	Potentially OpenAI and other SoftBank partners
Manufacturing	External foundries, packaging and supply-chain partners

SoftBank now formally groups Arm, Ampere and Graphcore within its AI Computing segment and says those companies are working together to strengthen its semiconductor business.

Probability of success

I would separate success into three levels.

Delivering a functioning next-generation product: likely

Graphcore has sufficient technical competence and now has a financially powerful owner. A new chip, system and software stack is quite plausible.

Estimated probability: 70–80%.

Delivering a competitive rack-scale AI platform: possible but unproven

This requires:

• current-generation dense tensor performance;
• large HBM capacity and bandwidth;
• a highly competitive scale-up fabric;
• robust PyTorch compatibility;
• strong LLM and mixture-of-experts performance;
• production reliability;
• manufacturable systems at meaningful volume.

Graphcore’s previous IPUs were architecturally interesting, but current AI economics are dominated by transformers, memory bandwidth, collective communications and mature software libraries. Google’s published TPU v4 analysis, for example, reported materially higher performance than similarly sized Bow IPU systems on its selected workloads.

Estimated probability: 40–60%.

Becoming a broadly adopted Nvidia-class full platform: low in the near term

Nvidia’s advantage is no longer just the GPU. It includes:

• CUDA and libraries;
• NVLink and networking;
• Grace CPUs;
• BlueField DPUs;
• complete rack systems;
• OEM channels;
• cloud availability;
• developer familiarity;
• model-framework optimisation;
• global production volume.

A technically strong Graphcore platform could still fail commercially if customers perceive migration risk or insufficient software maturity.

Estimated probability by 2028–29: perhaps 15–30%.

A more realistic outcome is that Graphcore becomes a strategically important component within SoftBank’s AI infrastructure rather than independently displacing Nvidia.

Who backs Graphcore now?

SoftBank Group

Graphcore has had one controlling backer since July 2024: SoftBank Group, which acquired it and made it a wholly owned subsidiary.

SoftBank provides far more than ordinary venture funding:

• access to capital;
• ownership links to Arm;
• ownership of Ampere Computing;
• data-centre ambitions;
• relationships with OpenAI and Oracle;
• potential access to large infrastructure programmes;
• long-term strategic tolerance that public markets may not provide.

SoftBank’s official materials describe Graphcore as part of its AI Computing segment alongside Arm and Ampere.

This is Graphcore’s strongest advantage. A chip start-up cannot finance repeated leading-edge silicon generations cheaply. SoftBank potentially can.

Arm

Arm is not merely a financial backer; it is a potential technology and ecosystem partner.

It could supply:

• CPU instruction-set architecture;
• CPU cores and compute subsystems;
• coherent interconnect IP;
• security technology;
• chiplet interfaces;
• software ecosystem integration.

However, Arm is a publicly traded company in which SoftBank remains the controlling shareholder. It must still act according to its own governance and commercial interests; it is not simply Graphcore’s internal CPU department.

Ampere Computing

SoftBank’s group-company listings now include Ampere as its Arm-based CPU-design business.

Ampere could provide:

• high-core-count server CPUs;
• platform validation;
• firmware and server experience;
• CPU inference capability;
• cloud-native deployment knowledge.

SoftBank and Ampere have already announced validation work aimed at improving CPU-based AI inference infrastructure.

Historical investors

Before the acquisition, Graphcore had one of Europe’s most distinguished investor groups. Backers included:

• Atomico;
• Sequoia Capital;
• Microsoft;
• BMW i Ventures;
• Bosch;
• Dell Technologies Capital;
• Samsung;
• Ontario Teachers’ Pension Plan;
• Fidelity International;
• Baillie Gifford;
• Schroders;
• Sofina;
• Amadeus Capital;
• Molten Ventures, formerly Draper Esprit;
• M&G;
• Pitango;
• Foundation Capital.

Graphcore’s Series D and Series E announcements document many of these investors.

They are now historical shareholders rather than current co-owners, because SoftBank acquired the whole company.

Who are Graphcore’s competitors?

Graphcore competes at several different layers, so there is no single competitor list.

1. Nvidia: the principal competitor

Nvidia is the benchmark Graphcore must beat or meaningfully differentiate from.

Nvidia offers the most complete vertically integrated commercial platform:

• GPUs;
• Grace CPUs;
• NVLink and NVSwitch;
• Spectrum-X and InfiniBand networking;
• BlueField DPUs;
• DGX and rack-scale systems;
• CUDA;
• collective-communication libraries;
• inference and model-serving software;
• cloud availability.

Graphcore does not necessarily need to outperform Nvidia everywhere. It must identify workloads where it offers clearly superior:

• cost per token;
• energy per token;
• latency;
• memory efficiency;
• sparse or irregular computation;
• rack-level scalability.

Without a decisive advantage, customers are unlikely to leave CUDA.

2. AMD

AMD is Nvidia’s closest merchant-silicon GPU competitor.

Its strengths include:

• Instinct accelerators;
• large HBM capacity;
• EPYC CPUs;
• open ROCm software;
• existing hyperscaler relationships;
• established manufacturing and OEM channels.

AMD can offer CPU and accelerator integration without requiring customers to adopt a completely unfamiliar architectural model. Graphcore may be more innovative architecturally, but AMD has much greater market scale.

3. Google TPU

Google’s TPUs are among the most mature vertically integrated alternatives:

• custom accelerators;
• optical and electrical interconnects;
• large TPU supercomputers;
• XLA compiler;
• JAX and TensorFlow integration;
• Google Cloud;
• direct optimisation for Google’s internal models.

Google does not normally sell TPU cards as general merchant hardware, but competes directly for cloud AI workloads.

4. Amazon Trainium and Inferentia

AWS designs:

• Trainium for training;
• Inferentia for inference;
• Neuron compiler and runtime;
• managed AWS infrastructure.

Amazon’s advantage is captive cloud demand. Customers already on AWS can adopt its accelerators without buying physical systems or building new operating practices.

5. Microsoft Maia

Microsoft is developing its own AI accelerators and complete systems for Azure. Its main advantages are:

• enormous internal AI demand;
• OpenAI workloads;
• Azure distribution;
• control of data-centre design;
• capital and supply-chain scale.

Microsoft was once an investor in Graphcore but now also represents a platform competitor.

6. Meta and custom hyperscaler silicon

Meta is developing internal accelerators to reduce dependence on merchant GPUs. These products do not necessarily compete in open hardware markets, but they reduce the addressable market available to Graphcore.

The same applies to custom ASIC programmes developed with Broadcom or Marvell for hyperscalers.

7. Cerebras

Cerebras uses wafer-scale processors rather than conventional packaged accelerator dies.

Its strengths are:

• very large on-wafer compute and memory;
• simplified model distribution;
• high inference throughput;
• complete systems and cloud services.

Cerebras is a particularly relevant competitor because it sells an architectural alternative rather than another conventional GPU.

8. Groq

Groq focuses principally on deterministic, low-latency inference.

Its strengths include:

• compiler-scheduled execution;
• predictable latency;
• high token throughput;
• purpose-built inference systems.

Groq is less directly focused on broad training infrastructure, but it competes strongly where inference economics dominate.

9. SambaNova

SambaNova develops reconfigurable dataflow processors and complete rack-scale systems.

This is one of Graphcore’s closest architectural comparisons because both companies emphasise:

• dataflow;
• compiler mapping;
• complete hardware/software systems;
• alternatives to GPU execution.

Recent accelerator research compares Graphcore IPUs directly with SambaNova RDUs and other non-GPU systems.

10. Tenstorrent

Tenstorrent develops AI processors, RISC-V CPUs and licensable semiconductor IP.

Its strategy differs from Graphcore because it can sell:

• finished accelerators;
• chiplets;
• CPU IP;
• AI IP for integration into third-party designs.

Tenstorrent may therefore compete both for systems and for design partnerships.

11. Intel Gaudi

Intel’s Gaudi accelerators have competed on:

• Ethernet-based scaling;
• competitive pricing;
• open framework integration;
• enterprise channels.

Intel’s strategic commitment to individual accelerator families has varied, but its manufacturing, CPU and enterprise capabilities remain relevant.

12. Emerging specialists

Other challengers include:

• Etched, focused on transformer inference;
• d-Matrix, focused on memory-centric inference;
• Fractile, developing inference silicon;
• Untether AI, focused on energy-efficient inference;
• custom ASIC programmes from Broadcom and Marvell;
• Chinese suppliers including Huawei and Cambricon.

These companies often target narrower workloads than Graphcore’s intended general platform, but a specialist can win commercially by being exceptionally good at one high-volume task.

Graphcore’s strongest differentiation

Graphcore’s most credible path is not “build another GPU.”

Its potential differentiation is:

• fine-grained massively parallel execution;
• distributed local memory;
• compiler-controlled placement;
• efficient sparse and irregular processing;
• hardware–software co-design;
• integration with Arm and Ampere;
• a clean-sheet rack architecture.

This could suit:

• mixture-of-experts routing;
• graph workloads;
• agentic execution;
• reinforcement learning;
• scientific AI;
• low-batch inference;
• workloads with complex control flow.

However, it must also perform extremely well on ordinary transformer matrix multiplication. Interesting edge cases will not fund a leading-edge platform by themselves.

The decisive tests

By the time Graphcore reveals its next platform, I would judge it against eight questions:

1. What silicon process and packaging technology does it use?
2. How much HBM is available per accelerator and per rack?
3. What are the scale-up bandwidth and all-to-all performance?
4. Can standard PyTorch models run without extensive rewriting?
5. What is the measured LLM training and inference performance?
6. Who are the first production customers?
7. Can systems be manufactured and deployed in volume?
8. What is the cost and energy per useful token?

Without independent answers to those questions, Graphcore’s platform remains an ambitious development programme rather than a proven competitor.

Overall verdict

Graphcore now has the best financial and strategic conditions it has ever had.

It has:

• credible processor architects;
• a substantial compiler heritage;
• SoftBank capital;
• access to Arm and Ampere;
• a growing international engineering organisation;
• a parent company explicitly pursuing integrated AI infrastructure.

But SoftBank ownership does not automatically solve:

• CUDA compatibility;
• advanced packaging capacity;
• HBM supply;
• developer adoption;
• production reliability;
• model optimisation;
• customer trust;
• Nvidia’s product cadence.

My central conclusion is:

Graphcore is capable of delivering an important component of a next-generation SoftBank AI platform. It is not yet reasonable to assume that Graphcore will independently deliver the complete platform or displace Nvidia.

Its most plausible success is as the accelerator and compiler centre of a broader SoftBank–Arm–Ampere AI-computing system, initially deployed in controlled SoftBank or partner infrastructure before being offered more broadly.

About Me, About BLU//SAS

It is a trend these days that when applying for roles and jobs these automated systems asks for a website link, so here’s me setting up one to showcase my career if nothing else…

My career interests are in the area of Bristol, Linux/Unix, Systems, Automation and Security – hence the domain blusas.co.uk. You can read more on my career on my Career page here: https://www.blusas.co.uk/career/

This is a secondary site where I will be posting articles related to my interest in everything else! Contents will vary as interests develops, evolves, and changes in the next decade or two, of course…