What Is MRAM?
MRAM stores data using magnetic properties instead of electric charge (like DRAM) or trapped electrons (like Flash). The core unit is a magnetic tunnel junction (MTJ): a thin insulating layer sandwiched between two magnetic layers. A small current sets the junction to high or low resistance — high = "1", low = "0".
Key properties:
- Non-volatile: Data persists without power. No refresh cycle required.
- Byte-addressable: Like DRAM, individual bytes can be read/written. Unlike Flash (page-based).
- No refresh power: DRAM must be refreshed continuously to keep its data, which draws standby power. MRAM is non-volatile and needs no refresh.
- High endurance: Vendors report write endurance far above NAND Flash and, for some MRAM types, close to DRAM-class levels; exact figures vary by product, so check the vendor datasheet.
- Process integration: Samsung and other foundries can embed MRAM directly into logic dies; for Samsung, eMRAM at 28nm FD-SOI is documented in Sources, and smaller nodes are roadmap targets.
The Memory Bottleneck in On-Device AI
Modern AI inference is dominated by a single problem: the von Neumann bottleneck. Compute (CPU/GPU) and memory are physically separate. Every neural network operation requires data to shuttle back and forth — weights, activations, KV caches in transformers.
This data movement is extraordinarily expensive compared to the actual math:
In Horowitz's widely cited energy estimates (ISSCC 2014, roughly 45 nm process), reading a 32-bit word from DRAM costs well over 100 times the energy of a 32-bit floating-point multiply. When model weights must be streamed from DRAM, data movement can therefore account for much of an inference workload's energy, although the share varies by model, hardware and process node. Faster CPUs/GPUs don't help if the memory bus is the wall.
For on-device AI on phones, laptops, and edge devices running on battery, this bottleneck is the primary obstacle to longer inference time without draining the battery.
Operation | Energy cost | Relative |
|---|---|---|
| 32-bit DRAM access | 500+ pJ | >130× more than a 32-bit FP multiply |
| 32-bit on-chip SRAM access | ~5 pJ | ~1.4× a 32-bit FP multiply |
| 32-bit floating-point multiply | ~3.7 pJ | baseline (1×) |
How In-Memory Computing Attacks the Problem
In-memory computing (also called processing-in-memory or PIM) solves this by moving the compute into the memory array itself. Instead of loading weights and activations into a separate ALU, the MAC operation happens directly in the crossbar of memory cells.
How it works:
- Neural network weights are stored as MRAM cells (or other analog-compatible memory).
- Input activations are applied as row voltages.
- The memory array's analog properties compute the matrix-vector multiply in a single pass — the core operation in transformer inference.
- Results are read out and quantized back to digital.
Where MRAM Stands Today
Samsung SAIT Nature Paper (January 2022):
- Published demonstration of a 64×64 MRAM crossbar array performing in-memory computing.
- Accuracy achieved (Nature abstract): 93.23% on MNIST handwritten digits (software baseline 95.24%) and 93.4% on face detection; an emulated eight-layer VGG-8 network reached 98.86% (software baseline 99.28%).
- Power efficiency: directionally confirmed to benefit from eliminating data movement, but specific quantified power reduction vs DRAM not disclosed in public materials.
- Limitation: Highly specialized for classical ML tasks (digit classification, object detection). Not yet demonstrated on large transformer inference.
- SemiFive + ICYTech PNM MRAM Edge Chip (May 2026):
- Successfully achieved tape-out on Samsung Foundry 8nm (8LPU) with embedded MRAM.
- Tape-out = design submitted for manufacturing. Silicon has not yet returned, benchmarks have not been published, and the product has not shipped.
- Claimed capability: On-device inference for models up to 2 billion parameters without network connectivity.
- Target: Text summarization, translation, conversational reasoning on edge AI, humanoid robots, automotive.
- First commercialization targeted for Asia; no North American timeline announced.
MRAM vs HBM vs DRAM vs LPDDR5: Trade-Offs
What MRAM offers that HBM doesn't:
- Non-volatility: weights survive power-off, enabling instant-on inference without reloading from disk.
- Zero refresh power: eliminates the standby current overhead.
- Direct die integration: no separate memory package required.
- Suitable for intermittently-powered edge nodes and wearables.
- What HBM offers that MRAM (currently) doesn't:
- Much higher bandwidth: the JEDEC HBM4 specification allows up to about 2 TB/s per stack, versus MRAM's embedded bandwidth (not published).
- Proven density at scale for training large models.
- No reliance on specialized MTJ process integration.
- Available today in production AI accelerators.
Memory Type | Peak Bandwidth | Non-Volatile | Standby Power | Form Factor | Best For | Status for AI |
|---|---|---|---|---|---|---|
| HBM4 (High Bandwidth Memory) | Up to ~2 TB/s per stack (JEDEC spec) | No | High (refresh) | Stacked on interposer | AI training, high-end inference | Production, proven at scale |
| LPDDR5X (mobile DRAM) | Varies by device and bus width | No | Refresh needed (low-power self-refresh modes exist) | Wirebonded package | Edge AI, mobile inference | Current standard for phones/tablets |
| MRAM (embedded eMRAM) | Not yet published | Yes | Near-zero | Embedded in SoC die | Always-on edge AI, specialized workloads | R&D, tape-out (May 2026), not consumer yet |
| Standard DRAM | Varies by module and channel count | No | Medium (refresh) | DIMM, SO-DIMM | General computing, inference on desktops | Production, everywhere |
Bottom line: MRAM and HBM are not competitors today. HBM targets high-bandwidth AI accelerators (GPUs, TPUs in data centers). MRAM targets edge inference and specialized in-memory compute where non-volatility and SoC integration matter more than raw bandwidth.
The Software Alternative: TurboQuant KV-Cache Compression
While hardware researchers work on MRAM in-memory computing, software engineers are attacking the memory bottleneck from a different angle: compression.
Google TurboQuant (ICLR 2026, published March 2026):
- Google Research reports that TurboQuant compresses the KV (key-value) cache in transformer inference to 3 bits — down from typical 16-bit or 8-bit formats.
- At least 6× reduction in KV cache memory footprint (Google Research, on needle-in-a-haystack tasks). In long-context inference the KV cache can be a large share of memory.
- No accuracy loss reported by Google on its own benchmarks, including needle-in-haystack evaluations (long-context retrieval); the arXiv paper reports quality neutrality at 3.5 bits per channel.
- Up to 8× speedup in computing attention logits on H100 GPUs (4-bit TurboQuant vs 32-bit unquantized keys), per Google Research; this is not an end-to-end speedup.
- Google Research reports that no training or fine-tuning is needed. Using it in an existing inference pipeline still requires a library implementation.
- How it works: Two-stage process — PolarQuant (quantize in polar coordinates using Lloyd-Max centroids) + QJL (Quantized Johnson-Lindenstrauss transform, adds error-correction to preserve inner-product accuracy at extreme compression).
- Community implementations exist; in the sources we checked, Google's blog post does not link official code. The figures above are reported by Google Research; compression ratios differ between community implementations.
- Why it matters: A software-only memory reduction needs no new hardware, so it could help today's devices once an inference library supports it. It is a pragmatic response to the memory bottleneck while MRAM matures.
If MRAM Reaches Consumer Devices
Consumer smartphone and PC adoption of MRAM would reshape on-device AI in three ways:
- Instant-on inference: Phone boots, AI model weights are already in non-volatile MRAM on the SoC, no reload from storage needed. Instant start for voice assistants, real-time translation, on-device reasoning.
- Battery longevity: No standby refresh drain on the memory subsystem. For always-on AI features (background listening, privacy-preserving analytics), energy savings are multiplicative.
- Larger models on a fixed power budget: If in-memory computing delivers a large enough energy-efficiency gain over LPDDR5 plus separate compute, phones could in principle run larger on-device models within the same battery budget. This is speculative; no such gain has been verified in a commercial product.
Timeline & Honest Outlook
What is documented (company roadmaps and announcements, which can change):
- What is NOT confirmed:
- We found no announcement of MRAM integration by major smartphone or SoC vendors (such as Apple, Qualcomm, MediaTek).
- MRAM bandwidth and density specs for consumer VRAM-scale (8–16 GB) are not publicly available.
- Power efficiency gains for large-model inference (30B+) have not been measured in silicon.
- Cost parity with LPDDR5 or HBM is not on any published roadmap.
- Realistic timeline (editorial estimate, not a vendor roadmap):
- 2026–2028: Edge AI SoCs (robots, automotive, IoT) with small MRAM in-memory compute units. Limited 2B-scale models. Asia-first deployment.
- 2028–2030: Potential smartphone integration as a non-volatile cache or specialized AI accelerator tile (not main memory replacement).
- 2030+: Mainstream consumer-phone deployment as a DRAM replacement would require solving density, bandwidth, and cost challenges that are not yet solved. We do not expect it before 2031–2035.
Year | Milestone | Status |
|---|---|---|
| 2019 | Samsung announces commercial eMRAM shipments at 28nm FD-SOI | Announced (Samsung, March 2019) |
| 2024 | Samsung 14nm eMRAM production | Roadmap target (2023 announcement) |
| 2026 | Samsung 8nm eMRAM production; SemiFive/ICYTech tape-out | Roadmap target; tape-out announced (May 2026) |
| 2027 | Samsung 5nm eMRAM process available (roadmap) | Announced target |
| 2028–2029 | Potential first edge AI SoCs with MRAM in-memory compute shipping (SemiFive, others) | Plausible but unconfirmed |
| 2029–2031 | Possible consumer smartphone MRAM integration (non-volatile cache or specialized AI die) | Speculative |
Frequently Asked Questions
Is MRAM available to buy now for my PC or phone?
No. MRAM is in production at several vendors, for example for industrial and data-center products (see the Everspin Form 8-K above). For consumer AI, it is R&D only. The SemiFive/ICYTech chip is still at tape-out stage — silicon not yet returned or benchmarked. Consumer deployment realistically 3–5+ years away.
Will MRAM replace my GPU's VRAM?
Unlikely in the near term. MRAM excels at low standby power and non-volatility, which matter on battery-constrained edge devices. HBM solves a different problem: maximum bandwidth for data-center training and large-batch inference. For consumer phones and specialized edge AI, MRAM may become a component (embedded cache or accelerator tile). VRAM for a gaming GPU or data-center accelerator will remain HBM or GDDR for the foreseeable future.
What's the difference between in-memory computing and vector database?
Different layers. Vector databases store embeddings and retrieve them by similarity (used in RAG pipelines). In-memory computing performs the neural network's core operation (matrix-vector multiply) inside memory itself, eliminating data shuttling. You could use both together: in-memory compute for the inference engine, vector DB for retrieval.
Can I use TurboQuant compression on my local LLM running today?
Not yet in mainstream inference libraries, but community implementations exist. TurboQuant is academic work (ICLR 2026). We found no confirmed TurboQuant support in Ollama or LM Studio, and Google's blog post does not link official code, so check GitHub for community implementations; compression ratios and accuracy vary between them. The core idea — aggressively quantizing the KV cache — is reproducible in custom inference code.
Does MRAM work with transformers?
Samsung demonstrated it on classical ML tasks (digit classification, face detection). Transformer-scale inference (7B+ models) in MRAM in-memory compute has not been published. It's plausible but unproven. The SemiFive chip claims 2B-parameter capability; we'll have real benchmarks when silicon returns and ships.
Is MRAM the same as 3D XPoint (Intel Optane)?
No. 3D XPoint was a storage-class memory technology developed by Intel and Micron (now discontinued). MRAM stores data magnetically; 3D XPoint works by a different mechanism. Both target the same problem space — fast, durable, non-volatile storage — but use different approaches.
How much power does MRAM save compared to DRAM?
For standby: MRAM is non-volatile and needs no refresh, so it avoids the refresh power DRAM draws; we found no vendor-neutral per-gigabyte figure, and the saving depends on device and capacity. For active inference with in-memory compute: Samsung's Nature paper presents analogue in-memory computing as a potentially low-power approach, but we found no quantified saving versus DRAM in it. Commercial silicon numbers will come when a product ships and is benchmarked independently.
Will MRAM replace DRAM?
Not as main memory in the foreseeable future. MRAM's strengths are non-volatility and no refresh power, which suit embedded caches and low-power edge accelerators. DRAM and HBM still hold advantages in density, bandwidth, and cost per bit. The realistic role for MRAM is a complementary non-volatile tile or in-memory-compute unit, not a DRAM replacement.
Did Qualcomm solve the memory bottleneck in 2026?
Qualcomm announced HBC (High-Bandwidth Compute) under its Dragonfly brand at its 2026 Investor Day — a near-memory computing design that bonds compute with memory in 3D-stacked silicon, with Qualcomm claiming roughly 6x higher bandwidth per watt than HBM and 200x higher capacity per watt than SRAM (vendor claims, not independently verified). Meta has signed a multi-generation deal to use the Dragonfly C1000 server CPU; in the sources we checked, Qualcomm has not named customers for the AI250 or AI300 accelerators that use HBC. Three distinctions still matter: HBC is near-memory (compute beside memory), not MRAM in-memory computing (compute inside the memory array); it targets data-center AI accelerators, not phones; and commercial sampling of first-generation HBC (AI250) is expected in mid-2027, with HBC Gen 2 (AI300) in 2028 — so nothing has shipped. It is a different approach to the same memory-wall problem, not an MRAM breakthrough.
Sources
- Samsung Newsroom: Samsung Electronics Starts Commercial Shipment of eMRAM Product Based on 28nm FD-SOI Process -- press release, 6 March 2019
- EE Journal: Ainekko Merges with Veevx (company press release) -- merger announcement, early 2026
- MRAM-Info: SEMIFIVE and ICY Tech tape out an edge AI SoC on Samsung 8nm eMRAM -- tape-out on Samsung Foundry 8nm (8LPU) embedded MRAM, posted 8 May 2026
- Nature: A crossbar array of magnetoresistive memory devices for in-memory computing (Jung et al., 2022) -- 64×64 MRAM crossbar, MNIST 93.23%, face detection 93.4%
- Everspin Q3 2026 results (SEC Form 8-K) -- revenue $20.6 million vs $14.1 million; MRAM product sales $16.9 million vs $12.7 million
- Everspin Q2 2026 results (SEC Form 8-K) -- $40 million contract with a U.S. prime contractor
- arXiv:2504.19874: TurboQuant, Online Vector Quantization with Near-optimal Distortion Rate -- KV-cache quantization; quality neutrality at 3.5 bits per channel
- StorageReview: Qualcomm unveils Dragonfly data center roadmap -- HBC bandwidth and capacity claims, AI250 and AI300 sampling dates, Meta C1000 CPU deal
- Horowitz, "Computing's energy problem (and what we can do about it)", ISSCC 2014 -- approximate per-operation energy at 45 nm: SRAM access, DRAM access, floating-point multiply
- Google Research: TurboQuant, redefining AI efficiency with extreme compression -- 3-bit KV cache, at least 6x memory reduction, up to 8x attention-logit speedup on H100 (vendor-reported)
- MRAM-Info: Samsung on track to start 14 nm eMRAM production by the end of 2024, 8 nm by 2026 and 5 nm by 2027 -- Samsung Foundry roadmap targets (2023 announcement)
- StorageNewsletter: JEDEC and Industry Leaders Collaborate to Release JESD270-4 HBM4 Standard -- HBM4: up to 2 TB/s over a 2048-bit interface
- EE Times Asia: SEMIFIVE, ICY Tech Achieve Successful Tape-out of 8nm eMRAM-Based Edge AI SoC -- companies' claim: on-device inference for models up to 2B parameters