Over the past two years, AI trends have shifted from software competition to a full-scale race in AI infrastructure, with big tech companies investing hundreds of billions of dollars to build data centers and processing systems to support AI, leading to a dramatic surge in chip demand.
Many might view GPUs as the core of AI, but in reality, the industry faces deeper issues—namely, memory constraints or the Memory which is becoming the most critical bottleneck in the industry.
Where exactly are these bottlenecks located? Who stands to win in the AI Super Cycle, and what risks should investors monitor? In Digital Frontier Talk EP.72, which explores AI chip stocks that have surged thousands of percent, what opportunities remain? The program invited Bom Atthaphon Jongraksak, an independent researcher and owner of the DinoFolio page, to discuss and find answers on this topic.
Following massive investments by big tech companies competing to build AI infrastructure, the sudden spike in demand caused severe shortages of memory chips, especially DRAM used in GPUs. This led to factory gate DRAM prices rising as much as 7% weekly and doubling in the first quarter, directly impacting IT equipment prices like PCs and smartphones, which had to be adjusted upward.
This shift caused shares of memory chip manufacturers such as Micron, SK Hynix, and SanDisk to soar hundreds to thousands of percent (some yielding up to 4,000%), making the memory chip sector highly volatile recently.
Current computing technology is based on the Von Neumann Architecture, which separates processing (Compute) from storage (Memory). AI systems processing massive data must shuttle data back and forth between these parts, creating a bottleneck known as the Memory Wall.
The Memory Wall causes AI systems to spend considerable time transferring data between processors and memory, limiting GPU performance. No matter how powerful the GPU, it cannot achieve full potential, making memory chips the most critical component in modern AI systems.
The solution to the Memory Wall is High Bandwidth Memory (HBM), designed to have much higher bandwidth than traditional DRAM, enabling faster data transfer and more energy-efficient GPU operation. However, manufacturing HBM is highly complex, requiring 3D stacking of wafers, drilling tiny 30-micron holes, and copper connections, where any defect raises costs immediately.
This complexity has made SK Hynix the market leader after years of investment in HBM development, while many competitors underestimated market readiness and missed critical opportunities as the AI era emerged.
Moreover, even if sufficient memory chips are produced, they must be integrated with GPUs via 2.5D CoWoS (Chip-on-Wafer-on-Substrate) packaging technology, currently offered only by TSMC worldwide.
TSMC leads this technology with the largest production capacity, making this capacity another critical industry bottleneck. Even if chip manufacturers produce more HBM, without CoWoS packaging, full AI systems cannot be delivered. Currently, production lead times are as long as 72 weeks (about a year and a half), with new factories expected to reach full production only by 2028, predicting severe bottlenecks from next year onward.
As AI evolves into Agentic AI capable of decision-making and commanding subordinate AIs, the need for central processors (CPUs) will increase to manage and coordinate GPUs and subsystems. Many foresee CPUs becoming the next major bottleneck if demand outpaces production capacity, positioning CPU manufacturers like Intel and ARM as potential chokepoints.
Due to HBM limitations, chip makers are experimenting with alternatives, such as using NAND Flash as supplementary memory in some tasks by connecting NAND Flash storage chips directly to GPUs, bypassing RAM to relieve HBM pressure.
The industry is also exploring technologies like Silicon Photonics, which uses light instead of electrical signals to transmit data, reducing speed and distance limitations, and Processing-in-Memory (PIM), which integrates processing directly within memory units to reduce data transfer between Compute and Memory—potentially a significant future architectural advancement.
Another approach is to redesign the entire architecture. One startup, Cerebras, attempts to break conventions by using short-term memory (SRAM) built on large Wafer Scale Engines (WSE) that integrate processors and SRAM on a single wafer. This technology bypasses the need for HBM or Nvidia GPUs and can process answers at speeds up to 750 words per second. OpenAI is investing heavily to test this technology.
China is pushing CXMT to become a major global memory chip manufacturer. Although its technology still lags behind leaders in some areas, increased production capacity from China could ease supply pressures and introduce more competition in the long term.
Read more
The memory chip industry has historically been cyclical with frequent shakeouts because factories require 3-5 years of advance planning. If future demand from major cloud providers falls short, a chip oversupply could occur, triggering price wars that reduce company valuations by 30-60%.
Despite concerns about bottlenecks and stock prices, in terms of real-world technology adoption, the world remains at the beginning of this cycle. Future device form factors will evolve into new types such as smart glasses, robots (Physical AI), or AI capable of generating revenue independently, driving continued long-term growth in chip infrastructure demand.
Today's AI chip industry resembles Russian nesting dolls—opening one layer reveals another hidden bottleneck. Solving one problem immediately reveals another limitation.
Initially, attention was on GPUs; now the focus has shifted to Memory and HBM, while addressing memory issues has led to new constraints in advanced packaging like CoWoS. Entering the full Agentic AI era, demand for CPUs, networking, photonics, and data management systems is likely to become the next bottleneck.
This means the AI Super Cycle is not about a single company but competition across the entire supply chain. Those controlling limited supply and unable to rapidly expand production will have bargaining power and greater opportunity to benefit from this investment cycle.
Key factors to watch closely include investments from big tech cloud providers, AI infrastructure spending trends, chip prices and production capacity at each supply chain stage, new AI technologies that could change winners, and risks of overinvestment that might turn chip shortages into oversupply and price wars.
Therefore, investors seeking AI stocks should look beyond GPUs to identify upcoming bottlenecks, manufacturers involved, and those able to scale production promptly.
Follow the Facebook page: Thairath Money at the link below -