AI is rapidly evolving from systems that generate answers to systems that can reason, plan and take action toward a goal. Rather than responding to a single prompt, Agentic AI can carry out a sequence of tasks over an extended period of time.
Consider travel planning. Generative AI might suggest destinations and create an itinerary based on a user’s preferences. Agentic AI takes this further by searching and comparing flights and hotels, checking availability, coordinating schedules and revising the plan as conditions change.
This shift from generating an answer to completing a multi-step task introduces repeated model interactions and a growing volume of context and intermediate data. Keeping that information readily accessible changes what AI systems require from memory.
Agentic AI works toward a goal through an iterative process that combines LLM-based reasoning with external tool use. It can break a goal into a series of tasks, determine what to do next and adapt its plan as new information becomes available.
One useful way to understand this process is the ReAct (Reasoning and Acting) framework, in which reasoning and action are interleaved through a repeating cycle:
Reason → Act → Observe → Reason
At each step, the agent evaluates the current state of the task and determines what to do next. It may call an external tool to search for information, perform a calculation or retrieve data, then observe the result. That result becomes new information for the next reasoning step, helping the agent decide whether to continue with the current plan or revise it before taking another action. The cycle repeats as the task progresses.
This iterative process can invoke the LLM repeatedly throughout a task. Messages, tool calls, tool results and other intermediate information can be carried forward into subsequent model calls as context, allowing later decisions to reflect what has already occurred.
In multi-agent systems, this information flow can expand further. Different agents may divide responsibilities such as research, planning and execution, while retaining and sharing intermediate results and task history as needed.
To understand how growing context affects memory, it helps to look at how a transformer-based LLM processes information during inference. The model generates text token by token, with each new token attending to information from previously processed tokens. Rather than recomputing the attention keys and values for those tokens at every step, the model temporarily stores them in a Key-Value (KV) Cache for reuse.
The KV Cache can be thought of as a working notepad that keeps previously computed information close at hand. It reduces redundant attention computation, but it is not the model’s long-term memory or a permanent record of conversation history. As the active context becomes longer, more KV data may need to be retained and accessed.
In Agentic AI, each model call can build on information from earlier steps. New messages, tool results and other intermediate information can be added to the context of subsequent calls as the task progresses. Depending on the system architecture, previously processed context or repeated prompt prefixes may be cached and reused rather than recomputed from scratch.
When many users, sessions or agents operate concurrently, these states must be managed across many active sequences. Each sequence may require its own context, KV Cache and intermediate data.
Longer context and higher concurrency place greater emphasis on two fundamentals of system memory: capacity and bandwidth. Capacity determines how much working data can be held in memory, while bandwidth determines how quickly that data can be accessed. Scaling memory, however, introduces its own constraints. Larger configurations can increase power consumption, heat generation and physical footprint inside the server.
System memory for Agentic AI must therefore balance four dimensions: capacity, bandwidth, power efficiency and space efficiency.
Originally developed for mobile devices, LPDDR has evolved around a combination of high bandwidth, low power and a compact form factor — characteristics that align naturally with these AI server requirements.
High data rates help move AI data efficiently, while low-power operation can ease the power and thermal burden of larger memory configurations. A compact package also makes better use of limited board space, leaving more room to configure memory capacity within the system footprint. Together, these characteristics make LPDDR well suited to AI systems that need to increase memory capability without proportionally increasing power and space demands.
But AI server memory needs rarely remain static. Different models, workloads and deployment scenarios can call for different memory configurations, and those requirements can change over a server’s lifetime. Traditional solder-down LPDDR offers limited flexibility to respond once the memory has been mounted on the motherboard.
SOCAMM2 (Small Outline Compression Attached Memory Module) addresses this limitation by bringing LPDDR into a detachable server memory module. This modular approach can retain LPDDR’s advantages in bandwidth, power efficiency and space efficiency while making memory easier to adapt to changing system requirements.
With solder-down LPDDR, memory capacity is largely determined at the system-design stage. Increasing it requires additional packages and board area, while replacing or upgrading memory after deployment is difficult.
With SOCAMM2, memory modules can be replaced or upgraded without replacing the motherboard. This may allow memory capacity to be tailored to different AI workloads while simplifying servicing and lifecycle management.
Its horizontal form factor also creates opportunities for system-level optimization. The additional design flexibility around processors and accelerators may allow for more versatile heatsink placement and front-to-back airflow design in dense AI servers.
By combining LPDDR’s efficiency with a modular design, SOCAMM2 enables more flexible memory configurations for diverse AI workloads.
In recognition of its technical and design advantages, Samsung’s LPDDR5X SOCAMM2 module was named a CES 2027 Innovation Awards Honoree.
Looking ahead, LPDDR6 is expected to provide the foundation for the next stage of SOCAMM2, which may enable further improvements in bandwidth, capacity and power efficiency while retaining its modular architecture.
These advances can give system designers greater flexibility in supporting a wider range of server configurations as AI memory requirements continue to grow.
As Agentic AI workloads become longer, more iterative and more concurrent, SOCAMM2 provides a scalable path for system memory to keep pace with evolving AI infrastructure needs.
As the platform advances toward LPDDR6, SOCAMM2 may continue to scale with the evolving needs of next-generation AI infrastructure.