Semidynamics sees a demand for big European AI processing companies, especially as he sees customers looking for alternatives to US and Chinese suppliers. The company plans to fill that need and is developing AI accelerator racks aimed at inference. He admits there are obstacles, such as funding and infrastructure. Semidynamics CTO Iakovos Stamoulis addresses those issues in a Q&A with JPR, as well as practical constraints on building AI infrastructure including memory shortages.
Iakovos Stamoulis (Source: Semidynamics)
Barcelona-based Semidynamics is developing AI accelerator racks aimed at inference. CTO Iakovos Stamoulis sees an opportunity for European technology that extends beyond Europe’s own sovereignty ambitions. Here, he talks with JPR’s David Harold about demand for alternatives to American and Chinese suppliers, the obstacles to scaling European chip companies, and the memory and power constraints shaping AI infrastructure.
What does sovereignty mean from a processor designer’s perspective?
Iakovos Stamoulis: In Europe, for us, sovereignty means something European, something that Europe can control. But what I found from speaking with people outside Europe—and that includes the Middle East, Taiwan, and places like that—is that they’re also really concerned about sovereignty. For them, it means something non-American and non-Chinese.
So it looks like Europe is in a very good position. People in Taiwan were asking for European technology because they see Europe—and that includes the UK—as a more trusted partner to do business with. They’re looking to diversify beyond US technology and, in their own region, Chinese technology. I think that’s a great opportunity for Europe. We’re seen as a reliable and trusted partner. That’s a great opportunity.
Can Europe turn that demand into a large, successful AI processor company?
Stamoulis: There is definitely demand for big European AI processing companies. However, to create big European companies, you need bigger funding as well. One of the issues is that a lot of European funds tend to be smaller than their US counterparts or those in other regions. That’s one of the problems in scaling up technology in Europe. When it comes to creating big hyperscalers or other really big companies, Europe has an issue. It comes down to funding and also our fragmentation in Europe: We’re not one Europe; we’re multiple countries in reality. We’re very good at creating start-ups and foundation technologies, but we need a strategy to scale those companies—for example through larger pan-European funds and sovereign-technology programs.
Beyond sovereignty, there are practical constraints on building AI infrastructure. Do you expect the memory shortage to ease soon?
Stamoulis: My feeling—and that’s more of a personal view—is that for at least the next two years, that’s not going to go away. We’re still going to have shortages until more capacity has been built.
There is such a big demand for AI data centers all over the world, for more and more capacity, and there’s a lot of memory use in AI. While we’re still building bigger and bigger infrastructure, I don’t expect these shortages will go away until memory manufacturers can catch up. Making new fabs and increasing capacity at memory suppliers takes a long time.
Memory has historically been a cyclical market. Does that make suppliers cautious about adding capacity?
Stamoulis: Yes. Historically, with memories, they’ve had very low margins and there have been oversupply issues. Now that the situation has been reversed, they’re a bit more cautious about increasing supplies and making huge investments in additional supply. In the past, there was a risk of overinvestment, and they didn’t see similar demand coming.
What is driving the move towards more memory per accelerator?
Stamoulis: We see that frontier models keep growing from generation to generation, and we see it clearly in open models such as DeepSeek and Kimi. As they try to solve more complex problems, models become bigger, and that requires AI accelerators and GPUs with more memory.
Another advantage of having more memory per GPU is that it decreases the amount of communication required between systems when you want to scale up. There’s less communication between separate GPUs. More can be handled locally. That is one reason we chose LPDDR-based memory for our accelerators: It lets us offer large memory capacity per accelerator at a lower cost than HBM.
With AI workloads changing so quickly, how do you balance specialized hardware against flexibility?
Stamoulis: In this typical computer architecture trade-off, you either make something more specialized, in which case it’s less flexible but can be more efficient, or you make something more general-purpose. In that case, it’s probably a little less efficient, but it’s more future-proof in the long term.
With the rapid change in AI models, new networks and new choices come all the time. We’ve seen in the past that some of the initial AI accelerators were mostly CNN engines, and these became obsolete really quickly. We made a conscious choice to go to a more general-purpose architecture. To mitigate the problems around memory efficiency, we have developed technologies, including our Gazzillion memory technologies, that hide latency and address the issues CPUs have with moving memory around. That’s one of the biggest challenges in AI.
As memory capacity increases, what becomes the next constraint?
Stamoulis: We don’t really know what the next problem is. Part of it could be interconnect, although by having more memory, we’re reducing the interconnect bandwidth we need.
It is my expectation that power will be an important next issue. All these data centers require a lot of power. That puts a strain on power delivery and on cooling, including water use. There are constraints there, but there’s also a cost problem. Power use is directly related to the cost per token. You need to balance subscription fees against the power used to provide those subscriptions.
Unlike a business where Microsoft makes Windows and there’s a marginal cost in replicating it, in AI, a substantial cost of your services is power consumption. You really need to bring that down to make it profitable.
What do we think?
Semidynamics has had a substantial change in both business model and competitive ambition, which parallels Arm’s evolution from licensing IP towards supplying silicon. Both companies recognize that some customers want complete solutions rather than the components from which to build them. Semidynamics is approaching the opportunity through AI acceleration, with a particular emphasis on innovation in memory handling.
RISC-V is arguably becoming secondary to the Semidynamics story. The open ISA provides architectural flexibility, but the company’s real differentiation lies in its ability to move data efficiently, hide memory latency, and maintain processor utilization.
We have argued before that AI infrastructure has become too focused on headline compute performance. TOPS and FLOPS matter, but they tell us relatively little about how efficiently a system executes real workloads when processors spend substantial time waiting for data.
Semidynamics’ decision to use LPDDR rather than HBM is interesting in this context. HBM offers greater bandwidth, but LPDDR can provide more memory capacity at a lower cost, potentially allowing larger models and KV caches to reside locally while reducing communication between accelerators. The question is whether Semidynamics can compensate sufficiently for LPDDR’s lower bandwidth through its memory architecture. Ultimately, success will be measured in tokens per second, per watt, and per dollar, rather than peak theoretical performance.
We tend to think of European AI sovereignty as a defensive ambition, reducing dependence on American technology. Stamoulis suggests something more commercially interesting: Europe could become a trusted alternative for countries seeking to reduce their dependence on both American and Chinese suppliers. Imagination Technologies’ CEO Markus Mosen recently told us something similar.
There is an interesting convergence between the technical and commercial arguments. If AI infrastructure is increasingly constrained by memory capacity, power consumption, and cost, there may be room for alternative architectures that address those limitations without attempting to beat Nvidia at its own game.
Increasing compute performance remains important, but keeping that compute productively occupied, reducing data movement, and improving inference economics may be just as significant.
Semidynamics is using RISC-V as the foundation for a broader approach to AI computing in which memory architecture and system efficiency are the defining features. The ISA enables the strategy. It is not the strategy.
WHAT DO YOU THINK? LIKE THIS STORY? TELL YOUR FRIENDS, TELL US.