This is a guest post for the Computer Weekly Developer Network written byDominik Tomicevic, CEO ofMemgraph
The original title in full for this piece is: From Better Reasoning to Faster QFS, An LLM Just Can’t Match An SLM-Knowledge Graph Combo
Memgraph is a graph database company specialising in high-performance, real-time connected data processing. The organisation supports an open source community of 150,000+ members and serves global 2000 customers, including NASA, Cedars-Sinai and Capitec Bank.
Tomicevic writes as follows…
Hardly a week passes without another breakthrough in the LLM (large language model)/reasoning engine space. The latest I’ve seen is ERNIE X1—another impressive development from China, claimed to match DeepSeek R1’s performance at half the cost.
So, LLMs must be the future, right?
They’re getting cheaper to train and if they keep getting smarter, why even consider their SLM (small language model) competitors? Well, neither an LLM nor an SLM alone may give you everything you need. But as I’ll argue, in most practical enterprise AI scenarios, the best approach is an SLM enhanced by something more.
Centrality of context
The word “context” here is deliberate because it lies at the core of the entire model debate. To be clear, for very general, homework-level problems, an LLM works fine – but the moment you need a language-based AI to be truly useful, you have to go with an SLM.
Still sceptical? Just look at the reasoning engine builders – that’s exactly what they do. R1 uses a “mixture of experts” framework with an amazing 671 billion parameters, yet only a maximum of 37 billion are activated to answer any given question. Why? Because, at its core, systems like these are really a hidden federation of SLMs (though they’d probably reject that label), each specialising in different areas of knowledge.
If you think about it, this makes perfect sense. An MRI of the human brain shows that only specific regions activate when discussing a particular topic – there are dedicated structures for language, memory, motor functions and more.Until (or if) true AGI is achieved, no single knowledge model can excel at everything. Experience shows that an SLM trained for a specific task will outperform a general model in that area. Likewise, by narrowing the domain and focusing on a smaller task set, an SLM can be trained to match or exceed an LLM’s performance on granular problems. When you feed an SLM the right context for its specialised task, you get peak performance – QED.
Granular? Think, use case. Thinkyour problem domain – the way your company mixes paint, builds IoT networks and schedules deliveries. The AI doesn’t need to recall who won the World Cup in 1930, you need it to help you optimise for a particular problem in your corporate domain.
So, an SLM can be trained to detect queries about orders in an e-commerce system and, within the supply chain, gain deep knowledge of that specific area- making it far better at answering relevant questions. Plus, for mid-range and smaller operations, training an SLM is significantly cheaper (considering GPU and power costs) than training an LLM.
Tomicevic: RAG powered by graph technology bridges structured & unstructured data, so AI systems can retrieve the most relevant insights with lower costs & higher accuracy.
Aha! Problem solved, right? Just go SLM and enterprise AI becomes practical and delivers ROI.
Well, not quite. The problem is how you get that supply chain data into a focused small language model. Until the basic architecture that both LLMs and SLMs share – the transformer – evolves, updating a language model remains difficult. These models prefer to be trained in one big batch, absorbing all the data at once and then reasoning only within what they think they know.
That means updating or keeping an SLM fresh, no matter how well-focused it is on your use case(s), remains a challenge – the context window still needs to be fed with relevant information. This is where an additional element comes in: graph technology. Organisations repeatedly find that a knowledge graph is the best data model to sit alongside a domain-trained SLM, acting as its constant tutor and interpreter.
The way forward for genAI?
Again, it comes down to context.
In high-stakes environments, SLMs combined with structured data sources like a knowledge graph can outperform a single domain-specific LLM by delivering faster inference, lower operational costs and more controlled outputs with structured reasoning. This agility makes SLMs especially valuable in data-sensitive industries, where continuous adaptation is essential – and arguably only achievable with the constant supervision and feeding of a graph.
A knowledge graph that harnesses the latest data parsing and search techniques – such as vector search, dynamic algorithms and especially RAG (retrieval-augmented generation) or GraphRAG – will significantly enhance context feeding, making it more precise than ever.
RAG powered by graph technology bridges structured and unstructured data, allowing AI systems to retrieve the most relevant insights with lower costs and higher accuracy. It also enhances reasoning by dynamically fetching data from an up-to-date database, eliminating static storage and ensuring responses are always informed by the latest information.
By combining semantic and structural data, RAG within a graph-supported SLM enhances reasoning for query-focused summarisation (QFS).
There is a strong case for the future of generative AI being focused SLMs, continuously updated by a knowledge graph. Despite the constant stream of LLM breakthrough headlines, it’s hard to see how an LLM can truly provide the same level of precision and adaptability.