Posts

Image
How Can We Change Agent Behavior?  (Revisited) Ask 25 language models to write a metaphor involving time, and most of them will give you variations of “Time is a River,” and occasionally “Time is a weaver.” The models seem to be drawn into these attractors that make their responses predictable. Can architecture overcome this pull? In my first blog post “Are All AI Models Secretly Speaking the Same Language?”, we encountered the idea that embeddings created a multi dimension vector space that captured the relationship between words and ideas. It seemed that the geometry of the vector spaces of different AI models (and even the brain) all share a certain symmetry. I attributed this to the fact that they reflect the same underlying reality. I recently came across an article in MIT Technology Review, “ LLMs are stuck in a groupthink groove. This startup is trying to get them out. ”. It talks about how the current LLMs are prone to a kind of Hivemind, where different models all come up ...
Post a Comment
Read more
Image
 Where Should Orchestration Reside? I ended the last post on a question: maybe LoCAL was about externalizing the LLM internals and making it less opaque… I think it might be time for LoCAL2? LoCAL2 is still a Loosely-Coupled Agent Language model. Agents and tools talk to each other using the same message bus. We still have a critic and memory agent, as well as a few other tools for the LLM to use. The big difference from LoCAL1 is that it doesn’t have the scaffolding layer orchestrating when an Agent should act and what it should do. In LoCAL2, we let the Gemma4 model do its own orchestration natively, thinking through the problem, deciding when to look up memory, do a web search, document lookups, or use other tools. As I was implementing this, two things occurred to me. First, this looks a lot like the hub-spoke architecture that I went to a lot of trouble to get rid of. That looked like we regressed back to an earlier version of LoCAL1. The second was this had the potential of ...
Post a Comment
Read more
Image
Can Intelligence be Externalized? In my previous posts, I explored the idea that intelligence may emerge not only from sheer parameter scale within a single model, but from the dynamic interactions between models and their environment. That line of thinking led me to build a prototype multi-agent framework I call LoCAL (Loosely-Coupled Agent Language model). The original architecture was straightforward: a collection of specialized role players communicating through a pub-sub message bus, entirely coordinated and driven by a central Core orchestrator. The Refactoring Initially what I was not happy about was that the Core Agent was doing all the heavy lifting. All the other agents just reacted to messages from Core. So, I started refactoring the codebase, systematically moving behavioral rules out of Core into the peer-to-peer interactions between the agents themselves. The first pass reduced the central code by 20%. A second pass cut another 25%. The refactoring kept going until a rad...
Post a Comment
Read more
Image
Where is Meaning? In my last post, I explored the idea that AI models may be converging on the same underlying map of meaning. It seems that meanings don’t so much reside in the words themselves but in how they relate to other words. Once I put it that way, it seems obvious. “ Of course, words don’t have any intrinsic meaning in isolation, it is only their relationship with other words that they take on meaning. ” So, perhaps   meaning resides in the relationships . It seems that our large capacity brains allow us to map these relationships and make use of them to show intelligent behavior. This fact has formed how we approach making LLMs more intelligent: increase their size. If you want more reasoning, more abstraction, more generality, you scale up the model. More parameters. More data. More compute. The story of progress has largely been the story of making a single monolithic system larger and more capable. And clearly, this works. Other Paths In nature, there are examples tha...
Post a Comment
Read more