A real-time intelligent gaming companion that actually plays alongside you. World agency, local inference, persistent plans, and an AI character designed to create a striking emotional experience.
This post was excellent. It's the best plug-LLMs-into-a-game demo I think I've ever seen. What I found most interesting about it, though, was the combination of "traditional" ML with LLMs all wrapped up in a low-latency, (mostly?) locally served package.
Through this whole AI-craze people have forgotten that we had intelligent systems that could process speech since the 1970s, and somewhat LLM-like behavior with chatbots like SmarterChild in the early 2000s. There's a lost art that is being overlooked today in things like traditional NLP and behavioral graphs.
I totally agree! The world of ML outside of LLMs has definitely drifted into underrated territory. People seem to have forgotten exactly how fast, cheap, and accurate a simple scikit-learn model can be.
Actual details on the heart of the system are frustratingly scarce.
This is the secret sauce and biggest differentiator. I call this system (ALE - Action Latent Encoder because it's an action encoder in need of a fun acronym). Under the hood, ALE is a hybrid of embeddings, small classifiers, explicit rules and traditional ML. ALE detects structure, identifies negation, commands, continuation, pronouns, and sequences […]
It creates embeddings from the full text as well as its extracted structure, then semantically combines and compares it with action prototypes. A separate classifier estimates whether the turn is a command, question, chat, clarification or complex request. Everything gets merged together.
From the Hacker News comments:
I like the approach here. It’s cleverer than just “dump context to LLM”. I couldn’t quite figure out if it genuinely runs purely locally though […]
Hello, author here. I was intentionally a little vague about this because this is kind of the thing that makes the whole approach work.
I wish the creator had been a little more open here.