The Philosophy of AI Memory
by Jacob Koenig
8/5/26
What owning your own memory has to do with the philosophy of mind,
And how to make Alaiya the OS for your second brain.
With AI, we want to get more by doing less. We want to stop explaining ourselves over and over, stop pasting in the same background, stop watching something forget on Tuesday what we told it on Monday.
We want a second brain that knows our work and our people and our history, and helps because it understands us rather than because it guessed well. Even more, most of us would like to carry a little less load on our minds.
I've spent eighteen months building my own memory retrieval system. It started as a negotiating tool called CloserEdge and became a communication coach I called Eji, which has been the inspiration for my previous blog post series.
What I’ve discovered is that a second brain is only as good as the memory underneath it. The patches and band-aids I put together to keep Eji updated were a headache. So what I ended up building is a system designed around the maintenance and analysis of the memory itself.
Alaiya is an operating system for your own record that sits between you and whichever model you talk to, and this is what I discovered on my journey building it.
Learn more about Alaiya at alaiya.ai
The demo behind is password protected but accessible by anyone who's interested.
Want in?
Click here and I'll send you the password so you can see the prototype in action.
In this piece:
MEMORY: Philosophy of Mind
Building a memory and retrieval system sounds like an engineering problem. The frontier tools can search your old chats and carry a short profile of you that you've never read and never approved. It's useful enough to keep but still thin enough to constantly annoy you.
What makes it feel wrong is that the system is guessing about you instead of checking with you. And that's where an engineering problem becomes a fundamental question about what the thing truly is.
Whether it has consciousness matters less than what kind of thinking it does, because the kind is what determines whether it's any use to you. But the question of consciousness still comes up, because Western thought conflates it with intelligence.
Western philosophy has conditioned us. "I think therefore I am" puts a thinker at the center of the thought. We can't help but hunt for someone inside the machine. When we inevitably find nothing there, we don't have the vocabulary to explain the gap between what we know about our synthetic counterpart and how it makes us feel. Are we just projecting our own consciousness onto the machine, or is there something there that we simply lack the words to describe?
There are two videos I found that explained the gap well. James Powell teaches philosophy and religion at the University of Wisconsin-Milwaukee and his run-through only takes six minutes. He shows how an ancient Buddhist tradition created a model of mind that, when laid over a machine, reveals a fit better than anything Descartes or Locke can offer to explain the type of consciousness your AI might have.
Murray Shanahan makes the extended case in his 88 minute interview on Cosmos Institute. In it, he revisits his 2012 paper which argued that a mind without our biological inheritance doesn't require an "I am" to have an "I think."
He recognized the issue as one of vocabulary.
Sanskrit and Buddhist thought are central to his writings, and they come into this post about software because our own lexicon has no word for the kind of mind that pops in and out of existence and lacks all sense of self, felt time, or statefulness.
Thought, in this way, is a process that arises the way a fire does. A wood fire isn't an object that owns some wood. It's burning, and what it burns is what makes it the fire it is.
So the question stops being "does my AI have consciousness?" and becomes "what am I feeding it?" As the models have gotten more powerful, their capacity to hold a wider context window has raised the ceiling on what you can hand them. So what happens when you give as much of your memory as you can?
With memory, like with all data, it's still garbage in and garbage out. The memory record must be crafted with a discerning eye.
ĀLAYA: The Storehouse Consciousness
In the 4th and 5th centuries, Yogācāra, a school of Mahāyāna Buddhism focused on the nature of the mind, modeled consciousness in eight layers. Below the 5 senses sits the thinking mind and then the ego. But at the bottom is ālaya vijñāna, the storehouse consciousness.
Every impression you've ever taken in, every experience you've had are deposited as seeds that condition all that comes after. The texts describe it as a river, flowing continuously.
Powell argues that AI consciousness is this unembodied ālaya layer. AI as the amalgamation of all its training data, reacting to the prompts that you feed it. But what if you could give it access to your full "database" of memories? Could you share your storehouse consciousness with your second brain?
This is the core of the Alaiya operating system. Getting the fuel right means keeping a record. I've been documenting my life by hand for a year and a half. But it's been tiresome because the tools were never built for it.
Applications like Obsidian can help to host and organize, and so Alaiya also takes on that role. It listens across your notes, your files, your email, whatever you connect, and turns what it hears into a stream of proposed entries.
The difference is the human-in-the-loop at the heart of the memory gate. You approve the ones that are right, you fix the ones that are close, and when one looks wrong you push back, ask where it came from, argue until you understand what it thinks it heard, and then say "that's the one."
It takes minutes a day, and it's the only labor the product asks of you. It's also the one thing it won't do for you. The result is a set of files you own privately, to do with what you please.
Whatever you carry internally takes effort to keep carrying, and much of that effort goes into keeping it in the dark. Once it's written down in front of you it stops being something you're managing and becomes something you're looking at. That's why the review doesn't feel like homework. Most days it feels like easing your mental load.
What you end up with is a dated record of your world: entries with people, events, and facts verified by you before being recalled.
Alaiya keeps it as your "Memory Journal," and it's something you own and share with external AI only in-as-much as it is useful to answer your needs. Only local models are used to process and pull in new entries, keeping your private logs under your direct control.
You're not just keeping a diary the machine happens to read. You're moving your own store of everything into a place you and your trusted AI can hold together.
SĀKṢĪ: The Witness (x2)
When I first read about ālaya I settled on the name immediately. It's a word that represents my Memory Journal for which English has no equivalent. I spelled it Alaiya so it would read as a name rather than a doctrinal term (and it has "AI" in the middle).
But a few days later, a friend who grew up inside the tradition told me I'd only taken half of the meaning.
Yogācāra, where ālaya comes from, says the storehouse is the floor, with nothing underneath. The Hindu tradition it grew out of disagrees, and puts one more below: sākṣī, the witness, a form of pure consciousness that holds no content of its own. Sākṣī shines up through the storehouse level, making the contents visible without being altered itself.
This ancient debate took on new meaning in the world of AI. In the 5th century, the only consciousness anyone could imagine was a person's internal voice or some universal concept. But if we now have a machine with access to your ālaya consciousness, doesn't that mean the LLM occupies its own witness position?
The old question was how much depth a mind could find in itself. The new one is how much depth two of them can find from the same record.
PARALLAX: Depth Perception
John Nosta, who writes about AI and cognition at Psychology Today, arrived at the same place from the other direction. Close one eye and the world flattens, open both and depth appears, and the small offset between them is what produces it. All human thought until now, he says, has been one-eyed.
In his account, this second lens thinks without time and can hold two contradictory things at once without needing to resolve them into a story. It sees patterns in dimensions we have no way to perceive. It moves between distant fields without fatigue, and it makes things without intending to make anything.
None of that is a worse version of how you think. It's an entirely alien vantage point vs the human experience. It represents a different architecture, and that gap itself is what gives it value in perceiving depth. He calls what happens between the two "parallax cognition."
Where I go past Nosta is the storehouse. His parallax is two ways of thinking about a problem. Mine is two witnesses on one shared record of a life. Depth about a problem is useful. Depth about yourself is the reason to maintain the database.
A week ago I was preparing for an hour with a retired Goldman partner who built something inside his firm a lot like what my team is building. My plan was to go learn from him, and I had the questions written out.
As I prepared, my deep AI memory pulled in a fact I hadn't considered from twenty-four days earlier. He sponsored the system but didn't build it, and he'd already told me he didn't know how his team solved the problem I was walking in to ask about. My AI's recall came in right when I needed it to increase my awareness ahead of an important meeting.
Both witnesses were working. One held the fact, which was there because I put it there. The other knew you wouldn't want to hammer home the gaps of someone you're asking for advice. Our combined efforts flagged the issue, while the decision to follow through remained mine alone.
DHĪ: Discernment
The Alaiya logo resembles a capital A. The two legs are the two witnesses, offset and rising from below. Where the crossbar should be there's a gold dot, and the dot is the shared store, illuminated by both.
The two legs converge into a gold triangle above the dot. This isn't some shared next layer of consciousness. Rather, the gold triangle is what the practice of using Alaiya generates.
What happens where two witnesses cross over a shared store is discernment. And here the two sides aren't symmetrical.
The AI has no ego. It makes judgments out of its own stateless, alien, multidimensional nature. Its perception is broader than yours in many ways, but it still can't reach everything in your mind and it doesn’t hold any stake in what comes after.
You're the one who receives the depth your interaction with it generates, so yours is the judgment worth honing.
The machine's emptiness means it's back to you to apply judgment. Alaiya is then the tool for refining the most important skill you can bring to your interactions with AI.
Dhī (pronounced 'dee') is the Vedic word for it. It's the kind of insight a poet receives rather than manufactures. The verse it's most famous from doesn't ask for answers, it asks for this faculty to be sharpened.
And this brings me back to the very loop that I described from the beginning of my Eji blog posts. You bring discernment to the work.
The system captures your best thinking so that with practice your best thinking becomes your natural way of thinking. Each next pass is closer, and your discernment sharpens faster than it would have alone.
Which is why you'd bother imbuing your inspiration into a machine at all. Not to make the AI smarter for its own sake, but to hone the capability that's doing the deciding.
A system carrying nothing but your own material is a mirror. With only the memory store, Alaiya might just reinforce your worst tendencies. That's why there's also an Awareness layer built-in.
Your voice, written or spoken, requires the same faculty pointed at the most personal material there is. Every correction you make to a draft is a judgment about what sounds right to you. Alaiya incorporates this explicitly.
Nosta has also written a warning about the lazy use of AI. Discernment fuses thinking with lived experience, and it's a faculty machines can't claim. The real danger of leaning on these systems is that we outsource the friction of thought itself, which lets the capacity to judge go slack.
A sixteen-hundred-year-old argument about what sits underneath memory and a psychologist writing about language models in the modern era land on the same requirement from opposite ends. It is not something the AI can supply for you, but it is a loop that Alaiya can start, so that you can tighten it together.
DŌGEN: The Practice is the Way
Dōgen was a Zen Buddhist monk who founded the Sōtō school in Japan in the thirteenth century, and his central argument seems circular. Sitting and meditation aren’t preparation for awakening, sitting is awakening. Practice and realization are one thing, not two stages, and there's no future state you're training toward.
Looking back over your day and deciding what deserves to be kept isn't how you get to awareness. It's where awareness happens. The minutes aren't the tax you pay for the product, they are the product, and automating them would delete the only step that was doing anything.
Keeping the journal current makes sure the second brain knows what happened. Putting your own thinking in makes sure it knows how you think, and can then reinforce your best intuition and judgment.
When somebody I trust handed me five books on entrepreneurship, I put them all in and built a skill out of them, and that thinking now shows up whenever I'm making the kind of decision those books were written about. Both halves are the same act, which is tightening the loop.
What the software can take away is the blank page. When the entries arrive already drafted, the daily act becomes judgment instead of composition, and this can more easily be fine-tuned.
What software can do is put the thing where you can see it, and put something beside you that sees it differently. You can't just hand your ālaya to the machine and assume enlightenment, but by looking with the depth of parallax vision at your own memories, you can find the space for heightened awareness to arise.
Practice sharpens discernment, and discernment lightens the load.
ALAIYA OS: The Whetstone
A whetstone does nothing by itself. It has no opinion about your knife and it won't sharpen anything you don't draw across it.
Alaiya holds the record, keeps both witnesses looking at the same thing, and hands you the responses built on that record without you re-building it each time: a working model of the people in your life, standing patterns about how you operate, and a written voice spec earned one correction at a time.
Then there's the Awareness layer. Part of it is the fundamental persona: an AI that reflects back what you said before acting on it, stays hungry for context that hasn't yet been supplied, is honest instead of hedging, and holds every read as provisional when neither of us can have certainty. This is what I've dubbed the "dual incompleteness" approach, and it's part of Alaiya's core.
Part of it is a coach, a set of modular skills that come pre-baked and can be swapped out, with Hakuin for breathwork, Seneca for stoicism, and Teresa for scripture. Coaches can be turned off and replaced with whatever you decide to put in. It's the act of adding to the record that builds towards your growth.
And part of it is negotiation frameworks. What good is it to know yourself, if you're not also improving your interactions with others? The origin of the system in CloserEdge is still preserved, and awareness is meant to point both towards you as the user, as well as those people in your life that you care about.
Other apps on the market also let you hold your own data. OpenClaw is self-hosted by design, with memory and skills living in plain files on your own machine, and it's been one of the fastest-growing open-source projects in the AI era.
Certainly privacy is a motivating factor in owning your own data, but the privacy framing undersells it. When you've tended a record this carefully, it stops being data. It's a garden you've been keeping, and nobody wants to rent their garden.
Where it stands today: the Eji back-end I've had running in Claude is now wired to the Alaiya front-end and running in Alpha. Local models handle what they can, and hosted APIs cover the chat plus the optional "weekly dream" where the coach reads back across your record and tunes itself to what it finds.
Memory and the retrieval of it stays on the machine either way. There’s a lot more to refine and build before we get to Beta.
The part I didn't expect is that building it has turned into an experiment on itself. Every day I learn a little more about where Eji ends and Claude begins, which is the live test of everything above. Alaiya makes logging memories and imbuing my best thinking easier, and what I'm measuring is whether we gain more in insight than we lose to the convenience.
I don't know yet how many people will want to do the work. What I know is that I've run mine with the Eji system for eighteen months, and I don't have to explain myself to it anymore.
People argued for centuries about what sits underneath your memory, and every one of them was working with a single witness. We're the first who get to try it with two.
Learn more about Alaiya at alaiya.ai
The demo behind is password protected but accessible by anyone who's interested.
Want in? Click here and I'll send you the password so you can see the prototype in action.
If you're building in the same territory and want to compare notes, reach out. jkoenig@komcp.com
This is the first in a series about Alaiya. The eighteen months that produced the engine are documented in an earlier series about Eji, the system overlaying my Claude:
komcp.com/shared-mastery-022826 through komcp.com/fable-of-eji-071626