Neo4j Live: AgentMemory for .NET: Persistent AI Agent Memory Engine

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Neo4j Live: AgentMemory for .NET: Persistent AI Agent Memory Engine · Channel: Neo4j · 1:04:49 · Watch on YouTube

Topics: AI agents, AI evaluation and benchmarks, Data, databases and embeddings, RAG, search and information retrieval

Summary

Jose presents AgentMemory for .NET, an open-source persistent memory engine for AI agents built around Neo4j and knowledge graphs. He explains how it distinguishes working, semantic, episodic, procedural, reasoning, temporal, prospective, forgetting, and bi-temporal memory, and how a router selects relevant memory types for retrieval. A live demo shows memory creation, recall, forgetting, graph exploration, and performance debugging, alongside benchmark results and the project’s current capabilities. The talk closes with future directions, including custom memory models, skill learning, and ways to measure agent consistency and alignment.

Key points

AgentMemory for .NET was rebuilt for the .NET ecosystem and is designed to work with Neo4j.
The project supports nine memory types, including prospective memory for reminders and forgetting.
A memory router selects the memory systems relevant to a user’s question and combines their results.
The graph schema represents people, organizations, locations, and objects, along with relationships, facts, preferences, messages, and provenance.
The demo app provides a live view of recalled graph data, memory traces, session comparisons, and performance.
The system supports shared memory and person-specific memory, as well as requests to forget information.
The speaker reports optimizing simple retrieval to about 200 milliseconds and describes additional query optimizations.
The project includes benchmarks intended to evaluate different memory types, in addition to LongMemEval.
Planned work includes custom memory models and schemas, learning from agent experience, and measuring behavioral consistency and alignment.

Chapters

Start	Chapter	Summary
00:02	Introductions and the motivation for agent memory	The hosts introduce Jose and he shares the science-fiction inspirations behind building persistent agent memory.
06:16	Why .NET and Neo4j	Jose explains his familiarity with Cypher and why he sees Neo4j as suitable infrastructure for an agent’s memory.
07:17	Introducing the study-coach demo	A recorded study-coach session illustrates retrieval before an agent response and memory updates afterward.
08:56	Agent memory, chat history, and RAG	Jose distinguishes context windows and replayed chat history from vector retrieval, RAG, and graph-based retrieval.
13:09	Memory types and routing	The talk surveys nine memory types and explains how the router chooses which systems to query for each question.
18:19	Memory organization and retrieval trade-offs	Jose discusses person- and organization-specific memory and compares files, vector databases, and graph-based approaches.
23:04	A study-planning example and project integrations	A student’s exam and travel plans show how graph retrieval can connect events, while Jose outlines supported integrations.
25:44	The graph schema and provenance	Jose describes the entity-and-relationship schema and how memories link back to the messages they came from.
27:49	Live demo: recall and debugging	The demo brain shows memory retrieval, voice interaction, graph visualization, provenance, and performance traces.
40:14	Personal and shared memory, including forgetting	Jose demonstrates separate user and shared memories, along with forgetting and hard-forgetting operations.
43:22	Performance and memory benchmarks	He reviews retrieval optimizations, performance measurements, and benchmarks for evaluating different memory types.
49:39	Current release and planned extensions	Jose summarizes package availability and outlines plans for custom memory models, skill distillation, and learning from experience.
52:17	Behavioral and experiential continuity	The talk proposes memory for an agent’s principles, decisions, behavioral consistency, and ongoing relationship with users.
54:54	Audience questions and closing	The speakers answer questions about querying and protecting stored information, ontology, model choice, and local deployment before wrapping up.

Quotes

Solutions have to be built. — 05:45

did we get any new information should we remember something — 08:22

So I made it safe by default. — 19:23

the graph is only as good at what it has — 24:07

memory shouldn’t foil sillingly you should be able to see it and measure it — 54:20

Concepts and entities

AgentMemory for .NET, Neo4j, Jose, Cypher, Microsoft Agent Framework, Semantic Kernel, Demo Brain, RAG, memória episódica, memória temporal, memória prospectiva, memória de consistência comportamental

Full transcript (original language)

00:02 Hello everyone. [clears throat] I’m excited to be here with you today. So I’m Zim. I’m from Neo forj um running developer relations in ina and I’m thrilled to be here together with my um good friend Jose talking about agent memory for uhnet persistent AI agent memory engine. So um I would love to have you guys drop

00:33 um your questions here in the chat. Uh what’s what’s your focus on? What brings you here. So and of course any any uh questions you have we would like to um to yeah to share and discuss with um with Jose. So let’s uh uh let’s welcome Jose and give him a warm uh applause for joining us today. So, hi Jose. Welcome. You’re muted. >> Not anymore. Hi Say, glad to be here.

01:05 Thanks to you, Antonio Forj for inviting me to host me live in your show. >> Amazing. Thank you so much for um for joining us uh today. So, I would love to begin with a quick introduction around like who you are and what you do. >> Yeah. Um it’s easy. Uh I’m a software engineer architect and I don’t just build agents, I build the tools of the people build agents with. So I’ve been collaborating with uh Microsoft on

01:35 sematic kernel since 2023 since all these craziness about AI agents and workflows started. uh been contributing as well to autogen and lately to Microsoft agent framework and as well since last year I started building some uh I would say flagship open source projects of mine like uh for example agent eval uh and now well um you know that one because it comes close to you um agent memory >> amazing and um I would be very much

02:06 curious to know like what drives your excitement and interest working in in agent memory particularly why inn net and why with neo forj um in particular >> that’s a very good question can you share my slides because I came prepared for that question >> okay let’s uh let’s do it um just a second here we go >> yeah perfect so um I will go back this

02:37 one because you already introduced me so what built me what drive me to create this. So, uh well, every robot that I love as a kid was defined by two things. Um that really hit me. What it remember and at what it valued 1,800 what 1977 um something happened. I was born in 71. Yes, I’m I’m I look younger than I look. So, I’m a small kid in a dark cinema and

03:08 a little blue and white robot draws on the screen carrying a hidden message that will save the galaxy. Uh, this one memory that saves everyone. I walked out of that memory out of that cinema thinking I want to build that then how in 2001 a machine with an inner conflict and a cube that couldn’t see inside its head and keep that words inside its head later on asimov uh I nearly read every story from him his

03:38 posetronic brain this uh if anybody read Asimov uh it’s amazing writer his positron brains had the values built in before anything else. And then it came all the parts that I love, anime, manga, robots. Uh, one that hit me is Astro Boy from Samuka. It was a robot built to protect people. And then this one was fun. Anybody remembers short circuit in 86? Number five. Speed reading books in the

04:09 house saying more input, more input. I always been a builder. architectures, complex systems, making them work. He never stopped wanting to build that brain. Crazy, I know. And last, today, it was science fiction. Until today, grabs were the national medium. Ideas connected by relationships that grow stronger or weaker over time, like we humans do. In year 2000, not that far away from HL, right? One year after um a

04:43 person named Emil sketch a graph on a napkin on a flight to Mumbai and this changed everything. So thanks Emil. Years before anybody called it a graph database 26 years later here we are [snorts] that napkin is where [laughter] my agent keeps his memory. Um I and then something happened this year. Uh there was an open- source agent memory for Python. Saints William uh and then I talked to my friend say here so he can

05:15 you can tell your part of the story if you want and I started to do a a port to learn uh how this was working because for me this oh my god this is open source I need to learn how this works and it became a full rebuild because the net world does not have the same that Python so today you will see the memory part an agent that remembers you like a coach that knows you the values part comes at the end and I already say that three times. U also Rasimov say it really really well.

05:45 Science fiction writers foresee the inevitable and although problems and catastrophes may be inevitable, solutions are not. Uh these next are mine. Solutions have to be built. So I’m trying to build one. And I will like to show you a bit what I built. It’s this part is a >> yeah maybe before we go forward um Jose say like can you outline briefly like

06:16 why did we go withnet and why are you particularly going forward with neoj >> yeah uh well I said it before I love cipher right so I think this what you built is pretty cool um so what neoforj has built in fact and it’s the leading database provider Right. So, and it’s very solid and and and very stable and you can host it in local, you can host it in a cloud. So, that’s what

06:46 drew me to it. I’ve been playing and exploring Neo forj for years to be honest. Um, so Cypher it comes very natural to me as a language. So that made it the the obvious choice and also that you are investing strongly on on that area on uh improving making performance improvements. Um so to me is is the perfect um infrastructure I would say the perfect uh provider to build the brain on. Uh that’s what draw me. Does

07:17 this respond to you? >> Definitely sounds exciting. So let’s go ahead. >> Yes. All right. So this is what I built. Uh let me show it. Uh so look over the video. It should be starting. Mhm. Do not disturb. So this is my demo agent playing a study coach, right? It’s remembering a fictional student named Lena. Uh this is a real session recorded and I want uh my agents to remember like

07:50 this, right? And when memory fails, uh, nobody tells you uh, nothing tells you usually. So what I wanted is a system that can tell you if it fail, right? And, and this is it. And what you see, you will see it in in a little bit live. This is the moment a video. You can see for example the phases where uh, we are doing two phases. One is when I ask something to the agent to my LLM and the agent that manages the LLM it’s a pre which is hey with this question that the

08:22 user has or another agent has hey remember look into your memory anything which is related to this and bring it back that’s what the retrieval part is and then we are doing the post hey did we get any new information should we remember something and and then it’s the remembering phase that’s at the moment how these things work and what you saw in the screen. Cool. Just wait there’s more cool in it. So um let’s talk first about some basics. What is agent memory?

08:56 So a good friend of mine asked me if I would uh explain a bit the basics. So I say okay because not everybody here knows about some basics. So let’s go through them. What is agent memory? How is it different from a chat history or rag? Um so uh in this case we can have a context window in your usual setup when you use an LLM, right? Which is what the model sees on the call. And then we have a chat history that is replayed and provided to the

09:28 agent every single time until it does not fit its context memory which each model has one. Some have bigger, some have less. At the moment is about 1 million tokens. Also we have uh some kinds of memory uh that usually use embedding. So this is vector database and we do the embedding process to transform a word into a meaningful vector that can be stored in that database, right? And it’s uh

10:00 retrieved by meaning um and it’s basically finding the nearest thing that coincide. It’s for finding things that mean kind of the same, right? And there is rack which is simply retrieval something that I want from any location. Uh rack does not explicitly imply a vector database. There are many methods and the other which is the one that brings us to graph databases is relationships. like humans do. We have a knowledge

10:32 graph in the uh graph database that there are things that have a space some relations to other things and we simply traverse the graph and with that we find what we want that’s ragged on here for example on Lina which have a tonin located in Munich right so things the nodes and how they are related the relationships are stored especially Here humans we have neurons and we have

11:04 synapses and they connect uh the neurons right so it’s it’s a very similar uh metaphor and then we have graph rack which is retrieval that follows the same attribute so it’s basically the same graph is basically a custom graph defined and explicitly done for a concrete purpose uh and you say it you are more expert in in graphite than I am but basically this is one of the ways to

11:35 do that and it’s a very simple explanation so it’s a custom graph for a concrete task which can be books can be medical can be finance can be um anti-money laundering here in Switzerland this is quite important and there’s a lot of focus on that lately but you can tell me say if that’s correct would you add something >> totally like these uh ro resonate with with our core use cases. Um, additionally like one of the very interesting cases is of course life

12:06 science and pharma like >> things around drug discovery. So how this using graph ragra uh basically accelerates um the process because we can better understand how all these diseases, viruses um are connected with each other and with which procurements and um yeah this us basically >> quicker um drug discovery accordingly and additionally there’s a lot of um

12:38 momentum around um aerospace as Well, like think about you’re building your >> I don’t know maybe your aircraft an aircraft is made of different components. You need to understand how these components are connected with each other with which um kind of um of of u yeah systems and so on. So you need to to understand the connections and relationships >> with it’s a way to reason things. So let’s move forward. Uh and this all comes to agent memory right which is uh

13:09 basically um deciding uh how to uh um write what to recall what to forget. Um but the thing which I’m using for this is basically uh a knowledge graph right or or or a graph or let’s be clear a different set of graphs but we will get close to that because it depends of one important thing the types of memory. Um

13:39 so we have here what technology and structure do we use and and then as well now comes what do we use to store that and this comes to the next part the memory types. Yeah. Um if we point uh at the brain uh each region here in the brain is a memory type that is represented through a nice visualization of a graph. And in the demo you will see them light up live as the agent recalls and learns. Um

14:10 and and then we have uh several types of memory. Uh the most usual ones are working semantic sodic basically everybody almost is using this one. Working is the conversation right now. Uh semantic is the facts and preferences that stay true the relations. For example, Lena prefers short morning sessions. Episodic is what happened and when it’s kind of a history of what happened in the different episodes of the conversations that we had and then we come procedural which is

14:42 how do I do this for example a coding agent needs a procedural but if you have a agent that want you want it to help you to cook something that’s also a process that is the kind of memory that we are talking here the routine that work then we have reasoning the agent’s own steps and tools and for that We have just seen that grabs are incredibly useful. And then we have temporal. When was this fact true? Not just when it was learned, right? That’s different thing.

15:12 And perspective. So that’s interesting one. And almost nobody’s using it. Is remembering to do something later, a reminder like some facts or some things that you have to do like a to-do list. I should do this by today. Right? These kind of things. and then forgetting which is also called abstension letting go forgetting usually this is not everybody’s using that all the memory systems usually or a lot of them are using a file system so they do not

15:44 forget they simply add more facts that override the the old ones but the old ones are still still there but this is very tricky also it’s called abstension memory which is not exactly a type of memory but it’s a traoral memory because you want to forget a among all the different parts and the best way to do that is through a decay function but that’s a different story and then we have by temporal which is two blocks right one when a fact was true and the other when the system learned it and

16:14 then when it stopped being true because I may have bought a car two months ago and then last week I sold it that’s no longer true that I have a car that’s an example all these nine types of memory are supported in agent memory before.net H and well uh these are the nine that I track others like share memory agents between agents images and audio are next right but the plug-in model that I will speak

16:45 a bit is how when >> one [clears throat] one question in between let’s say um like when do you use which kind of memory like do you use all memory components um al together >> yes and no >> okay >> I wrote them so every retrieval of every different uh kind of memory is a different system and I simply assemble them together so I created what you see

17:16 in the in the core it’s a memory routter it’s what is driving that so I identify on the question the types of memory that should support to respond that question that the user makes and then derive that to the corresponding. So I make a custom cipher query on each different memory system and then I bring them together. >> Mhm. Cool. >> It’s it’s interesting. Yeah. [clears throat] Okay. >> Nice. Um we have should we do the

17:49 questions now or should >> we can proceed because there will be some a couple of things. Um so uh basically what I said is we support the questions uh so all nine types of memory uh the only so I I wanted to look at what the others are doing and I think it’s pretty interesting that I support N of 9. I’m not saying aging memory for the net is the best. So everybody has their own uh capabilities but I think it’s an interesting exercise as a

18:19 standalone developer to try to make a system better than what company full companies are doing. So that’s a bit the the challenge I have. Um and at the moment I think it’s not going that bad. You will see. Uh I like to highlight the the work of SEP and graffiti. They cover six uh types of memory and SEP is the one that keeps both clocks of time by default. So by temporal uh so almost nobody remembers to do something later with a AI agent, right? So if ask about

18:52 the falls uh well um so if anybody has a question I can highlight if there is any question in the chat. I cannot see the chat. So you can see it say it but I cannot. Uh >> yeah there is there is two. So maybe we can uh we can highlight this now. So can see it here. Has any work been accomplished towards person centric architecture and structuring than the other project? >> That’s a very that’s a very abstract

19:23 question right but you will see um one thing I wanted is to be able to talk with my agent that has this memory and to remember me. So I made it safe by default. So it’s something I put a lot of effort uh and it can correlate and identify uh the memory for a concrete interlocator and this can be uh global memory shared by everybody or uh concrete memory [clears throat] that I can only access right so it has a key and the key can be a person or can be organization so it’s kind of what

19:54 everybody is talking about about organizational memory so technically this can be it you just need a proper identifier the other question Um the other question the other part of the question is has any been done specific to knowledge organization using historical methods not abstracted away but very clear. >> Okay. I think I responded that with uh

20:27 to pensic structuring using historical methods. uh what do you mean with historical methods that would be interesting to understand not abstracted so well um if you explain that I can tell you if yes or no um okay so let’s move on so this is what I built so the memory types in in there right um and and there the other thing is that uh a colleague of mine this I had an interesting discussion uh the like three

20:59 four days ago hey why don’t we use this uh wiki Right. So I wanted to bring that in. There’s some ways to store the information. One is in files you know uh clo mostly does that uh wiki that’s interesting right for storing different kinds of information. Created nodes reading fixing is very human. There is almost no infrastructure. You write it you store it that’s it. An example of that is cloud.mmd memory.mmd karpathis llm wiki uh and let us memory file

21:30 system which is quite good by the way. Um and then we have vector databases are great for finding a paraphrase a phrase searching in millions of memories that’s super scalable. Mem zero, open source, lang mem, magior. Uh, and then we have grabs and vectors, which is great for places, people, events, any anything you want for time owners where it came from. And you can also benefit from graphs. You can put vectorization and and do

22:01 vector search inside uh neo forj agent memory uh also agent memory for the net the one I’m presenting rapidly set and cogni do that right and they are really good but they fail silently sometimes wiki and files only the top is loaded the context can only hold so much because you load that into the context and what happens when two lines disagree which is current and then there is no owner so you don’t know who is editing

22:31 that right a vector database it will fail when it is not similar so it doesn’t come back it always will give you a confident of five and time usually is ignored on that dimension so it has its problems you can improve that with ranking by the way that’s a Google recruiting question but it only gets five to 10% more of accuracy and the best is routes and vectors al together but sometimes extraction can miss a at right time so yeah that’s a bit uh

23:04 the ecosystem right and then there is one question for three memories so in September my exam is on 12th of December so my statistic exam is on 12th of December at TU manion so if we go for files the the memory file grew on it stop is loaded the October line sits past the cut plan revision in Munich is on the 11th So it misses that or it has the potential to miss that. Then on vector exam revision finds exam at the weakest spot. It’s better. But the

23:36 hamburg trip is not similar. So it’s not retrieve. It retrieves something similar but a wrong plan. So that would be an example of cases that were used it. And then we have graph or graph plus vector walk from Lena to her December evens exam 12 or Munich and the conference 10 to 13 in Hamburg and then it tells you hey you’re in Hamburg until the 13 let’s move the revision earlier or later right so that’s an example of what happens

24:07 when you ask the agent a question and the memories are either in files in vector database or with a graph database like Neo forj Let me move faster because I need to show a couple things. So also the graph is only as good at what it has you know to it right. So it needs to be measured. So I built the graph agent memory for um so new forj built the graph agent memory for python and typescript. Um and uh

24:40 then inn net I also reimplemented that with vector only memory or as a graph rack provider as well. So it has the methods for graph rack that builds a graph but never writes one um or creates one from scratch. So I basically built it natively for uh Microsoft agent framework initially and then semantic kernel and as an MCP and as MIT also one important thing it’s it is also a connection to NMS something very cool

25:10 that uh Neo forj has created which is basically um agent memory in the cloud right new service I think at the moment is free so everybody can explore just search for neoforj nums ns and you will see it’s pretty pretty pretty easy to use or if you are in net you can use hme memory fornet all right then the basic schema of that powers or started powering um our brain is this

25:44 one it’s called pole plus o which means person organization location even an object basically the entities And then there are the relations right so it’s an entity it’s related to another entity uh message can mention an entity uh a fact is about the entity or uh preference is about entity and then it has a reason trace because uh provenence which is interesting name sounds very complicated but it says where does this fact come

26:15 from only as simple as that and then every memory points back to the message it came from so you have traceability. Let’s move on. Paul E on Lina. The example I gave, it’s very simple. Lena, hey, I’m Lina studying data science at TU Munchin here in Munich. Well, it participated in uh some quiz with some confidence, some intervals. It’s a student of TU Munchin and it’s located in Munich. So with that phrase, we have created that and we can reason over that

26:46 and is participating on a statistic exam. So this is the poly representation. I think it’s a >> maybe an interesting question Jose like how do you start building the schema and why did you go for this particular schema format like why did you go with Paul? >> That’s a very good question. Um basically this is the default that uh William Lion used to implement neo for uh a memory right. Uh so I

27:18 [clears throat] I I use the same at at the moment what I wanted is to have full tracibility. I know and so this what I build is fully compatible uh with um your uh schema and basically I wanted to be fully compatible with your schema and also it’s the schema using numps right because it it maps so it’s basically the same. I added a few more things on top but the the core schema is this one and this is the one use. Make sense? >> Mhm. Sounds good.

27:49 >> Yeah. And I will skip this one because I will not show how to code it, right? But I will go to [laughter] demo brain. [clears throat] Uh um but demo brain what I’m going to show is a flight recorder. It’s kind of a debugger or integity laboratory that I’m using to develop and try and and explore agent memory fornet which is open source. What I’m going to show demo brain is not. So with that you can chat with the brain you can debug the turns and you can see where the time goes. So

28:23 let me show it. Okay. Uh can you see that? Yes. So for example here I loaded me Jose and I did and I can type hey what projects I work uh

28:55 and then it goes uh it’s doing the retrieval um getting the memory and then if I say anything else it’s storing right in this case the only thing is episodic what was said and when uh and everything here it happens in real time in fact uh I could also configure that uh when mem speed that’s quite and then I have uh the rendering so at

29:26 the moment these particles can be uh adapted one particle is one unit when I say one unit is one token or we can do it a bit more and we can do for showcase because it’s a show right and yeah come on for particles and the next question would be a bit more interesting here so I can ask hey uh what do you think about agent memory

30:02 And now there’s some more things going on. Some more particles. Uh didn’t want to make it overload. And what you see by the way is real time. Uh and also that’s I put a bit of voice because uh the performance I work a lot in performance. uh uh to optimize this it reach along this way under 200 milliseconds and this makes this useful for for that. Hi um would you like to

30:33 use a memory as your memory? Well, in fact you are using it already but you can tell me why would you like to use it and what do you think about it? Also uh don’t overextend it. You have 200 uh letters to use and talk in a funny way that the words rhyme. So the recall 400 >> pick it. No luck required. Typed memory has me inspired with graph provenence in time. My recall stays truly prime. So

31:05 yes, I’m glad I’m wired. Your memory keeps me well minded. >> Okay. Uh let me disable that. Uh say uh was this here? Uh could we hear that properly? >> Yeah. Um it was um audible. >> Okay, perfect. Beautiful. So let’s let’s go back. Uh [clears throat] so uh can you tell me what you think

31:37 about me and what you know about me? And by the way, uh he knows a bit about me because he loaded a couple of things before. So yeah, uh just make me a short summary. Tell me some positive things of what you know about me, what do I do, what are my hobbies. Um yep, go ahead.

32:08 By the way, it’s a reasoning model behind. So sometimes it takes a bit more time and it rhymes. I’m speaking like the AI now. [laughter] Okay. He did not write. Hey, uh, can you say that again? I didn’t hear you. Okay. I think I lost the

32:39 Can you say that again? I didn’t hear you. Yeah, I think I uh broke it. So, let me do one quick thing and restart it. And meanwhile uh

33:10 I will show something. Uh, does anybody remember this scene from No. Okay, that’s a fun one, right? Um, so let me show you how this works here. Um

33:41 this is loaded. So let me do the other thing. It’s warming up the voice. Hey uh say in few words what do you think about me and my hobbies, attitude, um passions. Um you have like uh 400 letters maximum. Uh and say that out loud uh and rhyme a few words. Make it funny. >> Jose, here’s the deal. No fluff. All real. You’re a Spainborn Zurich code

34:12 wizard with tech skill and heart. Building agent agents like Intent Forge, pure open- source art. Poor J Ninja who loves sci-fi nights. Harlock and Azimoth fool your space flights. Evidence-driven challenge me too. That’s why his poems honest not fluffy goo. Verdict, brilliant, curious, slightly mischievous. 10/10 will debug with you again. >> All right, that’s pretty cool. Uh, okay then. Uh, let’s do the thing. So, I have here a book, right, or some chapters. Uh, and if you were aware, I’m loading

34:45 the full uh Alice in Wonderland, which is a book I like uh from Louis Coral. So, I’m going to load the book uh and we can count the seconds. Oops. Okay. No. Uh, teach Alice. Okay. It will take a bit more than the 10 seconds that it took uh number nine, number five, sorry, to to read the book. But also bear in mind that the um um what

35:19 part of the models I’m using are in my computer. So it’s it’s not at fast speed, but they have a very good responsiveness. So I’m losing accuracy for that. So now it’s uh feeding the the book of Alice into the brain. Right. Okay. And it’s done. So it was quite fast more or less the same. So what do you know about Alice in Wonderland? Can you summarize the story uh and what it meant for you? Um in a short uh uh one or two paragraphs, maximum 400 letters.

36:00 Okay. I think the audio there’s something there. Uh, it’s reading, but that’s more or less it and it even gives you. So, what do you think about the Mad Hatterer, which is one of my favorite characters. I I love that part. And then uh one important thing is I can make it forget um so I can tell um my favorite character is the Matt Hatterer um because it’s kind of um stands above

36:34 everything. Uh is a stoic personality and raises above everything uh even all the problems and challenges that he has which are quite a few. So also he’s fun, he’s passionate, and he’s kind of driving the full story in Alice in Wonderland. So kind of to me the the one of the characters that lead the story is is to me more the mad hatter than really Alice.

37:07 Also, let me show you one thing. Um, at the moment I show you how it works. By the way, uh, do you like it? Say it. It’s amazing. I I love the the the [clears throat] real-time voice um interaction and then how you get like the the >> Yeah. the um the the the things >> fueled from the the the graph from the brain. >> Yeah. >> It’s pretty responsive. So I can ask

37:37 even why this Android because I like traceability. I want to know and this is the other part that I like. So I still need to work a bit on that but I can tell hey uh humor memorable teaching. So here is what is in the graph. So I’m transcribing it according to the question that I made and I can look at the constellation of information that I have in the current graph. Right? So I can do a bit of transability and memories that it was given right um for

38:08 for this concrete response and also I can see what is shared or show everything or forget everything which at the moment I don’t want also it’s interesting that when I’m here I can also uh see aside from the knowledge which is a real uh graph visualization right that I can check and do a bit of navigation if I So I could say hey let’s see Asher uh what is it related to or intent forge or

38:40 SDLC is not dead or math doctor another proxy product [clears throat] um I could even navigate through that and and see what it knows or I could see the trace and that’s a different thing which I like to do so I can see for example how much time it was on on the request or storing information uh basically it’s kind of a network trace or the effort time for debugging and assessing the performance right the recall total it was 308

39:10 milliseconds and most of the time it was on the agent because I know at the moment I’m using the set theor GLM 5.3 flash which is not as good as some but it’s pretty fast and it’s pretty pretty cheap and then I’m see how much time it took for the other card. So I can see that also um can see some traces and all the traces are loaded. So I can also see what went on on this session and

39:42 reloaded and so it’s kind of a memory debugability that I have uh tool and I can also see a performance view of what happened on the different responses. Right? So I have this one. I can see that the recall happened in 384 milliseconds. Usually it’s quite fast. I can see where uh the most of the time went and I can see as well uh where the time is going right. So most of it on on the

40:14 agent and the factory is quite performant. Neforj behaves really well and the medics are also pretty good and I have as well operation leaderboard so I can see where should I put my effort in improving that. So I think it’s pretty pretty pretty cool and I can have some filters that I can or even compare sessions. So let’s go back to the brain. Um I think I show you that. Let’s do one thing. I can add a new person or I can

40:45 say Sam. I can say hey I am Sam and I like to surf and I like to watch movies. I can always reason why the question was done. Okay. Uh tell me what do I like to do? Okay. And I can simply forget

41:16 everything. Hey, who I am and what do I like to do? Oh, by the way, I should take the math history away. Okay. Uh, I am Sam. What do I like to do? Uh, do you know anything about me? Hm. By the way, Alice in Wonderland is loaded in a shared memory. So I can have a general memory and then a concrete

41:46 memory for me. So this is at the moment uh fully possible and and then I could even uh forget uh some some part. So hey uh I’m Sam. I like to surf and I like to eat chocolate. Okay.

42:20 People’s cook the puppy eat the cake. Okay. That’s >> So what are these different views, Jose? Like you can see the graph. >> Yes, I can see the graph related to the question, right? What it recovered uh for responding to the question and I can see the nodes in the graph. I could even tell them uh to to forget but at the moment uh it has not built enough memory. So I will simply skip that part

42:51 for now. But um if I go for example to me like again you can see there’s a bit more information, right? So I could uh say market mind and I could forget and I could ask hey what is market mind not afraid usually the one tell me so it does not remember so uh there is the

43:22 forgetting part and the hard forgetting uh implemented so we can tell it uh to hard forget something if needed be which usually is not that kind to the agent but yeah for testing purposes I think it’s okay. So that’s about it. Uh and uh then uh the performance which uh your colleague uh Michael Hunger uh was very keen. Hey what’s the performance uh Michael uh I don’t know if you’re around

43:53 but this is how I brought it. Initially it was uh higher than that. Um I was like taking several seconds. So it was kind of it was working but not usable. I’m talking that about when we met in uh Berlin in global AI conference that uh say was organizing right and I met some people from Neo forj banger and we were talking about this about the TCK and then the performance was really uh not that not that uh good as today

44:27 on about midepptember it was about 30 seconds the full around and about uh almost uh two three 4 seconds or even more uh the retrieval part which was terrible but it was working. Um but today the retrieval for a very simple request is about 200 milliseconds and uh as well um some other uh the the memory layer load

44:59 the neo forj I work a lot in the cipher uh queries and now it’s run from 300 it was a lot more before but uh so this is 25 July yeah so now it’s about 36 milliseconds right So uh and uh after some more optimizations thanks to the tool I show you uh it’s 24 milliseconds. So it’s widely highly optimized and the reason why I can use this in in voice interface when when it wanted to work but I think my system is kind of out of

45:30 memory the the voice model is loaded in my docker container so it may simply have uh exploded. I hope the um the Neo Forj local that I have because I’m using the local one is stays until the end but yeah so I’m pretty proud of the performance which is following and also about the precision it’s not that far away so what I did is I recall it rejected store so I did an observation I

46:02 also tried to explain what happens and experiment with other memories policies and thresholds and did a lot of analysis. So it’s a a lot of engineering mindset and one thing which is key I haven’t I have mentioned that I built something called agental where I’m pretty proud to have built [laughter] and it measures memory. So I reimplemented fully beginning of the year long meal. I can run a short long meal to have a grease a grasp of what

46:34 works and what not and how are we right and then I implemented a new uh benchmark on my own. Um which is type me eval. It measures concretely semantic working episodic procedural temporal perspective all the different kinds of memory in vertical because long meal measures mostly semantic and mixing memory from different places and it’s meant to see how do an agent reacts in a long context meal checks the memory in a

47:06 vertical slice. It does not care about the context. So usually if you want to use this uh with H&L you disable the chat history and you simply use your memory system. That’s the way to use it. But it has helped me to improve it quite a bit. And this is how we stand. And by the way if anybody wants to check Haval repo this QR brings you to it. It’s in aal hqal

47:36 and you can give it a start if you like it. I leave it for a moment and then I will go to the next slide. This is where we stand. So I will be very honest. Semantic is the top one working memory forgetting prospective episodic and then temporal the the last ones I’ve been adding them uh between August and September. They are still not uh top also. uh this measurement comes from

48:06 type name eval which may be quite hard for them. So it’s just a rule that determines that I will get better or not. So the sailing is almost perfection I would say. Uh and by temporal at the moment is off by default. So this is not enabled. That’s why you don’t see it light here. Um yeah long way uh what I mention I measure is about 76 to 90 at 4003 tokens

48:36 um when I use only the the memory on on the graph database which is not bad and hybrid is 8490 hybrid means I’m using um um the graph database and u chat history as well but then it it it occupies a bit more tokens 22,000. So you know you have to pay for that. And the others what have published is cip 71 to 90.2 uh mem 94.4 which is very high uh but

49:09 other people say it’s lower I don’t know and super memory inside mastra it’s around that one it’s 81 other’s 91 94 so yeah it’s pretty high on on the master. Yeah. Well, there’s a a war and everybody’s fighting along these long meal which to me is the wrong the wrong rule to measure them. But yeah, that’s just my opinion and doing things bit different. So uh what is done is

49:39 packages 1.6.1 I think since yesterday it’s in Nougat at the moment is inn net um um it it works perfectly on on ns. So it’s tested, it’s TCK compatible, it works with math, Microsoft GH framework, sematic kernel m, it has an MCP and also has some documentation in Microsoft learn and it has uh now today a recall of 0.2 seconds. So 200 millconds and the model is what takes the most. So that’s

50:10 why I did the med model local that loses precision but makes it a lot faster because of latency [clears throat] and and yeah then I built this demo brain and time eval so you can watch it and measure the performance and next I love the picture one of my favorite movies I plan extensibility to be able to bring custom memory models and graph rack schemas tailored to the job your agent does I think that’s a good and ka. So you can

50:41 bring oh now I want to learn uh martial arts. So you bring martial arts model right or one thing that is very cool I was discussing that I think one month ago with William Lion is about the skill distillation that is in Nams. So Nams is an amazing piece of art at the moment. It can distill skills for your agents from the memories. It’s pretty cool. And one thing I want to work is in autolearning. So improve from your own

51:13 experience. As soon as you have procedural memory, you can improve it and sharpen it and and learn how to do things better and then make it useful for coding agents because if we can achieve 90 95% of the currency with I mean 400 tokens or uh put 5,000 tokens, right? And if you use it with a normal way with files, it would be 100 or 200,000 tokens. I think we’re saving

51:44 some money here, right? Um, so that’s one of the goals. And then polish it, add the pending memory types completely, measure a bit more and improve the performance and the measurability also. This is what I mentioned about the values and a bit one idea I have around my brain and shaping itself is this is a research right once memory types are plugins the first ones I want

52:17 to build are not about what the agent knows they are about how it behaves what if agents could remember not just what they discussed but how they behaved while discussing it behavior Consistency memory is the umbrella made of two parts. I call them identity continuity. This is consistent character. Its principles its values. Remember right the values of the starter robots are what say the galaxy they deliver the

52:47 message right uh isak asimov it were the values and the lows the three robotic lows and then its decisions under pressure and drift detection alignment. That’s the biggest problem we all have. The same agent on Monday and Friday. And then experiential continuity, the relationship continuity. It’s the share history and moments that matter. Yeah. Matter to the agent. I don’t know if you are aware but clothes

53:18 and other high level reasoning are putting some thoughts and scoring them internally. They are doing that. All right. So it’s basically this experiential continuity as well this relationship continuity the share history and moments that matter no rebriefing every morning to your agent right not dating he already knows what you talk about >> um Jose we are slowly coming to an end how about we answer some questions from

53:49 the audience >> sure um I think that’s a bit it so my goal is basically to make alignment measurable also to provide a benchmark to check if the agent is aligned by checking the memory and how the agent with that memory responds and also auditable build on the extensibility model [clears throat] right so I think this can help on the alignment problem that companies like anthropic and openai are curious and if anybody’s working on it I

54:20 like to connect to you and talk to you I’m very interested on that and this is basically adding the proper values will determine the behavior. Uh and that said uh at the start you saw an agent remember Lena memory shouldn’t foil sillingly you should be able to see it and measure it because files can only do so much. Thank you say >> amazing thank you Jose. So, time to take

54:54 some questions from our lovely audience. So, just a second. Here we go. So, thank you. Thank you, Jose, for the very nice talk and uh an amazing live demo work today. So, you nailed it, which is great. sourcing it live [clears throat] and it was just amazing. So I’ll just go top down. Um the

55:28 question we have here uh from K Zota is it possible to ask questions about a database as its as it as in its contents? If yes, can I check myself if what happened is true with a query? Would such a query be provided? Can it block sensitive data?

55:58 >> That’s an interesting question. Um, this is one of the factors where the extensibility part would make its uh thing. One one thing is to talk to it and create some memories, right? using um um the the current uh memory uh types that we have. But maybe we could set up a concrete memory type for database or for AI infrastructure or for um IT infrastructure, right? Um and put it all

56:28 there uh a custom graph uh for that and add it as a memory type and plug it in and then yes, it could respond to that questions. it could store the information and it would have the queries for that. So yes, and it can block sensitive data. Well, if we build that into the schema, yes, it’s perfectly possible. >> Cool. Amazing. Um, another question from uh Harita.

57:00 >> What comes first, ontology or knowledge graphs? Is this built out of reading your data model or based on interviews with business? Um so yeah I can I can start maybe you can extend Jose. >> So um as we saw at the beginning of the or during the presentation Jose presented the the schema of his uh of his graph and of course that’s on on a

57:31 on a on a wider scale. This is the ontology like understanding what kind of nodes, what kind of relationships our graph should have and outlining briefly like what kind of um yeah exactly relationships and nodes we want to um to have on the graph to ingest uh data uh accordingly. So this comes first like this the first part like thinking about the schema and then using uh using technologies like

58:01 document intelligence which we just launched in public preview or your or or with our graph rack package you can also accordingly start building your knowledge graph and ingest that the content and the documents uh into the graph. What do you think Jose? uh uh I I think it’s a very wide uh detailed question. I would be more simple. I mean ontology is the model, the rules, the vocabulary, what makes the schema, right? And the knowledge graph is the actual facts connected according to the

58:33 chart model. So to me it’s kind of the chicken and egg thing which to me is kind of uh the graph uh schema is part of the knowledge graph. to me is uh ontology is part of of a graph uh because it’s what defines it, right? It’s what uh shapes it. Make sense? Sounds good. Um another question from Great Work

59:04 Great Work Alliance 642. >> Oh, >> I’m confused. Uh wait. Um >> yeah. Yeah. >> Yeah. >> It works. Okay. >> Yeah. I’m confused why reasoning model is being used um in this use case. No reasoning should be required for the type of retrieval being demonstrated so far. >> That’s that’s a good one. Uh I mean um it it it depends right. So I’m giving uh first reasoning model for what right? Um

59:36 I I have uh the embeddings model for the embeddings. So that’s definitely not a reasoning and the reasoning is done for retrieving and also for storing. So and I think it it it shapes a bit uh I have it in low right so it’s not reason that much basically what drives it is interesting story so I’m also a Microsoft MVP Microsoft AI MVP one of the four in Switzerland um and um Microsoft uh gives

1:00:08 gives to you a bit some benefit of Azure credits for presentations of open source which this open source at the end is contributing on uh Microsoft agent framework. By the way, if uh you went to my last slide on how to use it in math, it’s what points to the Microsoft Asian framework repository uh which explains how to use this um agent memory fornet which is part of the uh uh neoforj ecosystem but

1:00:39 also of the uh Microsoft ecosystem. So as part of my contributions to the AI ecosystem. So this thanks to this I’m being recognized as a Microsoft AIP aside from some conference talks like this one. Uh and they gave me some credits for Ashure. Um basically before um almost mid of September I was using uh OpenAI model deployed there. Uh well I I’m doing a lot of performance test

1:01:11 and doing a lot of benchmark to see if things work or not and this consume a hell a lot of money or credits that Microsoft gently gave me. So thank you Microsoft uh and and that’s the reason why I switched very quickly to another provider and this was the fastest model that they had. So that’s a bit the reason why I went with that. Hope it makes sense. Cool. And uh quick [clears throat] question regarding the brain app you built.

1:01:42 Uh can you run it locally? [snorts] >> If I can run it locally. >> Yeah. >> Um it’s running locally. [snorts] Um the at the moment the new forj database it’s local right. >> Oh perfect cool. >> The um the application is running locally. The embedding model is running locally. The only thing which is not running locally is is the uh is the model the reasoning model that I’m using. Um and yeah, I could run it locally but then we will need like one

1:02:13 hour more of session to let it think before it can respond because my computer can only do so much. So if anybody wants to sponsor me uh one of the new Apple machines for local inferencing, yeah, I guess we could. But still uh it could run the same model at about 60 or 70 tokens per second with less latency. But that’s the speed. Uh where I’m running them is in bit deer. Uh and the speed is more or less 200

1:02:43 tokens per second. Well, I guess we cannot beat that yet with local models, [clears throat and cough] but yeah, you can run it locally. >> Cool. So, I think that’s uh pretty much it. So, thanks so much, Jose, for joining us. Um just quickly before we wrap it up here um so big applause to our good friend Jose exciting work what you’re doing in um in Asian memory. So looking forward how we can keep the momentum um going so and collaborate more

1:03:15 and um yeah so coming up later this uh um this November we are running our developer conference called uh nodes so it’s an online conference it’s virtual so feel free to grab your cup of tea coffee in the evening depending where you are on the planet. And yeah, join us, tune in and listen for some cool uh stories and technical

1:03:46 sessions around graphs and AI and to keep uh learning to keep the conversation going um you’re welcome to join us of course on Neo Forj Graph Academy. So this is our um school. This is our academy where we um where we teach the latest um um learnings um in in graph technology. So we offer events, workshops and courses

1:04:17 talking about the latest um yeah advancements in in graph technology. To get started with Neo forj you also have uh we have also our community edition of aura. Let’s plug and play, spin up an aura instance and start building. And uh here we are. So it’s been great talking with you, Jose. Thank you to everyone who join us around the globe and looking forward to yeah to the next one. And uh yeah, let’s um

1:04:49 let’s keep the excitement going and see you at the next one. Thank you everyone.