Build Your Own Open-Source Dots with Pi and Telegram

Author: Hugging Face · Watch on YouTube

Video authored by the channel shown, not by IANIA; all rights belong to the creator. Transcript and summary generated automatically; the original-language content is preserved.

Original language: English

Original video: Build Your Own Open-Source Dots with Pi and Telegram · Channel: Hugging Face · 20:38 · Watch on YouTube

Topics: AI agents, Task and workflow automation, Local and open-source models

Summary

The video demonstrates how to build always-on, specialized agents using open-source models, Pi, and Telegram, hosted on a VPS or another machine that stays on. Each agent is configured in its own directory with its own context file, tools, and skills, such as Google Workspace access for a personal assistant or web research tools for a research agent. A lightweight Pi Gateway connects Telegram bots to Pi sessions, stores session information locally in SQLite, and routes messages to the appropriate agent. The presenter sets up two Telegram agents and suggests extending the gateway with additional tools, automations, or messaging platforms.

Key points

Agents can run on any suitable always-on machine, including a VPS, a home desktop, or a DGX device.
Each agent is organized as its own directory with an agents.md context file, tools, and skills.
The personal assistant example uses Google Workspace skills for tasks involving email, calendars, and Drive.
A message gateway enables access to agents through messaging platforms rather than only SSH or a terminal.
Pi Gateway listens for Telegram messages and routes them to the relevant agent and Pi session.
The gateway stores session information in a local SQLite database and supports starting new sessions.
Telegram bot setup includes creating a bot with BotFather and restricting access using a user ID.
The demo configures separate Telegram bots for a personal assistant and a research agent, using different models.
Possible extensions include scheduled automations, additional tools, and support for platforms such as WhatsApp.

Chapters

Start	Chapter	Summary
00:00	Open-source always-on agents	The presenter introduces the idea of specialized, always-on agents and previews personal assistant and research agent examples.
02:03	Agent directories, context, and skills	The video shows how agents are organized in separate directories and demonstrates adding context and Google Workspace skills.
06:44	How the message gateway works	The presenter explains how a background gateway can connect agents to messaging platforms and route messages to Pi sessions.
11:23	Installing and configuring Pi Gateway	The video walks through installing the gateway, creating a Telegram bot, restricting access, and configuring its agent and model.
16:00	Testing multiple Telegram agents	The personal assistant is tested, then a second Telegram bot is configured and started for research.
19:06	Extensions and next steps	The presenter summarizes the setup and suggests adding tools, automations, and support for other messaging platforms.

Quotes

build the same thing, but with open-source models — 00:31

Each agent is essentially its own directory — 06:44

both of these have something that is called a message gateway — 08:19

Your personal assistant running locally in a terminal harness, which is Pi. — 16:00

add more tools to your agents — 20:07

Concepts and entities

Dots, Pi, Pi Gateway, Telegram, Google Workspace CLI, SQLite, Firecrawl, OpenAI, OpenRouter, Kimi K3, VPS

Full transcript (original language)

00:00 Good morning, everyone. How’s it going today? So, recently OpenAI released dots, which are always-on agents that are very specialized in their specific use case. So, for example, if you can have a dot to be a research assistant, another dot to be a personal assistant with access to your email and your calendar, for example, and they can perform tasks for you, and they can ping you whenever they find something important, etc. Uh what I’m going to do today is I’m going to show you how to

00:31 build the same thing, but with open-source models, and how you can host it your own in your own always-on machine. Uh I’m going to be showing how to do this in my VPS right here, but the process is exactly the same in a VPS, in a desktop that you may have at home, or a DGX board, for example, that you might have at home if you want to run these models locally, okay? And it is very, very straightforward. The architecture is a little bit inspired by what uh Hermes and Open Claude do with the

01:01 message gateway. I’m going to be explaining a little bit more of that in a moment. But as you can see, this is what you’re going to have by the end. In my case, I created two different agents. Bon Andre right here is my research agent, and it has specific tools for research, and it has a system prompt that tells it that it is a research assistant. And right here, I have Bonnie, which is powered by the Stealth model, um Space Bonnie. And as you can see right here, this one has access to my Gmail account, to my calendar, and it

01:32 can create new emails, send them, read my emails, summarize my inbox, create new events in my calendar, etc. And you can create as many as you want, give them their specific tools that you want, run them on your own keys, with any model that you want, and I’m going to show you how to do all of that right here. So, without any further ado, let’s get right into it. All right. So, first of all, let me show you the agents that we’re going to demo right here, okay? And it is very, very

02:03 straightforward. As you can see, I am inside an agents directory, and I am actually inside of my VPS right here. Now, again, this is supposed to be an always-on machine, so my case it’s a VPS in this example, but this can be your home desktop if you leave it on, for example. It can be a DGX Spark if you have it at home or you want to run local models out of it. That can be also very cool. Uh in this case, it’s just a VPS, and but any always-on machine will be able to uh recreate this.

02:33 Um because we want our agents to be reachable anytime, not only when we’re in front of our computer. And right here, as you can see, I am in the home directory, and I have an agents directory right here, and inside of it I have a couple of agents. And my agents are essentially just directories, okay? It’s very, very straightforward. And if I open one of them, for example, I’m going to open the PA assistant, right? And I’m going to open it, and let’s take a look at what is inside here. I have an

03:05 agents.md file, so let’s take a look at it. Actually, I’m going to open Tau in this case. It’s the um the harness that I’m building right here. Um yep, yeah. I’ll not won’t upload immediately. But as you can see right here, I have an agents.md file, which is the context from this specific folder. So, I’m going to click on it to take a look at it, and as you can see, it gives my agent context about what he’s doing. So, uh you’re a personal assistant, you help the user organize their work with Google Workspace through the G um Google

03:37 Workspace CLI, uh so that it can have access to my Gmail, to my calendar, etc. Create new calendar events, send email, summarize my email, triage them, etc., right? So, that’s already very, very cool. I’m going to close this right here, and I have a bunch of skills as well that are going to be quite useful. I have research skills, which is the firecrawl powers, firecrawl scrape, and firecrawl search, so that my PA assistant can also search the internet. And something that I didn’t do right here actually uh is add the skills. So,

04:09 let me just add them. So, I’m going to just do NPX skills add and I’m going to add the skills from the Google Workspace CLI. And in order to have this, you’re going to have to install the Google Workspace CLI. Uh so, that I mean, of course, that’s probably another tutorial, but here you can see the bunch of skills that they have available. So, I’m going to want it to be able to access my calendar, my agenda. I want it to be able to also take a look at my Google Drive and to

04:40 upload files to Google Drive. I want it to be take a look at all my emails as well and be able to use that. And let’s say that that is all. Going to hit enter. I want to make sure that this is available for Pi, and I’m going to set that this is going to be installed only for this project because it’s going to be installed only for this specific agent. Going to say yes, install it with the sim link. Uh proceed with installation, and there we go. Now, if I open my assistant again, you will see

05:10 that here I have a bunch of skills right now, and they include the Google Workspace. Again, if you want to use this harness right here, it is Tau. It is one the one that I use for coding most of the time. It’s literally just a port of Pi in Python. And you can also, of course, look at this in Pi. Uh let me just trust this folder. And as you can see right here, we have all my skills that I told you before. And uh there you go. Now, this one right here is all right in my research agent.

05:40 I’m sorry, my PA agent that I can ask it like things like, “What are the recent uh unread emails that I have in my inbox. I can send that right here and it will answer for me. Oof, probably shouldn’t have used Astra for this simple question, but there you go. And there you go. It can read my emails and of course I do not want to be into need to connect to Pi via Herder every single time that I want to do this.

06:11 That’s why we’re going to be connecting this to Telegram in a moment. But as you can see it works, it knows about my email, it help it can help me organize my stuff and this is just one agent, right? If I go back up I have other agents right here. Let me just clear this. I have other agents right here. I have my researcher and for this one I can add different skills, different stuff, different agents.md file to give it a different personality, etc. Okay? So, there you go. That is essentially what my agents are. Each agent is essentially its own directory

06:44 with its specific tools, specific skills, and specific agents.md file and you can just put them all together like this, okay? Now, that now that you have done that, how I would encourage you to to imagine the different agents that you want to create, the different tools that you want to give them, and the different skills. Once that is done, we are ready to actually connect it to our gateway and let’s take a look at what the gateway actually is now. Okay, so let me show you how we’re going

07:15 to be building the gateway that is going to allow us to connect to our always-on assistant not only using Herder or SSH, but using a multiple multiple different platforms such as Telegram, WhatsApp, email, Slack, etc. Okay? And just to give you a little bit of precedent and a little bit of context about this, this is actually what made the agents such as Hermes or Open Cloze so important and so popular, right? The idea behind Hermes

07:47 and Open Cloze and the thing that really clicked with most people is not only that they have a lot of integrations with a lot of different tools, uh but the fact that they are sort of your personal assistant that you can talk to from any platform that you would usually use to talk to your actual human assistants, okay? So, for example, you can talk to Hermes and Open Cloze via Telegram, via chat, via via WhatsApp, email, etc. And what we’re going to be doing with

08:19 our commas or with our dots open source dots is we’re going to be building a similar system, okay? So, uh both of these have something that is called a message gateway, which is a background process in the always-on machine, be it a VPS, a spar DGX spark, etc. And this always-on machine’s going to have this uh background process that is going to be constantly listening to whichever integration it has with multiple

08:49 platforms. And the example that I’m going to be doing right here, it only supports uh Telegram right now, but that can be a really actually cool project for you. I’m going to show you how you can uh um implement the same thing but with other different uh platforms later on. Um so, what’s going to happen is that this process right here is going to keep listening to our Telegram messages. Uh when Telegram sends a message to our gateway, this gateway is going to create an ID. And that ID is going to uh have a

09:21 uh few uh elements. It’s going to include the source, so it’s going to the source is going to be Telegram, okay? Uh it’s going to include the name of the agent, so it’s going to be, for example, my PI agent. And it is going to be include the ID of my session from Pi, okay? And that way, that message is going to be stored in my SQLite, okay? So, if there is no session in my SQLite associated to that, it’s going to create

09:51 a new session in my SQLite database. By the way, the SQLite database runs locally in my always on machine, so in my VPS in this case. And it is going to create a new session. Then, whenever it receives a new message from Telegram, it’s going to see that it comes from my PA agent. It’s going to see that it’s the same session, so it’s going to send that and route it to that session in that particular directory, okay? And remember that Pi works in each directory, so that way uh since it will be sending that message to the specific

10:23 Pi session in that specific directory, it’s going to run with our skills, with our tools, with our context that we defined in uh the last chapter, okay? And that’s essentially all that is going to be happening. And if at some point I do {slash} new on Telegram to create a new session, then it’s going to get right here, it’s going to start and it’s going to identify that uh it’s that’s a session that is not yet in my SQLite database, so it’s going to create a new entry for a different session, and that way we’re going to

10:53 continue the conversation in my sessions in Pi, okay? That’s essentially all that is going on right here. It’s very fun, very straightforward, and the good news is that you do not need to code the gateway yourself. I actually have prepared this nice little thing for you. Uh it’s very straightforward. I started it using it a few months ago when um Termux came out. I when Termux was in its peak uh popularity, because I wanted something very minimalist. And uh I

11:23 started using it, and it’s basically just a very minimalist gateway that you can run in your VPS. I’m going to show you in just a moment how to run it. And it runs in the background and it you you can create a bunch of different bots right here. And as you can see it’s very very lightweight. It took me like 34 commits to finish this up until the current state. And yeah, uh let me show you how to actually use it. So, uh what we’re going to do is we’re going to go straight to our VPS again in order to set this up. We still have to

11:53 have access to our uh machine via SSH. So, right here, let me just show you. And right here, I have access. And what I’m going to do first of all is I’m going to install so UV tool install uh Pi Gateway, okay? And this is going to install this uh this tool that I just showed you. And just to be sure that it’s it exists, we’re just going to take a look at it. There we go. Pi Gateway is installed. And what I’m going to do now is I’m

12:24 going to go into my PI agent uh assistant right here. And there we go. So, here is my PI agent as I showed you before. Uh what I’m going to do is PI uh Pi Gateway and I am going to just initialize it right here. So, Pi Gateway init. All right. So, we’re going to go to Telegram right here and I’m going to show you I’m going to link in the description the exact ID of the bot that you have to call to create your own bot. But uh it’s called Bot Father like this,

12:55 okay? And once you have found it and it looks like this and you’re sure that it is the same uh ID as you see right here. We’re going to do {slash} new bot, okay? Because this is going to be a Twitter um a Telegram bot. All right. So, there you go. So, we’re going to start a new bot. How are we going to call it? I’m going to call him Bunny. Because I’m going to be using a bunny uh space bunny uh stealth model. Um then it says, “Now let’s choose some username for my bot.” Uh

13:26 it’s going to be also bunny. I’m just going to add a lot of weird characters because it has to be a unique one. And there we go. Oops. Uh it’s not supposed to end at test. It’s supposed to end in bot like that. And there we go. And uh now it has sent me this thing right here, which is the token to or to use this uh this uh bot. And this is the thing that I’m going to copy and I am going to paste it right here.

13:56 And here, also very important in case you’re building your own gateway for Telegram, you want to block uh which IDs have access to it. Um So, in this case, my user ID uh in order to look it up, you’re going to go to your user info bot. Just look it up like this with this exact ID. And just ask it uh {slash} start. And it will send you your ID. Okay, in my case, my ID is right here. So, I can just paste it. And there you go. Now, it is

14:27 going to ask me in which directory do I want this to uh start this um this Pi session this Pi uh bot to start. And of course, I want it to be inside my P A assistant uh directory because I want it to have all the context that we mentioned before. So, I’m just going to hit enter. And after that, it asks me which model I want it I want to use with it. And in my particular case, as I told you before, I’m going to be using uh space bunny from Openrouter. And there we go. I think in level it’s going to be the same one. And this one

14:58 it also asked me to name it. I’m going to name it bunny. There we go. And now, what I can do is I can do Pi gateway and I can do start and I do {dash} I to name the instance and I say bunny. And there you go. Now, Pi gateway is Oh, sorry. Uh it’s not stop, it’s start. And there we go. Now, it started Pi gateway. I can also take a look at Pi Gateway status. And as you can see, my Pi Gateway is running. I can also take a look at which

15:30 bots are running by doing this, and I can say instances. And as you can see, I am running Bunny right here. And what I can do right now is I can just go right here, go back to my bot, and let’s click on this Bunny right here. Click and start. And there we go. My Pi assistant right here is started. So, let’s actually just chat a little bit with it. Let’s say, um who are you?

16:00 And what are you capable of doing? As you can see, it’s working a little bit. And there you go. Your personal assistant running locally in a terminal harness, which is Pi. It has access to my Google Workspace, to my Gmail, to my calendar, to my drive, to my contacts, et cetera. And there you go. I mean, now I can just text it every day. And I would encourage you to clone the Pi Gateway repository that I showed

16:31 you right here. Explore it a little bit, and try to build on top of it. Something that I would like to add in the future, for example, is automations. So, if this thing runs automatically every day, so I can, for example, schedule automations for every day at in the morning, check my email, summarize it, and triage it for me, and organize it for me, so I can archive things more easily in the morning, right? So, these kinds of things. Same thing for the research one. And then, let’s just start off the start off the research one. So, I’m just going to go back, just going to go to the

17:01 researcher one, and let me just go right here. I’m just going to go Pi Gateway, and I’m going to initialize another one right here. Uh for this second one, I’m going to initialize a new bot right here. So, this one’s going to be a new bot. It’s going to be a researcher. I’m going to call this one Andre. And there we go. Now, I need to add an ID for it. So, I’m going to call it Andre. I’m just going to add a bunch of

17:33 different things cuz it has to be unique username. And there we go. Just going to copy this again. My Telegram user, so only I am allowed to contact like this. It has to be inside researcher. And what model am I going to use for this one? For this particular one, let’s use Yeah, let’s post it for this one. I want to use Kimi Kimi K3. Okay, so let’s go with Kimi K3 right here. Thinking level default, and this one is going to be

18:05 called Andre. And there we go. So, now if I do Pi Gateway and I take a look at my instances, my instances, you can see that I have Bunny and Andre working. And Andre, if we take a look at this, let me just open the Andre bot right here. I can just ask it. Hello. Who are you and what are you capable of doing?

18:35 Oh. Oh, my bad. I forgot to start it. So, I’m going to do Pi Gateway start and we’re going to start Andre like this. And now I agent is running and I can take a look at it. It says the Pi Gateway got connected. I’m just going to send my message again. And there you go. Pi is working and this one right here is my research agent. It has FireKrawl. It has I can later on add other plugins and other tools. So, there

19:06 you go. It’s very straightforward. It’s a very easy way to have a bunch of different agents right here. All of them specialized in your phone or to continue on the go and every single one is specialized in the whole thing. Every single one actually, just like the ones from OpenAI, has its own computer because it actually has its own workspace right here. As you can see, this is just it’s just a workspace for it where it can create, write files,

19:37 perform research, etc. So, it is very very useful. Um so, there you go. I hope that this has been useful. It is a very very short tutorial and very straight very quick demo of how you can build this thing yourself. Please do let me know if you want a more detailed explanation of how the Pi gateway works. For example, if you want to take a look at the actual architecture and how to create it, if you want to maybe expand it a little bit. Some things that I would leave you as homework would be to, for example,

20:07 add more tools to your agents and add them via CP or via custom tools to your Pi agent, for example, or you can also explore a little bit the repository that I have right here and try to add your own platforms that you want. So, for example, it can be you can add what support for WhatsApp, for example. But in that case, you would have to learn figure out how to to connect WhatsApp to the to the gateway. You can always take inspirations from from open-source repositories that already do that, such

20:38 as OpenFlow or Hera news, by the way. Um so, yeah, there you go. I hope that this has been useful to you. Thanks a lot for watching and please do let me know what you want to see next. Have a great day and I will see you in the next one.