This AI assistant plans everything for me

I do a lot of walking. That is where I plan: a business, an investment, a YouTube video, a book I just listened to. I used to run those walks as voice calls with ChatGPT. The personalization is good. The rest of the product falls apart once the call gets long.

So I built a voice agent that can write to Notion while I talk. It creates, updates, and archives tasks. It writes notes. It has the internet, so it can research or check a claim. Gmail, Google Calendar, or any MCP server can sit on the same call.

GPT voice calls drop the transcript

Call mode is still GPT-4o. We are on 5.1. That is one problem. The worse one is memory. I talk for 40 minutes to an hour. Sometimes hours. GPT does not keep the transcript. It stores a stub that says audio transcription. The details are gone.

Their Notion connector also timed out on me. I stopped using it and built the agent myself.

ElevenLabs agents, GPT-5.1 underneath

ElevenLabs runs the speech. Transcription in, audio out. The model behind mine is GPT-5.1. You can pick any model. It deploys as a web app or a mobile app.

The demo I ran was a brainstorm on complete life freedom. I asked it to open a new Notion note covering time freedom, location freedom, financial freedom, and freedom from authority. It did.

It does not wait until I hang up. While we talk it keeps writing tool calls into that same note. A typical session is 30 minutes to an hour. The document grows during the walk.

eleven.ai for Calendar, Linear, and GitHub

The simple path is eleven.ai. Create an agent. Pick a voice. Add the first integration. Google Calendar is there. So is Linear, which I already use for software work, and GitHub.

On a walk I can talk through a feature. The agent writes a Linear issue with requirements, acceptance criteria, and the tests that have to pass. Linear syncs that to GitHub. Coding agents pick it up while I am still outside.

Their built-in Notion integration was weak. I am not using that path for Notion. They also ship a mobile app.

Agents platform: medium reasoning, low temperature

The full setup is the ElevenLabs Agents platform, not the creative one. Prompt, model, temperature. I run GPT-5.1 on medium reasoning. Replies take a couple of seconds. Gemini 2.5 Flash, GPT Nano, or Mini are faster. I take the delay. High reasoning did not change much for this job.

Temperature is low. Expressive mode writes long answers and cuts across what I am thinking. The prompt says keep answers short so the conversation keeps moving. The same prompt can become a nutrition coach. Swap the system message.

Workflows exist. I have not used them much. The idea is a fast model for the greeting, then a handoff to GPT-5.1 high when I say I want a brainstorming session. Each sub-agent gets its own goal and knowledge base.

YouTube scripts, Russian, and a speech count

I made this to get back into posting. I gave the agent a link to the channel so it can scrape what I already talk about. GPT wrote a style document: real-world experience, automation, leverage, systems, business. That context is what makes the scripts usable.

I also have a Russian teacher. I am learning Russian. ElevenLabs does about 70 languages. And a speech coach. I say "um" and "er" a lot on camera. One line in the prompt: count them, correct me, give me the number at the end. Phrasing notes work the same way. Real time.

The same shape works for any coaching vertical. Scrape what already works in a community, put it in the knowledge base, let people call it. One example I looked at was a porn-blocking app doing about $500k a month. I am not shipping that. The pattern is the useful part.

A Twilio number so the agent can call first

Most of these products wait for you to open the app. Twilio gives the agent a US number. If this were a nutrition coach and someone skipped their macros, it can call them. Coaching is often accountability. The voice is good enough that a lot of people would not know it is a model.

I could train it on every YouTube video I have made. I probably will not. What I said two years ago is not what I think now. There is a call-analysis tab if you want to train on the conversations you already had.

Notion tools, Firecrawl, and MCP

The tools tab is the actual product. Notion can add a task. It can list recent conversations so I do not repeat myself. A blank memory every call is why these things feel dumb.

Firecrawl is the search layer. Ask for the top ten podcasts on life freedom and it pulls the pages into context. MCP is the rest: Airtable for a top-ten Instagram list, GitHub, even a Cursor agent from the same walk. HubSpot, Zendesk, and Salesforce are pre-built if you want a CRM. A website embed is a script tag.

I did not build my agent in that UI. That walkthrough is for non-technical setup. The Agents platform has an API. I vibe-coded the whole thing in about three hours. Claude Opus 4.5. I built it the day before I recorded this.

AirPods beat another chat window

The interface change is the point. Wearables and glasses will matter. @cluely already shares the browser screen. Right now I put AirPods in and talk: project manager, life coach, Notion assistant, brainstorming partner.

You can load a book and call it. You can load Naval Ravikant transcripts and ask about your actual situation. Same idea with Warren Buffett interviews. I am surprised the Agents platform has not picked up more. You have to pay for it. A lot of people are building n8n Telegram bots instead. Voice is the better interface.

Five freedoms I care about: time, location, money, commitment, authority. Most days I walk or I work on a small project I actually want to touch. Newsletter is Newsletter. A working session is Cal.com.

Transcript

Hey guys, hope you're well. Today going to go over an AI voice assistant for brainstorming and planning that I built with full access to Notion. So this agent can create, update, archive tasks within my Notion system, can create and update notes and do brainstorming sessions, which is something that I've been doing quite a lot to be honest. It has internet access, so it's also able to research topics and kind of verify information or kind of pull in information into its context. And you could also do things like plug in, you know, Gmail or Google Calendar or like literally any MCP server can actually be plugged in there.

The reason that I built this is because I do a lot of walking and for like planning or just like kind of shaping ideas. You know, it could be about a business, it could be about investments, it could be about YouTube. Sometimes I'll just listen to a book and mull over it walking around. But more recently, I've spent a lot of time doing calls whilst I'm on these walks, particularly doing calls with GPT. I've tried all of them, and GPT tends to be my go-to. It's very good, particularly that it has all context on you from all of your previous chats, which I do find to be very useful in personalization.

But it does have a number of issues. Number one, GPT in call mode is GPT-4o, which is now like now we're on like GPT-5.1, which of course is much better, but the main issue being that it loses all context if you're doing like longer calls. like I'll speak to GPT sometimes for hours but in most cases for about 40 minutes to an hour on these walks um because the transcripts are so long it doesn't save the transcription it just says like you know audio transcription but it doesn't actually have any details which means that information is completely lost so in order to and and one other thing as well is that it you can actually connect different tools within GPT for Notion.

It doesn't work well at all. It kept timing out for me. So, I decided to build this myself. Now, I'll walk you through what we have here. By the way, it's all built with ElevenLabs. ElevenLabs have built this ElevenLabs agent platform that at least I haven't seen anyone talking about, but I personally think it's really great. So this is all built with ElevenLabs components and it's ElevenLabs running on the back end doing all of the audio transcription and also creating the audio and but the actual model itself running is 5.1 but GPT-5.1 but you can pick any model that you want.

So this can also be turned into like a mobile app or I can deploy as a web app very easily. Okay. Okay. So if we come in here connecting high GPT, can we do a brainstorming session on how to achieve complete life freedom? Love that topic. Yes, let us do it. When you say complete life freedom, do you mean mostly time and location freedom, financial freedom, or more inner freedom and psychology stuff? Also, should we treat this as a brand new note in Notion for a fresh deep dive or continue an existing session you already have?

Please can we create a new note in Notion and it will be on time freedom, location freedom, financial freedom, freedom from authority. Nice. I will spin up a fresh outline for this freedom note now. Nice mix of freedoms that gives us a clear shape to ground this. Describe your perfectly normal free day. Not a So like you get the idea, but basically I would run through this. We I can speak for like you know 30 minutes to an hour. And the other very important thing is it doesn't wait for the end of the conversation to add everything into the Notion note as we are having the conversation.

It's constantly making tools to that same Notion note to kind of build out a document or summary of the call. Now this is I think such an interesting area because these things are so easy to build. I'll go through some of them in a minute. Like in terms of other examples that I've built, I also developed a language teacher because I'm currently trying to learn Russian, a speech improver. So you may have noticed in my videos, I tend to say are a lot. And you could even just add in a single prompt, for example, in this one, and just say, also correct my speech if ever I say um or are keep a count of how many times I do it and give that to me as feedback.

You can also ask it to like improve your phrasing or give you suggestions on your communication. And again, it all works in real time. On the language thing as well, by the way, this currently works in I think 70 different languages is what ElevenLabs are able to do. So, you could again, you could release a SAS as like a um you know, a language coach or whatever it is. And I'll go on to a few more ideas like that in a minute, but just to get you started on like how you can actually do this. There's two ways to do it.

The more simple one is probably this. So you can go to eleven.ai and it's going to walk you through like a really I'll walk through the process. Create new agent. Pick a voice for your agent. Yeah. And it's going to bring you into this and then you can just do add your first integration. You can add Google calendar Linear which I have another agent that I use for managing software some of our software projects. uh and it will write up Linear and GitHub issues that I then just pass directly over to the AI agent. So again, I can be out walking, talk about some of the features that I want.

We then turn those into tasks into GitHub issues with requirements, acceptance criteria, what needs to be done, what tests need to be passed and so on. This will all get written up, put into Linear, which automatically syncs over to GitHub and then it can be given to AI agents to start working on whilst I'm walking on a beach somewhere. So these are just really interesting workflows. Notion. Although the Notion integration here I found didn't work very well. So I'm currently not using this but this is their like kind of non-techy way of setting it up and they do have a mobile app.

So this is eleven.ai but if you want the full experience you would come into ElevenLabs and instead of the creative platform you come into the agents platform and this is the example of the one that I showed you. So you come into agent here you have the prompt you can choose the model that you want to use. So GPT-5.1 like I have mine I think on medium yeah medium reasoning but if you wanted much faster replies like you may have noticed it took a couple of seconds to actually reply to me you can do these faster models like Gemini 2.5 flash or even GPT Nano or Mini they're going to be much faster but I prefer to work with a smarter model and I'm happy to have the tradeoff of having a bit of a delay in the responses in order to get much better answers you could also do high reasoning as well but I don't find that there's too much of a difference in terms of the quality.

But again, that's just my own experience. Here, I turn the temperature down a lot. If you have it as more expressive, I find that the answers it gives you are much longer, whereas I don't like that because I find that it interrupts my kind of trail of thought. So, I want it to be very precise and very succinct. You'll also find that that's kind of written in my prompting here as well. Keep the answers very short in order to keep the conversation flowing. So, yeah, you can put in the system prompt. And again, I could literally just replace this with like you are an AI, nutrition, health coaching expert, like a fat loss expert.

You basically could launch a new SAS just by doing that. Then you have workflows. I haven't actually played around with these. They seem kind of simple, but basically you give a conversation goal and you can create different sub-agents. So you could start off the agent with a much faster model in order to get replies much quicker and then if I say I want to do a brainstorming session, it would hand over to a smarter model. for example, like GPT-5.1 on high reasoning. So you could have multiple different sub-agents that it kind of hands off to and each of them would have a specific goal and their own knowledge base.

So this is quite interesting. It's just not something that I've played around with. This is like a brainstorming agent as I mentioned that I use for like running through ideas and like now because I'm trying to get back into the habit of creating YouTube videos. The reason that I made this was for brainstorming YouTube videos, for kind of shaping out different ideas for videos that I have and crafting together potential scripts. So, I gave it access to my YouTube channel just as a link so that it can scrape that and it understands what I normally talk about.

And then I had GPT write up a document here which is like the style of my videos that I I like to make videos based off of my own real world experiences talking about automation, about leverage, about being clever, about thinking in systems and kind of different approaches to business. So it understands all of that context which also makes a massive difference as well. I did have a few other ideas. I'm not releasing any of these. They're just kind of fun personal side projects for me. But I think that there's so much value in what you could build on this platform.

I found a guy that was doing about 500k per month on a anti-masturbation app for like blocking pornography basically which would be a complete career turn for me at least considering what I was working on for the past few years. One idea that I had there was you could like scrape the whole of Reddit. There's some particular subreddits called like nofap which is like anti-masturbation subreddits and you could take the most upvoted content in there, the most upvoted comments, most upvoted threads, pull them all together in a knowledge base and train your large language model here on what was working for other people.

be it motivational kind of content in order for people who are trying to quit pornography for example would be able to call this assistant and it would be fully trained on what had worked for other people but you could you could apply that to anything like health and fitness nutrition any form of coaching I think this is really interesting for now the thing that you'll probably be thinking to that as well is like that they actually need to open the app and start the call in order to get this value but Actually, you can plug in here.

If you come down to phone number, you can connect via Twilio. Twilio is an SMS and calling API. You can actually give this a phone number. So, it will have a US phone number. And if someone hasn't checked in, for example, like if you're a nutrition coaching app and someone hasn't checked in and sent their macros in for a day, you could set this to automatically go and call them and the AI agent would speak to them. So it can actually proactively nudge them which I think is so interesting and again just really changes changes how we interface with these things.

Whereas normally I need to be the proactive person to kind of open the app and engage with it. Now that has completely changed. And if you think about a lot of coaching particularly in like the health gym and training and things like that a lot of it is down to the accountability side of things is going to be a lot of the value that people receive out of these services. But now with this, by giving it a phone number and being able to text people, you don't even need to tell it's an AI. I mean, that kind of depends on your point of view, but I would actually think that given the quality of the voice in these models, a lot of people would have no idea.

And they would think that they're being chased up by a person telling them to it could be like a life coaching thing related to specific goals, to specific tasks. I could even train this on all of my YouTube videos that I've ever made. The one reason I probably wouldn't do that is because my opinions have changed quite a lot compared to a lot of the things that I was talking about two years ago, but still very very interesting nonetheless. Then under analysis, you can also go and analyze all of the calls as well to kind of use this for training.

And then in tools, this is where you actually give it like the features. So these are where I give access to Notion and the ability to add tasks. It can also list recent conversations which helps to give context about things that have been spoken about already. Again, this instead of needing to like repeat myself about things, I want it to be able to pull information just kind of summaries from our like previous conversations and it it makes the model seem like much more intelligent. Otherwise, if it's starting with like a blank memory every single time, you find that that you repeat yourself quite a lot.

And this is just basically it appending its own things to Notion. I also have Firecrawl integrated. If you don't know Firecrawl, it's a searching tool specifically designed for LLMs. So I could say like top 10 podcasts on Life Freedom and it will go and pull in that information and the AI will have access to it as well. You can also connect MCP servers. I think most of you should be familiar with MCP servers by now. Essentially, it's a protocol for giving to LLMs to be able to interact with different tools similar to like well it is actually API based but just for LLMs to be able to understand and interact with things.

So I could connect for example Airtable over MCP and say can you go and look over and find our best our top 10 performing Instagram accounts from our Reels reposting farm as an example and you can see some that like GitHub was one that I connected as well. I think that they also had some yeah here maybe some pre-built integrations that you can connect as well. So yeah custom MCP server Cursor so you could actually launch Cursor agents again just on a call you can be out and about walking HubSpot as a CRM Zendesk Parallel Salesforce they they do seem to be like quite actively developing this agent side of things as well.

And then in terms of like getting this on your website, you can literally just copy the script and put it on there. You can like change some some of the settings here. So it's really easy to integrate. All in all, by the way, I should say I actually haven't built this at all through the UI. This was just for me to like walk through for anyone who's kind of non-technical. My app is completely vibecoded because they have an API for the agents platform. So you don't even need to go through and do any of this. You can just go through and vibe code the entire thing.

And like in total it took about three hours in order to build this. Again, I literally just built it yesterday just as like a fun little side project. It's built with Claude Opus 4.5, which is currently my go-to model for programming. But we're in such an interesting time with all of these models coming out and these really powerful, very well-funded companies all in competition with each other trying to acquire market share that Gemini 3 came out, really great model. I do also use it a lot as well. then 5.1 and 5.1 Codex came out the next day and then about a week later we've got Opus 4.5.

So, a lot of new tools for me to be able to play with. In terms of what I'm working on right now, um my life setup is I would say that I have actually now achieved like complete life freedom and I'll make a video on that as well. Not just in terms of like financial freedom, location freedom, time freedom, but also like commitment freedom. I've really sorted out that area of my life as well and freedom from authority. So, they're like the kind of five different pillars of freedom that I've kind of come up with myself of just kind of like philosophy, a way of living my life that I found to work really well for me.

But currently, most of my time is spent just following my passions. If I wake up in the morning, I either go for a walk or I I work on something that I feel inspired to work on. Generally, like a smaller little side project, something like this. there there's so many interesting tools like you know people would say the phrase like keeping up with AI is like a full-time job with all these developments and changes and I've I've just kind of made it my full-time job to do that because I'm I find it very interesting. The other like big changes that I think we're going to see is how the changes in how we interface with AI and I really feel that this is one of them.

So we're seeing Meta and people looking at wearables and kind of AI glasses and I do think that they are interesting. There's also startups like Cluely which give access to your like browser screen so that the AI has context of what you're doing and obviously with AI glasses it's got a camera so it has like a real-time feed of what you're doing at all times. But even now just the simple thing of like putting in your AirPods and building out these like AI agents and just having access to these tools and being able to interface with them hands-free in a different way as a project manager, as a life coach, as a Notion assistant, as a brainstorming agent.

There's hundreds of different things that that you could do. You could plug in books into it and literally have calls with the books rather than needing to read them. There's just infinite things that you could do. You could train on your favorite influencer or whatever you want to call it. Like Naval Ravikant, for example, you could just upload transcripts of all of his YouTube videos and podcasts and do direct calls with him to get advice on your personal life situation. You could do the same thing for investing data.

download all the information Warren Buffett has ever put out and have calls with an AI version trained on Warren Buffett's interviews and thinking processes. So many different ways of doing this. And to be honest, I'm surprised that this ElevenLabs, this agents platform hasn't picked up more traction than it has. The only reason I think it may not have is because you have to pay for it. On n8n, for example, I know that a lot of guys are building out these kind of Telegram bots. I just think this is such a far superior way of interfacing with AI.

Yeah, I think that is that's everything from me. I will get back into making more videos as well. I hope that was useful. Any questions? If you've built anything similar or have ideas or things like that, as always, feel free to leave a comment.