Transcription
OpenAI just dropped their new voice mode, and I personally think this is the most underrated AI release of the year, and it seems like nobody is talking about it. You can even speak with it on your phone, and it can literally control your computer and get work done for you. And on top of that, it can run multiple tasks in parallel while you're speaking with it. So, in this video, I'm going to show you what's new with it, where you can use it, and then five different use cases you can run right now so you can understand just how powerful this is. So, without further ado, let's dive in.
All right. So, before we dive into a demo as well as show these different use cases, we need to understand what is new inside of ChatGPT voice mode because we have had previous voice modes, but this is much more capable. And once we understand the foundation of how this works, you'll be able to see some of the use cases you could apply yourself.
Now, the first big thing that's different with the new voice mode is that previously, it was like speaking on a walkie-talkie. And what I mean by that is we would speak to ChatGPT voice mode, and it would have to wait until we finish talking in order to process what I just said. Well, now with the new voice mode, it can talk and listen at the exact same time. So, if we need to interrupt ChatGPT, it will immediately stop talking and then begin processing exactly what we're saying. Basically, mid-conversation, mid-sentence, we could interrupt, and it will adjust automatically. It's basically like we're talking to a normal human instead of this back and forth we're so used to when it comes to voice AI models, where you have to cut it off when it goes on a run-on sentence, and it just throws off the whole process of the conversation.
Next up, this voice mode now lives on the desktop app. So, we could use this in things like ChatGPT chat mode, the new work mode that they just released last week, and then Code Interpreter. And there's something I'm going to talk about later in this video that is extremely powerful if you're using Code Interpreter, and you could basically speak to ChatGPT voice mode from your phone in order to trigger Code Interpreter on your desktop. But we're going to talk about that and dive deeper into that later in the video.
Now, there's a couple more features here where when you truly understand how this works, it can begin to feel like magic that we can speak with an AI agent like ChatGPT voice mode and have it do countless amounts of things for us in real time. Now, one thing we could do is we could give it one command while we're speaking with it, and it can perform many tasks at the same time. So, what I mean by this is we could start multiple different threads inside of ChatGPT, and we could almost use voice mode as like this coordinating agent to understand what's going on in parallel across those different conversations and tasks. And I'm going to show you later on in this video exactly how that works because it's genuinely mind-blowing, and it kind of gives you a taste of the future of how we're going to interact with AI in our own operating systems.
Next, we could have voice mode use our actual tools. So, any of the different plugins or connectors that we have configured inside of ChatGPT, we can now perform tasks across. So, whether that's Gmail, your calendar, Slack, or any of the different tools that you use on a day-to-day basis, we can speak with voice mode, and it can go and perform actions across those as well as pull information from them.
Next up, voice mode can actually see our screen on our computer. So, let's say you need help navigating a specific website or application you're trying to use, you could have ChatGPT voice mode walk you through exactly where to click on the screen or explain complex topics for you in real time.
This next feature allows you to work from your phone in order to control your computer. This allows you to use voice mode on your phone in order to pull different files from your computer and basically run any of the different tasks you would do inside of ChatGPT or ChatGPT work or even Code Interpreter.
Now, there's a few places we can access this with a couple of caveats. So, make sure to stick with me. First of all, if we come to chat.openai.com, we have the voice mode right here that we could activate. But this does come with some limitations that I'm going to talk about here in a second.
Now, the next place to use it, and I highly suggest this over chat.openai.com, is the ChatGPT desktop that is inside of the desktop app. On top of that, we could also access this on our mobile app on our phone, which comes with some pretty cool capabilities, which I'm going to show you later on in this video.
Now, let's quickly talk about the difference of using chat.openai.com versus the desktop. So, if we're using this on our phone or the web application, it does not have the ability to run different tasks for us and use our actual screen, pull information from our computer, etc. Think of this like using normal ChatGPT mode. You could talk back and forth with it, and the model sounds really good, but it doesn't have the full capability. Whereas in the desktop app, we can run tasks and specific agents and basically do everything that I'm going to show you in this video.
Now, let's move on to testing this out in real time. So, first of all, I'm inside of the ChatGPT desktop app. And up top, you can see we have two modes. We have chat mode and we have work mode. Voice mode works in either of these different platforms. But if we want the full capabilities, I'm going to toggle on work mode. So, this way, it's able to work inside of different folders on my computer.
All right. So, for this first use case, I'm going to speak with ChatGPT voice mode in order to pull context from a previous task that I ran actually last week. And this is a conversation right here. So, I was using ChatGPT work in order to create a travel plan for me. It's using this connector called Ease My Trip in order to pull flight information and find the cheapest and the best flights for me coming back from Europe. On top of that, it even generated this ChatGPT site for me, breaking down everything that it found when researching the best travel plan for me. So, what I'm going to do is I'm just going to toggle this on right here.
"Hey, what's going on? Could you do me a favor? I want you to look at that Europe trip flight that you were helping me book. Um, can you pull some of the information on what you think will be the best?"
"I'll check the trip details and see what flight options fit best for your Europe trip. I don't have the details in this chat. What are your departure city, destination, and travel dates?"
"Yeah, so you actually had this in the previous conversation. Can you pull from that? The name of the conversation was called flight planning."
"Okay, I'll check that. I'm locating that flight planning task now. I found the trip details. Barcelona to Redmond on August 14th. You wanted him to lie flat without paying for business all the way. The best value route was through Boston, but I'm checking updated availability since fares can shift."
"Okay, cool. Once you find the best flight for me, can you actually pull up um inside of Ease My Trip so I could book this?"
"Got it. I'll see if I can bring that up there. I'm seeing a snag. The cheapest live option now arrives too late for a same-day connection, which would mean overnighting. I'm pricing that against a faster through ticket that keeps the trip simpler. The lowest risk value option is a one-stop through ticket via London and Seattle, landing in Redmond the night of the 14th."
"Yeah, that's fine. Can you just go ahead and pull it up so I could book that?"
"Sure, I'll pull up the lower cost flight option in Ease My Trip. Barcelona to Boston on August 14th. It requires an overnight in Boston, then a separate Redmond flight the next day."
"Okay, perfect. I think I'm going to book this. Thanks."
All right. So, there you go. You saw ChatGPT voice mode in action. It went ahead, it pulled from a previous thread conversation that I had, and this was actually like last week, so it was able to use that as context. And then it used this application here that I have connected and even pulled this up in my browser. So, I can go and book this flight automatically right now.
All right. So, for use case number two, I'm going to use ChatGPT voice mode in order to find a specific file that's living on my computer. So, I generated this a couple of days ago, and let's say we don't want to have to go and like try to search for this all across our computer. Instead, let's offload this task to ChatGPT in order to do it. So, as you can see right here, it's living on my desktop, this co-work setup teaching HTML pack that I have open right here. So, I'm just going to come over to ChatGPT and have it find this for me.
"Hey, can you go on my computer and I want you to find the ChatGPT co-work setup HTML that you generated for me a couple of days ago."
"Got it. I'll look for that file and confirm what I find. Also, while you're while you're doing that, what was the last task that we were running inside of ChatGPT?"
"I'll check that, too."
Now, as you can see, it's running and doing multiple tasks at the exact same time, which is the power of using the new ChatGPT voice mode. And it's pretty cool when you have multiple different tasks you have it running at once.
"I found and opened the simple co-work setup teaching pack on your desktop. Last created July 17th."
"Yeah, this is exactly..."
"This is exactly what I was wanting. Thank you."
"Great. Glad it matches what you needed. Now, what was that task you said that you found? Like the last task you ran for me?"
"Let me pull up the exact last task for you. It was the voice mode update chat where you asked if the new voice mode had launched recently and then to see your most recent Gmail message."
"Perfect. Thank you so much."
Now, one thing I want to mention that's pretty cool is you could see this little ChatGPT voice blob, and we could basically put this anywhere on our computer. So, let's say I'm editing this video right here, for example. You could still see ChatGPT voice mode, and I could just basically come here and say, "Hey, I want you to look at my screen. What does it look like I'm working on right now? I need help with this."
"Okay, I'll take a look. I'm reading what's on screen now. You're actively editing a roughly 7-minute project in CapCut with the clip's basic settings open."
"Yeah, could you actually break down how I can apply a filter? I can't figure out how to do it."
"I'll check that real quick. In CapCut, click filters in the left sidebar. Then choose one and drag it onto your clip in the timeline to apply it across the whole video. Drop it onto the main video track, then adjust its intensity in the right panel."
"That fixed it. Thanks."
So, for this next use case, I'm going to use it to generate images and videos with voice mode using the Highfield MCP. Not only am I going to generate one video or image, I'm going to generate multiple at the exact same time since we're able to run these voice mode tasks in parallel. And if you don't know what Highfield is, it's an all-in-one AI creative platform that has access to all the top image and video models. So, we could generate basically anything we want in terms of videos. And the best part about this is we don't even need to be inside of this user interface here. We could simply just talk to voice mode, and it will go and generate all these different videos using whatever model it thinks is best.
Now, it's really easy to set this up. So, I'm going to show you how to do that right now. First of all, we're going to go to the link in the description to use the Highfield MCP. From here, we're going to see this landing page right here. And what we want to do is we want to select ChatGPT. And simply, we're going to click this "Add Highfield Plugin" right here. Next, I'm going to click on "Install Plugin" and sign into my Highfield account. If you don't already have one, make sure to sign up for an account in order to use the MCP. I'm just going to click on "Allow" here. And then, just like that, we now have Highfield installed. It's really simple to do. Then make sure to do the exact same thing inside of the desktop app. So, basically, just come to plugins, search up Highfield, click on this right here, and then make sure to enable it.
Now, let's come back into ChatGPT and test this out right now.
"Hey, so I want you to use the Highfield MCP in order to generate two different videos for me in parallel in separate chats. I want this to be a product video of a Rolex. And I want this to almost be like an exploding animation where the different parts of the Rolex kind of come apart that we could use for a scrolling website. Then the second video that I want you to generate is going to be more of like a luxury product video advertisement that shows a Rolex up close on somebody's wrist."
"Sure. I'll set those up. I'm checking the video setup and prepping both Rolex concepts. One exploded view and one wrist-worn ad. I've split them into separate generation tasks. So, the exploded view website animation and the luxury wrist close-up ad are both running in parallel now."
All right, so a few minutes later, and we now have both of these videos generated. So, let's take a look. I mean, this looks pretty cool. It's zooming in on this Rolex Submariner, and then right there, you could see the different parts of this watch kind of coming apart. I think this would look really good and smooth on an animated website. I mean, that is really good. And this only took one prompt, and not even like a prompt. I basically just told ChatGPT voice mode in order to go and do this. And then here is the second video that it generated. This is more of like an up-close um product shot of the Rolex Submariner on somebody's wrist. I mean, again, that looks super solid as well.
So, now I'm just going to come back here and say, "Hey, I just took a look at these, and they look really solid. What I want you to do now is turn these into an animated website using the animated website skill that you have."
"I'll start shaping that into an animated site. I'm mapping the animations into a simple product page flow."
And so, now what this is doing is it's locating the animated website skill that I've downloaded inside of ChatGPT, which is basically a set of instructions to take videos and animations like that and turn it into an actual scrolling website. So, as we scroll, these animations are going to play. So, it just really makes a stunning website for somebody to look at. So, it's been running for a couple of minutes. So, let me come back to voice mode and just ask, "Hey, how's it going? Did you finish the website?"
"Let me trigger the build real quick. Yes, the website is ready, including the exploded view sequence and the wrist film closing section."
"Perfect. Could you open it up for me?"
"Sure, I'll pull it up. I'm opening it now. The animated Submariner site is open."
All right. So, it pulled it up inside this browser inside of ChatGPT. What I'm going to do is I'm just going to copy this URL and view this in my browser so you can see a full view. And man, this looks super good. I mean, it's the exact same animation that we just created with the, you know, the Rolex kind of exploding here. I like the typography. I like the font of this website. And then if I scroll to the very bottom, you can now see that it even shows this wrist shot, which I could actually view right here. All this was generated with ChatGPT voice mode and the Highfield MCP. We literally did not even have to type a single word or a prompt in order to generate this. This is all done with voice mode.
Now, for this next use case, hold on to your seat because this is where it gets extremely powerful. It's blown me away after playing around with this, showing what's capable with voice mode. So, in order to show you what I mean, take a look at this folder right here on my desktop called Opus 5 animated websites. Inside of this folder, we have a couple of different thumbnails that I generated just yesterday. And what I'm going to do is I'm going to use voice mode from my phone in order for ChatGPT to go onto my computer and find that exact folder and then analyze those images for me. Now, the reason why this even matters is because anything that we can do on our computer inside of ChatGPT work or Code Interpreter, we can now do directly from our phone using voice mode. And I want you to let that sink in because it can be extremely powerful if we use this correctly.
So, what I'm going to do is I'm simply going to click on this voice mode and then I'm going to come here and say, "Hey, I want you to go on my computer and look at the Opus 5 animated websites folder. I then want you to tell me what images I have inside of there."
"Sure, I'll take a look. I see six images: Insane websites, Opus 5 websites, two copies of 15K websites, and a lighter clawed colon animated-website graphic."
"Amazing. Can you do me a favor and I want you to rename this folder to say Opus 5 websites?"
"Okay, I'll take care of that. I found the folder and am updating its name now. Done. The folder is now called Opus 5 websites."
Now, I wasn't able to screen record both my phone and my computer at the exact same time, but literally right when my voice agent told me that it changed the name, I actually saw this name of the folder change. And I know this is a really basic use case, but basically anything you could have Code Interpreter do on your computer, you can now trigger ChatGPT to do it with the voice mode from your phone.
All right, so I'm going to show you exactly how to set this up now before we move on to the next use case. So, what we're going to do is we're going to come inside of our ChatGPT desktop app. Then at the very bottom left where you see your account, you're going to click on this and then click on settings right here. From here, underneath coding, you want to click on connections. And then we're now going to see this right here where we can control our Mac from our phone. So, now we're going to click this add button. Now we're going to see this QR code right here. I'm going to pull up my phone and then scan this. And then now inside of the ChatGPT app on our phone, what we're going to do is we're going to come over to the left-hand side, swipe to the left, and then you're gonna see this new remote button right here with this little green icon. I'm going to click on this, and then now you could see the different projects that we have that we've ran in the past. So, for example, I could then go to the Rolex animated website that I created, and we can view any of these sessions that we ran on our desktop app now on our phone. And in order to use voice mode and have it go perform a task for us, we just click on this little voice icon right here. It's going to open this up. It's connecting locally to your desktop. And then we could chat with it.
"Hey, what's up?"
"Hey, what's up?"
"What's on your mind?"
"Yeah. Can you do me a favor and go inside of my IG carousels folder on my desktop and then let me know what's inside of there?"
"Sure. I'll check that folder. I'm checking that folder now. Inside your IG carousels folder, there are four sets. Five cla..."
So, there you could see exactly how easy it is to trigger this from your phone. And this is going to be really powerful if you're on the go and want to have Code Interpreter or ChatGPT go and do something if you're not at your computer.
Now, I'm going to quickly talk about how to add a plugin and then I'm going to show you a hack to connect ChatGPT to 9,000 plus different applications so you could work across your entire tech stack. First of all, if I come over to plugins on the left-hand side, there are a few different public plugins that we can connect to directly inside of ChatGPT, but this is kind of limited right here and it won't have all the different apps that you want to use. A really good workaround for this is using the Zapier MCP, which connects to 9,000 plus different applications and it's really easy to use. So, to show you this in action, this is this MCP server that I have configured. I then have it connected to Beehive. I also have it connected to School, and these are a couple of different applications that I use every single day to run my business, but there's no way to connect them inside of ChatGPT without using this MCP server. And to show you exactly what I mean, let me use voice mode in order to pull email newsletter information from Beehive. And it's going to do so using the Zapier MCP.
"Hey, I want you to check inside of Beehive via the Zapier MCP and give me some relevant information about my email newsletter list."
"Okay, I'll check the Beehive details now. I'm checking the connected Beehive data source now. Your active newsletter, Brock Masara, AI for Non-Techies, shows 1,694 free subscribers and no paid subscribers with about a 29% average open rate. The latest delivery numbers are 449 delivered, 131 opens, and nine clicks. There's also a similarly named publication that's empty right now."
"Perfect. Thank you so much."
And so, as you can see, it was able to access Beehive even though it's not connected directly inside of ChatGPT. And it's able to do this with a Zapier MCP. If you guys want to try it out, there will be a link in the description. And it's as simple as coming here, clicking on creating a new MCP server, and then we're just going to select whatever agent we're using. So, for this use case, we're going to be using ChatGPT. So, we'd select this, and then we just go through these steps right here, and it's going to walk you through exactly how to connect this. And then once you've configured that, all you need to do is add our different applications that we want to connect. And there are, again, 9,000 plus that we could connect to.
Now, this next use case I'm going to show you is really helpful for automating specific tasks that you do every day or every week, for example, and that is using it to generate scheduled tasks inside of ChatGPT. So, I'm just going to come here and say, "Hey, I want you to create a scheduled task where every single day you pull from my Gmail and at 8 a.m. you remind me everything that I need to reply to for the day, as well as I want you to pull from my Google Calendar so I can know what to look forward to throughout the day."
"Okay. I'll check your Gmail and calendar and see how to set that up. I'm checking the connected mail and calendar capabilities so that the daily briefing can cover both replies due and your schedule."
So, what it's doing now is it's pulling from both of these different apps because I have connected them, and it's going to pull all the relevant information for me. I could then even use voice mode to draft email replies to all these emails that I need to reply to so I don't personally need to do it myself.
"Your daily briefing is set for 8 a.m. It'll highlight emails that need replies and summarize your day's calendar in one snapshot without changing any messages or events."
"Perfect. Can you run this scheduled task right now so I could see how it works in action?"
"About it. Running it now. Today's briefing is ready, and I'm checking if we can trigger it immediately for a live preview. Your briefing shows a few items worth replying to, including a couple of creator partnership invites, a Slack invite, an account reconnection notice, and a package pickup reminder. Your calendar is clear today, so you've got some room to tackle those."
"Perfect. Thank you. And this will run tomorrow."
"Let me confirm it's set for tomorrow, too. Yep. You'll get the briefing tomorrow morning at 8:00 a.m. and every day after that."
"Perfect. Thank you."
All right. There we go. That is the new ChatGPT voice mode. You should now have a sense of how powerful this is and different ways that you can use this right now. I'm personally absolutely stoked about this. It really feels like we now have this like Jarvis moment in AI where we can now use our phone and speak into it and can literally have ChatGPT going and performing tasks on our computer. If you enjoyed this video, subscribe to the channel for more videos like this. Let me know what you think in the comments, and I'll see you in the next one.