Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. learn more
OpenAI has expanded the number of applications its desktop apps can work with, including allowing Advanced Voice Mode to work with other apps, and is moving closer to ChatGPT using computers .
The desktop application introduced integration in November with four original submissions. During Day 11 of its “12 Days of OpenAI” event, OpenAI announced several integrated development environments (IDEs), terminals and text apps that it will support.
ChatGPT now supports BBEdit, MatLab, Nova, Script Editor, TextMate as IDEs, VS Code for VSCode Insiders, VSCodium, Cursor, WindSurf, the JetBrains family of IDEs of Android Studio, AppCode, CLion, DataGrip, GoLand, IntelliJ IDEA, PHPStorm, PyCharm, RubyMine, RustRover and WebStorm. It also added the Warp and Prompt terminal apps as an integration. These applications join VS Code, Xcode, Terminal, iTerm 2 and TextEdit as integrated apps.
But coding applications aren't the only applications that ChatGPT desktop apps get. OpenAI also added Apple Notes, Notion and Quip to its integrations. Advanced Voice Mode can work with these applications, considering the context of projects in the integration.
OpenAI confirmed that users must give ChatGPT permission to access these apps.
Allows ChatGPT to use your computer for you
App integration with AI chatbots is, of course, nothing new. In October, GitHub Copilot has been added coding platform integration. Connecting applications to ChatGPT or Copilot brings context from those platforms into the chat experience. Developers can encourage ChatGPT for some coding help for a project they have on VS Code, and the chatbot understands what they've been working on.
Kevin Weil, chief product officer at OpenAI, said during a live stream that the development of the desktop app will help the company get closer to a more agent-like user experience for ChatGPT.
“We've been putting a lot of effort into our desktop apps,” Weil said. “As our models become more powerful, ChatGPT becomes more functional. ' means we go beyond just questions and answers; ChatGPT starts doing things for you.”
He said the desktop apps are “a big part” of that transformation.
“Being a desktop app allows you to do a lot more than just a browser tab,” he said. “That includes things like, with your permission, being able to see what's on on your screen and be able to automate a lot of the work you do on your desktop. We'll have a lot more to say on that as we head into 2025.”
If OpenAI allows ChatGPT to see more of your computer, ChatGPT gets closer to Claude Anthropic Computer usage feature, which allows Claude to click around one's computer, navigate screens and even type text.
OpenAI already declared a medium similar feature for mobile phone version of ChatGPT, although the chatbot cannot yet access computers or phones in the same way. Users can share their screens with the chatbot so they can see what they are reading or watching. Microsoft and Google also developed comparable properties with Copilot vision and Astra project of charge.
How to get in
On MacOS, users who want to open ChatGPT while using other applications can use “option + space” to pull up ChatGPT and select the application they need via a button on the chat screen . Another shortcut, “option + shift + 1,” will bring up the top most used application.
From that window, users can also access Advanced Voice Mode in the same way. Voice mode automatically detects context from the application.
Integrations are available to ChatGPT Plus, Pro, Team, Enterprise and Edu users. However, Enterprise and Edu subscribers must ask their IT administrators to turn the feature on.
Source link