Sam Altman’s OpenAI has made a big improvement to ChatGPT’s Advanced Voice Mode by adding vision capabilities.
This means that ChatGPT can now understand and respond to live video input and even share screens during conversations. Users can now chat with ChatGPT using their smartphone camera allowing the chatbot to “see” what they see.
Also, read| Microsoft Quits OpenAI’s Observer Role Amid Growing Antitrust Issues
The team also showed how ChatGPT can support screen sharing, and understanding an open message on a smartphone.
Just in time for the holidays, video and screen sharing are now starting to roll out in Advanced Voice in the ChatGPT mobile app. pic.twitter.com/HFHX2E33S8
— OpenAI (@OpenAI) December 12, 2024
With this update, ChatGPT can understand and discuss visual context in real and users can now access this feature through a new video icon in the mobile app and screen sharing is available through a separate menu option.
This new feature is available to ChatGPT Plus, Pro and Team subscribers, and will be available to Enterprise and Edu users in January.
Also, read| OpenAI Whistleblower Suchir Balaji, Who Accused Company Of Copyright Infringement, Dies By Suicide
OpenAI has also introduced a festive “Santa Mode” for voice interactions. This limited-time feature allows users to chat with Santa Claus adding a fun and seasonal touch to the chatbot. The “Santa Mode” will be available until early January.
Screenshare while using Advanced Voice for instant feedback on whatever you’re looking at. pic.twitter.com/d4Xm36dwOX
— OpenAI (@OpenAI) December 12, 2024
The update was demonstrated by OpenAI’s CPO, Kevin Weil, who showed how ChatGPT can help with tasks like making pour-over coffee.
Now you can chat with ChatGPT over video and voice in real time. pic.twitter.com/6LySLJcFy5
— OpenAI (@OpenAI) December 12, 2024
During the demo, the team pointed the camera at the action and ChatGPT understood the principle of the coffee maker and explained brewing.