Microsoft has introduced a groundbreaking new feature in Copilot Studio that allows AI agents to interact with websites and desktop applications just like humans do. This innovative capability, available through an early access research preview, enables agents to click buttons, select menus, and type into fields on the screen. With this new skill, agents can handle tasks even when there is no API available to connect to the system directly.
If a person can use the app, the agent can too.
Swamped at work or down to your last brain cell? Let Microsoft 365 Copilot Chat do the heavy lifting. Learn more about your go-to AI chat: https://t.co/1h6PGANjzN pic.twitter.com/7jDqxqmVJb
— Microsoft 365 (@Microsoft365) April 16, 2025
The agents adapt to changes in apps and websites automatically, adjusting in real time using built-in reasoning to fix issues on their own. Computer use in Copilot Studio runs on Microsoft-hosted infrastructure, so organizations don’t need to manage their own servers.
Enterprise data stays within Microsoft Cloud boundaries and is not used to train the Frontier model. This helps accelerate deployment, reduce maintenance, and lower infrastructure costs. Some high-value use cases for this technology include:
Morning brain has met its match, thanks to Microsoft 365 Copilot in Teams. What are your favorite prompts to use when your get-up-and-go got up and left? pic.twitter.com/b0H2XK6Ote
— Microsoft Teams (@MicrosoftTeams) April 16, 2025
Copilot is becoming a layer across how we search, read, and create. From search to browser to desktop, it's now part of how ideas take shape. With @bing and @MicrosoftEdge, it’s now supporting creativity at every step 👇
— Yusuf Mehdi (@yusuf_i_mehdi) April 16, 2025
– Automated data entry: Enterprises can automate the process of inputting large volumes of data from various sources into a centralized system.
– Market research: Marketing teams can automate the collection of market data from various online sources for analysis.
Ai-driven computer use in Copilot
– Invoice processing: Finance departments can automate the extraction of data from invoices and input it into accounting systems.
Computer use agents are transforming robotic process automation. They overcome traditional limitations like the fragility of UI elements and can handle complex dynamic interfaces. This makes automation accessible to people beyond professional RPA developers.
In Copilot Studio, computer use addresses common RPA challenges by making automation smarter and more intuitive. The tool continues working seamlessly even as buttons or screens change. Users can describe their tasks in natural language without coding, and test and refine prompts with real-time side-by-side video of the reasoning chain and planned UI automation.
The agent sees what is on the screen and makes smart decisions in real time, even in complex or constantly changing environments. Creators can view a history of computer use activity, including captured screenshots and reasoning steps. Copilot Studio is designed to help organizations achieve their AI and operational goals by streamlining processes, enhancing productivity, and driving innovation.
More about this new announcement will be shared at Microsoft Build in May 2025.
Image Credits: Photo by Christian Wiediger on Unsplash
Cameron is a highly regarded contributor in the rapidly evolving fields of artificial intelligence (AI) and machine learning. His articles delve into the theoretical underpinnings of AI, the practical applications of machine learning across industries, ethical considerations of autonomous systems, and the societal impacts of these disruptive technologies.
