> "It would need access to our browser, an ability to drive that. It would need our credit card information to pay for the tickets. It would need access to our calendar, everything we're doing, everyone we're meeting. It would need access to Signal to open and send that message to our friends," she said. "It would need to be able to drive that across our entire system with something that looks like root permission, accessing every single one of those databases, probably in the clear because there's no model to do that encrypted."
Whittaker added that an AI agent powerful enough to do that would "almost certainly" process data off-device by sending it to a cloud server and back.
"So there's a profound issue with security and privacy that is haunting this sort of hype around agents, and that is ultimately threatening to break the blood-brain barrier between the application layer and the OS layer by conjoining all of these separate services, muddying their data, and doing things like undermining the privacy of your Signal messages," she said.
I've been thinking about building a robot that can use a camera to look around, use motors to go in different directions, and when it sees a human, it could also ask if they've seen John Connor, and if the person is being "difficult" then press a button to terminate them.
The interesting thing is that the three laws of robotics says that robots shouldn't harm humans, but I don't really see a way for an AI agent to understand that by "pressing a button" they actually hurt the human.
You have stumbled upon the point of the three laws of robotics, which is that they are part of a series of stories showing why they don't necessarily work
To wit, the three laws are actually a formulation of three laws of tool design: a tool must not harm its user; a tool must be fit for purpose and do what the user wishes, as long as that doesn't harm the user; and a tool should be sturdy and reusable so long as that doesn't interfere with the tool's safety or usability.
These design principles make sense when you are talking about a non-sentient object, but intelligent, adaptable beings cannot be so easily constrained.
At some point (~50 years from now ?) they could even form their own type of life. If they can mine for resources, think, do actions and reproduce. "synthetic life"
I underground that this is nothing more than a proof of concept but imagine what Apple itself could do with this idea if they truly embraced the concept and cut all the internal red tape that currently prevents them from doing so. This is what “Apple Intelligence” should be but never materialized (and at this point I have doubts it ever will, although I am curious what they’ll show off at WWDC this year).
> I am curious what they’ll show off at WWDC this year
Apparently, not much is planned, per [1]. I'd be very cautious about AI agents like these; from a user level, this has so many security vulnerabilities.
> "It would need access to our browser, an ability to drive that. It would need our credit card information to pay for the tickets. It would need access to our calendar, everything we're doing, everyone we're meeting. It would need access to Signal to open and send that message to our friends," she said. "It would need to be able to drive that across our entire system with something that looks like root permission, accessing every single one of those databases, probably in the clear because there's no model to do that encrypted."
Whittaker added that an AI agent powerful enough to do that would "almost certainly" process data off-device by sending it to a cloud server and back.
"So there's a profound issue with security and privacy that is haunting this sort of hype around agents, and that is ultimately threatening to break the blood-brain barrier between the application layer and the OS layer by conjoining all of these separate services, muddying their data, and doing things like undermining the privacy of your Signal messages," she said.
--Meredith Whittaker earlier this year.
I've been thinking about building a robot that can use a camera to look around, use motors to go in different directions, and when it sees a human, it could also ask if they've seen John Connor, and if the person is being "difficult" then press a button to terminate them.
The interesting thing is that the three laws of robotics says that robots shouldn't harm humans, but I don't really see a way for an AI agent to understand that by "pressing a button" they actually hurt the human.
You have stumbled upon the point of the three laws of robotics, which is that they are part of a series of stories showing why they don't necessarily work
To wit, the three laws are actually a formulation of three laws of tool design: a tool must not harm its user; a tool must be fit for purpose and do what the user wishes, as long as that doesn't harm the user; and a tool should be sturdy and reusable so long as that doesn't interfere with the tool's safety or usability.
These design principles make sense when you are talking about a non-sentient object, but intelligent, adaptable beings cannot be so easily constrained.
At some point (~50 years from now ?) they could even form their own type of life. If they can mine for resources, think, do actions and reproduce. "synthetic life"
> If they can mine for resources, think, do actions and reproduce. "synthetic life"
Essentially the story of the Horizon series of video games: https://en.wikipedia.org/wiki/Horizon_(video_game_series), and I'm sure many other sci-fi novels.
Or like in Futurama, the apparition of "Robosexuals"
I underground that this is nothing more than a proof of concept but imagine what Apple itself could do with this idea if they truly embraced the concept and cut all the internal red tape that currently prevents them from doing so. This is what “Apple Intelligence” should be but never materialized (and at this point I have doubts it ever will, although I am curious what they’ll show off at WWDC this year).
> I am curious what they’ll show off at WWDC this year
Apparently, not much is planned, per [1]. I'd be very cautious about AI agents like these; from a user level, this has so many security vulnerabilities.
[1] https://www.macrumors.com/2025/05/30/the-macrumors-show-last...
> I am curious what they’ll show off at WWDC this year
Fool me once...
Interesting project, if anything it shows what Android or IOS may support in the near future.
>iOS apps are sandboxed, so this project uses Xcode's UI testing harness to inspect and interact with apps and the system. (no jailbreak required).
What are practical limitations of this? Maybe you can't submit this app to the store?
It's not an app that runs on device at all. It's an program that runs on your mac.
in case if anyone wants to understand how it works: https://github.com/kiranz/phoneagent/blob/add-docs/explanati...