your follow-up makes no difference to the main point under discussion : if you ask two different llms about a legal fact pattern and they independently reference the same law, then it is, in fact, more likely that the law is more relevant to the fact pattern than if only one llm mentioned it.
i propose the following benchmark task that i think can serve as a baseline of whether these local automation systems can really save time:
starting with a bare ubuntu desktop system with plenty of RAM and CPU, setup three ubuntu VMs for secure development and networking skills learning (wireshark, protocol analysis, etc etc):
one ubuntu “virtual” desktop to simulate a working desktop that an end-user or developer would use. its networking should initially be completely isolated.
one ubuntu server to simulate a bastion machine. route all “virtual desktop” traffic through this “bastion”. it will serve as a tap.
one ubuntu server to serve as edge node. this one can share internet access with the host. route all bastion traffic through the edge node.
use this three vm setup to perform ordinary tasks in the “virtual desktop “ and observe the resulting traffic in the “bastion”. verify that no other traffic is generated on or from the host outside of the expected path virtual desktop -> bastion -> edge.
i claim this is a minimal “network clean” development setup for anyone wanting to do security-conscious development.
extra credit: setup another isolated vm sever to act as the package manager ; ie mirror anything to be installed on the “virtual desktop” onto this package server and configure this server as the install point for apt on the “virtual desktop”.
i doubt an AI can set this up right now. (i’ve tried)
no ties to my other accounts would be ideal from my perspective
i understand that it’s easier to get a stronger “trust” signal by being more invasive
but hopefully the product will be so valuable that users will value their accounts as assets (like on hn) that they won’t want to compromise with bad behavior
that’s good to know thanks but creates more special cases to manage if i just want to backup my stuff so i can manually recover when i need to (on lost device say).
i just ran into a situation activating a new device in which apple were trying to send to a device i had forgotten to “properly” remove from that icloud account.
and also another situation in which the 2fa code would flash on the remote device and disappear in a fraction of a second. i eventually captured it with screen recording but every time i did it the code was not accepted.
my conclusion: apple had silently ruled that i would not be allowed to activate using that particular icloud account. no idea why. i tried a different one and things went through ok.