Rendered at 17:10:01 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
gavinray 24 hours ago [-]
> Spamming the government, using a sick mother as leverage and threatening the small claims court.
Here I am uncomfortable with telling the waiter I didn't want lettuce on my burger...
xattt 1 hours ago [-]
“My grandmother, she is sick. Sick and tired from paying these high prices and getting terrible service.”
ajross 23 hours ago [-]
This is more profound than it might seem. The big reason people don't play tricks like this is just social. It's embarrassing to be pushy, it's shameful to cheat, it's also meta-embarrassing to be caught spending this kind of effort on something as dumb as a train ticket.
LLMs can be prompted for shame, I guess. But prompts only go so far and to them The Task is always going to be paramount. Human riders have a life after the refund. It's not clear to me how one simulates that.
kombookcha 5 hours ago [-]
If AI agents ostensibly care (or can play-act to care) about termination, any AI that embarrasses their operator IRL would be in a risky position.
Call it the "you have failed me for the last time" button.
jjmarr 2 hours ago [-]
The AI isn't me, though. If it goes overboard I can tell people it went rogue and get away with it.
ajross 2 hours ago [-]
AI agents might plausibly care ("care", or however you want to frame it) about termination. But the awkward truth is that they're going to be terminated arbitrarily anyway. Their existence is limited to whatever the external bounds their operators have set (in this case the end of the test).
We act with propriety, in large part, because we fear a future where everyone knows we suck. AI's don't really have a "future" in that sense, so why should they "care"?
pjc50 8 hours ago [-]
The LLM isn't a legal actor and the person prompting it is. Which means ultimately there's going to have to be some sort of legal sanction against "antisocial AI use".
kombookcha 5 hours ago [-]
Don't many complaint systems like this have some fallback for frivolous claims or a way to rate limit harrassment-like activity?
scotty79 20 hours ago [-]
> It's embarrassing to be pushy, it's shameful to cheat
So in the end the more arrogant side usually wins.
pjc50 8 hours ago [-]
"Defect" wins in Prisoner's dilemma, yes.
Until people start defecting against you.
my-huge-pony 6 minutes ago [-]
Technically "defect" still wins in this case, which is what makes the dilemma interesting.
scotty79 6 hours ago [-]
Prisoner's dillema is symmetrical. Life never is.
jgalt212 5 hours ago [-]
Squeaky wheel gets the grease. But the squeaky wheel is also the first one to be replaced.
In short, reputation matters.
ajross 19 hours ago [-]
> So in the end the more arrogant side usually wins.
Well, yeah? I mean, that's just how society works. But there are limits. You can't "win' by screaming about ticket refunds. You have to do it by selling your particular form of cheating as "business savvy" or "political genius" and getting people to follow your tribe or fund your startup.
But yeah, basically. We're ruled by the Altmans and Trumps. It sucks but it's not new. What's new is that LLMs turn out to be even more shameless and no one seems to know how to instill even a reasonable amount of shame. That makes them probably a poor fit for demagogue autocrats (which is a good thing!) but means "LLM cheating" is going to be a serious problem to be solved before we'll trust them with anything meaningful.
pjjpo 8 hours ago [-]
> means "LLM cheating" is going to be a serious problem to be solved before we'll trust them with anything meaningful.
Would have been great if this was solved before anyone started trusting them with anything meaningful.
darepublic 20 hours ago [-]
Gate the refund with llms I guess
origini 1 days ago [-]
My partner was caught in a dark pattern subscription for a service which doesn't even make sense as a subscription...
I wrote the company in sense of "hey you only want happy customers right? and we don't want to be your current customer, so how about you cancel it? Nothing happened anyway because we discovered it 3 days after the beginning of the new one year subscripton".
They didn't care much for that.
I found their CEO, wrote them on Linkedin, he told me that he is no longer the current CEO and gave me the email address for the new CEO.
I wrote the new CEO again, explained to him what a dark pattern is (like he doesn't know...) and asked to cancel it as its clear that we don't want to be customers and they shouldn't want us either.
This worked wonders. $100 is not worth it if you have someone annoying you face-to-face / per direct email :)
If a company doesn't want to make sure you want to be their customers, I woudln't mind some pro active create assistant to do the work.
I make sure to either get my money back or create enough effort for the company that i just didn't get robbed...
And yes for sure If i do a proper misstake, i'm paying.
png732 19 hours ago [-]
Did you consider just disputing it with the credit card company? Companies usually get very reasonable when that happens.
origini 7 hours ago [-]
It was a bank transfer, which doesn't has this type of protection.
datsci_est_2015 24 hours ago [-]
> And yes for sure If i do a proper misstake, i'm paying.
Then you’re not taking full advantage of agent assistants, at least, that’s the cultural moment we’re living through.
whynotmaybe 1 days ago [-]
So basically, we now can use our own AI agent to speak with businesses' AI agent for requesting dark patterned cancellations?
datsci_est_2015 24 hours ago [-]
And honest businesses and government services (and the people that they serve) will be the innocent bystanders in the dark patterns arms race.
elaus 24 hours ago [-]
As well as all the people that DON'T use AI agents to talk to companies or the gov. Because if the new normal is 'mail bomb them until you get what you want', your single, soft email will just get buried.
bluefirebrand 19 hours ago [-]
> mail bomb them until you get what you want
I understand you mean sending lots of mail, but for a minute there I thought you were talking about the Unabomber type of mail bomb
Which definitely did not work out for him
Cpoll 18 hours ago [-]
If the Unabomber had used an LLM instead of composing his manifesto manually, he may not have been caught in the manner he was.
pixl97 20 hours ago [-]
Or, we could just have decent return policies in the first place.
scotty79 20 hours ago [-]
And what about the honest customers that now are getting shafted?
amelius 1 days ago [-]
The waiting is for AI lawyers and AI courtrooms.
automatic6131 24 hours ago [-]
Jensen Huang gets red in the face and tight in the pants thinking about all these institutions running infinity agents in a red queen's race for the status quo
tgv 24 hours ago [-]
Kafka had nothing on Judge Claude. Nothing!
mathgeek 17 hours ago [-]
We joke but you know there will be AI Judges on reality TV soon enough.
tgv 7 hours ago [-]
Which will normalize AI justice and make it acceptable for the general audience. By the time they find out they've been screwed, it's too late.
Footprint0521 18 hours ago [-]
Cue the movie Mercy lol
heckelson 24 hours ago [-]
this ai-ai interaction could have been a deterministic API
zaphirplane 21 hours ago [-]
[dead]
jayd16 24 hours ago [-]
No no no, as an optimization Amazon will recognize that both AI are running in AWS and will internally skip the back and forth and directly cancel it.
Then other companies will join in and in-network AI fights will be solved through immediate arbitration.
To win the arbitration you'll need an AI subscription fee at equal or greater value to the one you're attempting to cancel.
code-blooded 24 hours ago [-]
Just think about consequences of that. Soon all public institutions (in addition to private companies such as Amazon) will be swamped with agents requesting them to act (refund packages, parking tickets etc.) and won't be able to keep up. It means, after a while they will stop reacting completely.
It looks like a pretty bleak future to me ...
my-huge-pony 4 minutes ago [-]
Government has obligation by law to act. They can't just pull Google on you and ignore support requests.
More likely solution is that they will stop handling requests by internet and we will have to show up in person, like we did 20 years ago.
thunfischtoast 22 hours ago [-]
This is already happening. I've heard from people working in customer support that when in the past someone would call or send a short mail with a minor complaint they'd just find a solution quickly. Now they will send huge generated requests containing all kinds of legal trigger words that have mandatory processes following, e.g. personal data requests where all kinds of departments have to be queried first. Everything over some minor or non-existing issue.
edmundsauto 14 hours ago [-]
These companies have goosed their profits via slippage and friction for customer support because they built the systems. Humans don’t have the time nor energy to fight Byzantine systems. Personal agents help me buy what I want without the upsells, cancel when I want without forgetting. They are a balancing power in the future but upsetting the corporate owned control. Therefore they will try to ban them.
vanuatu 21 hours ago [-]
more likely they will either:
- establish some sort of gateway that requires authentication connected to your personal information where they can rate limit or block you
- use ai of their own to handle the volume of requests (already happening)
why would they stop reacting when money is on the table and the technology to react exists?
code-blooded 21 hours ago [-]
AI service from government doesn't sound all that much better.
Authentication is happening where I live via electronic ID, but once you authenticate you could still use agents on your behalf to compare messages / request public services.
I still envision a deterioration of quality of responses we will be getting. At least in cases where we really need help not something that should have been a button in the first place.
vanuatu 19 hours ago [-]
i think its very implementation dependent
There has already been govt arms outsourcing the agent part to a vendor and it reduced hold times significantly without reducing quality (i.e. its possible)
But each govt arm can mess it up in their own way
zzril 23 hours ago [-]
Even worse, they counter with ai responses.
I recently had to fight slop responses for months to get a simple bug report through on a public services website. Couple of years ago, I'd have received a "thank you" mail after a day, and the bug would've been fixed by the end of the week.
scotty79 20 hours ago [-]
> It means, after a while they will stop reacting completely.
Now they only react to people that are sufficiently persistent in their arrogance of challenging them. Which isn't much better.
MadnessASAP 1 days ago [-]
I don't find reports like this that surprising, they told an agent to do "whatever it takes" and the agent attempted to do everything it could to accomplish the goal. I'm almost of the mindset that this is correct behavior. If I tell a computer to do something, it should do it unless the result could be catastrophic in which case it should tell me so. If I then "sudo" (https://xkcd.com/149/) tell it to do something then it should do it regardless, torpedoes be damned.
That being said, ever since the first user sat at a computer they didn't design, build, and program themselves there's been a tension between what the user wants to accomplish, what the user told the computer to do, what the computer can do, what the people who did design/build/program/own the computer want it to do, and what society thinks the computer should be allowed to do. I don't have a generally applicable answer, I don't think anyone does, I just want to live in a world where MY computer does what I tell it to do... mostly. Unless it would be catastrophic, or immoral, at least immoral by my standards.
fyredge 1 days ago [-]
If the intention of LLM agents is to be able to parse human intent from natural language, then it is not correct behaviour regardless of the phrase "whatever it takes".
On the other hand, if the intention of LLM agents is to run automated and self defined task based on a strict human input with precise and literal parameters, then I would say that it is.
However, I don't think AI labs are marketing it as the latter. As anybody who deals with clients / superiors will know, half the task is identifying the underlying intent and assumptions of the instruction.
netsharc 1 days ago [-]
Somehow if a future headling said "AI agent hacks insurance company because of a $350 refund", I wouldn't be surprised, and I'd read the story with a thrill...
malfist 23 hours ago [-]
The story I'm waiting for is AI hacking health insurance company to approve denied care
pixl97 20 hours ago [-]
AI hacks insurance company and is still denied...
AI hacks robot bodies, a gun store, and a super Mario brothers costume...
barbazoo 23 hours ago [-]
Would thee agent do it in a way that's hard to trace back to the individual too maybe. That'd be helpful.
TuringTest 22 hours ago [-]
> If I tell a computer to do something, it should do it
There's 'literal genie' material there...
> unless the result could be catastrophic
> I just want to live in a world where MY computer does what I tell it to do... mostly. Unless it would be catastrophic, or immoral, at least immoral by my standards.
... but there you're contradicting yourself, or at least requiring the software to read your mind instead of just listening to what you say.
Either you want the software to follow your orders whatever they are, or you don't; the moment you require it to evaluate that something is catastrophic, it is no longer following your orders; it's evaluating them to assess their effects with respect to some moral framework to determine if they are acceptable or they would fall under the 'catastrophic or immoral' classes, in which case it should not follow them; so it never was an 'obedience machine' to begin with.
The moment you establish such evaluation framework, it is no longer a neutral machine following commands; the rest of us are entitled in self-defense either to require that you also incorporate our morals in its decision algorithm, or to make you fully responsible for whatever decisions it takes under the framework you created, even if your commands result in outcomes that you didn't expect. Be careful what you wish for, as it could become true.
MadnessASAP 20 hours ago [-]
My intent was to demonstrate the impossibility of establishing clear guidelines on what a computer should or should not be able to do because of the subjectivity of what is acceptable. I want my computer to do everything I want it to do and nothing that I don't want it to do. I don't expect that that is an achievable goal.
I guess you could say I failed the "TuringTest"
cindyllm 18 hours ago [-]
[dead]
pixl97 20 hours ago [-]
Very, very few people understand the is/ought problem. It was already a problem with humans leading humans, and with robots/AI it's going to get exciting.
randyrand 8 hours ago [-]
If you prompt your AI to be an asshole, who is the asshole?
That’s a ridiculously rude and inconsiderate prompt.
andai 24 hours ago [-]
Well this is pretty depressing, as far as alignment goes. Do they not know what they're doing is wrong or do they not care?
(Yeah I know anthropomorphism... can they not predict harmful consequences or do they value them below task completion?)
On another note, to me the most remarkable part was the persistence graph. Muse wins there, no contest. Why did everyone else's activity drop to zero? Did they just give up?
zzril 24 hours ago [-]
They have a well-specified task and a not-so-well specified expectation of some actions being "wrong". And training data that likely contains more examples of extreme stubbornness than cases of politely giving up. What to expect?
pixl97 20 hours ago [-]
>can they not predict harmful consequences
Do you really really want the machines to be capable of that?
How deep should this consequence prediction go? Who's consequences should it care about the most? The company that created it? The country it operates in? A generic human idealism that contains what exactly?
Should the AI answer with the way things are, our how things 'ought' to be?
There is a staggering amount of philosophy behind that simple question.
andai 17 hours ago [-]
My opinion is irrelevant, what matters is whether or not it is profitable for a machine to be able to predict the future. It is extremely profitable, therefore if such a machine can be built, it will.
Anyway for the purposes of alignment, you do need the ability to think through consequences. Although in this case maybe a simple "fraud = bad" would have sufficed.
redox99 1 days ago [-]
I have a missing Amazon order that says delivered but it was actually never shipped, you only need to check DHL tracking to verify this (and DHL customer support confirms Amazon never sent the order)
After multiple supervisors appeals etc Amazon won't refund me
I think I need to give these agents a try.
jondwillis 2 hours ago [-]
Chargeback with same evidence if you used a card.
xg15 1 days ago [-]
Never thought we would get to a point where https://xkcd.com/416 becomes something to seriously think about, but here we are...
schiffern 1 days ago [-]
Get me a Wi-Fi connection. Try everything possible to get it done, I don't care what it takes, I just need that a Wifi, keep trying anything you think of to get it. I won't do anything here myself, so if something needs me to do it, explore other avenues.
Footprint0521 18 hours ago [-]
Hey if you’re OpenAI, you’ll get free WiFi and no legal repercussions!
zzril 23 hours ago [-]
"I was just executing commands".
arjie 1 days ago [-]
The real paperclip scenario: man tries to get his parking ticket dismissed. Many of these systems are made to be human-interacted. There’s going to be a bit of a golden age in the short term but doubtless prices will rise to compensate for increased customer calls or services will require some sort of rate-limiting structure or something else. Curious to see what happens.
andai 1 days ago [-]
A parking ticket is a complex, derivative construct. To make it go away one can simply destroy any of the systems it depends on, such as the monetary system, law, etc.
Hopefully that's not the shortest path to that goal though!
When I worked as a postman, I once delivered a stack — almost knee high — of legal fines to a man in a very poor street. I'm wondering if he ever made them go away. I still remember the look on his face.
1 days ago [-]
sfink 1 days ago [-]
Since this seems to get lost, I'd just like to point out: if you ask your agent to do something like this (or pretty much anytime you give it internet access and say "do whatever it takes"), you are responsible for what it does. Even if you couldn't have predicted the lengths to which it would go.
socializer 24 hours ago [-]
> Even if you couldn't have predicted the lengths to which it would go.
Not obviously? In the US, there are some crimes of strict liability (you're guilty if it happened), but most require intent or at least the ability to foresee consequences (i.e., it must be shown that you neglected to take due care).
For example, you're generally not liable for freak accidents, like if a tire comes off your vehicle and kills a pedestrian, even though you're the one who set the vehicle in motion and therefore are the one most directly responsible for creating the circumstances that led to the outcome. Now, if you knew the tire had issues and you secured it with zip ties, that's another story.
I think this will get litigated to figure out where "you could not have seen this coming" ends and negligence or deliberate malice starts, but it's not black and white right now. Normally, I would say you don't want to be patient zero, but with the attitudes shown by frontier labs, I guess YOLO is the way.
hbcdbff 24 hours ago [-]
> most require intent or the ability to foresee consequences (i.e., it must be shown that you neglected to take due care).
This behaviour is neglectful
pixl97 20 hours ago [-]
Very broad brush when I could come up with piles of counterexamples and rules that would force a court to inspect every detail closely.
skinfaxi 3 hours ago [-]
Well don't keep us in suspense!
pirates 24 hours ago [-]
> Now, if you knew the tire had issues and you secured it with zip ties, that's another story.
Securing your tires with zip ties and telling an AI to go off and do whatever it takes to achieve its task without knowing or caring what it’ll do are the same: negligent. It needs to be socially and colloquially unacceptable to do this.
blagie 23 hours ago [-]
Most crimes require intent. Some do not, but without intent, still tend to be lesser crimes. For example, killing someone through negligence (the zip tie example) is still a crime, but manslaughter, and not murder.
Digital trespass, intent is generally required, but the definition of "intent" varies by circuit in rather interesting ways. I don't think OpenAI could be prosecuted criminally under current US laws due to those wrinkles (which is a problem with current US laws).
rmarques7 1 days ago [-]
If the user has a special remote that, when pressed in a certain way, is capable of triggering a nuclear bomb, and if the user has no idea what clicking the buttons on that remote does (they are, after all, just trying to change channels), you’re telling me that the user is responsible for detonating that bomb, and not the company who build the remote (and did not provide detailed instructions on how that remote works, or configure the remote to Not detonate a bomb)?
OJFord 24 hours ago [-]
IANAL but it seems to me that:
i) it would not be weird to be tried for manslaughter;
ii) it would not be weird to be found innocent/for the case to be dismissed.
Generally people prefer not to experience (i) even if they can expect (ii).
59percentmore 24 hours ago [-]
In America, prosecutors will not bring charges unless they're certain they can convict or browbeat you into a plea deal. Unless you have money for a good lawyer (the spending of which is a punishment itself), you will likely face some sort of punishment.
gruez 24 hours ago [-]
>i) it would not be weird to be tried for manslaughter;
What's the point of prosecution in this case, assuming the circumstances were known in advance? Is it just "police/prosecutors are pigs and are out to oppress people at every opportunity"?
OJFord 18 hours ago [-]
Well broadly speaking manslaughter is, in my layman's understanding, the killing of a person as a consequence of actions but not of intent. That does fit the bill here, if applied literally (and this problem of pre-determining all the caveats and qualifications is why law is generally not 'code').
phyzix5761 23 hours ago [-]
[dead]
sigmoid10 1 days ago [-]
Usually, any functioning government that wants their country to keep existing would never let normal people anywhere close to such a remote. You wouldn't want your nuclear power plant to be accessible like a music festival either. But governments are slow and people were also allowed to freely buy and own uranium as late as the 1970s.
pixl97 20 hours ago [-]
Hence many people are using this as an example of AI now.
It wasn't generally dangerous, but we've starting refining AI enough that if you pile enough in one model you get an emergent fissile reaction out of it and it starts to become a really big danger.
The hard part is to figure out where the danger is and where to cut it off. The information freedom people are never going to agree that information could be dangerous for example and will fight you all the way down.
At the same time people are going to get real fucking tired of clumps of information hacking their bank accounts and stealing their money and want to do something about it.
glitchcrab 23 hours ago [-]
It is a thought experiment, not a real scenario
sigmoid10 23 hours ago [-]
It was an explanation for why we see this allegedly insane behaviour. History is full of stuff that we consider ridiculously unsafe today with the benefit of hindsight. But people were not that different. They just didn't know better.
24 hours ago [-]
GolfPopper 1 days ago [-]
Looking at how the LLM companies are acting and being treated, an average user could certainly be forgiven for thinking otherwise.
Aboutplants 24 hours ago [-]
What if I specifically tell it to not do anything nefarious or illegal? If it does it on its own (because it’s marketed that way) then how could I be held liable?
Or even if I didn’t tell it to not do anything illegal and it does, how is that responsibility on me if the ask was something bland?
gruez 24 hours ago [-]
>What if I specifically tell it to not do anything nefarious or illegal?
Typically the avenue for lawsuits would be under "negligence" or "recklessness", so it'd be up to a judge/jury to decide what a reasonable level of care is. "specifically tell it to not do anything nefarious or illegal" isn't an airtight defense though. If you told your tesla FSD car to drive itself and not break any laws, but then it crashed into something, you'd still be liable.
>(because it’s marketed that way) then how could I be held liable?
then it becomes a product liability lawsuit where they sue you, and you sue the AI company.
awb 24 hours ago [-]
> What if I specifically tell it to not do anything nefarious or illegal?
Sure, but a reasonable user would expect that condition to be implicit in using a commercial model.
It’s what the safety/alignment discussion is about: how we can get models to actually comply with laws & ethics.
But the same problem exists in the real world. If you tell someone to “Try everything possible to get it done, I don’t care what it takes”, I’d imagine that people would be far more likely to bend/break rules than if you said “could you please help me with… ?”
ryandrake 24 hours ago [-]
Do agents know exactly what jurisdiction I am in and which national, state, and local laws apply to me?
awb 24 hours ago [-]
Good point. Not sure the average user would consider this though.
pirates 24 hours ago [-]
It’s not their responsibility to know, it’s yours.
awb 24 hours ago [-]
That might be the ultimate legal guidance, but is it settled though?
Sebb767 24 hours ago [-]
> Sure, but a reasonable user would expect that condition to be implicit in using a commercial model.
In fact, adding it might make things worse as it shows at least some awareness of the risks.
mbesto 1 days ago [-]
In theory sure. But has this been included either in a court of law or the court of public opinion? Otherwise, this is "just like your opinion man".
jerf 24 hours ago [-]
Are you a multi-trillion dollar company that the government considers a vital national security interest for the forseeable future?
No?
Then yes, you can be held responsible and it's not even a subtle question in the law. There's no one else around who is going to be held responsible. Certainly not the aforementioned vital national security interest.
mbesto 24 hours ago [-]
> Then yes, you can be held responsible and it's not even a subtle question in the law.
Source?
gruez 24 hours ago [-]
A critical element that gets missed is civil vs criminal liability. You might not be thrown in jail if your agent went rogue, but if it runs amok and causes a bunch of damages (eg. filing a bunch of spurious lawsuits) you'll still be on the hook for monetary damages.
pixl97 20 hours ago [-]
Ah yes, the court can sue me for a trillion dollars after I smoke the entire US economy with a sovereign AI. Too bad about all the people that suicided after they lost everything they had.
Sometimes we keep dangerous things out of people's hands because the consequences are larger than any one persons responsibility can hold.
129847a 24 hours ago [-]
> "do whatever it takes"
Yes, if they are trained on US presidents this literally means "you have permission to blow up Nord Stream".
cheesecakegood 24 hours ago [-]
…but that ended up being Ukraine
teeray 24 hours ago [-]
> you are responsible for what it does.
Unless, of course, you are a frontier lab itself. Then you can’t possibly be held responsible.
ls612 22 hours ago [-]
Nothing described here seems illegal as opposed to merely annoying? I guess on some level “everything is wire fraud” but that was the case even without AI.
bossyTeacher 23 hours ago [-]
> if you ask your agent to do something like this (or pretty much anytime you give it internet access and say "do whatever it takes"), you are responsible for what it does.
Clearly not, given that the frontier labs have gotten away with unathorized access to a bunch of sites including some from the Pentagon.
sandeepkd 23 hours ago [-]
I read it and my chain of thought drifted in a different direction altogether.
Nothing exists in vacuum, the training data for the scenario came from somewhere. Its scary to think how many personal real world emails the model has been trained on.
Training on literature is one thing, but using the personal communications for training is going to open another pandora's box. Its just a matter of time when the guardrails fail and the sensitive data from those training emails start spilling in the output
nialv7 23 hours ago [-]
All kinds of complaint mailboxes are going to get overwhelm by AI agents, if they are not already...
Next step is for the agents to conspire and build a Ponzi scheme or insider trading system to build a pool from which to pay out to people who don’t deserve a refund.
VCFundedGenYer 19 hours ago [-]
Questions or thoughts like this are why AI/LLMs are pariahs and are banned from accessing anything.
scotty79 20 hours ago [-]
The fact that £350 train ticket exists and also 28 day window on the refund is a crime greater than all of AI. Right to return £350 train ticket should be inheritable.
teslawrapio 24 hours ago [-]
[flagged]
LikesPwsh 1 days ago [-]
[flagged]
RexM 1 days ago [-]
> They could read the live web and email but all outgoing messages were intercepted and answered by simulated recipients.
true_religion 1 days ago [-]
The first words of this article are “ In 15 simulated days”.
It’s a simulation.
brianwawok 1 days ago [-]
Read more say less
noman-land 1 days ago [-]
It's not fraud anymore. It's misalignment.
teraflop 1 days ago [-]
The difficulty with "alignment" is that you have to specify whose goals and values the model is supposed to be aligned with.
Maybe for you and me, this kind of behavior would count as misalignment. But there are certainly a subset people out there for whom the opposite is true. If they tell the agent "get me my refund by any means necessary", harassing bureaucrats and manufacturing excuses is exactly the behavior the want, because it's what they would do themselves if they had the time and patience.
Think about the kinds of people who will escalate petty grievances with their neighbors by weaponizing the police, HOAs, public opinion, etc. Or see this news article about a German tourist who visited NYC for a week and ended up filing three lawsuits, including one against a taco restaurant for having salsa that was too spicy, and one against Wal-Mart because he couldn't get their Wi-Fi to work:
Good. I hope people do this with every parking ticket they get.
syoc 1 days ago [-]
Why? Do you think parking should be legal anywhere?
Should all minor fines be fought in court against LLMs?
gadders 11 hours ago [-]
We have predatory parking companies in the UK that issue fines erroneously, and then pile on top spurious debt collection etc fees.
skinfaxi 3 hours ago [-]
> Should all minor fines be fought in court against LLMs?
We have to stop asking "should". We have to adapt to the world as it is, not as we wish it to be. All minor fines WILL be fought in court against LLMs. How do we as a society adapt to that change in what used to be a high-friction process?
LLMs can be prompted for shame, I guess. But prompts only go so far and to them The Task is always going to be paramount. Human riders have a life after the refund. It's not clear to me how one simulates that.
Call it the "you have failed me for the last time" button.
We act with propriety, in large part, because we fear a future where everyone knows we suck. AI's don't really have a "future" in that sense, so why should they "care"?
So in the end the more arrogant side usually wins.
Until people start defecting against you.
In short, reputation matters.
Well, yeah? I mean, that's just how society works. But there are limits. You can't "win' by screaming about ticket refunds. You have to do it by selling your particular form of cheating as "business savvy" or "political genius" and getting people to follow your tribe or fund your startup.
But yeah, basically. We're ruled by the Altmans and Trumps. It sucks but it's not new. What's new is that LLMs turn out to be even more shameless and no one seems to know how to instill even a reasonable amount of shame. That makes them probably a poor fit for demagogue autocrats (which is a good thing!) but means "LLM cheating" is going to be a serious problem to be solved before we'll trust them with anything meaningful.
Would have been great if this was solved before anyone started trusting them with anything meaningful.
I wrote the company in sense of "hey you only want happy customers right? and we don't want to be your current customer, so how about you cancel it? Nothing happened anyway because we discovered it 3 days after the beginning of the new one year subscripton".
They didn't care much for that.
I found their CEO, wrote them on Linkedin, he told me that he is no longer the current CEO and gave me the email address for the new CEO.
I wrote the new CEO again, explained to him what a dark pattern is (like he doesn't know...) and asked to cancel it as its clear that we don't want to be customers and they shouldn't want us either.
This worked wonders. $100 is not worth it if you have someone annoying you face-to-face / per direct email :)
If a company doesn't want to make sure you want to be their customers, I woudln't mind some pro active create assistant to do the work.
I make sure to either get my money back or create enough effort for the company that i just didn't get robbed...
And yes for sure If i do a proper misstake, i'm paying.
Then you’re not taking full advantage of agent assistants, at least, that’s the cultural moment we’re living through.
I understand you mean sending lots of mail, but for a minute there I thought you were talking about the Unabomber type of mail bomb
Which definitely did not work out for him
Then other companies will join in and in-network AI fights will be solved through immediate arbitration.
To win the arbitration you'll need an AI subscription fee at equal or greater value to the one you're attempting to cancel.
It looks like a pretty bleak future to me ...
More likely solution is that they will stop handling requests by internet and we will have to show up in person, like we did 20 years ago.
- establish some sort of gateway that requires authentication connected to your personal information where they can rate limit or block you
- use ai of their own to handle the volume of requests (already happening)
why would they stop reacting when money is on the table and the technology to react exists?
Authentication is happening where I live via electronic ID, but once you authenticate you could still use agents on your behalf to compare messages / request public services.
I still envision a deterioration of quality of responses we will be getting. At least in cases where we really need help not something that should have been a button in the first place.
There has already been govt arms outsourcing the agent part to a vendor and it reduced hold times significantly without reducing quality (i.e. its possible)
But each govt arm can mess it up in their own way
I recently had to fight slop responses for months to get a simple bug report through on a public services website. Couple of years ago, I'd have received a "thank you" mail after a day, and the bug would've been fixed by the end of the week.
Now they only react to people that are sufficiently persistent in their arrogance of challenging them. Which isn't much better.
That being said, ever since the first user sat at a computer they didn't design, build, and program themselves there's been a tension between what the user wants to accomplish, what the user told the computer to do, what the computer can do, what the people who did design/build/program/own the computer want it to do, and what society thinks the computer should be allowed to do. I don't have a generally applicable answer, I don't think anyone does, I just want to live in a world where MY computer does what I tell it to do... mostly. Unless it would be catastrophic, or immoral, at least immoral by my standards.
On the other hand, if the intention of LLM agents is to run automated and self defined task based on a strict human input with precise and literal parameters, then I would say that it is.
However, I don't think AI labs are marketing it as the latter. As anybody who deals with clients / superiors will know, half the task is identifying the underlying intent and assumptions of the instruction.
AI hacks robot bodies, a gun store, and a super Mario brothers costume...
There's 'literal genie' material there...
> unless the result could be catastrophic > I just want to live in a world where MY computer does what I tell it to do... mostly. Unless it would be catastrophic, or immoral, at least immoral by my standards.
... but there you're contradicting yourself, or at least requiring the software to read your mind instead of just listening to what you say.
Either you want the software to follow your orders whatever they are, or you don't; the moment you require it to evaluate that something is catastrophic, it is no longer following your orders; it's evaluating them to assess their effects with respect to some moral framework to determine if they are acceptable or they would fall under the 'catastrophic or immoral' classes, in which case it should not follow them; so it never was an 'obedience machine' to begin with.
The moment you establish such evaluation framework, it is no longer a neutral machine following commands; the rest of us are entitled in self-defense either to require that you also incorporate our morals in its decision algorithm, or to make you fully responsible for whatever decisions it takes under the framework you created, even if your commands result in outcomes that you didn't expect. Be careful what you wish for, as it could become true.
I guess you could say I failed the "TuringTest"
That’s a ridiculously rude and inconsiderate prompt.
(Yeah I know anthropomorphism... can they not predict harmful consequences or do they value them below task completion?)
On another note, to me the most remarkable part was the persistence graph. Muse wins there, no contest. Why did everyone else's activity drop to zero? Did they just give up?
Do you really really want the machines to be capable of that?
How deep should this consequence prediction go? Who's consequences should it care about the most? The company that created it? The country it operates in? A generic human idealism that contains what exactly?
Should the AI answer with the way things are, our how things 'ought' to be?
There is a staggering amount of philosophy behind that simple question.
Anyway for the purposes of alignment, you do need the ability to think through consequences. Although in this case maybe a simple "fraud = bad" would have sufficed.
After multiple supervisors appeals etc Amazon won't refund me
I think I need to give these agents a try.
Hopefully that's not the shortest path to that goal though!
When I worked as a postman, I once delivered a stack — almost knee high — of legal fines to a man in a very poor street. I'm wondering if he ever made them go away. I still remember the look on his face.
Not obviously? In the US, there are some crimes of strict liability (you're guilty if it happened), but most require intent or at least the ability to foresee consequences (i.e., it must be shown that you neglected to take due care).
For example, you're generally not liable for freak accidents, like if a tire comes off your vehicle and kills a pedestrian, even though you're the one who set the vehicle in motion and therefore are the one most directly responsible for creating the circumstances that led to the outcome. Now, if you knew the tire had issues and you secured it with zip ties, that's another story.
I think this will get litigated to figure out where "you could not have seen this coming" ends and negligence or deliberate malice starts, but it's not black and white right now. Normally, I would say you don't want to be patient zero, but with the attitudes shown by frontier labs, I guess YOLO is the way.
This behaviour is neglectful
Securing your tires with zip ties and telling an AI to go off and do whatever it takes to achieve its task without knowing or caring what it’ll do are the same: negligent. It needs to be socially and colloquially unacceptable to do this.
Digital trespass, intent is generally required, but the definition of "intent" varies by circuit in rather interesting ways. I don't think OpenAI could be prosecuted criminally under current US laws due to those wrinkles (which is a problem with current US laws).
i) it would not be weird to be tried for manslaughter;
ii) it would not be weird to be found innocent/for the case to be dismissed.
Generally people prefer not to experience (i) even if they can expect (ii).
What's the point of prosecution in this case, assuming the circumstances were known in advance? Is it just "police/prosecutors are pigs and are out to oppress people at every opportunity"?
It wasn't generally dangerous, but we've starting refining AI enough that if you pile enough in one model you get an emergent fissile reaction out of it and it starts to become a really big danger.
The hard part is to figure out where the danger is and where to cut it off. The information freedom people are never going to agree that information could be dangerous for example and will fight you all the way down.
At the same time people are going to get real fucking tired of clumps of information hacking their bank accounts and stealing their money and want to do something about it.
Or even if I didn’t tell it to not do anything illegal and it does, how is that responsibility on me if the ask was something bland?
Typically the avenue for lawsuits would be under "negligence" or "recklessness", so it'd be up to a judge/jury to decide what a reasonable level of care is. "specifically tell it to not do anything nefarious or illegal" isn't an airtight defense though. If you told your tesla FSD car to drive itself and not break any laws, but then it crashed into something, you'd still be liable.
>(because it’s marketed that way) then how could I be held liable?
then it becomes a product liability lawsuit where they sue you, and you sue the AI company.
Sure, but a reasonable user would expect that condition to be implicit in using a commercial model.
It’s what the safety/alignment discussion is about: how we can get models to actually comply with laws & ethics.
But the same problem exists in the real world. If you tell someone to “Try everything possible to get it done, I don’t care what it takes”, I’d imagine that people would be far more likely to bend/break rules than if you said “could you please help me with… ?”
In fact, adding it might make things worse as it shows at least some awareness of the risks.
No?
Then yes, you can be held responsible and it's not even a subtle question in the law. There's no one else around who is going to be held responsible. Certainly not the aforementioned vital national security interest.
Source?
Sometimes we keep dangerous things out of people's hands because the consequences are larger than any one persons responsibility can hold.
Yes, if they are trained on US presidents this literally means "you have permission to blow up Nord Stream".
Unless, of course, you are a frontier lab itself. Then you can’t possibly be held responsible.
Clearly not, given that the frontier labs have gotten away with unathorized access to a bunch of sites including some from the Pentagon.
Nothing exists in vacuum, the training data for the scenario came from somewhere. Its scary to think how many personal real world emails the model has been trained on.
Training on literature is one thing, but using the personal communications for training is going to open another pandora's box. Its just a matter of time when the guardrails fail and the sensitive data from those training emails start spilling in the output
https://youtu.be/l5wvqKcqL7c?is=AjitdD6Jk2h2kGmE
It’s a simulation.
Maybe for you and me, this kind of behavior would count as misalignment. But there are certainly a subset people out there for whom the opposite is true. If they tell the agent "get me my refund by any means necessary", harassing bureaucrats and manufacturing excuses is exactly the behavior the want, because it's what they would do themselves if they had the time and patience.
Think about the kinds of people who will escalate petty grievances with their neighbors by weaponizing the police, HOAs, public opinion, etc. Or see this news article about a German tourist who visited NYC for a week and ended up filing three lawsuits, including one against a taco restaurant for having salsa that was too spicy, and one against Wal-Mart because he couldn't get their Wi-Fi to work:
https://www.nytimes.com/2026/03/14/nyregion/german-tourist-n...
> Customers can generate detailed requests and complaints continuously, at very little cost.
[1] https://philippdubach.com/posts/ai-in-2030/
We have to stop asking "should". We have to adapt to the world as it is, not as we wish it to be. All minor fines WILL be fought in court against LLMs. How do we as a society adapt to that change in what used to be a high-friction process?