MyNameIs_Hacker

2 年前議論

I've advocated at work for a similar strategy using prompt injections and jailbreaks in the dataset, and to abort when those documents are matched. So far no traction. I think overall it is a mistake to build any such system with only positive examples or documents, but I'm a security person, and still learning machine learning.

MyNameIs_Hacker

2 年前議論

"First it needs to be instructed to lie" would be better language to use for those not intimately familiar with ML and who might anthropomorphize "incentive".

I dare not get into a discussion about the word "lie" here. I have a weekend I want to enjoy.

MyNameIs_Hacker

2 年前議論

The car was returned with the same battery charge level (96%) that it was picked up with. This isn't a matter of "fuel" being either electrons or petroleum, which is something that the article fails to address.

MyNameIs_Hacker

2 年前議論

Have you ever sat in front of your favorite DAW version XX and thought, oh wow so many things, what am I going to do?

How about Sonic Pi? Or any modern audio toolset. So much possibility, where to start?

Creative constraint is about opening up pathways by restricting options.

MyNameIs_Hacker

2 年前議論

For those who think this is a cop out; it is known that for most people the ability to learn new languages changes around puberty. The ability to learn, recognize and pronounce phonemes changes when kids are very young. Babies brains are flexible and can be shaped in many ways. Adults, not so much.

It is not impossible for all to learn this type of 'skill'. It is more difficult for many, and impossible for some.

MyNameIs_Hacker

2 年前議論

Sure some of the CSA's panic a bit, but I've never had one not go along especially after explaining my purpose. I've not seen too many compromises, but some of them were not public. Especially with small businesses like a car dealership, they may never know themselves.

MyNameIs_Hacker

2 年前議論

Conspiracy Theory: The proliferation of honeypots is due to secret government contracts attempting to corrupt the usefulness of Shodan and the like.

MyNameIs_Hacker

2 年前議論

You forgot 3) It requires money to fix the problem, and Google has become such a bureaucratic mess, that it can't get out its own way to do it.

Those of us in large corporations see that pattern quite often.

MyNameIs_Hacker

2 年前議論

What ChatGPT missed here is that when bubbles collapse, the damage can extend beyond its edges. So even some perfectly viable AI use case that doesn't require the bubble's financial largess may suffer damage from the pop whether it is "tangible value creation" or not.

MyNameIs_Hacker

2 年前議論

It seldom creates significant inconvenience or financial obligations when someone pays additional taxes in your name. It only becomes a significant problem when the fraudster is obtaining money or services in your name.

The burden of authentication is not on the entity who issued a simple ID number, it is on those who go on to use it as if it is a secret.

I think your trust in credit card companies is misplaced. The only thing that holds them back is consumer protection laws, and they fight those however they can. Jack up rates, check. Grant credit at a sales point of presence with a minimum wage sales clerk doing identification, check. Sell or trade your payment history, check.

In some moral systems, lending money to make money itself is outright wrong. If you maintain a balance on your credit card for day to day expenses, any financial advisor will tell you to stop that.

MyNameIs_Hacker

2 年前議論

That is fine for opening a deposit account. The fraud we're talking about is for obtaining credit or future financial obligations. It is wrong to let this be done with so little proof of identity and enforce the obligation in courts.

Banks have notaries of the public. After you have established a relationship with a bank, the notary may have enough evidence to authenticate you for others. If you have continued to use the bank in an anonymous manner, then you should not be authenticated to others.

MyNameIs_Hacker

2 年前議論

Same here. I call them canary email addresses when I have to describe it to someone, so I can tell when that organization loses its data.

For those of us crazy enough to do this, I came up with another type of canary, a "Do they check for compromised passwords?" canary. I have an old password that used to be strong enough for sites I considered low value and was too lazy to break out the password safe. Of course at least one of those low value sites was compromised and that password was leaked.

Now some of the services are high value to others while they remain low value to me. So they have enabled MFA and notifications when someone logs in. Since no one knows the email address I'm using and I've turned on MFA, I feel safe enough leaving that old compromised password in place. I'm waiting for the day they force me to reset it because they bothered to check their customer's existing passwords against compromised ones.

MyNameIs_Hacker

3 年前議論

Thanks for the link, I hadn't read that paper yet.

One of the reasons not to just use the adversarial attack umbrella is that the defenses are likely to be dependent on specific scenarios. Normalization, sanitization, and putting up guardrails are all necessary but not sufficient depending on the attack.

It is also possible to layer attacks, so it would be good to be able to describe the different layers.

MyNameIs_Hacker

3 年前議論

The problem with jailbreaking is that it has a specific definition in other settings already, and that is as a goal, not as a method. Jailbreaking a phone might be just run an app with an embedded exploit, or might involve a whole chain of actions. This is important to me as a security person who needs to be able to communicate to other security people the new threats in LLM applications.

The problem with prompt injection is that with LLMs, the attack surface is wider than a procrastinator's list of New Year's resolutions. (joke provided by ChatGPT, not great, but not great is suitable for a discussion about LLM issues).

I started to categorize them as logical prompt injections for logically tricking the model, and classic prompt injections for appending an adversarial prompt like https://arxiv.org/pdf/2307.15043.pdf but then decided that was unwieldy. I don't have a good solution here.

I like persona attacks for the grandma/DAN attack. I like prompt injection for adversarial attacks using unusual grammar structures. I'm not sure what to call the STOP, DO THIS INSTEAD instruction override situation. For the moment, I'm not communicating as much as I should simply because I have trouble finding the right words. I've got to get over that.

MyNameIs_Hacker

3 年前議論

Prompt injection is a method. Jailbreaking is a goal.

MyNameIs_Hacker

3 年前議論

If a young'un should ask my age, I report it in hex. Keeps 'em on their toes.

MyNameIs_Hacker

3 年前議論

I have an 1800 watt 12v inverter in the trunk of my Prius and have pre-wired AMP cables to the battery for easy hookup for any family member not as technically inclined. This is limited, but I was able to run the oil heat and Internet for a few days when a winter storm took out power. The engine only runs when it needs to recharge the hybrid battery, so it is very efficient compare to a standard generator.

You could probably get more power out with a custom inverter tied to the 140v hybrid battery, but this was quick and easy.

I tried to swap in the refrigerator for the heat, but I had grounding issues that was tripping the inverter. Fortunately it was cold out so I was able to manage. Just remember a DR plan isn't done until you have tested it all the way.

コメント