The "random factoids" were verbatim training data though, one of their extractions was >1,000 tokens in length.
> GPT4 never merely regurgitated
I interpreted the claim that it can't "regurgitate training data" to mean that it can't reproduce verbatim a non-trivial amount of its training data. Based on how I've heard the word "regurgitate" used, if I were to rattle off the first page of some book from memory on request I think it would be fair to say I regurgitated it. I'm not trying to diminish how GPT does what it does, and I find what it does to be quite impressive.
Anecdotally, there were also a few examples I tried earlier this year (on GPT3.5 and GPT4) of being able to directly prompt for training data. They were patched out pretty quick but did work for a while. For example, asking for "fast inverse square root" without specifying anything else would give you the famous Quake III code character for character, including comments.
That is pretty interesting and also I didn't realize you could share chats like that.
That GPT is so bad at tic-tac-toe and relatively good at other games like chess is one of the main things that contributes to me having a lower opinion of its ability to generalize than I would have otherwise.
I think any human with GPT's abilities in chess (but somehow no prior knowledge of ttt) would have zero issue becoming an expert with a single explanation of the game. Even very young children can learn to play ttt well and at least consistently make valid moves if nothing else.
Oddly just like the text version it is still really bad at tic-tac-toe. Gave it a picture of a completed game and "Who won?" It told me "X won with a vertical line through the middle column" when in fact O won and there was only one X in the middle column.
Very impressive with almost everything else I gave it though.
> "picture" doesn't think of his fellow humans as fully real, thinking people
I can't say I agree, this feels like a very uncharitable reading of his/her posts. Unless it was edited in after the fact they even said "You're absolutely free to believe either way, and I don't want anyone to force you to do anything."
While it would be insulting to call any individual person's preferences a result of brainwashing, I don't think it is a stretch to say that at a societal level preferences are shaped by mass-media and advertising. Improving access to and making people aware of less resource-intensive forms of comfort doesn't have to come from an authoritarian place. One of my major motivations for seeking out a more walkable area was urbanist YouTubers extolling the benefits. I suppose one could argue that things like bike lanes are hurting drivers but if a city's transit priorities stem from local politics and preferences I don't think it can be reasonably argued that making any particular transit method a priority is more authoritarian than another.
> In practice, comfort is mostly a function of stuff.
No question that it is a variable for most people but I'm not sure I buy that it is the most dominant one. All other variables being excluded, time to do what I want is at least as important for me as stuff (luckily I like my job so time/money aren't usually in conflict). And I think for many people "stuff" like cars and nice lawns aren't inherently drivers of comfort, but rather just possible reifications of goals like "pretty yard" or "fast/easy transit," both of which can be realized in less resource intensive ways. For the yard example, that might be a native garden or xeriscape (in some cases there are rules against these, which actually goes against freedom imo).
Comfort != "stuff". Yes in some cases stuff brings additional comfort, and what stuff does that varies by person but there isn't anything inherently contradictory in what they said imo.
I moved from an area where I needed a car to an area where I don't and doing so increased my comfort. If areas like this were more accessible I think a lot of people would willingly degrowth and become more comfortable at the same time. Of course people shouldn't be forced to lose their car or move to a denser area if they don't want. And I like my gadgets but it is pretty ridiculous that their lifespans are artificially shortened to prop up profits. I have a computer from 1984 that still works, I would bet a huge amount of money none of the devices I buy today will work in 2062.
I've noticed that one of the most common failure patterns I get from GPT4 for code generation is that it incorrectly asserts something and then corrects itself in the same response.
ex: "This code `(some-fn 1 2)` does x because y. That is incorrect because abc"
I wondered if this has to do with common StackOverflow post formats.
Definitely not the only one, an entire country decided that durability is important for the $1 denomination, I'll copy what I posted above:
I was recently in El Salvador, which uses the USD as an official currency (alongside BTC lol). Despite using dollars, the $1 coins are used instead of notes almost exclusively for that denomination (mostly presidential dollars and some silver Susan B Anthony dollars). I was curious and did a bit of research, apparently the reason is that because day-to-day transactions are done almost entirely in small amounts of cash the paper notes have a very short lifespan (apparently <1yr) there, while coins will last decades.
I was recently in El Salvador, which uses the USD as an official currency (alongside BTC lol). Despite using dollars, the $1 coins are used instead of notes almost exclusively for that denomination (mostly presidential dollars and some silver Susan B Anthony dollars). I was curious and did a bit of research, apparently the reason is that because day-to-day transactions are done almost entirely in small amounts of cash the paper notes have a very short lifespan (apparently <1yr) there, while coins will last decades.
I live in DC, it is ridiculously expensive but rent increases are capped, google says the formula is CPI + 2% but no more than 10%/yr. If she stayed in the same unit she might want to make sure her landlord isn't raising her rent illegally.