HackerTrans
TopNewTrendsCommentsPastAskShowJobs

davidmanheim

no profile record

Submissions

Building less-flawed metrics: Understanding and creating better systems

cell.com
3 points·by davidmanheim·3 yıl önce·0 comments

comments

davidmanheim
·11 ay önce·discuss
Yes - I'm fairly culturally conservative, but very much don't support the (so-called) conservative political leaders basically anywhere.
davidmanheim
·3 yıl önce·discuss
Here are some good lists; Rob Wiblin https://medium.com/@robertwiblin/things-i-recommend-you-buy-...

Rosie Campbell https://www.facebook.com/ardenlk/posts/10156553178262333

Sam Bowman https://medium.com/@s8mb/things-i-recommend-you-buy-and-use-...
davidmanheim
·3 yıl önce·discuss
...but progress was slower. Exponential growth means it grows faster now than it did then.

And yes, relative poverty matters greatly, but absolute poverty is a real thing even in the US. In 1964, the US launched the war on poverty, and poverty declined. Since the 1980s, some of that progress was reversed, but it's still far better than 1964. (And poverty now is relative, and includes very little literal starvation. That certainly wasn't true in the US in 1964, much less earlier - especially before WWII.
davidmanheim
·3 yıl önce·discuss
I don't know enough about heat pumps to comment about the substance of this report, but looking at the list of their other reports at the end,it seems worth pointing out the heavily ideological, global warming denying biases of the group that wrote this.
davidmanheim
·3 yıl önce·discuss
It's more specific than that once they actually operationalize the question:

This question resolves as YES if, in Meta's 2024 Q4 Quarterly Adversarial Threat Report, Meta claims that there was at least one "coordinated inauthentic behavior" that

specifically pertained to the 2024 US Presidential election, and Meta suspects was primarily conducted via AI.
davidmanheim
·3 yıl önce·discuss
> Nowhere do they define "AGI"

Ummm, maybe you should have looked? At the top of the very first prediction, here: https://www.metaculus.com/questions/5121/date-of-artificial-...

We will thus define "an AI system" as a single unified software system that can satisfy the following criteria, all completable by at least some humans.

Able to reliably pass a 2-hour, adversarial Turing test during which the participants can send text, images, and audio files (as is done in ordinary text messaging applications) during the course of their conversation. An 'adversarial' Turing test is one in which the human judges are instructed to ask interesting and difficult questions, designed to advantage human participants, and to successfully unmask the computer as an impostor. A single demonstration of an AI passing such a Turing test, or one that is sufficiently similar, will be sufficient for this condition, so long as the test is well-designed to the estimation of Metaculus Admins.

Has general robotic capabilities, of the type able to autonomously, when equipped with appropriate actuators and when given human-readable instructions, satisfactorily assemble a (or the equivalent of a) circa-2021 Ferrari 312 T4 1:8 scale automobile model. A single demonstration of this ability, or a sufficiently similar demonstration, will be considered sufficient.

High competency at a diverse fields of expertise, as measured by achieving at least 75% accuracy in every task and 90% mean accuracy across all tasks in the Q&A dataset developed by Dan Hendrycks et al..

Able to get top-1 strict accuracy of at least 90.0% on interview-level problems found in the APPS benchmark introduced by Dan Hendrycks, Steven Basart et al. Top-1 accuracy is distinguished, as in the paper, from top-k accuracy in which k outputs from the model are generated, and the best output is selected.

By "unified" we mean that the system is integrated enough that it can, for example, explain its reasoning on a Q&A task, or verbally report its progress and identify objects during model assembly. (This is not really meant to be an additional capability of "introspection" so much as a provision that the system not simply be cobbled together as a set of sub-systems specialized to tasks like the above, but rather a single system applicable to many problems.)

Resolution will come from any of three forms, whichever comes first: (1) direct demonstration of such a system achieving ALL of the above criteria, (2) confident credible statement by its developers that an existing system is able to satisfy these criteria, or (3) judgement by a majority vote in a special committee composed of the question author and two AI experts chosen in good faith by him, for the sole purpose of resolving this question. Resolution date will be the first date at which the system (subsequently judged to satisfy the criteria) and its capabilities are publicly described in a talk, press release, paper, or other report available to the general public.
davidmanheim
·3 yıl önce·discuss
Well, I do know something about those topucs, but here's a twitter thread from someone else who thinks the people worried about risks are silly, who's definitely an expert, with speculation on how he thinks GPT-4 could be made into a generally intelligent autonomous system: https://mobile.twitter.com/karpathy/status/16425988905738199...