pseudonom-

2 years agodiscuss

Dhall is the FP config language you're thinking of, I think.

3 years agodiscuss

Funny, I did almost the exact same thing: https://github.com/colehaus/hammock-public. Though I project to 3D and then put them in an interactive 3D plot. The other fun little thing the interactive plotting enables is stepping through a variety of clustering granularities.

pseudonom-

3 years agodiscuss

OpenAI was formed as a nonprofit with a specific charter ("OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.") and the capped-profit entity under which daily operations occur formed years later with the claim that it was instrumentally useful for pursuing that charter. The capped-profit entity remains a subsidiary of the nonprofit. The board in the dispute is the board of the overseeing nonprofit.

So there are many particulars that mean pattern matching to a standard board dispute will lose something. I think it's likely many of the primary actors have, at various times, had non-strictly-pecuniary motives. That one side won doesn't mean that the other side was always a farce.

pseudonom-

3 years agodiscuss

Are you aware of the history and governance structure of OpenAI?

pseudonom-

3 years agodiscuss

Unless I'm misunderstanding something, this is precisely why I don't use Home Manager. I've literally never had my NixOS setup break over the course of many years.

pseudonom-

3 years agodiscuss

https://arxiv.org/abs/2306.11644 is along these lines.

pseudonom-

3 years agodiscuss

Probably quotes like:

"It must be familiar, roughly C-like. Programmers working at Google are early in their careers and are most familiar with procedural languages, particularly from the C family. The need to get programmers productive quickly in a new language means that the language cannot be too radical."

And not including sum types despite having a sum-type-shaped hole in the language (`if err != nil`).

And some of the discussion about "why no generics" seemed kind of divorced from existing PL knowledge on the topic.

pseudonom-

3 years agodiscuss

Tim Dettmers recently (https://www.manifold1.com/episodes/ai-on-your-phone-tim-dett...):

"But what we found with these neural networks is, if you use 32 bits, they're just fine. And then you use 16 bits, and they're just fine. And then with eight bits, you need to use a couple of tricks and then it's just fine.

And now we find if you can go to four bits, and for some networks, that's much easier. For some networks, it's much more difficult, but then you need a couple more tricks. And so it seems they're much more robust."

pseudonom-

3 years agodiscuss

There are other mechanisms for dealing with vanishing and exploding gradients. I (maybe wrongly?) think of batch normalization as being most distinctively about fighting internal covariate shift: https://machinelearning.wtf/terms/internal-covariate-shift/

pseudonom-

3 years agodiscuss

> (Do bear or typeguard allow you to do this using decorators?)

You can push some of this directly into Python type annotations thanks to https://peps.python.org/pep-0646/.

e.g.

  @overload
  def mean(a: ndarray[float, Dim1, *Shape], axis: Literal[0]) -> ndarray[float, *Shape]: ...
  @overload
  def mean(a: ndarray[float, Dim1, Dim2, *Shape], axis: Literal[1]) -> ndarray[float, Dim1, *Shape]: ...

pseudonom-

3 years agodiscuss

There's some discussion in section 3.2 of https://arxiv.org/pdf/1910.10683.pdf

pseudonom-

3 years agodiscuss

> Ie even lower than my guess of 90% above.

These are not comparable numbers. You're comparing "fraction of people" vs "fraction of outcomes". Presumably an eye-roller assigns ~0 probability to "extremely bad" outcomes (or has a shockingly cavalier attitude toward medium-small probabilities of catastrophe).

comments