dlwh

3 tahun yang laludiscuss

Stanford CRFM | Bay Area | Full-Time | On-Site, Hybrid

Foundation models like ChatGPT, PaLM, and Stable Diffusion are transforming the world around us. The Stanford Center for Research on Foundation Models (CRFM; https://crfm.stanford.edu/), which is part of Stanford HAI, is an interdisciplinary initiative that aims to make foundation models more reliable, transparent and accessible to the world. We take on ambitious projects that seek to rigorously evaluate existing foundation models and to build new ones.

We are currently seeking a research engineer to join our engineering team. This is an unique opportunity to work with seasoned engineers who have spent many years in industry as well as PhD students, post-docs, and faculty at CRFM. You will contribute to cutting-edge research, publish papers, gain access to the latest foundation models, and be immersed in the vibrant CRFM community.

You will work on our open source software projects, including:

  - HELM, a framework for holistic evaluation of large language models 
  - Levanter, a framework for transparent and accessible large-scale language model training

For more information or to apply, please go to https://careersearch.stanford.edu/jobs/research-engineer-213...

dlwh

7 tahun yang laludiscuss

I highly recommend the book 1491, which goes pretty deeply into what we can infer about the pre-Columbian Americas.

dlwh

8 tahun yang laludiscuss

Thanks for the questions.

I am very unlikely to take on visualization. I don't acutely need it for what I do, and I am some-but-not-nearly-enough interested in visualization for its own sake. I started to read about the grammar of graphics stuff at one point and decided it was too far down the rabbit hole.

I have looked more into gpu stuff, and agree specifying a compute graph (and then implicitly optimizing it) is more likely to be the future. FWIW, this is basically what XLA (from TensorFlow) and whatever it was FB announced on Friday are doing.

I wrote my thoughts up recently on the Breeze mailing list here: https://groups.google.com/forum/#!topic/scala-breeze/_hEFpnI...

I'm starting to think it through but I'm not sure I have time for that either :(. A 4-month old and a startup take up a lot of time.

dlwh

8 tahun yang laludiscuss

Thanks for the kind words.

Breeze does a large chunk of (dense) compute via netlib-java, which calls out to "real" lapack if you set it up. Are things really faster than that? Or are you referring to the non BLAS/non Lapack things?

dlwh

8 tahun yang laludiscuss

Main author here.

Breeze has breeze-viz, which is very basic but at the time there wasn't anything else. I highly endorse using something else. I personally like http://sameersingh.org/scalaplot/

They're under the same aegis basically because they're all mine. ScalaNLP started out as really being just NLP, but it scope-crept. That said, Epic is a library for structured prediction first and foremost, and one of the main applications of structured prediction is NLP.

Breeze is basically like SciPy and large chunks of it power Epic. It's really the only thing that doesn't belong in the namespace.

dlwh

8 tahun yang laludiscuss

So, this is (largely) my project.

Not sure why this is on the front page of HN, but I'm happy to answer any questions.

I'm not really giving these libraries the love they need these days. I mostly started them in grad school before the deep learning revolution really hit my subfield (NLP), and I haven't had time to modernize them. They still have their uses, especially Breeze, which is used in Spark's MLLib and directly by a number of companies.

dlwh

9 tahun yang laludiscuss

Sure, but the FP side of Scala is definitely firmly in the advanced static type system camp.

dlwh

9 tahun yang laludiscuss

Semantic Machines | Software Engineers and Machine Learning Engineers | Boston, MA and SF Bay Area, CA (Berkeley) | http://www.semanticmachines.com/careers/

Semantic Machines is developing technology to power the next generation of conversational artificial intelligence: AIs that you can actually have a conversation with. Think Google Assistant or Alexa or Siri, but without having to carefully craft commands like you're talking to a Bash shell.

Our team has built much of the core technology underlying Siri and Google Now, and our founders (including both the former Chief Speech Scientist for Siri and the head of UC Berkeley's Natural Language Processing group) have multiple >$100 million exits under their belt.

We're looking to hire a few talented software engineers and machine learning engineers to help build out technology, by expanding our core NLP infrastructure, data processing pipelines, neural net clusters, and backend services.

Experience with natural language processing systems is a plus, as is experience with the JVM (especially Scala), but we're mainly interested in passionate engineers who can learn quickly and work effectively in complex systems.

Please reach out to me directly or email [email protected]. Thanks!

dlwh

11 tahun yang laludiscuss

Not a neuroscientist, but I do do NLP, and I only lightly skimmed the paper.

This doesn't really speak to UG.

First, you can believe in the structures they purport to show without accepting the existence of UG, by appealing to the existence of general mechanisms in the brain for assembling hierarchical structures, which is equally validated by this experiment.

Second, they looked at two languages with sentences of up to ~7 syllables each with at most two constituents (Noun Phrase Verb Phrase). You can't show any evidence for any hierarchy of interest in 7 syllables. They demonstrated that phrases exist and phrase boundaries exist, but it's entirely possible to have "flat' grammars without interesting hierarchy, especially in simple sentences. If they want to show interesting hierarchy, they should conduct experiments with more interesting structure (say, some internal PPs and some limited center embedding) and show something that correlates with multiple levels of the "stack" getting popped, or something.

It's still interesting work, but as usual oversold by the university press office.

dlwh

11 tahun yang laludiscuss

Prosopagnosia[1] is a real condition that affects ≈2.5% of people. You might check out "The Man Who Mistook His Wife for a Hat" if you're interested.

[1] https://en.wikipedia.org/wiki/Prosopagnosia

dlwh

11 tahun yang laludiscuss

to add another:

    System.out.println(0.0 == -0.0); // true
    System.out.println(java.util.Arrays.equals(new double[] {0.0}, new double[] {-0.0})); // false

(It's documented in the contract for Arrays.equals, but still kind of ridiculous.)

dlwh

11 tahun yang laludiscuss

statically checked, but still runtime-determined, matrix sizes can be done in Scala. Doing specialized implementations would require bytecode magic, though.

  case class Dim(size: Int)

  trait Matrix[X <: Dim with Singleton, Y <: Dim with Singleton] {
    def *[Z <: Dim with Singleton](other: Matrix[Y, Z]):Matrix[X, Z] = ???
  }

  object Matrix {
    def apply[X <: Dim with Singleton, Y <: Dim with Singleton](x: X, y: Y):Matrix[X,Y] = ???
  }

  val x = Dim(100)
  val y = Dim(readFromFile(...))
  val z = Dim(37)

  val A = Matrix[x.type, y.type](x, y)
  val B = Matrix[y.type, z.type](y, z)

  A * A // compile error
  A * B // ok

dlwh

12 tahun yang laludiscuss

(I'm one of the authors of Overmind.)

First, it's important to know that AI is a very broad term. Roughly, there's "modern" AI (machine learning and statistical modeling) and "classical" AI (mostly logic and search). Game AI (as I understand it) falls more into the "classical" AI camp, and classes these days are focusing more and more on modern non-game techniques.

kaybe's link is a good starter class for modern AI. Norvig quite literally (co)wrote the text book on AI, and Sebastian Thrun. The PI on Overmind (Dan Klein) co-taught a class on edX (https://courses.edx.org/courses/BerkeleyX/CS188.1x/2012_Fall...) that you might also want to check out. They're all based on the same underlying book, just different focuses.

Overmind itself is a collection of some reasonably sophisticated ML/modern AI, classical AI (which is more common in games), and a lot of horrible hacks. Really, tons of them.

If you want, I can try to dig up the lecture slides for the "advanced" AI class we taught once that specifically focused on the kinds of techniques one would want to write StarCraft bot AIs.

dlwh

12 tahun yang laludiscuss

Machine translation is no where near solved.

Machine translation is still one of the most funded areas in NLP, and the quality is still incredibly bad for many language pairs. DARPA has been running one machine translation grant program or another for more than a decade.

It is true that some of the best systems take a long time to translate sentences (on the order of 1 cpu-minute or more). But their quality is still not anywhere near human level for most languages.

Submissions

The Unreasonable Effectiveness of Fuzzing for Porting Programs

comments