dlwh · HackerTrans

dlwh

no profile record

Submissions

The Unreasonable Effectiveness of Fuzzing for Porting Programs

25 pointsby dlwhgeçen yıl4 comments

comments

3 yıl öncediscuss

Stanford CRFM | Bay Area | Full-Time | On-Site, Hybrid

Foundation models like ChatGPT, PaLM, and Stable Diffusion are transforming the world around us. The Stanford Center for Research on Foundation Models (CRFM; https://crfm.stanford.edu/), which is part of Stanford HAI, is an interdisciplinary initiative that aims to make foundation models more reliable, transparent and accessible to the world. We take on ambitious projects that seek to rigorously evaluate existing foundation models and to build new ones.

We are currently seeking a research engineer to join our engineering team. This is an unique opportunity to work with seasoned engineers who have spent many years in industry as well as PhD students, post-docs, and faculty at CRFM. You will contribute to cutting-edge research, publish papers, gain access to the latest foundation models, and be immersed in the vibrant CRFM community.

You will work on our open source software projects, including:

  - HELM, a framework for holistic evaluation of large language models 
  - Levanter, a framework for transparent and accessible large-scale language model training

For more information or to apply, please go to https://careersearch.stanford.edu/jobs/research-engineer-213...

7 yıl öncediscuss

I highly recommend the book 1491, which goes pretty deeply into what we can infer about the pre-Columbian Americas.

8 yıl öncediscuss

Thanks for the questions.

I am very unlikely to take on visualization. I don't acutely need it for what I do, and I am some-but-not-nearly-enough interested in visualization for its own sake. I started to read about the grammar of graphics stuff at one point and decided it was too far down the rabbit hole.

I have looked more into gpu stuff, and agree specifying a compute graph (and then implicitly optimizing it) is more likely to be the future. FWIW, this is basically what XLA (from TensorFlow) and whatever it was FB announced on Friday are doing.

I wrote my thoughts up recently on the Breeze mailing list here: https://groups.google.com/forum/#!topic/scala-breeze/_hEFpnI...

I'm starting to think it through but I'm not sure I have time for that either :(. A 4-month old and a startup take up a lot of time.

8 yıl öncediscuss

Thanks for the kind words.

Breeze does a large chunk of (dense) compute via netlib-java, which calls out to "real" lapack if you set it up. Are things really faster than that? Or are you referring to the non BLAS/non Lapack things?

8 yıl öncediscuss

Main author here.

Breeze has breeze-viz, which is very basic but at the time there wasn't anything else. I highly endorse using something else. I personally like http://sameersingh.org/scalaplot/

They're under the same aegis basically because they're all mine. ScalaNLP started out as really being just NLP, but it scope-crept. That said, Epic is a library for structured prediction first and foremost, and one of the main applications of structured prediction is NLP.

Breeze is basically like SciPy and large chunks of it power Epic. It's really the only thing that doesn't belong in the namespace.

8 yıl öncediscuss

So, this is (largely) my project.

Not sure why this is on the front page of HN, but I'm happy to answer any questions.

I'm not really giving these libraries the love they need these days. I mostly started them in grad school before the deep learning revolution really hit my subfield (NLP), and I haven't had time to modernize them. They still have their uses, especially Breeze, which is used in Spark's MLLib and directly by a number of companies.

9 yıl öncediscuss

Sure, but the FP side of Scala is definitely firmly in the advanced static type system camp.

9 yıl öncediscuss

Semantic Machines | Software Engineers and Machine Learning Engineers | Boston, MA and SF Bay Area, CA (Berkeley) | http://www.semanticmachines.com/careers/

Semantic Machines is developing technology to power the next generation of conversational artificial intelligence: AIs that you can actually have a conversation with. Think Google Assistant or Alexa or Siri, but without having to carefully craft commands like you're talking to a Bash shell.

Our team has built much of the core technology underlying Siri and Google Now, and our founders (including both the former Chief Speech Scientist for Siri and the head of UC Berkeley's Natural Language Processing group) have multiple >$100 million exits under their belt.

We're looking to hire a few talented software engineers and machine learning engineers to help build out technology, by expanding our core NLP infrastructure, data processing pipelines, neural net clusters, and backend services.

Experience with natural language processing systems is a plus, as is experience with the JVM (especially Scala), but we're mainly interested in passionate engineers who can learn quickly and work effectively in complex systems.

Please reach out to me directly or email [email protected]. Thanks!

11 yıl öncediscuss

Not a neuroscientist, but I do do NLP, and I only lightly skimmed the paper.

This doesn't really speak to UG.

First, you can believe in the structures they purport to show without accepting the existence of UG, by appealing to the existence of general mechanisms in the brain for assembling hierarchical structures, which is equally validated by this experiment.

Second, they looked at two languages with sentences of up to ~7 syllables each with at most two constituents (Noun Phrase Verb Phrase). You can't show any evidence for any hierarchy of interest in 7 syllables. They demonstrated that phrases exist and phrase boundaries exist, but it's entirely possible to have "flat' grammars without interesting hierarchy, especially in simple sentences. If they want to show interesting hierarchy, they should conduct experiments with more interesting structure (say, some internal PPs and some limited center embedding) and show something that correlates with multiple levels of the "stack" getting popped, or something.

It's still interesting work, but as usual oversold by the university press office.

11 yıl öncediscuss

Prosopagnosia[1] is a real condition that affects ≈2.5% of people. You might check out "The Man Who Mistook His Wife for a Hat" if you're interested.

[1] https://en.wikipedia.org/wiki/Prosopagnosia

11 yıl öncediscuss

to add another:

    System.out.println(0.0 == -0.0); // true
    System.out.println(java.util.Arrays.equals(new double[] {0.0}, new double[] {-0.0})); // false

(It's documented in the contract for Arrays.equals, but still kind of ridiculous.)

11 yıl öncediscuss

statically checked, but still runtime-determined, matrix sizes can be done in Scala. Doing specialized implementations would require bytecode magic, though.

  case class Dim(size: Int)

  trait Matrix[X <: Dim with Singleton, Y <: Dim with Singleton] {
    def *[Z <: Dim with Singleton](other: Matrix[Y, Z]):Matrix[X, Z] = ???
  }

  object Matrix {
    def apply[X <: Dim with Singleton, Y <: Dim with Singleton](x: X, y: Y):Matrix[X,Y] = ???
  }

  val x = Dim(100)
  val y = Dim(readFromFile(...))
  val z = Dim(37)

  val A = Matrix[x.type, y.type](x, y)
  val B = Matrix[y.type, z.type](y, z)

  A * A // compile error
  A * B // ok

12 yıl öncediscuss

(I'm one of the authors of Overmind.)

First, it's important to know that AI is a very broad term. Roughly, there's "modern" AI (machine learning and statistical modeling) and "classical" AI (mostly logic and search). Game AI (as I understand it) falls more into the "classical" AI camp, and classes these days are focusing more and more on modern non-game techniques.

kaybe's link is a good starter class for modern AI. Norvig quite literally (co)wrote the text book on AI, and Sebastian Thrun. The PI on Overmind (Dan Klein) co-taught a class on edX (https://courses.edx.org/courses/BerkeleyX/CS188.1x/2012_Fall...) that you might also want to check out. They're all based on the same underlying book, just different focuses.

Overmind itself is a collection of some reasonably sophisticated ML/modern AI, classical AI (which is more common in games), and a lot of horrible hacks. Really, tons of them.

If you want, I can try to dig up the lecture slides for the "advanced" AI class we taught once that specifically focused on the kinds of techniques one would want to write StarCraft bot AIs.

12 yıl öncediscuss

Machine translation is no where near solved.

Machine translation is still one of the most funded areas in NLP, and the quality is still incredibly bad for many language pairs. DARPA has been running one machine translation grant program or another for more than a decade.

It is true that some of the best systems take a long time to translate sentences (on the order of 1 cpu-minute or more). But their quality is still not anywhere near human level for most languages.

12 yıl öncediscuss

This article is basically nonsense. The "Deep" here doesn't mean "deep learning"/neural networks. They just mean non-superficial understanding of text (beyond keywords, named entity recognition, and the like.)

For example, the Illinois NLP group has a grant under this project, and their pubs are largely not using "deep learning." http://cogcomp.cs.illinois.edu/page/funding_view/43

13 yıl öncediscuss

> Yes it does. AFAICT it's working well. There are 5 releases in the 2.10.x line and I haven't had a compatibility issue between any of them.

This isn't quite true. I have a library (Breeze) that works with 2.10.3 but not 2.10.2. Something in name mangling changed, I think.

13 yıl öncediscuss

Make the prior art fee scale up by number of patent applications submitted by that entity, then. First one's free. Then $100, then start growing at some small-ish exponent.

13 yıl öncediscuss

Actually, NNs (specifically auto-encoders) are more like a nonlinear PCA. PCA can be seen as an "information bottleneck" where you try to represent an N dimensional space as a K < N dimensional space such that your ability to reconstruct the original space is maximized (that is, the "reconstruction error" is minimized.)

Autoencoders are precisely the same idea, except that you use nonlinear transformations (e.g. softmax or tanh) in addition to linear transformations (matrix multiplications). You can also use multiple layers in an autoencoder, just like any neural net. (The linearness of PCA makes multiple layers useless. You can always replace multiple layers with a single layer.)