Limitations of DALL-E(twitter.com)
twitter.com
Limitations of DALL-E
https://twitter.com/benjamin_hilton/status/1520032772072607747
109 comments
[deleted]
It’s kind of crazy to think that Alex-net was just ~10 years ago, and look at what DALL-E can do now. I can’t imagine what the ML landscape is going to look like in another 10-20 years. Excited to be working in the field to experience it all first hand.
Is it too late to join?
Now is the perfect time to join. There's tons of open chat-rooms filled with people excited about the space. Computing resources are basically thrown at you. I'm not an ML guy, but I find the terminology and explanations very approachable.
If you want to get started I can recommend the EleutherAI[1] discord. It seems to have a good mix of experience and excitement.
[1]: https://www.eleuther.ai/
If you want to get started I can recommend the EleutherAI[1] discord. It seems to have a good mix of experience and excitement.
[1]: https://www.eleuther.ai/
> Now is the perfect time to join. There's tons of open chat-rooms filled with people excited about the space. Computing resources are basically thrown at you.
Any pointers on where one has to stand in order to be targeted by said computing resources?
Any pointers on where one has to stand in order to be targeted by said computing resources?
Hmm, I’d feel like a johnny come lately
Better today than tomorrow.
You only have one life
I still think it's pointless to join. Most of the innovation comes from big companies with infinite resources.
I disagree. There is a ton of low hanging fruit in applying SOTA ML techniques to niche problems. It just requires a bit of business sense, but there is a lot of money to be made there. More than what I was making as someone doing ML research at Google Research.
So what?
I too wouldn't know how to draw anything related to "ophanim" unaided by a dictionary.
I'd also argue that its ability to come up with symbols that look oddly like letters but aren't actual letters could be a form of creativity.
Don't get me wrong, I'm not a GPT-3/DALL-E fanboy but I don't get why this is interesting. Anyone who has worked with a creative designer/artist/architect/etc. would know that there are so many "wrong answers" when it comes to creating visual imagery. Some of these aren't limitations of DALL-E but rather misinterpretations of our own expectations.
I too wouldn't know how to draw anything related to "ophanim" unaided by a dictionary.
I'd also argue that its ability to come up with symbols that look oddly like letters but aren't actual letters could be a form of creativity.
Don't get me wrong, I'm not a GPT-3/DALL-E fanboy but I don't get why this is interesting. Anyone who has worked with a creative designer/artist/architect/etc. would know that there are so many "wrong answers" when it comes to creating visual imagery. Some of these aren't limitations of DALL-E but rather misinterpretations of our own expectations.
Failure modes often tell you more about a technology or phenomenon than successes do.
The marketing and promotion will, of course, focus on successes. Failures suggest boundaries, limits, and weaknesses.
There's a similar case for humans. Studies of optical and auditory illusions, cognitive biases, and various forms of pathological behaviours, both individual and group, would be examples of this.
Or any equipment or system. The flight envelope of an aircraft is literally the boundary beyond which its behaviour is unpredictable and/or uncontrollable. For software, limits define the extent to which applications or AI algorithms produce useful results.
How much those limits can be pushed is itself an interesting question. The observations of failures of negation (a map of the world without Europe, a bowl of fruit without apples, a man not running) suggest that DALL-E has something of a "now don't think of an elephant" problem. It's also a problem that's been observed in search contexts --- I am recalling a case where Google and/or Amazon repeatedly failed when requested to find shirts without stripes, consistently presenting striped shirts.
Perhaps semantic negation is hard?
"If you please ... draw me a sheep."
http://www.supercoloring.com/sites/default/files/styles/colo...
The marketing and promotion will, of course, focus on successes. Failures suggest boundaries, limits, and weaknesses.
There's a similar case for humans. Studies of optical and auditory illusions, cognitive biases, and various forms of pathological behaviours, both individual and group, would be examples of this.
Or any equipment or system. The flight envelope of an aircraft is literally the boundary beyond which its behaviour is unpredictable and/or uncontrollable. For software, limits define the extent to which applications or AI algorithms produce useful results.
How much those limits can be pushed is itself an interesting question. The observations of failures of negation (a map of the world without Europe, a bowl of fruit without apples, a man not running) suggest that DALL-E has something of a "now don't think of an elephant" problem. It's also a problem that's been observed in search contexts --- I am recalling a case where Google and/or Amazon repeatedly failed when requested to find shirts without stripes, consistently presenting striped shirts.
Perhaps semantic negation is hard?
"If you please ... draw me a sheep."
http://www.supercoloring.com/sites/default/files/styles/colo...
>> Perhaps semantic negation is hard?
More to the point, it doesn't do semantics at all, only syntax, if you will. It has learned correlations between images and words but it has no way to map either the words or the images to their meanings. All it knows is that this word appears often in correlation with that image. So when it sees the word, it gives you the image.
What is the word "no" correlated with? Probably, absolutely anything. Most likely "no" is as probable to appear as the description of an image with apples as it is to appear as the description of an image without apples. But "apples" is likely to describe an image with apples, even if it has "no" in it. For example "An image of apples in a basket with no handle" has "no" and "apples". So- you get apples, "no" or not.
In a sense, nobody has told it that "no" means "no".
(Also, it doesn't know that "apples" means "apples". "Apples" is only the label of some sets of pixels in a space. If you scrambled the labels so that "apples" was correlated with images of elephants... "an image with no apples" would get you an image with elephants).
More to the point, it doesn't do semantics at all, only syntax, if you will. It has learned correlations between images and words but it has no way to map either the words or the images to their meanings. All it knows is that this word appears often in correlation with that image. So when it sees the word, it gives you the image.
What is the word "no" correlated with? Probably, absolutely anything. Most likely "no" is as probable to appear as the description of an image with apples as it is to appear as the description of an image without apples. But "apples" is likely to describe an image with apples, even if it has "no" in it. For example "An image of apples in a basket with no handle" has "no" and "apples". So- you get apples, "no" or not.
In a sense, nobody has told it that "no" means "no".
(Also, it doesn't know that "apples" means "apples". "Apples" is only the label of some sets of pixels in a space. If you scrambled the labels so that "apples" was correlated with images of elephants... "an image with no apples" would get you an image with elephants).
Not even syntax so much as vocabulary and associations.
"No Means No" would make a great title for some future paper on AI NLP making significant progress on this problem.
"No Means No" would make a great title for some future paper on AI NLP making significant progress on this problem.
One thing I don't like about this article is that it's showing singular examples of Dall-E's failures, when Dall-E really produces 8 possible answers per text prompt (and more if you iterate further). You can cherry pick the worst of the batch and write an article about how bad it is just as, I suppose, you could write an article about the best of each batch and how good it is. However, the fact that multiple drafts are created is similar to the way humans work when doing creative work. So I think it's more instructive to look at the successes. I'm sure if we chose any genius in human history and analyzed their life's work, we'd find tons of failure and false starts and bad attempts. But we don't let those failures detract from their later success. However, in analyzing AI, you can quickly jump to that kind of "it has to be perfect or its fake" analysis and I think it's a flawed way of looking at things.
Okay, so the author had to iterate on a couple of examples. But look at the speed. Most of these examples would take a human a day to a week of labor, for each one. The fact you can rattle off almost any random text prompt like "draw me a walrus in the style of pointillism with a fake looking mustache" and that it can accommodate such requests in one second is absolutely stunning.
Okay, so the author had to iterate on a couple of examples. But look at the speed. Most of these examples would take a human a day to a week of labor, for each one. The fact you can rattle off almost any random text prompt like "draw me a walrus in the style of pointillism with a fake looking mustache" and that it can accommodate such requests in one second is absolutely stunning.
DALL-E is really impressive, but many believe it shows we're making progress towards AGI (the explicit end goal of Open AI), or at least it's displaying some degree of "understanding" (whatever that means).
If I ask for a red box on top of a blue box on top of a yellow box, and the model gives me an image with two boxes side-by-side (albeit a very high quality one), it's hard to believe there's much "understanding" of (a) box, (b) colour, (c) relative positioning. Likewise if I ask for six penguins and it gives me four penguins, can we confidently say there's any "understanding" of what a penguin is?
Again, that's not to say it's not mind-blowing technology/progress in machine learning. It most certainly is. But I'm not ready to give up on every other angle and go all-in on large diffusion models as the path to AGI.
If I ask for a red box on top of a blue box on top of a yellow box, and the model gives me an image with two boxes side-by-side (albeit a very high quality one), it's hard to believe there's much "understanding" of (a) box, (b) colour, (c) relative positioning. Likewise if I ask for six penguins and it gives me four penguins, can we confidently say there's any "understanding" of what a penguin is?
Again, that's not to say it's not mind-blowing technology/progress in machine learning. It most certainly is. But I'm not ready to give up on every other angle and go all-in on large diffusion models as the path to AGI.
>Likewise if I ask for six penguins and it gives me four penguins, can we confidently say there's any "understanding" of what a penguin is?
What if you ask it to draw some Nazi paraphernalia and 3 sims?
https://www.pcgamer.com/bizarre-russian-propaganda-links-sim...
https://twitter.com/b_nishanov/status/1518637055596769282
What if you ask it to draw some Nazi paraphernalia and 3 sims?
https://www.pcgamer.com/bizarre-russian-propaganda-links-sim...
https://twitter.com/b_nishanov/status/1518637055596769282
> If I ask for a red box on top of a blue box
The model has to encode those words into a fixed length vector, an architectural design decision, not an inherent requirement. For example translation systems can translate such phrases perfectly, but they don't pass the representation through a single vector bottleneck.
The model has to encode those words into a fixed length vector, an architectural design decision, not an inherent requirement. For example translation systems can translate such phrases perfectly, but they don't pass the representation through a single vector bottleneck.
Does the fact that sometimes graphic artists misinterpret commission requests imply that they are not conscious? People misunderstand each other all the time, and then play an interrogative game to try to synchronize their minds. That is, they ask questions.
DALL-E right now doesn't have the facility to ask questions, but iterating on the request statement is a bit of a proxy for it. I'd argue that this is exactly what understanding does in fact look like.
That isn't to say that DALL-E is AGI or conscious or anything like that. It's just to say that the mere presence of occasional misunderstanding is not a marker of un-intelligence. It's precisely when you are able to access the richness of symbolic reasoning that misinterpretation is at greatest risk, e.g. your calculator never misinterprets you.
DALL-E right now doesn't have the facility to ask questions, but iterating on the request statement is a bit of a proxy for it. I'd argue that this is exactly what understanding does in fact look like.
That isn't to say that DALL-E is AGI or conscious or anything like that. It's just to say that the mere presence of occasional misunderstanding is not a marker of un-intelligence. It's precisely when you are able to access the richness of symbolic reasoning that misinterpretation is at greatest risk, e.g. your calculator never misinterprets you.
what question does DALL-E ask to clear up the difference between "draw four penguins" and "draw six penguins"? how many iterations does it need to get "small red box on top of larger yellow box"?
DALL-E is very good at matching words to pictures and combining the results in novel ways. It isn't intelligence and it isn't art, and never will be. It's a tool that humans can use to accomplish tasks. That has its own merit.
DALL-E is very good at matching words to pictures and combining the results in novel ways. It isn't intelligence and it isn't art, and never will be. It's a tool that humans can use to accomplish tasks. That has its own merit.
Not intelligent - maybe. Not art? How do you conclude that? The output looks like art to me.
It would have been cool if there were a few tweets with the caveat of "tho here's what DALL-E is good at". Maybe the twitter account has showcased that separately.
otherwise so what if DALL-E has limitations on some weird obscure prompts. As I am hearing about DALL-E the first time, I am still impressed with the images in this tweet thread.
otherwise so what if DALL-E has limitations on some weird obscure prompts. As I am hearing about DALL-E the first time, I am still impressed with the images in this tweet thread.
When something is new and mysterious, knowing what it is not is just as interesting as knowing what it is.
True, but the "blue cube over the red cube" limitation is caused by a deliberate choice to maximise other abilities of the model at the expense of exact word for word attribute memory. The reason is that CLIP is encoding the text to a simple fixed length vector, the more text you put in the more gets ignored.
Such failures are interesting, in the physicist's sense.
That it struggles with composition sometimes but not other times might indicate model has learned a particular distribution factorization which is good much of the time but makes it unable to properly account for conditional dependencies in certain queries. The issue of losing track of lots of details may also be related.
Its struggles with negation is also fascinating because that has been a thorn since the early days of GOFAI. Negation is trickier and more involved than what one might naively assume.
For those taking relief from such weaknesses, it's worth remembering that this is as bad as it is going to be and probably not for long. Model size is surprisingly small when compared to recent behemoths. That it can already do all that it does is certainly cause for concern.
That it struggles with composition sometimes but not other times might indicate model has learned a particular distribution factorization which is good much of the time but makes it unable to properly account for conditional dependencies in certain queries. The issue of losing track of lots of details may also be related.
Its struggles with negation is also fascinating because that has been a thorn since the early days of GOFAI. Negation is trickier and more involved than what one might naively assume.
For those taking relief from such weaknesses, it's worth remembering that this is as bad as it is going to be and probably not for long. Model size is surprisingly small when compared to recent behemoths. That it can already do all that it does is certainly cause for concern.
>> Its struggles with negation is also fascinating because that has been a thorn since the early days of GOFAI.
Well, ish. Negation (specifically, negation-as-failure, i.e. failure to prove) in logic programming introduces non-monotonicity, but reasoning remains sound and even completeness is guaranteed under conditions.
John McCarthy (father of AI and also LISP) was very enthusiastic about nonmonotonic reasoning because he considered it the obvious way to represent common sense which was, for him, the one thing missing from AI systems. In plain words, nonmonotonic reasoning means that you can change your mind as new information becomes available.
But, you have a point that DALL-E's (and other similar systems') inability to deal with negation is reminiscent of older issues with expressing negation. In particular, it reminds me of datalog, a subset of Prolog that does not allow negation as failure. The motivation for that is to guarantee termination. This guarantee comes at the cost of completeness, so there are programs that cannot be expressed in datalog (but can in Prolog).
There might be something similar going on with DALL-E, or maybe it's just a simpler case of not being able to derive negation from correlations, as I point out in another comment.
Well, ish. Negation (specifically, negation-as-failure, i.e. failure to prove) in logic programming introduces non-monotonicity, but reasoning remains sound and even completeness is guaranteed under conditions.
John McCarthy (father of AI and also LISP) was very enthusiastic about nonmonotonic reasoning because he considered it the obvious way to represent common sense which was, for him, the one thing missing from AI systems. In plain words, nonmonotonic reasoning means that you can change your mind as new information becomes available.
But, you have a point that DALL-E's (and other similar systems') inability to deal with negation is reminiscent of older issues with expressing negation. In particular, it reminds me of datalog, a subset of Prolog that does not allow negation as failure. The motivation for that is to guarantee termination. This guarantee comes at the cost of completeness, so there are programs that cannot be expressed in datalog (but can in Prolog).
There might be something similar going on with DALL-E, or maybe it's just a simpler case of not being able to derive negation from correlations, as I point out in another comment.
Natural language has some major complexities that people don't realize. One of the most surprising one is that plurality does not means more than one.
This reminds me of when I used to put a sentence or two through Google translate and then put it through Google translate again with a different language and then paste it on my Tumblr.
It's not actually anything. It's just computer-generated nonsense that kinda looks like something.
It's not actually anything. It's just computer-generated nonsense that kinda looks like something.
>So what?
DALL-E 2 is very impressive. So impressive that many people have wondered if all commercial artists will be out of a job as soon as DALL-E 2 is made publicly available. These limitations form a strong argument that DALL-E 2 is not ready to completely replace human artists just yet. This is particularly relevant if you're an artist, or if you're in the market for one.
DALL-E 2 is very impressive. So impressive that many people have wondered if all commercial artists will be out of a job as soon as DALL-E 2 is made publicly available. These limitations form a strong argument that DALL-E 2 is not ready to completely replace human artists just yet. This is particularly relevant if you're an artist, or if you're in the market for one.
It's a word I use every day. The term is used in the Jewish daily prayerbook, in the prayers before the "shema" among other places:
https://www.sefaria.org/Siddur_Ashkenaz%2C_Weekday%2C_Shacha...
https://www.sefaria.org/Siddur_Ashkenaz%2C_Weekday%2C_Shacha...
I think some limitations are interesting in terms of tempering expectations and understanding how DALL-E "thinks" or works in order to best utilize its strengths.
Of course, I would always expect limitations based on having to "understand" anything beyond word-image associations, like floor plans or solvable mazes, which have meaning well beyond the visual. Those limitations are indeed uninteresting. Might as well try "An image proving the Collatz Conjecture"
Of course, I would always expect limitations based on having to "understand" anything beyond word-image associations, like floor plans or solvable mazes, which have meaning well beyond the visual. Those limitations are indeed uninteresting. Might as well try "An image proving the Collatz Conjecture"
It's interesting how some of them are described as limitations. I can't understand the designer photograph prompt either, and not just because of the "operated" part. The mazes look great to me. And the million apples look like what I'd expect from a photo called "a million apples".
If it learns from humans and produces results like humans would... What's the definition of "limitation" here?
If it learns from humans and produces results like humans would... What's the definition of "limitation" here?
If it learns from humans and produces results like humans would.
No, humans can understand "A red cube, on top of a yellow cube, to the left of a green cube" and draw it.
Dall-E gives you nonsense -- look at the picture.
It fundamentally doesn't understand things that humans do. That is limitation.
Of course it might be great overall and do things that humans can't, but it still has a limitation.
Gary Marcus has been hammering this home at least since 2018, probably earlier. It's more of a memory/association/recall machine than a machine that understands, and understands logic in particular.
No, humans can understand "A red cube, on top of a yellow cube, to the left of a green cube" and draw it.
Dall-E gives you nonsense -- look at the picture.
It fundamentally doesn't understand things that humans do. That is limitation.
Of course it might be great overall and do things that humans can't, but it still has a limitation.
Gary Marcus has been hammering this home at least since 2018, probably earlier. It's more of a memory/association/recall machine than a machine that understands, and understands logic in particular.
This seems tractable though.
GPT-3 can complete this prompt:
GPT-3 can complete this prompt:
clarify this scene description:
A bowl of fruit containing no apples
The bowl contains bananas, oranges, and grapes.
> "how some of them are described as limitations"
[deleted]
I agree it doesn't understand. I believe we will have an AGI when you ask it to draw:
"seven red lines, all of them perpendicular, some in green ink, some transparent, in the shape of a kitten"
Then it will respond:
"FU, it cannot be drawn!"
"seven red lines, all of them perpendicular, some in green ink, some transparent, in the shape of a kitten"
Then it will respond:
"FU, it cannot be drawn!"
Or do something like this: https://www.youtube.com/watch?v=B7MIJP90biM
Now I want someone to feed this classic prompt to DALL-E
The maze that looks more or less like a maze at first glance is full of disconnected sections that can't be reached by each other. It's missed a really essential part of being a maze. (The hedge maze doesn't look like a hedge maze at all: I'm kind of surprised that DALL-E isn't able to figure out that consistent width of the passages is a part of hedge mazes).
The designer photograph one is complicated and I'm not surprised that DALL-E misunderstand it.
But you're cherry-picking for the apples. It didn't consistently give an picture of five apples when asked for five apples. When asked for a bowl of fruit with no apples, it gives a bowl of only apples. That's obviously strongly sub-human comprehension.
The designer photograph one is complicated and I'm not surprised that DALL-E misunderstand it.
But you're cherry-picking for the apples. It didn't consistently give an picture of five apples when asked for five apples. When asked for a bowl of fruit with no apples, it gives a bowl of only apples. That's obviously strongly sub-human comprehension.
Are you familiar with the empirical results that have been observed when you ask a human to 'draw a bicycle'?
https://www.booooooom.com/2016/05/09/bicycles-built-based-on...
People draw bicycle-like shapes, that don't quite work - they miss important details, or connect things that shouldn't be connected.
https://www.booooooom.com/2016/05/09/bicycles-built-based-on...
People draw bicycle-like shapes, that don't quite work - they miss important details, or connect things that shouldn't be connected.
This is from memory. Humans wouldn't make this (serious level) of mistakes if they had an enormous data-set of bicycles to draw from.
That’s an odd distinction to make. DALL-E is a trained model, everything it produces is ‘from memory’. It doesn’t go out and google for reference material based on the prompt.
> It doesn’t go out and google for reference material based on the prompt.
And note that even if it did (which is definitely architecturally possible given progress in retrieval and demos like WebGPT), it still wouldn't necessarily be 'copying', in the goalpost-moving excuse.
Facebook has a neat diffusion model which uses retrieval on a dataset of images for generation: https://arxiv.org/abs/2204.02849#facebook You can see the exemplars it uses: they both greatly improve the quality of generated samples, but also don't look that much like the generated sample.
And note that even if it did (which is definitely architecturally possible given progress in retrieval and demos like WebGPT), it still wouldn't necessarily be 'copying', in the goalpost-moving excuse.
Facebook has a neat diffusion model which uses retrieval on a dataset of images for generation: https://arxiv.org/abs/2204.02849#facebook You can see the exemplars it uses: they both greatly improve the quality of generated samples, but also don't look that much like the generated sample.
Yes, but it was still was trained on a large dataset for the purpose of drawing. It's not an AGI that just happened to be asked to draw bicycles.
Maybe a better comparison would be a trained artist, who likely wouldn't make such egregious mistakes.
Maybe a better comparison would be a trained artist, who likely wouldn't make such egregious mistakes.
Humans do, because they have spent a lifetime looking at and riding bikes. They draw from memory just as much as DALL-E 2 does.
This is a funny comparison to draw, but I don't think it's one with profound meaning: humans have trouble drawing bicycles, but the semantics of a badly-drawn bicycle are sound. This is in contrast to DALL-E, which might do an excellent job of drawing apples but otherwise misses the semantics of the query (i.e., "without apples").
In other words: humans mis-draw things because we don't always do a good job of visually apprehending the world; DALL-E mis-draws things because it doesn't really understand the problem statement.
In other words: humans mis-draw things because we don't always do a good job of visually apprehending the world; DALL-E mis-draws things because it doesn't really understand the problem statement.
The semantics aren't that sound - a lot of the bikes have the seat at the very back, too far from the handlebars.
None of them are good, but the semantics are correct: the seat is behind the handlebars, the handlebars curve correctly, &c. The fact that humans tend to draw the pieces too far apart is more a testament to our problems drawing scale and perspective than a semantic error.
(I think this would be apparent if participants were asked to draw a bicycle, followed by a bicycle without a wheel. The two drawings would be roughly as "good" in terms of quality, but the humans will correctly obey the prompt and omit one of the two wheels. I suspect DALL-E would struggle with that prompt.)
(I think this would be apparent if participants were asked to draw a bicycle, followed by a bicycle without a wheel. The two drawings would be roughly as "good" in terms of quality, but the humans will correctly obey the prompt and omit one of the two wheels. I suspect DALL-E would struggle with that prompt.)
I totally picked the examples - some are definitely deficiencies, but I wanted to concentrate on the debatable ones.
A counterpoint to the maze - it depends what we expect. If you wanted a maze designed, it would be of course a bad example. But if you wanted one painted, it's... good? If it was in a gallery, it could easily be a commentary on life with limitations, dead ends and problems we can't solve.
A counterpoint to the maze - it depends what we expect. If you wanted a maze designed, it would be of course a bad example. But if you wanted one painted, it's... good? If it was in a gallery, it could easily be a commentary on life with limitations, dead ends and problems we can't solve.
I’d argue an unsolvable maze is art only if the artist can conceive of and understands the metaphor. Or can at least comprehend that they are doing mazes with no solution.
The computer has no understanding of what it’s doing. And I don’t think you can argue that it’s interrupting a brief for a commission. Is it even producing at???
The computer has no understanding of what it’s doing. And I don’t think you can argue that it’s interrupting a brief for a commission. Is it even producing at???
I think it is the person perceiving that transforms the work into art. If they can perceive the metaphor, why does it matter if the artist cannot?
That could be true. But it’s a bit awkward no?
A human needs to be there at the beginning to either dream up a concept and then describe it in painful detail to the computer.
Or a human needs to be there at the end to sift through the results to pick out the ones that actually match some concept.
In trying to think of a useful application for this: if an artist can lock down the style of the output, and then pump in a script and further description to story board things or get near final output, that could be quite useful.
A human needs to be there at the beginning to either dream up a concept and then describe it in painful detail to the computer.
Or a human needs to be there at the end to sift through the results to pick out the ones that actually match some concept.
In trying to think of a useful application for this: if an artist can lock down the style of the output, and then pump in a script and further description to story board things or get near final output, that could be quite useful.
Great roundup. AI are great at technical optimization problems (for example, reducing the usage of electricity in a data center). For creative and subjective works, it seems like a machine-human hybrid model will be the way to go for the next decade. The AI spits out some rough drafts, then the human artist picks it up from there. All of these last-mile problems would be trivial for an artist to touch up.
I suspect that we'll see a big wave of Large Transformer models getting built into high cost professional tools.
Need to write some random background NPC dialogue for a game?
Need to generate a draft Cad design for "Airframe supporting a large turbojet engine with X newtons thrust"?
These are problems where the high cost of inference won't be significant, and there is an expert available to revise and discard the result if its not working.
I'd be curious if there is a good startup model for anything in this space, seems like you would already need to be a vendor of the underlying technical software to make a viable business (CAD,Game Engines,Creative Tools etc.)
Need to write some random background NPC dialogue for a game?
Need to generate a draft Cad design for "Airframe supporting a large turbojet engine with X newtons thrust"?
These are problems where the high cost of inference won't be significant, and there is an expert available to revise and discard the result if its not working.
I'd be curious if there is a good startup model for anything in this space, seems like you would already need to be a vendor of the underlying technical software to make a viable business (CAD,Game Engines,Creative Tools etc.)
Sounds about right based on my experience with GitHub copilot. It does make me fear a little bit. I mean I’m not experienced with cad but I could see you might train on a particular cad program, that program may have undiscovered bug X that allows a model to pass all the synthetic tests while optimizing for something like cost (which goes undiscovered because no human would ever design in such a way)… the reviewing engineers rubber stamp under pressure and then bam, 9/10 airframes fall apart after 1,000 miles. That’s the copilot analogue at least.
Yes, and that could inspire a massive creative explosion.
One application I'm pining for: I'd love Apple to introduce AI into Logic Pro X (LPX) such that it could take a 4, 6, 8 or 16 bar set of tracks and generate infinite variations, from which I can pick and choose ones to embellish.
Or, similarly, I have innumerable LPX projects that are just a single idea that, in the hands of someone more capable (or with more motivation) than myself, could be turned into full pieces. I'd love to be able to give LPX 2 or more of these projects and it generates transitional pieces between them.
All LPX upgrades are free, but I'd happily pay for this feature.
One application I'm pining for: I'd love Apple to introduce AI into Logic Pro X (LPX) such that it could take a 4, 6, 8 or 16 bar set of tracks and generate infinite variations, from which I can pick and choose ones to embellish.
Or, similarly, I have innumerable LPX projects that are just a single idea that, in the hands of someone more capable (or with more motivation) than myself, could be turned into full pieces. I'd love to be able to give LPX 2 or more of these projects and it generates transitional pieces between them.
All LPX upgrades are free, but I'd happily pay for this feature.
Some of the proposed models for human thought can't handle negation. An example of this is map-based models. Theories which propose human thought is a form of a production system (as in logic) get a boost from the fact they can explain negation as well as other things. So to see DALL-E can't handle negation is, to my thinking, an important difference between the way it "thinks" (or what passes for that) and how we do.
I strongly suspect the inability to do negation comes from the fact that the training dataset contains virtually no images tagged with negated descriptors. It wouldn't surprise me to see it correctly handle negations that might be more common in image tags (e.g. "a car not running" might be a sufficiently common phrase for "broken down car").
Seems straightforward to check something like LAION-400M's captions to see how often negations show up.
If it's simply a lack of data, that should be easy to fix. Given an image caption, every other caption is a relevant negation: 'a photo of a happy dog' is also not 'a photo of a sad cat', and can be turned into the caption 'a photo of a happy dog and not a photo of a sad cat'. So lots of obvious data augmentation and synthetic data approaches to apply there to boost negation.
If it's simply a lack of data, that should be easy to fix. Given an image caption, every other caption is a relevant negation: 'a photo of a happy dog' is also not 'a photo of a sad cat', and can be turned into the caption 'a photo of a happy dog and not a photo of a sad cat'. So lots of obvious data augmentation and synthetic data approaches to apply there to boost negation.
Its training data might not have negations in it - I'm not sure anyone has tried to explain what CLIP understands of text.
Besides that, we know it doesn't because models don't recurse because no matter how big they are, they still have constant execution times. And we gave up on systematically parsing text in favor of just vibing anyway (technical term).
That's just as well since most parsers didn't really work in practice; I think I vaguely remember one as kind of working (https://www.link.cs.cmu.edu/link/) but it can't handle bad grammar or misspellings like ML can.
Besides that, we know it doesn't because models don't recurse because no matter how big they are, they still have constant execution times. And we gave up on systematically parsing text in favor of just vibing anyway (technical term).
That's just as well since most parsers didn't really work in practice; I think I vaguely remember one as kind of working (https://www.link.cs.cmu.edu/link/) but it can't handle bad grammar or misspellings like ML can.
That's not true, the autoregressive LMs (GPTs) can generate variable length outputs allowing for step by step computation. Recent papers get them to generate the full chain of thought reasoning process to the solution (also called rationales in some papers), improving accuracy.
But this version of Dall-E is not autoregressive, they've gone for diffusion models in the generator and for a single vector embedding (no internal spatial grid structure) for the encoder. That explains why it's bad at negations, counting and stacking coloured blocks. I believe these skills were not the top priority of OpenAI this time, they understandably went for pretty images.
But this version of Dall-E is not autoregressive, they've gone for diffusion models in the generator and for a single vector embedding (no internal spatial grid structure) for the encoder. That explains why it's bad at negations, counting and stacking coloured blocks. I believe these skills were not the top priority of OpenAI this time, they understandably went for pretty images.
I agree GPT as a whole generates variable length outputs, but the model isn't generating them in a single pass, is it? I thought there was some sampling process implemented in traditional code outside the model that made it do that, and it wasn't learned. Have forgotten the details.
If Dall-E took variable length sequences as inputs, its execution time would still be O(N) on the length of the sequence and it would still have a fixed amount of memory to work with. Whereas a computer program of length N can do pretty much anything, and a human given a description can go off and think about it.
If Dall-E took variable length sequences as inputs, its execution time would still be O(N) on the length of the sequence and it would still have a fixed amount of memory to work with. Whereas a computer program of length N can do pretty much anything, and a human given a description can go off and think about it.
> and how we do
https://langcog.stanford.edu/papers/NF-cogsci2013.pdf
Fun fact: Young Children also don't understand negation. There are some parenting theories claiming that is why one should use avoid negations in commands.
https://langcog.stanford.edu/papers/NF-cogsci2013.pdf
Fun fact: Young Children also don't understand negation. There are some parenting theories claiming that is why one should use avoid negations in commands.
I don't get it? Most of the "limitation" examples are on par with humans, both linguistically and artistically.
So much so that it feels like a DALL-E ad :)
So much so that it feels like a DALL-E ad :)
...you know a lot of people who can't count past four? Who don't understand the concept of negation? Who don't understand what a modern supercart aisle looks like? Who lose track of elements in a description after four or five? Who can't get that when you ask for a "monk and a skeleton," you don't mean that the monk is also a skeleton? Who literally don't know what letters are?
A lot of people can't subitize past four. Counting is another matter: it's a symbolic, deliberate, step-by-step activity.
Arguably most neural networks don't do symbolic, step-by-step reasoning "natively", and maybe human beings don't either. But they can be told to, and then the same network becomes much smarter: https://arxiv.org/abs/2201.11903
Arguably most neural networks don't do symbolic, step-by-step reasoning "natively", and maybe human beings don't either. But they can be told to, and then the same network becomes much smarter: https://arxiv.org/abs/2201.11903
> Who don't understand the concept of negation?
I teach students of the first year of the university, and plenty of them have problems with reading comprehension, in particular to understand sentences with negations.
I teach students of the first year of the university, and plenty of them have problems with reading comprehension, in particular to understand sentences with negations.
This, pretty much. I sometimes feel we might be underestimating AI because most people on HN are well above average.
What comes to mind is all those cakes with bad inscriptions. First Google result: https://images.app.goo.gl/JZtBVCwmCFfcmV6F8
Yeah, a lot of people seem to have problems interpreting instructions for drawings.
Yeah, a lot of people seem to have problems interpreting instructions for drawings.
I would rather see a research into how much of "learned" material is it regurgitating ala Copilot verbatim Carmacks fast inverse square root.
Look, this criticism comes up every single time and has been addressed in many research papers already. Among other things they do nearest-trainingset-match searches for output to see how much is reproduced verbatim. It's generally very little. And the inverse square root thing is a very poor example because it involved deliberate prompt engineering. It's like saying humans are just memorizers because some human can recite a short poem after you explicitly asked them to by citing the first line.
If I had a nickel for every time someone asks this...
Perhaps we should investigate how many HN comments are regurgitated vs. novel?
If I had a nickel for every time someone asks this...
Perhaps we should investigate how many HN comments are regurgitated vs. novel?
That’s not too hard, just compare generated images to similar images from the training data.
It's pretty clear from the outputs that it's been heavily trained on stock photo libraries. Or maybe that's just the material it extracted the most info from in the training set. Stock image libraries have enormous numbers of high quality, high res images annotated with simple labels so they're ideal for this type of training task. I'm actually surprised nobody remarked on this yet because the resemblance is so strong.
E.g. look at what it outputs for welcome signs. It can do a great job for that. Many of the images have artistic use of focus, perspective and blur. That's because:
https://www.shutterstock.com/search/photo+of+a+welcome+sign
Likewise the "omg ai bias" examples are easily explainable as being due to training on stock photo libraries:
https://www.shutterstock.com/search/photo+of+a+nurse
Every single output they get for "photo of a <job>" looks exactly like what you'd get from stock photos.
This might also be why it struggles with counting.
https://www.shutterstock.com/search/an+apple
Easy.
https://www.shutterstock.com/search/5+apples
Hmmm ..... not so easy. The search is good: the results that actually have apples in them tend to have five, but, the actual labels don't include the counts.
Likewise with negation. How many images are annotated with "photo of a man NOT running"? My guess is, nearly none.
E.g. look at what it outputs for welcome signs. It can do a great job for that. Many of the images have artistic use of focus, perspective and blur. That's because:
https://www.shutterstock.com/search/photo+of+a+welcome+sign
Likewise the "omg ai bias" examples are easily explainable as being due to training on stock photo libraries:
https://www.shutterstock.com/search/photo+of+a+nurse
Every single output they get for "photo of a <job>" looks exactly like what you'd get from stock photos.
This might also be why it struggles with counting.
https://www.shutterstock.com/search/an+apple
Easy.
https://www.shutterstock.com/search/5+apples
Hmmm ..... not so easy. The search is good: the results that actually have apples in them tend to have five, but, the actual labels don't include the counts.
Likewise with negation. How many images are annotated with "photo of a man NOT running"? My guess is, nearly none.
Hmm my assumption is that they heavily curated the training images and captions with human labelers. I doubt for something like this they would just scrape stock images and use it directly.
These models need huge amounts of data. DALL-E was probably trained on the entire web like their other models (I didn't read the paper, maybe they say in there). At any rate the 'stock style' is pretty easy to spot.
The training data isn't available to the public.
Obviously. I meant this is what the researchers who created this model should do (if they haven’t already, I haven’t looked at the paper yet).
DALL-E is way better than me. Sure, I can count past four and I know what letters are, but I can’t draw a wizard koala as good as that. Also, there is criticism of how it draws faces, but it’s way better than I am.
Also when it comes to faces, that's intentional. Trying to generate realistic looking humans is explicitly forbidden, just like politics, violence, nudity and other stuff. Images matching those topics are likely filtered out of the training set (it's no public, so no idea what is actually in it).
It still has some ideas how humans look like, but that seems to come from classic paintings, not real world photos of humans.
It still has some ideas how humans look like, but that seems to come from classic paintings, not real world photos of humans.
You can see the future of search..
No longer is it a description of what currently exists, but an expression of what's been queried.
Which may demonstrate something entirely different.
No longer is it a description of what currently exists, but an expression of what's been queried.
Which may demonstrate something entirely different.
Yep, that's the truly impressive part with DALL-E2. If you take those sentences and paste them into Google Images, the results will always be much worse. Even if you find a query for which there do exist closely matching images, they will still look worse.
Meanwhile if you do it in reverse, take an exist image and try to recreate it in DALL-E2, you can get extremely closer. Over on Reddit they did a experiment:
https://old.reddit.com/r/dalle2/comments/ub0sfg/dalle_2_imit...
Meanwhile if you do it in reverse, take an exist image and try to recreate it in DALL-E2, you can get extremely closer. Over on Reddit they did a experiment:
https://old.reddit.com/r/dalle2/comments/ub0sfg/dalle_2_imit...
As someone who appreciates the craftsmanship and care that goes into human-created art, I found the abilities of DALL-E to be a little distressing. These limitations really point out how much DALL-E is still very different from humans, in a way that I find reassuring. Human reasoning still seems to be out of the reach of these kinds of models. What they really seem to be is very very good imitators which, after being trained on very large amounts of data, can imitate what they've seen. You could make an argument that that's essentially what humans do too, just in a more complex way. But I think it underscores why humans and machines will always be different. We'll always be trained on different inputs, and have our own unique way of combining those inputs, even as the abilities of AI rival and even surpass our own in many ways.
Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect. We probably won't hit upon the right way for a while, since large models are so expensive, but with some trickery on the architectural level and bigger models, they'll spit out better and better art until it's indistinguishable from human level. Most of your brain is doing the same kind of dumb learning these models are, just much better and faster. There are some fundamental architectural differences, especially relating to the higher-level functions we associate with "consciousness" (working and short term memory, attention, maybe the executive?), but for almost every task you stick with the big dumb blob of compute and it gets the job done better than your higher faculties ever could. Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away. Our biggest models are still a fraction of the size and a smaller fraction of the complexity of the bits of brain which perform the same function. I won't rule out anything until we see models of that scale.
>Honestly, I'd assume that pretty much any of these flaws are more fixable than you'd expect.
The first 90% of the project takes 90% of the time, the last 10% of the project takes the other 90% of the time.
I don't think it's wise to speculate at this point on what problems are "actually easy" or "actually hard", or how long it'll take to achieve certain milestones. I don't think that even the people at the forefront of this research have any way of knowing for sure.
The first 90% of the project takes 90% of the time, the last 10% of the project takes the other 90% of the time.
I don't think it's wise to speculate at this point on what problems are "actually easy" or "actually hard", or how long it'll take to achieve certain milestones. I don't think that even the people at the forefront of this research have any way of knowing for sure.
your projects take up 180% of the time?
To completly replace artists, the model would need to fully understand speech and have a good model of the world. The failure cases of the monk and the negation show pretty clearly that the model does something more akin to a keyword search rather than actual NLP. This alone will be a major problem for the last 10% - the more complex edge cases you introduce, the closer you need to be to AGI to handle them. This might come at some point, but the effort needed will skyrocket for less and less payoff, so I think we can safely assume that this will still take some time.
> Don't underestimate the extent to which your brain plows through difficulties by throwing neurons at the problem until it goes away.
We also mustn't underestimate the extent in which the human brain is a fuzzy ensemble of partial experts, and composes the first few salient answers provided rather than trying to find a "best" answer that would be too late. Or how often the human brain creates post hoc rationalizations simply to provide narrative continuity.
We also mustn't underestimate the extent in which the human brain is a fuzzy ensemble of partial experts, and composes the first few salient answers provided rather than trying to find a "best" answer that would be too late. Or how often the human brain creates post hoc rationalizations simply to provide narrative continuity.
An important caveat is that the model isn't actually limited to generating full images from a single prompt, even if this perhaps how the author is interacting with it. The model can also be used for editing existing images, so you would be able to stack prompts while choosing the best intermediate results.
It's very interesting, but for me, the fact that one has to resort to very complicated sentences, rare words or ambiguous words to show limitations (and even then, they're only near misses, not utter failures) is a testimony to how amazing DALL-E is.
> e.g. I had to try this one (“a medieval painting of a monk and a skeleton drinking together in a tavern”) a few times to get the monk to not be a skeleton! And he still looks a bit skeleton-y.
Well, the prompt is actually a bit vague - is it a monk sitting together with a skeleton, or is it a single entity that is both a monk and a skeleton? Seems like DALL-E tried to find some middle ground.
Well, the prompt is actually a bit vague - is it a monk sitting together with a skeleton, or is it a single entity that is both a monk and a skeleton? Seems like DALL-E tried to find some middle ground.
> Well, the prompt is actually a bit vague - is it a monk sitting together with a skeleton, or is it a single entity that is both a monk and a skeleton? Seems like DALL-E tried to find some middle ground.
My thought exactly. I'd be interested to know whether any attempts were made at explicit disambiguation (eg. "A monk sitting beside a skeleton", etc.).
My thought exactly. I'd be interested to know whether any attempts were made at explicit disambiguation (eg. "A monk sitting beside a skeleton", etc.).
is there a hosted version of dall-e that people can play with? free or paid.
thanks!
thanks!
Not specifically DALL-E, but you can play with MindsEye beta, which is quite easy to run, considering the complexity of all this stuff: https://multimodal.art/mindseye
It doesn't get close to the quality of the art pictures produced using DALL-E, but it's nice to play with.
It doesn't get close to the quality of the art pictures produced using DALL-E, but it's nice to play with.
thx will try!
No, it’s something we will keep hearing about and nobody will get to try it. OpenAI is anything but open. Looks like great tech, but it’s irrelevant if it’s totally inaccessible.
:). i see seemingly random people keep posting their experiments with it. i guess they’re all in the inner circle or something
All these posts about DALL-E are making me so excited to try it if I ever make it through the waitlist.
I wonder what Filip Pieknewski thinks of DALL-E
These aspects were much better in the prior version (GLIDE), which conditioned on text tokens instead of just a fixed length vector.