In normal NCA cells are pixels and they can perceive their neighboring pixels (cells can't move). In NPA cells are particles and they can perceive all particles in a support radius around them and these particles can move freely. Does this answer your question?
Indeed! The system has good regeneration capabilities but it certainly has limits.
The particles can only grow reliably if they start from the egg-like initial condition. If we switch the rules mid rollout, we would get a messed up morphology.
Thanks! Yeah I think it should be possible though it requires making the cell division/splitting a differentiable operation. But nontheless, this is indeed a very interesting and promising direction to pursue.
Texture sampling retrieves pixels by coordinate, while NCA grows them from local rules with no global lookup. The weights are actually ~3× smaller than JPEG-compressed texture maps, so it's not just memorizing the image either.
The more interesting part is that it's a dynamic process, you can damage the pattern while it forms and it repairs itself.
I think performance is not the only issue for scaling to larger grids. CUDA Convolution implementation already utilizes coalescing to improve performance.
The main bottleneck is that in larger grids, cells are further apart, and it takes more steps for them to be able to communicate.
The input to the NN is just the 3x3 neighborhood around a cell.
We can overlap two NNs on the same grid (through interpolation). Checkout https://meshnca.github.io
to see the effect. When the brush is in graft mode, it basically allows you to paint some regions that will follow a different NN rule.
If you're familiar with CAs (e.g. Conway's Game of Life), you can think of a NeuralCA as a CA where the update rule is given by a neural network. Here we optimize the neural net weights so that it behaves a certain way (e.g. grow a lizard from a single seed).
The NeuralCA both generates and maintains the pattern. Because the NCA was not exposed to damage or erasure during training, its regeneration capability is a purely emergent phenomenon. However, this ability remains somewhat brittle, particularly when the central regions of the pattern are erased.