In concept each 16-digit chromosome would work as the nearest neighbors, and finding “a range” of the nearest neighbors would be very fast.
In a data scientist’s perspective, this may seem to be a hack. I would be very appreciated to learn more from you about this. I can tell you that the weakness on this method is the multiple writes to the database. I assume vector database can gets this implemented with less writes.
In the past, I have used this for supervised training and yielded very good results. However, I think this would be inefficient in large scale networks. I am planning to use Go + Clickhouse to improve the performance in the next project.
Thank you for the reply. Most of my repos are currently in private, so it's difficult for me to point you to the code I wrote. However, I can list a simple example here.
Genetic algorithm usually deals with chromosomes. I have used binary and hexadecimal chromosomes in the past and found that the binary chromosomes are more flexible, especially with bitwise operations.
Let's say a chromosome with four chunks of 4-digit binary, with each digit as dimension of true/false value; we end up with something like 1011 1100 0010 0101. Then I stored these four chunks in four documents in a NoSQL database. Each document then also has the records of other 16-digit chromosomes, so that I can refer to those 16-digit chromosomes contain an exact/similar 4-digit chunk. This was the fastest method I could come up with the last time I worked on it; I am sure that there are more efficient methods out there.
Hopefully, this can shed some light on how the genetic algorithm works.
Having written some genetic algorithms and stored the chromosomes, which can be a representation of vectors in any type of database, I find this interesting and will some of them a try.
Please correct me if I am wrong, with genetic algorithm, DL, etc. The vector embedding happen on the software layer (inside the network). I imagine it would be somewhat tricky to separate the vector embedding and put it inside a vector database. It could also be a performance hit during the training process.
Perhaps he meant overhyped; I am just guessing. The creator of GPT-3 also said that the model is more hyped than it deserves because it's simply looking at the existing data to forecast the next words.
Hype or not, there are some questions that deserve some serious consideration. It is hard to quantify how "personalized" the content GPT-3 spits out actually is. It may output the same content on every N-th try on average. If so, we have copyright issues at hand.
Two people using GPT-3 who happened to feed it similar parameters and got similar results can sue each other for copywrite infringement. On the other hand, an author who genuinely created an article can sue someone using GPT-3 created content without that person knowing he has infringed on the author's copyright, and vice versa.
It's all wonderful on paper. In practice, a whole host of issues can pop up when using this indiscriminately.