Gay Talese starts "The Kidnapping of Joe Bonanno" (Esquire, August 1971) with this:
Knowing that it is possible to see too much, most doormen in New York have
developed an extraordinary sense of selective vision: they know what to see
and what to ignore, when to be curious and when to be indolent—they are most
often standing indoors, unaware, when there are accidents or arguments in
front of their buildings, and they are usually in the street seeking taxicabs
when burglars are escaping through the lobby. Although a doorman may
disapprove of bribery and adultery, his back is invariably turned when the
superintendent is handing money to the fire inspector or when a tenant whose
wife is away escorts a young woman into the elevator—which is not to accuse
the doorman of hypocrisy or cowardice but merely to suggest that his instinct
for uninvolvement is very strong, and to speculate that doormen have perhaps
learned through experience that nothing is to be gained by serving as a
material witness to life’s unseemly sights or to the madness of the city. This
being so, it was not surprising that on the night when the reputed Mafia
chief, Joseph Bonanno, was grabbed by two gunmen in front of a luxury
apartment house on Park Avenue near Thirty-sixth Street, shortly after
midnight on a rainy Tuesday in October, the doorman was standing in the lobby
talking to the elevator man and saw nothing.
Paul Strassmann wrote a book in 1990 called "Business Value of Computers" that showed that it matters where money on computers is spent. Only firms that spent it on their core business processes showed increased revenues whereas the ones that spent it on peripheral business processes didn't.
For vector generation I started using Meta-LLama-3-8B in april 2024 with Python and Transformers for each text chunk on an RTX-A6000. Wow that thing was fast but noisy and also burns 500W. So a year ago I switched to an M1 Ultra and only had to replace Transformers with Apple's MLX python library. Approximately the same speed but less heat and noise. The Llama model has 4k dimensions so at fp16 thats 8 kilobyte per chunk, which I store in a BLOB column in SQLite via numpy.save(). Between running on the RTX and M1 there is a very small difference in vector output but not enough for me to change retrieval results, regenerate the vectors or change to another LLM.
For retrieval I load all the vectors from the SQlite database into a numpy.array and hand it to FAISS. Faiss-gpu was impressively fast on the RTX6000 and faiss-cpu is slower on the M1 Ultra but still fast enough for my purposes (I'm firing a few queries per day, not per minute). For 5 million chunks memory usage is around 40 GB which both fit into the A6000 and easily fits into the 128GB of the M1 Ultra. It works, I'm happy.
Knowing that it is possible to see too much, most doormen in New York have developed an extraordinary sense of selective vision: they know what to see and what to ignore, when to be curious and when to be indolent—they are most often standing indoors, unaware, when there are accidents or arguments in front of their buildings, and they are usually in the street seeking taxicabs when burglars are escaping through the lobby. Although a doorman may disapprove of bribery and adultery, his back is invariably turned when the superintendent is handing money to the fire inspector or when a tenant whose wife is away escorts a young woman into the elevator—which is not to accuse the doorman of hypocrisy or cowardice but merely to suggest that his instinct for uninvolvement is very strong, and to speculate that doormen have perhaps learned through experience that nothing is to be gained by serving as a material witness to life’s unseemly sights or to the madness of the city. This being so, it was not surprising that on the night when the reputed Mafia chief, Joseph Bonanno, was grabbed by two gunmen in front of a luxury apartment house on Park Avenue near Thirty-sixth Street, shortly after midnight on a rainy Tuesday in October, the doorman was standing in the lobby talking to the elevator man and saw nothing.