I was considering paying someone to build something like this at some point. With two jobs, I eventually had no time to even organize what I find. It's just piles of links in text files.
Can I give your software a huge list of URL's to index? Or do I need to use browser automation to open them a few at a time with it caching and indexing them?
Cloud companies were made to sell others compute. Now, one is buying billions of compute from SpaceX, a rocket company. That sounds so backwards lol.
Great work by Musk and his companies to be in a position to sell billions to cloud vendors. I'd have probably missed that opportunity while trying to build great rockets or AI models.
People learning Python or searching for it will run into endless answers using PIP. Then, lots of advice on how to work around PIP's problems. Then, multiple alternatives they have to consider. I only recently started using UV after going through all that.
Packaging, concurrency, and type errors had me strongly considering switching to Go or Rust recently. These are such long-solved problems in other languages that I question why we should put up with it in Python. Then, I remember it was the ecosytem, including job market and AI performance, that made me use Python.
So, maybe a Python/Rust combo... There's the extensions the OP article mentioned and a Python interpreter written in Rust.
I'd normally call that a low-effort, troll comment. But, thinking on it, you may have a great metaphor.
They keep promising great performance out of models whose key ingredient (parameters) they are diluting. Many seem to be in a competition saying they're getting smaller and higher performance at the same time. Then, the homeopathic models don't perform as well as real models when independently tested. Again, spot on.
Good overview except for the last part. I've heard multiple things from people of the time:
1. In "If A1 was the answer, what was the question," thr author pointed out that features and assurance levels were mandated together. Buyers often didn't need specific features which made it more costly and slow to develop for nothing. The festures the market demanded weren't present. So, TCSEC-certified, high security was unmarketable.
2. In a similar vein, Lipner's "Ethics of Perfectiom" talked about how it took two to three quarters to make a significant change to the VAX Security Kernel. The market was wanting major features every quarter. They couldn't afford to lag behind all the competition in velocity.
3. Another person mentioned changes in DOD (other government?) purchasing policy to order COTS products from many vendors. Those vendors were also sometimes paying campaign contributions or hiring ex-Pentagon people to be favored. Their products weren't TCSEC A1. So, corruption and supplier diversity both forced government agencies to use insecure products which made secure products less competitive.
4. Similarly, the NSA started pushing lower-assurance like CC EAL4 and later Commercial Solutions for Classified. They were also selling GOTS gear guaranteed to get their approval. In these ways, they caused a surge of low-assurance competition with high-assurance vendors.
5. They promoted, required expensive certs for, and basically killed the Seperation Kernel Protection Profile. Spending millions on something that ultinately didn't matter to them doesn't inspire more EAL6+ certifications.
If I understand them correctly, they're saying to use standard, optimization methods after writing a fitness or evaluation function to score your possible solutions. Which is a normal, non-SAT way of doing optimization.
So, you could use it for any application you saw benefit from genetic algorithms, simulated annealing, or tabu search. You can even use those to optimize neural networks without backpropagation and with fewer, local optima. Many papers on this but it's computationally heavier.
They come in mobs or waves, too. Historically, they show up first while the reasonable people show up later, sometimes reversing those votes. Others here have noted that phenomenon.
Is that how the U.S. and U.K. economies grew to be significant? Mostly foreign investment for cheap labor with hardly any innovation or entrepreneurs of their own?
It can be worse in terms of justice. You might be able to charge or win in court against a street hustler. Most people can't beat a big company in court. They usually won't even try.
What needs to happen is for companies (or individuals) tired of that to pool money together to build new, memory products. Then, sell them to consumers first and for non-AI use. If not that, then round-robin scheduling of quantities so the units are spread around more.
If costs are high, they might reserve a certain percentage for big business at market prices (or just under) to cover the chip's mask costs.
After DDR5+ RAM, then GDDR5-6 RAM for use with AI accelerators. They might try to jump right in on a HBM alternative. That could be the percentage for AI buyers I just mentioned. Especially if they could put 40-80GB on accelerators like Intel ARC's.
If successful enough, they license MIPS' gaming GPU's to combine with this stuff with full, open-source stack and RTOS support for military sales.
With those results, I would've already done that in any models I got to train. There's also the principle that the LLM's are often better at what they saw last in their training set. That also justifies putting more logic, code, and math in at the end for an analytical or coding model. So, a few precedents for that technique already.
Can I give your software a huge list of URL's to index? Or do I need to use browser automation to open them a few at a time with it caching and indexing them?