Focus on automating human labour as much as possible. Problems in the world all trace back to lack of resources. A rich man can afford to be 'nice to the environment', a poor man can't.
The below all are easier machine learning problems than self-driving cars, yet no big tech companies or national initiatives are focused on aggressively applying machine learning to them.
Likely a couple of billion dollars, a year, and a 100 people lab would 'solve' each specific problem.
1. Robot that cooks meals, and clean the dishes afterwards. That saves billions of hours daily.
2. Robotic self-cleaning toilets. Saves another billion hours daily.
3. Robots that can dig up dirt and build a house from that dirt.
4. An app that can teach anyone anything like a teacher would - literally - a talking avatar and cameras and voice output and machine learning powered dialogue.
5. Home manufacturing 'box' that can make 95% of anything that anyone typically wants (some arrangment of 3d printer/laser cutters/pcb placement/wood router machines etc, that can take plastics/wood/metal/electronic components and output a gadget/furniture etc)
The above 5 give the equivalent of a 'basic income' for everyone (if distributed to everyone, and assuming the finished gadget is about the size and complexity of an automobile).
Then the inputs/ouputs problem of energy/raw materials/waste needs to be provided.
Disregarding scientific advances like fusion power etc (which require more than 100 billion maybe, or not possible), a drone distrubution platform for getting the energy / matter (input/waste) handled.
To do this (as above, 100 people, a year, 1 billion dollars)
6. p2p aerial surveillance system for air traffic managment of millins of drones. Basically, a sky pointing camera gadget that analyzes and broadcasts what it sees and process. Millions/billions of these camera gadgets airdropped every few 100 meters .
7. a drone that can carry 100 kilos and drop ship materails/waste p2p using the p2p air traffic control. The drones are battery operated with range a coupel of kilometers.
8. a drone that can mid-air 'refuel' the above drones. Basically a flying battery that recharges that larger cargo drones.
Summary - 'gadgetize' every problem (it becomes a self contained mechano-electrical desktop/fridge size thing that a team of 100 people can rapidly iterate on) and throw machine learning at it at heavily as possible. Seek to eliminate human labour as fast as possible.
1 rich person saving his money so his 3 children never have to work is bad for the economy.
It's better the 1 rich person is forced to consume that in his lifetime. By doing so that money is redistributed back to the economy, and his 3 kids are also productive.
In scenario 1 only 1 out 4 people work. In scenario 2, 4 out of 4 people work.
Also - it's hypocritical for 'pull yourself by your bootstraps' for children born poor, and 'guaranteed basic income' for children born rich. That is more dehumanizing than not being allowed to pass off inheritance.
About 10 million cars are produced in the USA every year.
An RV/Trailer type thing is like a car - so quite easily you could imagine producing that many of these 'tiny houses' every year with car style factories.
About 600k people are homeless in the USA. That's about equal to the number of RV's produced every year.
So that solves 'physical' homelessness quite quickly if there was an effort to. (that disregards that some homeless choose to homeless out of mental illness etc)
'I like they way my lawn is and i don't like construction noises'
vs
'I can't get a job because it's too expensive to rent or buy where the jobs are, so i live in poverty'
(or i can't start family because i can only afford a room-share, or any number of other effects from NIMBYism)
Something is wrong with that argument. Welfare + Taxation = Basic Income.
Already we have it, by definition - it's an equation.
So some part of your calculation is wrong.
Likely you're over-inflating the amount paid out (it's not going to by 15/hr person) and undercounting payments (maybe disabilty/housing/local programs).
UBI given at equivalent levels to today's welfare would be strictly cheaper because of the reduced adminstration cost.
(this though is different from the question of whether payments should be higher and paid by increased taxation.
And off-topic, but taxation doesn't need to be increased - it can be payed by helicopter money, but actually not cause inflation, because of geographic mobility deflating that aspect of the economy).
Removing the needs assessment also removes the geographic location requirement, and that allows to deflate the land value inflation that is the real cause of inequality today.
SAT, SMT, constraint programming lie on a continuum. SMT is SAT plus machinery from constraint programming. I'm not an expert on this, just info from this video. According to him constraint programming will win in the long run because of constraint programming has larger structures which can be exploited with global constraints (SAT is only a flat sequence).
- hashing followed by dot product in transformer you said
- you were doing dot products at each layer to introduce non-linearity in transformer (and neural nets in general). Polynomials are already non-linear, so you don't need that. Transformer and vw -interact are polynomials. Maybe the feedforward layers and skip connections are not actually needed.
- 12 layers ? vw -interact xxxxxxxxxxxxx is 12 layers. You need a lot of memory for that, but in principle vw interact can do any number of them
These results are coming from google and their massive compute resources. If they ran vw with -interact x^13 they might get similar results.
We're really talking about polynomial approximation here, both transformer and vw used in this way. And that is in theory able to approximate any continuous function (just like neural networks).
What is reformer?
Transformer where LSH is applied.
One type of LSH is SimHash. ngrams of strings, followed by 32 bit hash.
Vowpal Wabbit -n flag for ngrams.
vw -interact xxx -n2 -n3 and you get ngrams + 32 bit hash doing SGD over a vector.
This vector is equivalent to a 2 layer reformer.
Non-linear activation is not needed because polynomials are already nonlinear.
So vw + interact + ngrams (almost)= reformer encoder. (if reformer uses SimHash, then they are identical).
Transformer/Reformer have an advantage, the encoder-decoder can learn from unlabeled data.
However, you can get similar results from unlabeled data using preprocessing such as introducing noise to the data, and then treating it as noise/non-noise binary classification. (it can even be thought of as reinforcement learning, with the 0-1 labels as the reward using vw's contextual bandits functionality. This can then do what GAN's do - climb from noise to perfection).
Locality sensitive hashing is a way to put similar vectors into the same buckets - by omission etc. It does this by hashing, but the intent is to approximate nearest neigbours.
skipgram/ngrams turn features into other features by omission etc, and so makes similar things the same. The hashing trick then reduces memory usage.
So yes, you're right the hashing in locality sensitive hashing is different in intent, but my point is, that both these approaches are designed to be more memory and compute efficient.
And vowpal's feature interactions give you transformer layers.
Add up all these together, and they have about the same net effect.
They are doing the same thing - using less memory by hashing.
The hashing trick in VW hashes multiple same words into one integer, not the same as reformer, but similar to how reformer puts similar vectors together.
With VW's ngram/skipgram features, you get the same kind of effect - similar strings hash into the same hash.
So locality sensitive hashing = (is around about the same thing as) ngram/skipgram on strings plus hashing trick.
Vowpal Wabbit has been doing this 'hashing trick' since the 200s.
It also the feature interaction, which are the same thing as a layer in transformers (all against all matrix).
So it seems like they are still catching up to where John Langford and crew were over a decade ago.
And, the vowpal wabbit approach is extremely fast to train because it's only doing stochastic gradient descent on a linear function - linear regression. Transformers are much slower to train.
EDIT: Downvoters, please see my last leaf to see why they're effectively the same. The guy responding here seems unfamiliar with all the functionality of vowpal wabbit.
Random forests split the features This splits the outcomes.
So each tree in RF only looks at a few features. In this, each model looks at all the features.
RF can handle multiclass problems of tens to hundreds (maybe thousands). This MACH algo can handle multiclass problems of millions/billions (extreme classification).
The markets will sort it out though. That filtered out candidate has some inherent worth and companies can take advantage of that. Competent hirers will then always use multiple AI hirers and experiment with hiring to make sure they're not losing out on talent that's been inaccurately filtered out.
The below all are easier machine learning problems than self-driving cars, yet no big tech companies or national initiatives are focused on aggressively applying machine learning to them.
Likely a couple of billion dollars, a year, and a 100 people lab would 'solve' each specific problem.
1. Robot that cooks meals, and clean the dishes afterwards. That saves billions of hours daily.
2. Robotic self-cleaning toilets. Saves another billion hours daily.
3. Robots that can dig up dirt and build a house from that dirt.
4. An app that can teach anyone anything like a teacher would - literally - a talking avatar and cameras and voice output and machine learning powered dialogue.
5. Home manufacturing 'box' that can make 95% of anything that anyone typically wants (some arrangment of 3d printer/laser cutters/pcb placement/wood router machines etc, that can take plastics/wood/metal/electronic components and output a gadget/furniture etc)
The above 5 give the equivalent of a 'basic income' for everyone (if distributed to everyone, and assuming the finished gadget is about the size and complexity of an automobile).
Then the inputs/ouputs problem of energy/raw materials/waste needs to be provided. Disregarding scientific advances like fusion power etc (which require more than 100 billion maybe, or not possible), a drone distrubution platform for getting the energy / matter (input/waste) handled. To do this (as above, 100 people, a year, 1 billion dollars)
6. p2p aerial surveillance system for air traffic managment of millins of drones. Basically, a sky pointing camera gadget that analyzes and broadcasts what it sees and process. Millions/billions of these camera gadgets airdropped every few 100 meters .
7. a drone that can carry 100 kilos and drop ship materails/waste p2p using the p2p air traffic control. The drones are battery operated with range a coupel of kilometers.
8. a drone that can mid-air 'refuel' the above drones. Basically a flying battery that recharges that larger cargo drones.
Summary - 'gadgetize' every problem (it becomes a self contained mechano-electrical desktop/fridge size thing that a team of 100 people can rapidly iterate on) and throw machine learning at it at heavily as possible. Seek to eliminate human labour as fast as possible.