Ask HN: What's a promising area to work on in 2024 -ideally not crowded already?
42 comments
Do you have recommendations on good semi-advanced textbooks for fundamental AI research? I agree with you about this space... I would love to spend more time in it, even if it's just an hour or two per day, but nearly all the content out there is about deep neural networks and transformers.
I also have a sneaking suspicion this is what John Carmack is exploring.
I also have a sneaking suspicion this is what John Carmack is exploring.
The best books I would recommend are;
— The Master Algorithm by Pedro Domingos
— The Book of Why by Judea Pearl
— What Computers Still Can't Do by Hubert L. Dreyfus
— Society of Mind & The Emotion Machine by Marvin Minsky
— The Feeling Of What Happens by Antonio Damasio
— Language & Mind by Noam Chomsky
— A great mooc is also, the "Learning From Data" Telecourse from Caltech https://work.caltech.edu/telecourse
— The Master Algorithm by Pedro Domingos
— The Book of Why by Judea Pearl
— What Computers Still Can't Do by Hubert L. Dreyfus
— Society of Mind & The Emotion Machine by Marvin Minsky
— The Feeling Of What Happens by Antonio Damasio
— Language & Mind by Noam Chomsky
— A great mooc is also, the "Learning From Data" Telecourse from Caltech https://work.caltech.edu/telecourse
What do you mean by fundamental AI research? What kind of computational model?
What are some practical applications of complex systems?
What are some practical applications of complex systems?
> What do you mean by fundamental AI research?
Coming up with new/original models that solve the same exact problems that people are trying to solve with neural networks (computer vision, language translation, image generation etc) hopefully that are better performing & cheaper computationally.
> What kind of computational model?
I think answering that question would be what the researcher in this field is trying todo. I think a place to start would be exploring abandoned AI strategies like Genetics Algorithms & Markov Logic
>What are some practical applications of complex systems?
- Modeling Financial Markets [1]
- Drug discovery via Molecular Dynamics Simulations [2]
- Urban Planning [3]
- Modeling Epidemics [4]
- Cosmology
- Computer Graphics
- Much more ....
[1]: https://sites.santafe.edu/~wbarthur/elfarol.htm
[2]: https://youtu.be/PGqCeSjNuTY?si=uNP8o6DW1cSeAjo-
[3]: http://traffic.phys.cs.is.nagoya-u.ac.jp/~mstf/index_e.html
[4]: https://en.m.wikipedia.org/wiki/Yaneer_Bar-Yam
If interested, subscribe to these podcasts.
1: Simplifying Complexity: https://pca.st/podcast/1c42ee40-3735-013b-efb2-0acc26574db2
2: Complexity: https://pca.st/podcast/87eef2d0-cd12-0137-1e17-0acc26574db2
Coming up with new/original models that solve the same exact problems that people are trying to solve with neural networks (computer vision, language translation, image generation etc) hopefully that are better performing & cheaper computationally.
> What kind of computational model?
I think answering that question would be what the researcher in this field is trying todo. I think a place to start would be exploring abandoned AI strategies like Genetics Algorithms & Markov Logic
>What are some practical applications of complex systems?
- Modeling Financial Markets [1]
- Drug discovery via Molecular Dynamics Simulations [2]
- Urban Planning [3]
- Modeling Epidemics [4]
- Cosmology
- Computer Graphics
- Much more ....
[1]: https://sites.santafe.edu/~wbarthur/elfarol.htm
[2]: https://youtu.be/PGqCeSjNuTY?si=uNP8o6DW1cSeAjo-
[3]: http://traffic.phys.cs.is.nagoya-u.ac.jp/~mstf/index_e.html
[4]: https://en.m.wikipedia.org/wiki/Yaneer_Bar-Yam
If interested, subscribe to these podcasts.
1: Simplifying Complexity: https://pca.st/podcast/1c42ee40-3735-013b-efb2-0acc26574db2
2: Complexity: https://pca.st/podcast/87eef2d0-cd12-0137-1e17-0acc26574db2
What about you? Are you doing anything related to these fields? Why do you need the money to do work related to this?
> What about you? Are you doing anything related to these fields?
Yes. I have some ideas I would like to explore (currently premature speculation & not implemented or tested comprehensively) for;
1. Computationally efficient AI models for image classification (has nothing todo with neural networks). I believe the model also be generalized to other problems like regression problems Or generating music, art, text and even language translation. — But image classification is the basis of my research.
2. Some financial modeling techniques based on ideas in 1.
3. Thinking on how to build a good Iversonian language for mass adoption. (I have already thought of some important components must be built out)
> Why do you need the money to do work related to this?
It is very difficult to think about technical problems when you have to also concurrently think about financial problems (I am unemployed at the moment).
I want to create a "set-up" where I can almost exclusively think about these problems for a good amount of time without having to worry about my basic needs.
But I don't need millions. $150k can take me for at least a decade where I live.
Yes. I have some ideas I would like to explore (currently premature speculation & not implemented or tested comprehensively) for;
1. Computationally efficient AI models for image classification (has nothing todo with neural networks). I believe the model also be generalized to other problems like regression problems Or generating music, art, text and even language translation. — But image classification is the basis of my research.
2. Some financial modeling techniques based on ideas in 1.
3. Thinking on how to build a good Iversonian language for mass adoption. (I have already thought of some important components must be built out)
> Why do you need the money to do work related to this?
It is very difficult to think about technical problems when you have to also concurrently think about financial problems (I am unemployed at the moment).
I want to create a "set-up" where I can almost exclusively think about these problems for a good amount of time without having to worry about my basic needs.
But I don't need millions. $150k can take me for at least a decade where I live.
In the previous decade or so, everyone has been shifting to technological fields. Whether they are computer science or even things like marketing.
I think CS will be, if not already, very crowded.
I think it's psychology, particulary being a therapist. I know this is hard if you already graduate. But in the next few years unfortunately more people will get depressed and anxious. And they will need more therapist. This is because of social media and similar stuff (I think this topic has been discussed greatly. For more information see Social Dilemma movie.)
This will even increase greatly in the future, due to virtual reality. I think Apple Vision Pro will ruin society more and more.
I think CS will be, if not already, very crowded.
I think it's psychology, particulary being a therapist. I know this is hard if you already graduate. But in the next few years unfortunately more people will get depressed and anxious. And they will need more therapist. This is because of social media and similar stuff (I think this topic has been discussed greatly. For more information see Social Dilemma movie.)
This will even increase greatly in the future, due to virtual reality. I think Apple Vision Pro will ruin society more and more.
I like your take on this. I've been in awe of the explosive growth of software developer salaries, college CS departments, "learn to code" initiatives, etc. over the past 15 years and really wondering how long it can go on.
On the other side, my family has started consuming a lot of mental health care over the past couple of years, and it has not been easy to get in with therapists. The trend is for insurers to cover more of it. With that and teletherapy being the norm now, I think a lot more people are willing to sign up for more sessions, and the overhead of running a practice must be much lower (due to teletherapy, not dealing with insurers).
On the other side, my family has started consuming a lot of mental health care over the past couple of years, and it has not been easy to get in with therapists. The trend is for insurers to cover more of it. With that and teletherapy being the norm now, I think a lot more people are willing to sign up for more sessions, and the overhead of running a practice must be much lower (due to teletherapy, not dealing with insurers).
You're not wrong, but will people be able to pay for it?
Read up on capability based security, data diodes, and the Bell-LaPadula model. Computer "security" needs a serious upgrade, be one of the people making it happen.
Any architecture that could compete with LLMs is
going to reshape the ML landscape that is now
dominated by GPU-heavy algorithms like transformer attention.
Its likely there is something more fundamental that
could work without much training, like
finding exactly what training process changes and\
forming architecture around optimizing that.
I don't think people realize how much influence
'dumb stochastic parrots' have now, and the whole
progress in identifying what exactly happens inside
these networks is still a mystery: there hints
that by sheer size they forming structures capable
of cognition but only as side effect(finding out what
exactly is going there and replicating in simpler terms
would yield massive speed-ups).
So. Transformers work well due to scale, but fall short without it. Why this happens is not known?
[deleted]
With the advances in ML (not just LLMs, but mostly image generation/recognition), getting a cheap EEG machine to read your thoughts and visualize your dreams.
This was presented at NeurIPS 2023: Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity
https://arxiv.org/abs/2305.11675
https://arxiv.org/abs/2305.11675
Autonomous exploration & utilization of the world’s oceans
webxr
I heard this a few years ago and haven’t heard much. Care to share why you think it’ll grow?
JavaScript is going through another major renaissance, and I see a big future for it.
Traditionally in the last 20 years, Rule number 404 of development has always been “thou shall not use JavaScript on the server side”.
That rule has been flipped on its head in the last year or so, and I see a big future ahead for server side JavaScript.
Traditionally in the last 20 years, Rule number 404 of development has always been “thou shall not use JavaScript on the server side”.
That rule has been flipped on its head in the last year or so, and I see a big future ahead for server side JavaScript.
Last year? It was all in awe already in 2014 or so.
Server Components introduced in React 18
Well, I meant that nodejs exists since long ago...
Work time tracking tools to control developers. Companies would love to stop paying devs for scrolling social media, but so far no tool has gained anything close to widespread adoption. Developers tend to be able to argue themselves out of using them based on privacy concerns and sheer negotiating power.
Now that the market is more employer-friendly, could be a good time to introduce a killer surveillance app. Perhaps go around the privacy concerns by sending mouse movement patterns or anonymized keyboard use patterns instead of screenshots and use ML to detect working time?
Now that the market is more employer-friendly, could be a good time to introduce a killer surveillance app. Perhaps go around the privacy concerns by sending mouse movement patterns or anonymized keyboard use patterns instead of screenshots and use ML to detect working time?
It's interesting how in 2024, managers still believe that building software is simply the act of pushing random buttons on a keyboard while moving a mouse.
If the developer stops typing for a minute, or a few hours in order to use his brain to think about a hard problem, then the developer doesn't deserve to get paid because he's not actively using his computer?
If you really want to track developer hours accurately, you need to invent a microchip that will be implanted in the developer's brain.
This chip will track all electrical activity related to building your software, so that you get the best bang for the hourly rate you're paying the developer.
If the developer stops typing for a minute, or a few hours in order to use his brain to think about a hard problem, then the developer doesn't deserve to get paid because he's not actively using his computer?
If you really want to track developer hours accurately, you need to invent a microchip that will be implanted in the developer's brain.
This chip will track all electrical activity related to building your software, so that you get the best bang for the hourly rate you're paying the developer.
So I can get paid for all the time I spend thinking about the problem while I'm sleeping and in the shower? Sign me up!
Why not? If a solution suddenly pops up while you're walking in the woods, you don't want to get paid for it?
Is the developer getting paid for the software he's building, or for the number of key-presses recorded on his keyboard?
That's why paying developers an hourly rate doesn't make sense.
If you like hourly rates so much, then become a manager at a factory, and leave the tech industry to people who are aware of knowledge work.
Is the developer getting paid for the software he's building, or for the number of key-presses recorded on his keyboard?
That's why paying developers an hourly rate doesn't make sense.
If you like hourly rates so much, then become a manager at a factory, and leave the tech industry to people who are aware of knowledge work.
Almost all of my best work is done while I'm out taking a walk. I leave my desk and the problem solves itself.
Same. It's been proven time and time again that hard problems can be solved by simply walking away while the brain keeps working on it in the background.
As a hardcode vim/vimium/wm user, my mouse usage is near zero.
Yeah, "hard-core vim [etc]" positively correlates with high-salary, and opinions of your own. Firing you is the point!
That is good, too much mouse movement typically indicates that the developer is not typing.
That it's useless - or actively destructive! - does not mean it can't be sold. I hope OP was being satirical, but I fear he wasn't.
> or actively destructive!
Business considerations aside, monetary penalty attached to procrastination/doomscrolling might do wonders for the developers' mental wellbeing.
For the penalty, I would suggest an extension of the model my recent client had me implemented for his blue collar workers: every missed hour (rounding up) is deducted from the pay.
Business considerations aside, monetary penalty attached to procrastination/doomscrolling might do wonders for the developers' mental wellbeing.
For the penalty, I would suggest an extension of the model my recent client had me implemented for his blue collar workers: every missed hour (rounding up) is deducted from the pay.
Good luck finding skilled developers to work under these conditions. Workers will go to the companies that respect them and offer the most freedom. Example, I've been working remote for years now and will never go back into the office.
> monetary penalty attached to procrastination/doomscrolling might do wonders for the developers' mental wellbeing.
Heh. Does HN count? This is pretty much the last "social" site I allow myself.
> every missed hour (rounding up) is deducted from the pay.
Wow! Where's that legal? It wouldn't be in my jurisdiction.
Heh. Does HN count? This is pretty much the last "social" site I allow myself.
> every missed hour (rounding up) is deducted from the pay.
Wow! Where's that legal? It wouldn't be in my jurisdiction.
I consider these low hanging fruit that no one is looking at.
- Fundamental AI research. (Everyone currently has a tunnel vision on deep learning, there is very little fundamental research focusing on new models that might be less data intensive, and less computationally expensive than deep learning)
- TLA+ For high level description of systems. Both hardware, software & physical phenomena. The creation of relevant educational material.
- An Iversonian language designed for mass adoption. (This will be needed to facilitate the modeling of complex systems. Imperative languages and mathematics have are not sufficient)
- Smart Contract powdered CFDs. (Contracts for difference that allow people outside the US to speculate on US/Global Equities)
- A security focussed OS, Smartphone, Laptop & Hardware Wallet that people who care about security can truly trust.
- Complex Systems. The field is so wide and so exciting. Very few complexity theorists exist. And what I like about it is that alot of the concepts can be practically applied in the real world. This is how Science is going to be done in the 21st century.
If you would like to talk further about these, feel free to send me an email (in bio)