Do you think (or care) about the ethics of this sort of behavior? Do you consider it unethical and if you do, under what conditions would you decide to do it anyway?
I frequently visit your website to get design inspiration for my own. Thanks for being so detail-oriented and all your writing in general!
Edit: Actually, while I have you here: do you think that the modal popups for links (the ones that pop up when you hover on a link) should be a standard browser feature? I'd be curious to see if a web extension could replicate it more generally for all sites.
> I’d rather fight for collective ownership of the machines.
I would love if we could force the big tech companies to release their models + weights since they're fundamentally products built on the collective labors of humanity (at least some of which is licensed under the GPL or the CC-BY-SA).
If I could hit a button and abolish copyright and the notion of intellectual property, I would.
"I took their free carrots and now several years later, their carrots are a global ~monoculture that have been modified to grow faster but taste much worse. I don't like their carrots anymore but most other carrots are grown by small-scale local farms and can't be bought for cheap because the farmers never managed to get competitive economies of scale."
"I wish I'd supported the crazy folks who did carrot science in public and distributed seeds and allowed everyone to breed them so that we could all find better varieties for the common good! They still seem to be eating well."
(I see your very practical point, but I do think making the locally suboptimal choice in the hope of better long-term outcomes is a valid philosophical position.)
I'd say you're right about any given individual channel: the activation of a single voxel doesn't tell us much about all the fancy computation happening in that ~1 mm^3 of tissue.
But the pattern of activity of thousands of voxels across cortex does contain reliable information! And a decent amount of it too, at least in sensory cortices.
Caveat: brain-computer interfaces are not quite my field, but I think the consensus is (judging from some conversations with folks who know more):
Neuralink is doing interesting BCI research, with decent hardware, but it's not really a step-change above and beyond the rest of the field.
There's definitely a lot of promise in using BCIs for rehabilitation of patients with brain injuries but their input-output capabilities are still incredibly crude: for example, we can't reliably "write" to the brain to make people perceive things beyond very simple stimuli (e.g. a phantom touch sensation, or a visual phosphene).
This is understandable: the brain has a bajillion neurons and we only have ~1,000 electrodes that aren't particularly precise in how/where they zap the brain---and even if they were, we don't really know well enough how the brain works to "control" perception finely.
Other problems for BCIs include (i) "representational drift", where the brain's code changes over time, so you need to keep fine-tuning your interface in some sort of closed loop fashion and (ii) damage/scarring to neural tissue.
> Is there enough signal for this to really work?
I'm not quite sure what Neuralink's marketing claims are, so I'm not sure what you mean by "this" here. But intracranial electrodes do have a surprising amount of signal, especially relative to non-invasive methods (I'm currently collecting some iEEG data myself!)
I really want the sci-fi future where we have brain-computer interfaces that augment our cognition and perception, but we're nowhere close---though we're getting better.
Yeah, there's a ton of criticism of fMRI as a method, largely because of a lot of results that are statistically unsound (to say the least)!
I tend to think of fMRI data as some highly nonlinear transform of whatever neural activity is occurring in a particular region of the brain, at pretty coarse spatial resolution (~1-3 mm) and pretty bad temporal resolution (~5-15 s).
Sure, it's no direct measure of neurons firing, but that doesn't mean there isn't information in the signal that we can interpret and maybe use (see [1] for a recent example of reconstructing seen images from brain activity)
As a cognitive neuroscientist, I tend to abstract away a ton of the details (neurons, molecules) and focus on more general computational principles: how do we get complex behavior from many simple interacting units---voxels in fMRI, for instance?
Regarding the specific paper you posted, I saw some of the discourse around it but haven't read it carefully myself (it's not my area of expertise). I saw some recent re-analysis of that data [2] that argues that the result isn't valid, but need to look at it more carefully.
If folks are interested, I recently published a paper [1] demonstrating that fMRI activity in the visual cortex is remarkably high-dimensional!
Specifically, using a linear approach (like PCA, but slightly fancier), we find that stimulus-related information is present along many, many dimensions of the neural response---much more than previously expected/reported.
As a recent grad who also refused to use LLMs, the last sentence in the article was one of the primary reasons why:
> “I’m here to learn how to do things,” she adds. “I don’t think outsourcing it to a large language model is the goal of a PhD for me.”
I wanted my cognitive abilities and technical skills to improve, not just produce output more efficiently. IMHO, abstracting over these low-/mid-level skills and focusing on "high-level ideas" is worth it for experts who've already internalized the deep knowledge and know-how; for a novice like me, I need to suffer through the details before understanding things better.
Other more idiosyncratic reasons:
(i) I try to use only FOSS tools on principle, and frontier models aren't;
(ii) When I graduated, LLMs weren't quite as great as they are today and I wouldn't trust their output for anything important;
I would happily use LLMs to learn new things though! I've tried some local LLMs, but they weren't particularly impressive last time. I should re-evaluate now; it's been several months.