From the article:
"The project’s binary distributions didn’t include the required MIT license notice for the llama.cpp code they were shipping. This isn’t a matter of open-source etiquette, the MIT license has exactly one major requirement: include the copyright notice. Ollama didn’t."
I'm always puzzled by how often I see this happen in the open source community. Can the llama.cpp take legal action against Ollama for this? What's the point of these licensing requirements if they aren't (or can't be) enforced meaningfully?
Note: I'm not blaming the llama.cpp team for inaction here, I just want to see them get some justice!
I said before that the function shape changes as you're updating the parameters that govern the function but that's actually very misleading, (sorry), since the kernel parameters are only indirectly governing the function. What the parameters directly govern is the joint probability distribution P(f(x1), f(x2), ..., f(xn)). So the function f is implicitly defined by how likely the entire sequence of f values are.
So how does it change shape? Well this part is actually something I don't fully grasp myself yet. But I can sketch a crude bayesian interpretation, which is how I think of it. Not completely correct but works as a placeholder until I fully work out the math of updating the parameters.
Basically, from a bayesian perspective we can condition the joint distribution of function outputs as a likelihood conditioned on the kernel parameters theta: p(f(x1), f(x2), ... | theta).
Then we can derive the posterior distribution over theta p(theta | f(x1), f(x2), ...) like so:
So we fit the theta parameters based on how well it fits the observed data we feed our bayesian model.
FWIW, I recommend chapter 14 of Richard McElreath's Statistical Rethinking for a better introduction of GPs. This article kind of glosses over a lot of the intuition and introductory concepts that you need to really grok it.
> Why is the 'true process' changing here? I understand our best guess or model is changing with new observations, but the true process should not be changing. If it actually is, then the formulation should be changed to isolate the parameters that is feeding back to it.
He's not saying the true process is changing, just the functions that are being sampled from the GP. The true process refers to the true, underlying function so it's deterministic if you have correctly identified all its inputs.
> So is the shape of each function changing?
Yes, the function changes shape as you get more data because the parameters governing that function (that we define in the kernel) are updated with new observational data, so that over time it converges to the 'true' process/function we are trying to discover.
> What is the 'distribution' over the functions doing? Is that also changing? Is the said 'distribution' just flat mean of these functions?
I think you're confused because the example given with cheese is really confusing when we're trying to understand the functions as arising from a multivariate distribution. So, I'll try to clarify that part. GPs are typically used to represent some function where the input is time or distance. This is why its called a 'process' - because the variables in a random process are indexed by space or time. So in this 1D example, in the X domain, [x1, x2, x3] represents something like fixed increments of increasing cheese. f(X) represents the gold amount. Now imagine gold can take any value from 0-100. Now plot all possible values of f(x1) on the x axis of a grid, f(x2) on the y-axis of the grid, and f(x3) on the z-axis of the grid. We have 100^3 points in this 3D grid. If we select one point, it's x,y,z coordinates correspond to the f(x1), f(x2) and f(x3) gold amounts. The dimension index, corresponds (typically) to something like time, or distance. In this example it's cheese.
In a GP, we're modeling the sampled f(X) point as if its from a 3D multivariate normal distribution. So sampling one point gives us the gold amount for cheese amount 1, 2, and 3. This is the 'function', and as we sample more points, we get more 'functions' that give us varying gold amounts for cheese amount 1, 2, and 3. And because it's a multivariate distribution, we can capture correlations between dimensions, so the amount of gold you get for cheese-1, should influence how much gold you get at cheese-2 because its close by. This relationship is defined by the covariance function of the gaussian.
> GP(m(x), k(x, x')) What is 'x' here? (Sigh! We need to learn to define the variables before using.) I can infer that x' is not derivative of x.
x refers to some amount of gold, and k(x, x') just means that the kernel consumes any two values in our X vector (i.e. [x1, x3] or [x1, x2]).
> "In the context of GPs, a kernel or covariance function k(x, x') = Cov(f(x), f(x')), encodes which function values should vary together." It does not seem the 'f' here is intended to be the specific 'f' introduced at the beginning of the article.
I believe it is the same f actually. He's saying the kernel function takes in two values of x (cheese), and outputs the covariance between their output gold amounts. This illustrates his previous point that the "closeness" between x values should be reflected in the gold amounts.
> The plots now have y and x, and x1 and x2. How are these related?
y is gold. x is cheese. x1, x2 correspond to the first two x-values in the linear plot.
> And with k(x, x') = Cov(f(x), f(x')), what is 'f' for the various kernel functions being plotted.
f(X) is the approximation of the "true" process we're trying to learn from observational data. The observations are tuples of cheese and gold amoutns, so f(x), f(x') is just the corresponding gold amount, we don't actually model that function explicitly. The gaussian distribution we are sampling from for functions just models correlations between our variables, so it represents the function implicitly.
> In 2017, Amodei hired Page Hedley, a former public-interest lawyer, to be OpenAI’s policy and ethics adviser. In an early PowerPoint presentation to executives, Hedley outlined how OpenAI might avert a “catastrophic” arms race—perhaps by building a coalition of A.I. labs that would eventually coördinate with an international body akin to NATO, to insure that the technology was deployed safely. As Hedley recalled it, Brockman didn’t understand how this would help the company beat its competitors. “No matter what I said,” Hedley told us, “Greg kept going back to ‘So how do we raise more money? How do we win?’ ” According to several interviews and contemporaneous records, Brockman offered a counterproposal: OpenAI could enrich itself by playing world powers—including China and Russia—against one another, perhaps by starting a bidding war among them. According to Hedley, the thinking seemed to be, It worked for nuclear weapons, why not for A.I.?
Brockman and his wife were the biggest donors to Donald Trump's Super PAC, MAGA Inc., in 2025 with each of them donating US$12.5 million. Brockman and his wife also donated $50 million to Leading the Future, a super PAC dedicated to AI deregulation that he helped found with Andreessen Horowitz co-founders Marc Andreessen and Ben Horowitz.
This is awesome. How do you integrate morphology into the simulation? Does morphology effect movement (via area friction or mass impact on momentum) or metabolism (via area/volume ratio)?
That presumes that languages didn't evolve independently across different communities. The fact that different ancient languages have completely different grammatical structures, for example, provides some evidence of this.
> In practice, if you improve your productivity at work, it's going to be mostly for your employer's benefit (other than bragging rights and some personal satisfaction).
This is true if you're bagging groceries, but certainly not true for most software developers. To quote the post again:
"I'm sympathetic to the argument and agree that upper management and shareholders capture most of the value from work. But as much as I sympathize with the idea of deliberately being unproductive to "stick it to the man", I value spending my time on things that I want enough that I'd rather get my work done quickly so I can do things I enjoy more than work. Additionally, having been productive in the past has given me good options for jobs, so I have work that I enjoy a lot more than my acquaintances in tech who have embraced the "antiwork" movement."
> But GP is making a point that it should be perfectly acceptable to not want to (significantly) improve at work while meeting the bar for staying employed.
I don't see any point in Dan's post against this idea... it's the GP who is elevating this into a moral choice, i.e. from his/her response to mine:
> Yeah but he doesn't recognize that his entire argument is personal choice elevated to moral imperative.
He literally discusses all your points in the first paragraph:
> The top reasons I see people say that productivity doesn't matter (or is actually bad) fall into one of three buckets:
1. Working on the right thing is more important than working quickly
2. Speed at X doesn't matter because you don't spend much time doing X
3. Thinking about productivity is bad and you should "live life"
Here's his argument for your last point:
> The last major argument I see against working on velocity assigns negative moral weight to the idea of thinking about productivity and working on velocity at all. This kind of comment often assigns positive moral weight to various kinds of leisure, such as spending time with friends and family. I find this argument to be backwards. If someone thinks it's important to spend time with friends and family, an easy way to do that is to be more productive at work and spend less time working.
Personally, I deliberately avoid working long hours and I suspect I don't work more than the median person at my company, which is a company where I think work-life balance is pretty good overall. A lot of my productivity gains have gone to leisure and not work. Furthermore, deliberately working on velocity has allowed me to get promoted relatively quickly4, which means that I make more money than I would've made if I didn't get promoted, which gives me more freedom to spend time on things that I value.
The image says "Spend time with those you love. One of these days you will say either 'I wish I had' or 'I'm glad I did'."
But another way to interpret this graph is to spend less time attempting to cultivate relationships (which yield less and less value over the long term) and spend more time developing interests and skills that keeps you happy and occupied as on average we tend to age in isolation.
blog: saeranv.github.io
work: github.com/saeranv/
contact: saeranv [at] gmail.com