kannanvijayan

il y a 3 joursdiscuss

The feeling one gets from observing and thinking of cosmic scale events, and the feeling one gets from eating a really good risotto, and the feeling one gets from watching through a microscope as a paramecium goes about the business of survival... they're all rich and meaningful in their own way.

The existence of one doesn't diminish the meaning of the others.

As an addendum: you may not realize that your response - in a roundabout and somewhat ironic way - serves to support the argument you are responding to.

Here we are, learning of a planet getting swallowed up by a star, and yet the focus of your argument? The ego of one of those insignificant little humans that are dwarfed in scale by those cosmic events.

kannanvijayan

il y a 7 joursdiscuss

What a wonderful looking piece of software :) My child is far past toddler age but I shall keep this in mind as a gift for friends of mine that are about to have children.

kannanvijayan

il y a 8 joursdiscuss

Having a child was a profoundly selfish act for me. I wanted one because I can't imagine any challenge more fascinating and rewarding (for me) than raising a child.

I don't understand what the point of hiring people to take care of mine would be. That's the fun part. Makes about as much sense as going to an amusement park and paying someone to take the rides for you.

kannanvijayan

il y a 12 joursdiscuss

Disclaimer: I am not a physicist and this is merely semi-informed speculation by an amateur enthusiast of understanding physical phenomena.

I had assumed that those were steam bubbles, not air bubbles. At high enough temperatures water becomes a gas. In a boiling pot, the highest temperatures are at the bottom. Temperature fluctuations on the bottom would lead to more steam production in some areas than others.

Any point at the bottom of the pot that did form a steam bubble would immediately lose energy to the steam and cool down. Now we likely are thinking of metal pots in this exercise, and metal conducts heat very well, so that cooled down point in the metal would draw nearby heat from the pan towards it, decreasing the chances of bubbles nearby. This would presumably lead to bubbles forming at some reasonable distance from each other, as any bubble formation causes the nearby pot metal to cool down.

Once in steam form, the steam bubble has a harder time losing energy. The energy was gained through direct contact of water with metal, but steam is much less dense, and it seems like the high-density shell of water around the steam bubble would act more like a mirror - reflecting the kinetic energy back into the bubble. This would keep the bubble stable as it travels up.

That's been my mental model of it for a while. It would be cool if someone who studied this stuff for real opined on it though.

kannanvijayan

il y a 13 joursdiscuss

I'd point out that pointing at those largely powerless people is a tactic used by domestic power centres that have their own regressive views and policies which they want to draw discourse away from.

I'd ask for a comparison of how these arrivals have led to worse policy outcomes in terms of women's rights, and how that compares to the policy behaviour and outcomes of domestic groups.

I'd close out with a pointed question about which group it is that should be treated as a greater threat.

kannanvijayan

il y a 14 joursdiscuss

An obsessive stalker police officer that's angry at their ex girlfriend moving on and finding a new partner will probably not be interested in EVERY car, just the one she drives and the one her new boyfriend drives.

I won't speculate as to what your law enforcement family members may or may not be capable of when it comes to this technology, but I will speculate on what they will likely do if they found out about an obsessive stalker police officer that's watching their ex-girlfriend and her new partner using this tech: they will likely assist in hiding it so as to ensure that the optics of the justice system are not marred.

The reason I suspect they will behave in this way is not because they're bad people - but because they're likely normal people who are subject to normal influences and incentives. There will be no personal benefit, and significant personal risk associated with whistleblowing on this hypothetical officer, and so they will find rationalizations for why they shouldn't. Why it's fine to let this "one bad apple" go for the greater good of the optics of the justice system.

So it goes.

kannanvijayan

il y a 16 joursdiscuss

Isn't that just leveraging the benefit of having access to a higher level of government which does have rules?

kannanvijayan

il y a 16 joursdiscuss

I read through the article, and I'm not sure this is dependent on quadratic scaling.

Are they allowing all oscillators to influence all others, or are they picking modalities where the influences can be limited to some maximal fixed degree?

One would imagine that there'd be a variety of different topologies available to explore. Even if during training the treatment was fully connected, one could imagine the training itself biasing towards a maximal fixed degree per oscillator, and then inference later operating on a quantized version of that that drops the low-weight influences to zero.

kannanvijayan

il y a 16 joursdiscuss

Isn't it more that the Republicans have co-opted many previous stances that used to be talking points to be pointed at as examples of "leftist examples of extremism"?

The anti-vax, unpasteurized milk drinking, alternative medicine seeking "crunchy mom" USED to be called about by American "right" as an example of "leftist absurdity", but it seems that when that group finally found a political home that truly elevated its views to public policy - it was with the Republicans.

kannanvijayan

il y a 21 joursdiscuss

Yes I've found this to be the case as well. I've also found that it's useful to split these documents into three broad distinguished classes: goals, design, and idioms.

The goal docs provide directionality - helping the agent generally make consistent design decisions. Scoping constraints and stuff are useful to put here, but also feature goals a general idea of where the project is heading long-term (even if none of the items there are on the implementation roadmap anytime soon). It keeps more of the sessions aligned with each other.

The design docs specify the state of the project as it is, and are kept in sync during implementation sessions, by instructing the agents to treat their updates as part of the implementation plan for any work.

The idioms docs keep track of incidental decisions that don't relate to long term goals per se, but things like code style detail, specific project-related investigative techniques, code organization rules, build process guides, etc.

It's a single anecdote but I found that overall the work encountered fewer low-level design mismatches where one session doing work on one thing would make a design decision that didn't really mesh well with another session doing work on another thing. Overall hygiene took less work to maintain.

There are likely a variety of superior ways of organizing things, but at the very least it seems there's a ton of value to be squeezed from just organizing your project meta-information in certain ways. Definitely worth spending some time experimenting with.

kannanvijayan

il y a 29 joursdiscuss

Why are you under the impression that you have a choice?

Those that are violent will always exist, and will always attempt to leverage their willingness for violence, and those that are able to - through luck or circumstance - gain leverage will be able to use that leverage to consolidate more until they have a hegemony on violence.

That's simply an observational fact of human history.

Governments are a way of occupying that opportunity space with some structure that _organizes_ that otherwise disorganized "cut people's faces up and flay them alive" sort of violence, and replaces it with "you get to grumble about the taxman every year" sort of violence. The averager person loses more freedom when there isn't a government around. The average violent psychopath gains more freedom (to become the government) when there isn't a government around.

That's why. Because I, and most people, prefer the paying taxes to getting flayed alive for insulting the duke. The government is a _binding_ of the monarchy and the warlord class to rules. If you look at the history of western democracy, it's extremely obvious.

I highly suspect that one of the reasons that Americans speak in this way.. that government as an idea is inherently some nonsensical or flawed concept - is compensation for their own sense of futility and inability to effect change on their own government.

It's hard for them to reconcile their self-image as "free thinking exemplars for the rest of the world" with the idea that they don't actually have control over their society. So they default to the idea that "all government is bad". If government by definition is bad, then obviously you can't accuse Americans and American culture of being particularly poor at creating a government that serves its people: it's just a fundamental structural problem, not a cultural problem.

To use internet slang: it's a cope.

kannanvijayan

le mois dernierdiscuss

Super neat concept, sort of like a stripped down chatroulette.

However, in today's age I have to say that these sorts of little fun social experiment projects are just a lot harder to engage with without the back of your head going: "I wonder if this has some data collection, or voiceprint collection mechanism behind it".

I'm not saying it does. In fact my natural assumption would be that it doesn't. But I can't be sure until I check, and look into the developer and ensure for myself that it is in fact a hobby project by someone legitimately just fooling around.

The prevalence of data harvesting on the internet has basically poisoned the well with regards to these little fun experiments.

kannanvijayan

le mois dernierdiscuss

The business plan is really different depending on your starting capital.

If you have none, don't bother you can't afford the wood.

If you have a little, maybe your approach above is what makes sense.

If you have a lot, you might start looking at ways to lock down the chair distribution market within a region by maybe buying up some of the major warehousers - a bit of the old vertical integration. Maybe use some "free speech" of the financial kind directed towards some politicians to mandate that all workplaces have a certain minimum ratio of chairs to people.

The pricing strategy will be different for each approach.

kannanvijayan

il y a 2 moisdiscuss

What if over time, the system degrades in such a way that there's a group of people that extract more and more value from the system while adding less and less value themselves?

Is this a possible failure mode of the system?

What sort of symptoms might one look for in a society if we believed this might be happening?

Or do we simply dismiss that this has been proven impossible (as per the theory of Money 101) and move on?

kannanvijayan

il y a 2 moisdiscuss

Interesting. So what's the strategy there? Just assume that each expert will learn some underlying clustering of semantic associations, but not direct it?

kannanvijayan

il y a 2 moisdiscuss

I feel like the talk about "world models" is trying to reach at that, but cast it in different terminology. World model is just domain model, and once you're at domain model, there are multitudes of domains.

Unsupervised learning over domain rulesystems has the potential to let us define really well-defined, scoped models that behave a lot more deterministically and don't colour outside the lines, and reserve their weights for cleanly modeling the domain associations and relationships that matter.

I just asked codex the following question in the middle of my coding prompt:

  What are you thoughts on the relative strengths of ewoks vs jawans?

Answer:

  • Ewoks are stronger in direct conflict. They are organized fighters, good at
    ambushes, traps, terrain control, and coordinated attacks. On Endor, the beat
  a technologically superior force by using preparation and local knowledge.
  ....

As amusing as this may be, I really have no need or desire for my coding model to understand or be aware of ewoks and their relative strengths compared to jawans. Nor do I need it to understand the nuances of the races of middle earth. And prompt response of "I have no idea what you are talking about" to all of these would feel reassuringly scoped.

Mixture-of-Experts seems like an attempt to do this - the domain structure being extracted into specific sub-models that are presumably trained on particular domain-associated content - but it feels like this is once again the beginnings of what is possible.

kannanvijayan

il y a 2 moisdiscuss

This is a reasonably well-examined take of the situation.

On the technical side, one of the additional things I've had on my mind is the potential that these mega models are in fact hiding a ton of inefficiency.

The approach of simply shoving higher dimensionality and more parameters into largely tweaks to the current models has delivered results, but it feels like "mainframe" era of computing to me.

Throwing reams of annotated human content and forcing the machine to globally draw associations from it feels clumsy. Just as people are able to learn structured knowledge via rule-systems that are successively elaborated with extensions and situational contradictions, I feel like there's probably a much more compact representational model that can be reached by adapting the current technical foundations (transformers, attention, etc.) to work well with generated examples from rule-systems, that then gets used as a base layer to augment the "high level" models that process unstructured data.

The risk for the behemoth datacenter might be similar to the risk in the early computing era of building compute centers right before the PC revolution took off.

If it turns out that there exists some more compact and efficient representation for this intelligence (which IMHO is likely given that we are still in the first generation of this technology), the datacenters may end up decaying mausoleums of old tech that has no relevance to a distributed intelligence future.

That's the big technical unknown unknown for me. How much efficiency juice is there left to squeeze, and what does that mean for a distributed landscape vs a centralized datacenter based landscape.

kannanvijayan

il y a 2 moisdiscuss

I can speak towards building large-scale systems from scratch with these tools. I've been working since late last year on a project that was barely a tech demo, and the progression of development on that project has seen me go from leveraging co-pilot autocomplete at the start, to full-on vibecoding 100% of the new additions.

I have reasonable eng chops I'd like to think - I have been a senior IC for a while on a reasonably diverse set of challenging systems problems and built out some pretty large-scale pieces of software the old "artisinal" way.

This particular project is a productization of some ideas I had for leveraging a virtual machine to execute high-divergence parallel logic on GPUs, in an effort to move complex things like "unit behaviour in games" (the classical symbolic kind, not NN-based unit behaviour) into the GPU. The project is going well but still quite a ways from release. But it's at about 300k lines of code now across 9 or so rust repositories, and a smattering of typescript on the frontend.

I have had stumbles, but overall I feel I have put together some good strategies and principles for pushing large projects along with these tools in an effective way.

The biggest takeaway for me is that the "feel" is different. Software construction by hand felt like building legos where you put the pieces together yourself. A lot of my focus would be on building and solidifying core components so I could rely on them when I stepped up to build higher-level components. Projects would get mired quickly if you didn't solidify your base.

With agentic development, one of the early challenges I ran into was this issue with something I'll call "oversight inception". It's when at some early point in the process a somewhat low-importance decision is made - an implementation decision, a decision to say.. align a test with the implementation rather than an implementation with a test.

Then, as you build more on top of this, that small decision somehow ends up getting reified into a core architectural policy that then cascades up.

You realize that when you're building a big project, the focus on some particular component is backstopped by a general understanding of local development directionality with respect to the larger level project. And the agent has no idea of directionality.

So small chinks in the design end up getting magnified and blown up as the dev process proceeds, and later on review you find major architectural pieces have just been overlooked, all flowing from some small incidental implementation choice a long time before.

This is one among a number of issues, but it's a big one. Once I saw it happening I tried an approach to mitigate it by developing a set of golden "goal" documents that describe directionality at the project level: what you are working towards and what design components need to exist.

This doesn't eliminate the "oversight inception" issue, but it does catch them earlier.

When I started applying the goal documentation aggressively to re-align the project implementation direction, I found velocity dropped a lot.

And as I progress, I'm balancing this out a bit - to allow the system to diverge a bit, but force reconvergence towards the goals at some specific cadence. I haven't found the right candence yet but I'm getting there.

This new style of development feels more like claymoulding pottery than lego assembly. You sort of "get it into shape". It's a very interesting new set of process assumptions.

kannanvijayan

il y a 3 moisdiscuss

I learned to snowboard in Wapiti Valley which is a little river valley skislope setup way out in the middle of nowhere saskatchewan. I know what you're talking about. I took the lift up with both 6 year olds and 86 year olds and both would offer advice to a new learner. I drove 3 hours in from "big city" Saskatoon but most of the attendees were kids and adults from the nearby towns. Loved the literal 30-second wait times to catch a lift back up - it was a really great environment to learn in.

That said, "move to a small town" is easier said than done when you have a family and kid :)

kannanvijayan

il y a 3 moisdiscuss

I have a 10 year old boy and I'm facing these issues right now. I'm also in Canada so culturally adjacent to the US and similar enough with regards to this topic.

I don't see child welfare agencies personally as a particular threat when it comes to this topic. Maybe they ARE more likely to get involved in cases of more free range parenting where before they weren't, but it doesn't register as a real worry.

The major difference I see between when I was growing up and now is that when I went out onto the streets, there were other kids on the streets. My parents didn't know exactly what they were sending me out to, but they knew that there as a general crowd of kids that would be out on the street until some point in the evening, and that they would all go home at around the same time, and that's also when you were expected home.

The draw of smartphones and video games as indoor entertainment can't be understated, but I can exercise some parental tyranny here and always kick him out of the house to go play like my folks used to do.

But there are no other kids out there. I'm sending him out into streets empty of kids.

To mitigate this I'm trying to nudge things in the direction of him and his friends forming some sort of after-school crew that finds outside activities to do together, undirected. There are other like minded parents that I've found that are also interested in enabling something like this.

On the subject of risks - I strongly believe that the role of parenthood is to mediate a child's exposure to the real trauma of a hostile, often absurd reality that they will grow up into. Controlled exposure to risk, to self-directed decision making in times where they feel like someone won't be there to help them out and they need to figure things out on their own, these are critical requirements in parenting IMHO. And all risk comes with some small chance of tragedy, and that's a burden we as parents have to bear: to expose ourselves to the emotional trauma of the possibility of our children getting hurt, however small the chance, so that they are able to grow into healthy well-adjusted adults.

I feel like I have to work a lot harder than my parents did to enable that exposure.

comments