I'd say all of these problems have answers in CUE.
> I want to be able to write a "generic environment" module for each application and then parameterize it accordingly for each environment.
This is pretty solidly in the target use case range, i'd say - managing variations of the same "object type" over some dimension is a lot of what's targeted by the way that CUE treats directory hierarchies when loading files: https://cuelang.org/docs/concepts/packages/#instances
The main thing you have to consider in designing a layout is that you have to take a compositional approach to how you define individual config instances. That is, you can't start from prod's config, then override a value or two for staging.
If i were to do it - i have not, this is not how i currently use CUE - my first approach would probably be by defining defaults at the "policy" level (per the above link), which effectively allows you to get exactly one "override"-ish behavior.
Lots of possible approaches to this, though.
> but you can't quite emulate functions as far as I can tell
If this is the sense in which the comment is intended - CUE is capable of some kinds of computation. It's just not Turing complete.
My 2c - thus far, i've found the language features enabled by this constraint much more useful than the expressiveness lost - at least, for the purposes i've chosen.
That said, it's not like we're planning on replacing all the "export JSON" buttons in the Grafana UI with "export CUE." One of the interesting properties of defining schemas in CUE is how it allows us to remove schema-defined default values from a dashboard's JSON. The JSON representation can actually look a lot more like a concise CUE representation.
> versioning
Versioning of the Grafana schema is the essential design goal of the "scuemata" system that is under discussion in that epic issue
Like, editing something other than raw code in an editor? Yes, this is also something directly enabled by the schema (again, see the Polly doc, the "Produce" heading). For data-intensive tasks, such editing experiences are the only way to see your logic in the context of data, and therefore IMO prerequisite for confidence
> reproducibility
Yup. This already isn't "hard" to do today, but reproducibility gets more complicated at scale - those questions about how to map what's on disk to what's in your Grafana (or whatever app) in my parent comment become more complicated, leading to friction, leading to staleness.
Very excited to see someone doing this! Right now, Grafana is [planned to] relying on an anemic CUE->Typescript translator for getting its schema to the frontend - https://github.com/sdboyer/cuetsy. (Somebody also pointed me to Project Cambria recently, which could be an interesting compilation target for what we have https://www.inkandswitch.com/cambria.html)
Being able to work with CUE natively in TS, though, would be a huge gamechanger for what we can do with CUE in Grafana
CUE is currently centered around its CLI, but AIUI, that's not the long-term goal. i read in some CUE issue somewhere that the goal is shifting towards enabling frameworks rather than driving people to the CLI, though i don't have a link.
Our use of CUE in Grafana is an example of framework-style usage. It is a hard requirement that users never have to install the CUE CLI to perform any of our planned CUE-related tasks; rather, the needed tooling is baked into Go packages we export, and things like grafana-cli. (Avoiding a dependency on the CUE CLI also gives us a defense mechanism against breaking changes)
re: Grafana (i'm the author of the linked issue) - i'm quite excited, i do think there's a world of possibilities here.
Two-way sync with a git repo is one possible path, and we've talked a lot internally at GL about how to best support it. My sense is that we can do it with relatively little friction and likely will - but if you're just syncing with a git repo, there's still a lot of arbitrary, opaque repo layout decisions that still have to be made (how do you map a filesystem position for a dashboard to a position in Grafana? In a way that places the dashboards next to the systems they're intended to observe? With many teams? With many Grafana instances?) which induce new kinds of friction at scale.
Fortunately - and not mutually exclusively with the above - by building the system for schema in CUE, we've made a composable thing that we can make into larger systems. That's what we're starting to do with Polly: https://github.com/pollypkg/polly
sadly, gb is one of the extant systems i haven't had a ton of time to explore. but...
i guess the analogue to what you're describing would be
1. <add an import path>
2. `dep ensure`
3. `git commit -am "all of your vendor'd source`
a bit more detail, starting at a high level: gb is a replacement toolchain. dep is focused strictly on dependency management. there is no notion of a `dep build`.
with dep, there's no explicit command to actually fetch a dep; the import graph is still queen, as is customary in go. so there's no direct analogue to `gb vendor fetch github.com/some/pkg`. if you want to add a dep, import it in your source code, and run `dep ensure`. (there's some flux in exactly how that works right now, but what i'm describing is the state we're moving towards)
`dep ensure` is really the workhorse command, and pretty much the only thing you'll ever need to run. (in fact, the only three subcommands we currently plan on having are `init`, `ensure`, and `status`).
Force pushes - by far more common than straight-up repository removal - are handled without problem; we let you stick with your old version. (At least, that's how it should be - there might be a couple more test cases to write. I know I designed for this problem early on).
Repo removal, renaming, or whatever, are still problems, for sure.
Today, dep populates vendor/ with dependencies, and works equally well whether you decide to commit them or not.
yeah, this is one of the main pain points there. the reason it's so difficult to solve in glide is intrinsically tied to the engine glide uses for version selection - otherwise we'd have dealt with it long ago.
There are a host of other reasons that we could happily debate for years (no thanks), but bottom line: you can't use it everywhere you can write/compile Go code.
dep mgmt, at least, is poised to improve a lot in the coming months. we've got growing consensus around some metadata files, and i'm nearly done with my SAT solver (github.com/sdboyer/vsolver), which will be in glide, and maybe others, soon.
> and then be able to compose the things you’ve proved separately
This has always struck me as the really challenging, problematic part about correctness proofs. I've no formal training in this area (sadly), but how does one go about being able to just compose your proof components? Often as not, that seems like a pipe dream - http://queue.acm.org/detail.cfm?id=2889274
> Language package managers are good for quickly sharing code written purely in that language with other developers, but unacceptable for anything else.
Right, so, what almost all of the FLOSS world is doing, almost all of the time. Miiiight be worth optimizing for that case. Especially if a proper PDM would compose well with larger systems...which it would!
> I want to be able to write a "generic environment" module for each application and then parameterize it accordingly for each environment.
This is pretty solidly in the target use case range, i'd say - managing variations of the same "object type" over some dimension is a lot of what's targeted by the way that CUE treats directory hierarchies when loading files: https://cuelang.org/docs/concepts/packages/#instances
The main thing you have to consider in designing a layout is that you have to take a compositional approach to how you define individual config instances. That is, you can't start from prod's config, then override a value or two for staging.
If i were to do it - i have not, this is not how i currently use CUE - my first approach would probably be by defining defaults at the "policy" level (per the above link), which effectively allows you to get exactly one "override"-ish behavior.
Lots of possible approaches to this, though.
> but you can't quite emulate functions as far as I can tell
Function-like capability is present, just in a form that's less familiar. I think of them as "function structs." This post has a bunch of examples https://github.com/cuelang/cue/issues/139#issuecomment-55677.... It seems there's a plan to add a more comfortable notation (https://github.com/cuelang/cue/issues/943), but it's fundamentally possible now.