Comptime – C# meta-programming with compile-time code generation and evaluation(github.com)

150 pointsby bj-rn7 माह पहले66 comments

Comptime – C# meta-programming with compile-time code generation and evaluation

https://github.com/sebastienros/comptime

70 comments

spicyusername7 माह पहले

I started using C# recently for a hobby project writing a game engine on top of Monogame, and I have been very surprised at how nice of a language C# is to use.

It has a clean syntax, decent package management, and basically every language feature I regularly reach for except algebraic data types, which are probably coming eventually.

I think the association of .NET to Microsoft tarnished my expectations.

andix7 माह पहले

Modern C# and .NET are great. It still suffers from the bad reputation of the Windows-only .NET Framework. It's still a quite heavy platform with a lot of features, but the .NET team invested a lot of time to make it more approachable recently.

With top level Programs and file-based apps[1] it can be used as a scripting language now, just add a shebang (#!/usr/local/share/dotnet/dotnet run) to the first line and make it executable. It will still compile to a temporary file (slow on first run), but it doesn't require a build step anymore for smaller scripts.

[1]: https://learn.microsoft.com/en-us/dotnet/csharp/fundamentals...

actionfromafar7 माह पहले

Also, if you compile Ahead Of Time (AOT) you can cut down on the features and get basically as small a subset as you want of the libraries. IMHO C# and dotnet are really starting to become very impressive.

olvy07 माह पहले

There's also bflat [0]. Not an official Microsoft product, more of a passion project of a specific employee.

"C# as you know it but with Go-inspired tooling that produces small, selfcontained, and native executables out of the box." Really impressive. Self contained and small build system.

[0] https://github.com/bflattened/bflat

andix7 माह पहले

AOT requires a lot of fiddling around, and might break the application unexpectedly, with very weird errors. It is mostly targeted to Blazor (WASM) and for serverless functions.

The default runtime and JIT are fine for most use cases.

orphea7 माह पहले

  > AOT requires a lot of fiddling around, and might break the application unexpectedly, with very weird errors.

It hasn't been my experience. Native AOT does come with some limitations [1][2], but nothing awful. Mostly it's that you can't generate code at runtime and you have to tame the code trimmer. Just don't ignore code analysis warnings and you should be good.

  > It is mostly targeted to Blazor (WASM) and for serverless functions.

Making your CLIs start fast is also a huge use case.

[1]: https://learn.microsoft.com/en-us/dotnet/core/deploying/nati...

[2]: https://learn.microsoft.com/en-us/dotnet/core/deploying/trim...

actionfromafar7 माह पहले

C# AOT may sometimes require fiddling around, but in my experience a lot less fiddling around than what my alternative used to be, which was to use C++.

neonsunset7 माह पहले

vips7L7 माह पहले

Unions have a proposal: https://github.com/dotnet/csharplang/issues/9662

thdrtol7 माह पहले

I love C# and .NET and use it every day.

Sometimes I wonder if the .NET department is totally separated from the rest of Microsoft. Microsoft is so bad on all fronts I stopped using everything that has to do with it. Windows, Xbox, the Microsoft account experience, the Microsoft store, for me it has been one big trip of frustration.

andix7 माह पहले

Microsoft is huge, it's many companies inside one company.

.NET seems to be somewhere close to Azure, but now far away from Windows or the business applications (Office/Teams, Dynamics, Power Platform). Things like GitHub, LinkedIn or Xbox seem to be de facto separate companies.

Edit: .NET used to be tied closely to Windows, which gave it the horrible reputation. The dark age of .NET ;)

jauntywundrkind7 माह पहले

Makes me think of Boo language; Boo was so good at metaprogramming and multi-phasr programming! A very fine .NET language that was so far ahead of the curve, with having the tools of that language be usable at runtime.

Alas many of the docs are offline now. But it had great quasiquotes, which let you write code that gets turned into AST that you can then process. Good macros. A programmable compiler pipeline. So much. Alas, obscured now. https://boo-language.github.io/

daeken7 माह पहले

Boo and Nemerle both were really showing what was possible in .NET back in the early days. I still miss the metaprogramming they had, not to mention their pattern matching (which C# has closed the gap on, but is still way, way short.)

andix7 माह पहले

The syntax looks a bit like F#. But F# only has a few features that generous people might consider meta programming.

There was a lot of fuss about meta programming around 10-15 years ago, but it never got a lot of traction. Maybe for a good reason? I think a lot of the problems it solved, were also solved by functional programming features that slowly appeared in C# over the years.

CharlieDigital7 माह पहले

C# meta programming game is strong. Source generators are :chefs_kiss:

daeken7 माह पहले

I love (and heavily use) source generators, but the development experience is godawful. Working with the raw Roslyn types is painful at best and this is compounded by them having to be written against .NET Standard, severely limiting the use of newer .NET functionality.

Eventually I want to write a good baseline library to use for my source generators -- simplifying finding definitions with attributes, mapping types to System.Type, adding some basic pattern matching for structures -- but haven't found a way to do it that's general enough while being very useful.

zigzag3127 माह पहले

I agree, .NET Standard limitation unnecessarily complicates development experience. I think it's because some tools (Visual Studio) is still use legacy .NET Framework. I don't understand why they didn't integrate them via out of process architecture into these tools, since source generators didn't exist in the legacy framework anyway.

I sometimes generate code from plain CLI projects (avoiding source generators altogether), as whole debugging and DX is so much better.

daeken7 माह पहले

Yeah, I went with that approach for most of the code generation in the emulator I'm currently working on. Source generators handle a few core things, but more advanced compilation tasks went to just ahead of time generation; couldn't get my parser combinator library to play nicely with .NET Standard, so that was just a dead-end.

tryfinally7 माह पहले

You add PolySharp to your source generator project to get back some of the modern C# features. https://github.com/Sergio0694/PolySharp

octopoc7 माह पहले

.NET standard isn’t the biggest issue with making source generators. You can’t add dependencies to your project, which is an absolutely huge oversight IMO.

olidb7 माह पहले

You can add dependencies, it's just important, that you also add the dependencies to the project where the source generator is used:

SourceGenerator.csproj:

  <ProjectReference Include="..\Dependency.csproj" />

FinalProject.csproj:

  <ProjectReference Include="..\Dependency.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />
  <ProjectReference Include="..\SourceGenerator.csproj" OutputItemType="Analyzer" ReferenceOutputAssembly="false" />

And I also recommend to add "<LangVersion>latest</LangVersion>" to SourceGenerator.csproj to use new features like raw string literals.

pjmlp7 माह पहले

Yeah, it is kind of sad that it was the community that had to step up for some T4 like experience instead of string concatenation.

That isn't as cool as Aspire and AI features.

mexicocitinluez7 माह पहले

I feel the exact same way. They can be insanely powerful, but the syntax is insanely off-putting.

And the documentation is still pretty sparse.

> Eventually I want to write a good baseline library to use for my source generators -- simplifying finding definitions with attributes, mapping types to System.Type, adding some basic pattern matching for structures -- but haven't found a way to do it that's general enough while being very useful.

I'd use it in a heartbeat. The new ForAttributeWithMetadataName function has been a big help, but everything else feels like a totally different language to me.

olidb7 माह पहले

Add "<LangVersion>latest</LangVersion>" to the csproj to get more useful features like pattern matching and raw string literals.

torginus7 माह पहले

I used to think so too, but I do have to complain about Microsoft doing the Microsoft thing - they have a brilliant idea (and execution), but they bury it in so much boilerplate that 99% of people who would use it are put off by it.

The amount of stuff you have to wade through here compared to something like comptime in Zig (Roslyn API, setting up the project, having VS recognize it and inject it in the compiler, debugging etc) makes usage of these an absolute pain.

torginus7 माह पहले

While this looks cool, I don't see any code generation capability in the examples or the tests, only compile time evaluation.

This is more like constrexpr than a macro system.

smcnc7 माह पहले

I think Zig really shines here: https://ziglang.org/documentation/master/#comptime

pjmlp7 माह पहले

D was already doing it in 2010 thereabouts, an then there is the whole reader macros in Lisp and Scheme.

jibal7 माह पहले

And Nim, which has multiple levels of metaprogramming.

Tiberium7 माह पहले

There's also Nim, it shines even more in compile time and has proper compile time macros :)

Zambyte7 माह पहले

A key difference is that in this C# package, `[Comptime]` is an attribute (annotation? not sure on the C# term) applied to methods. In Zig, the `comptime` keyword can be applied to pretty much any expression. In the C# package, if you want to do factorial at runtime and at compile time, (I think, from reading the README) you need to define the same function twice, one with `[Comptime]` and once without. Contrast this to Zig, where if you have a regular runtime factorial function, you can just execute it at compile time like:

    const x = comptime factorial(n);

Another limitation of the C# package is it only works with primitive types and collections. Zig comptime works on any arbitrary types.

hahn-kev7 माह पहले

You don't. The way it works is that it intercepts the call site when the input args are constant. If they're not then it won't be replaced and it will call the original method. C# source generators can't replace method definitions, however a call site can be changed to another method via source generators.

NuclearPM7 माह पहले

You don’t what?

ygra7 माह पहले

You don't have to write the method twice. Source generators can only add new code, they cannot take away a method you declared, so it will still be there at runtime when called with non-constant arguments.

estimator72927 माह पहले

Decorations in [square brackets] are Attributes in C#

pyrolistical7 माह पहले

I’m annoyed they called it comptime when it isn’t the same as Zig’s more powerful comptime.

You can think of zig’s comptime as partial evaluation. Zig doesnt have a runtime type reflection system, but with comptime it makes it feel like you do.

orphea7 माह पहले

  • Supported return types:
     ◦ Collections: ..., List<T>, ...
     ◦ Note: Arrays are not allowed as return types because they are mutable. Use IReadOnlyList<T> instead.

I don't understand. Why is List<T> allowed then if it's mutable?

[deleted]7 माह पहले

andix7 माह पहले

Array also implements IReadOnlyList if I'm not mistaken.

I think C# doesn't really have immutable collections, they just can be typecasted to IReadonly* to hide the mutable operations. But they can always be typecasted back to their mutable base implementation.

The only real immutable collections I know of, are F#s linked lists.

maltalex7 माह पहले

Immutable collections exit, they were just added later. See System.Collections.Immutable:

https://learn.microsoft.com/en-us/dotnet/api/system.collecti...

zmj7 माह पहले

Those collections are more like copy-on-write than actual immutable. System.Collections.Frozen is the real thing.

rawling7 माह पहले

Isn't Frozen something you do to a set or dictionary to say, I'm not going to add any more values, please give me a version of this which is optimized for lookup only?

andix7 माह पहले

I do a lot of .NET programming, and I've never seen them getting used. :O

ygra7 माह पहले

Roslyn, the C#/VB compiler, uses them extensively, but in other code they're indeed quite rare.

orphea7 माह पहले

Adding to the sibling comment, there are also Frozen{Dictionary,Set}: https://learn.microsoft.com/en-us/dotnet/api/system.collecti...

randomNumber77 माह पहले

One man's constant is another man's variable.

neonsunset7 माह पहले

mfro7 माह पहले

This seems like the kind of feature that should be built into MSBuild.

eterm7 माह पहले

It's a lot less ergonomic but there are source generators in C# :

https://devblogs.microsoft.com/dotnet/introducing-c-source-g...

That said, for more complex results, you'd typically load a serialization on start.

I can see the value in this tool, but there must be a fairly limited niche which is too expensive to just have as static and run on start-up and cache, but not so large you'd prefer to just serialize, store and load.

It also needs to be something that is dynamic at compile time but not at runtime.

So it's very niche, but it's an interesting take on the concept, and it looks easier to use than the default source generators.

Const-me7 माह पहले

> there are source generators

Last time I tried them discovered source generators in the current .NET 10 SDK are broken beyond repair, because Microsoft does not support dependencies between source generators.

Want to auto-generate COM proxies or similar? Impossible because library import and export are implemented with another source generators. Want to generate something JSON serializable? Impossible because in modern .NET JSON serializer is implemented with another source generator. Generate regular expressions? Another SDK provided source generator, as long as you want good runtime performance.

tubs7 माह पहले

You can source generate JSON serdes from your own source generator but you do need to generate the jsontypeinfo metadata yourself.

richardw7 माह पहले

Not terribly niche. All config that isn’t environment-specific and is used in inner loops or at startup. It’s even got a test for serialised values so can be used to speed your case up:

https://github.com/sebastienros/comptime/blob/main/test/Comp...

But you need to be sure you won’t want to change without compiling.

eterm7 माह पहले

Well it also needs to be something that you need to generate/calculate, otherwise you would just write by hand the code that comptime outputs.

piskov7 माह पहले

Also t4 templates before that for at least a decade

pjmlp7 माह पहले

And much more developer friendly.

mexicocitinluez7 माह पहले

I use source generators pretty extensively but don't really understand how this is different (or what it's solving that source generators can't).

I'm pretty new in the source generation area and only do enterprise dev, so I'm sure I'm missing something or just don't have the use cases.

andix7 माह पहले

Please don't. Nobody wants even more complexity in MSBuild ;)

ygra7 माह पहले

The use cases are different. While MSBuild tasks run during build (and partially when loading a project), typically the IDE is oblivious what happens there. The source generator runs directly inside the compiler infrastructure and thus you didn't get error highlights for code that would otherwise be only generated during build but not as you type. This makes it much more friendly than pure build-time generation of code.

rubenvanwyk7 माह पहले

How is Comptime different from default AOT compilation? I assume it can work with third-party libraries that AOT don’t work with yet?

andix7 माह पहले

It's a code generator that runs during compile time. It's a source generator that adds some generated code files to the project. So it runs way before AOT or JIT. Once AOT/JIT run, Comptime is already invisible to them, they only see the generated code from Comptime.

betaporter7 माह पहले

I believe this is like `constexpr` for C#.

faithlv7 माह पहले

This is amazing, after using Zig, I started to write exactly this for C# too but never really had motivation to finish it.

andix7 माह पहले

Does this finally allow reading string literals from files and include them into the binaries?

I've seen a Go project that was heavily using https://pkg.go.dev/embed for loading some template files, and got a bit jealous. Back in the days of .NET Framework it was common to compile file contents into resource files, but it was always quite cumbersome.

eterm7 माह पहले

C# / .NET has always had that abililty.

  <EmbeddedResource Include="/path/to/file.txt" />

Then you can read that file from the assembly with:

  assembly.GetManifestResourceStream

I'm not sure what's awkward about that, but it's nothing new.

Discoverability for this is in visual studio, if you go to file properties there's a drop-down where you choose between Content, EmbeddedResource, Ignore, etc.

That determines what happens in compile, whether it is copied to output directory, compiled into the binary, etc.

andix7 माह पहले

The Stream is awkward. It needs to be processed either once at startup or lazy on access, or every time it's used (allocations!). Also discovering multiple files is awkward.

[deleted]7 माह पहले

floucky7 माह पहले

Can't you already do that with embedded resources?

andix7 माह पहले

Yes, you can. But it's a bit cumbersome. The API to read them is not really intuitive, they are only accessible as a Stream, so they need to be either read every time into a string (new allocation, slow), or you need a helper that reads them once and keeps them in memory. I think there are also a lot of gotchas around naming and listing them.

In modern code I don't see them that often anymore.

orphea7 माह पहले

I see your point and I don't necessarily disagree but better API for embedded resources is just a couple of simple extension methods away. I wouldn't even bother adding an external package for it.

mgaunard7 माह पहले

So it's like C++ consteval?