HackerTrans
TopNewTrendsCommentsPastAskShowJobs

_g6sn

no profile record

Submissions

Kefir: Solo-developed full C17/C23 compiler with extensive validation

kefir.protopopov.lv
64 points·by _g6sn·10 maanden geleden·21 comments

comments

_g6sn
·10 maanden geleden·discuss
(I'll address the parallel build part in addition to what has been said, but please respond to my email address/sourcehut mailing list for any further communication on these issues).

I should have communicated this more clearly in README. Kefir requires relatively recent GNU Make versions (see dist/Dockerfile for full and complete set of dependencies), as it basically uses make for all project lifecycle tasks. I remember there were some parallel build issues with older GNU Make versions. This could be an issue.
_g6sn
·10 maanden geleden·discuss
Thank you. On that subdomain, nothing was generated -- I just sat down and typed it in the editor for no particular reason. Should have validated it, at least

Edit: The external source archive page was. Although, with a very simple bash script, but nevertheless
_g6sn
·10 maanden geleden·discuss
Yes, indeed, libc behavior (at minimum) is defined by the standard itself, and the compiler has to ensure that the library is functional. I have explicitly acknowledged it in my comment above ("this is my problem as a compiler developer").

Here I am not speaking about the compiler optimizing based on known libc behavior, but libc "specializing" based on assumed compiler behavior. Which is, again, absolutely reasonable -- both compiler and libc typically go hand in hand. However, as my compiler "parasites" on the existing libc implementations, I would have preferred that such assumptions would be validated to the extent it is possible at compile time, preferably with some graceful failure.

Glibc provides quite profound libc part even to non-GNU C-compatible compilers, but in some places it might fail silently and disruptively.

I am deeply thankful to all developers of libc implementations for Linux and BSD systems, and I am in no way accusing anyone or demanding anything. In the project's README I explicitly note, that any standard library function not being usable will be treated as a bug in Kefir itself. In the above rant, I just expressed a mere wish that some things could be a bit smoother.
_g6sn
·10 maanden geleden·discuss
I am going to hijack this comment for a little rant about libc and independent compilers. Existing libc implementations have varying levels of hostility towards non-gcc/clang compilers. Glibc is probably the worst offender here, musl is the most compliant (but even there are some assumptions).

I am in no way accusing any of libc developers, they got much higher priority things to do than supporting obscure compilers, and this is my problem as a compiler developer, but nevertheless. For instance, glibc may simply override __attribute__ keyword with an empty macro, or omit packed attributes for non GNU C compilers (breaking epoll ABI along the way). Of course, strictly standard-compliant compiler may not have attributes at all, but in my opinion #warning/#error directive would have been more appropriate than silently producing ABI breakage.

Although, I have not engaged with glibc developers on these topics, and mostly documented encountered challenges as patches in my own test suite.
_g6sn
·10 maanden geleden·discuss
I have addressed compiler inefficiency in the sibling comment. This is indeed a problem. Empty arrays of such size should be compile-able (there is sparse representation for arrays). However, I would say that this use case is not particularly practical, at least in none of the projects from my test suite this has been an issue.
_g6sn
·10 maanden geleden·discuss
(I thought that the announcement has completely faded, so haven't even checked the replies).

I'll immediately reveal some issues with the project. On the compilation speed, it is unfortunately atrocious. There are multiple reasons for that:

1. Initially the compiler was much less ambitious, and was basically generating stack-based threaded code, so everything was much simpler. I have managed to make it more reasonable in terms of code generation (now it has real code generator, optimization pipeline), but there is still huge legacy in the code base. There is a whole layer of stack-based IR which is translated from the AST, and then transformed into equivalent SSA-based IR. Removing that means rewriting the whole translator part, for which I am not ready.

2. You've outlined some appealing points (standard compliance, debug info, optimization passes), but again -- this came at the expense of being over-engineered and bloated inside. Whenever, I have to implement some new feature, I hedge and over-abstract to keep it manageable/avoid hitting unanticipated problems in the future. This has served quite well in terms of development velocity and extending the scope (many parts have not seen complete refactoring since the initial implementation in 2020/2021, I just build up), but efficiency of the compiler itself suffered.

3. I have not particularly prioritized this issue. Basically, I start optimizing the compiler itself only when something gets too unreasonable (either, in terms of run time, or memory). There are all kinds of inefficiencies, O(n^2) algorithms and such simply because I knew that I would be able to swap that part out should that be necessary, but never actually did. I think the compiler efficiency has been the most de-prioritized concern for me.

Basically, if one is concerned with compilation speed, it is literally better to pick gcc, not even talking about something like tcc. Kefir is abysmal in that respect. I terms, of code base size, it is 144k (sans tests, 260k in total) which is again not exactly small. It's manageable for me, but not hacker-friendly.

With respect to marketing, I am kind of torn. I cannot work on this thing full time, unless somebody is ready to provide sufficient full-time funding for myself and also other expenses (machines for tests, etc). Otherwise, I'll just increase the workload on myself and reduce the amount of time I can spend actually working on the code, so it'll probably be net loss for the project. Either way, for now I treat it closer to an art project than a production tool.

As for compiled code performance, I have addressed it here https://lobste.rs/s/fxyvwf -- it's better than, say, tcc, but nowhere near well-established compilers. I think this is reasonable to expect, and the exact ways to improve that a bit are also clear to me, it's only question of development effort

P.S. JKU is a great school, although by the time I enrolled there the project has already been on the verge of bootstrapping itself.

EDIT: formatting