PeCaN · HackerTrans

il y a 7 ansdiscuss

Huh, I had no idea it worked like that. That's bizarre.

il y a 7 ansdiscuss

It doesn't get in that situation, because malloc() can return null on Solaris (i.e. it never¹ overcommits).

While in general I think this is vastly better than the somewhat insane Linux OOM killer, you can get in awkward situations where you can't start any processes (including a root shell) because you're out of memory.

I rather like the FreeBSD solution to this, which is to not overcommit, but after a certain number of allocation failures it kills the process using the most memory. This prevents situations where you can't start any processes.

There's no one-size-fits-all solution to handling low memory conditions, but the Linux solution manages to almost never do what you want which is kind of impressive in a way.

¹ I seem to recall hearing somewhere that you can allow allocations to overcommit on a per-application basis on later versions of Solaris, but don't quote me on this.

PeCaN

il y a 7 ansdiscuss

This makes sense but is somewhat annoying since a web browser is not the only program running on my computer (but apparently wants to be) and eats up RAM that could be used for the OS file cache.

I wish there was some sort of allocation API specifically for allocating caches so that recently accessed files could kick out a web browser's cache of a not-so-recently accessed tab or vice versa.

PeCaN

il y a 7 ansdiscuss

welcome to MySQL

PeCaN

il y a 10 ansdiscuss

The thing you have to understand about Mike's work on the new LuaJIT GC design is that it's heavily constrained by the Lua API. It's not a very interesting GC, though it does improve slightly on the standard tricolor incremental marking GC. See my reply to your other post.

PeCaN

il y a 10 ansdiscuss

I implemented something heavily based on that once¹, though no work has gone into optimizing much. It's not particularly state-of-the-art, but it's a good design for what it is. I'll probably use the quad-color strategy in my next incremental GC.

Note that while Mike's is strictly non-copying/non-compacting (because of Lua's API) mine does copy between generations and compacts the oldest generation. This makes arena management simpler and faster. I think that's the only significant difference between my implementation and Mike's design.

I suspect it's still a bit buggy, though it passes my test suite™. Non-trival GCs can be tricky. And to reiterate, it's not especially fast, because no work has gone into making it fast. (It's not slow either, because the design is good, but don't expect performance competitive with current production GCs².)

1. https://github.com/alpha123/yu

2. Well, it probably smokes Ruby and Python, but those go solidly in the ‘toy GC’ category.

PeCaN

il y a 10 ansdiscuss

As far as FL/FP/FPr go, http://www.call-with-current-continuation.org/fp/ is the most mature and complete implementation I'm aware of.

There's #proglangdesign on freenode where a bunch of us have been on an array language/function-level streak lately. Some people there might be aware of more.

PeCaN

il y a 10 ansdiscuss

Well, there's:

- Dependently typed languages (ATS, Agda, Idris)¹ should be fairly familiar if you're a Haskell veteran.

- Array languages (APL, J, K, and more obscure ones like Nial) are pretty enlightening if you're a functional programmer (at least they are for me). Most of these trace to Ken Iverson and his Notation as a Tool of Thought. They are a bit brain-bending at first, largely because of the density, but they're fun to use and the density makes comprehension easier after a while.

- Function-level languages (FL, FPr, J) – a somewhat obscure and very advanced sister to functional programming. If you're familiar with Haskell Arrows, there are many parallels. They are, essentially, a more convenient point-free style. Most (all?) of these trace to John Backus (of Fortran and Backus-Naur Form fame) and his 1977 Turing Award lecture Can Programming Be Liberated From the von Neumann Style?

J (http://jsoftware.com/) combines array and function-level programming, and IMO is a very good language to learn to expand your horizons if you're a veteran functional programmer.

--

¹ Coq kinda, but it's more of a theorem prover than a programming language. Agda sort of fits that too.

PeCaN

il y a 10 ansdiscuss

I implemented my own, but I'll definitely check out jemalloc. I happen to have a use case for allocating large chunks aligned on large boundaries (high-performance garbage collector). jemalloc is common enough that I suppose it's not a particularly bad dependency.

Adding Lisp-style AST macros to C is never gonna happen[1], though it would be cool. I'm of the opinion that text macros and ‘real’ macros complement each other anyway.

One of these days I'll get around to implementing a less-portable-more-cool-C. There's Jai[2], but I don't really agree with all of its design (understandable, since I'm not a game developer).

--

1. Just kidding, people have done it already; e.g. https://github.com/eudoxia0/cmacro but it'll probably never be part of the standard.

2. https://github.com/BSVino/JaiPrimer/blob/master/JaiPrimer.md

PeCaN

il y a 10 ansdiscuss

Not as far as I'm aware. POWER, x86(_64), even Itanium all just bump a stack pointer[1]. SPARC has a funky stack, but I don't know enough about it to say how hard VLAs would be. (It's a darn cool architecture though. I wish acquiring a modern SPARC processor wasn't ridiculously expensive.)

The thing that makes VLAs hard to implement is that on some embedded and special purpose processors the stack is actually a physical, fixed-size location on the chip. I recall, but can't seem to find, some architecture that didn't use an explicit stack pointer.

1. Itanium has 2, but for purposes of VLAs it may as well have a conventional stack.

PeCaN

il y a 10 ansdiscuss

This is inherited from early Lisp implementations, where it was common to define a

  struct car { ... };

which would get cumbersome to type.

PeCaN

il y a 10 ansdiscuss

As long as we keep __builtin_apply out of the standard, GCC is loaded with good ideas.

PeCaN

il y a 10 ansdiscuss

They were made optional in C11. Compilers for DSPs or whatever where VLAs aren't practical don't have to implement them.

Just because an _optional_ feature is impractical on a few niche architectures doesn't mean it's a mistake.

PeCaN

il y a 10 ansdiscuss

free(NULL) is already perfectly legal (specced to be a no-op) as noted by the parent. The problem is that `char * s;` is probably _not_ NULL, so you'd try to free some random address on the stack. Of course it is easily fixed with `_cleanup_free char * s = NULL;`, but it's a somewhat difficult error to spot sometimes.

EDIT: I super misread that, my bad.

PeCaN

il y a 10 ansdiscuss

Yes, that's exactly why they'd use _Auto and a header to redefine the keyword. Existing code that does auto x = 3; (implicit int type) wouldn't break, but #include <stdauto.h> auto x = 4.2f; would be the new meaning.

PeCaN

il y a 10 ansdiscuss

I've only somewhat recently become a hardcore C programmer, but I actually quite like it. Wrote my own incremental generational garbage collector recently—the things you can do in C are fun. Here are some things I'd really like:

Not language, rather standard library, but I would like to see a new allocation API with explicit control over alignment, separation of reserving and committing addresses, and generally updated for the fact that memory systems are different now than they were in 1979.

aligned_alloc is alright, except there's no aligned_realloc. Linux always reserves addresses and doesn't commit until it needs to, but that can't be relied on for portable code.

In short: we're in a 64-bit, virtual memory world. I don't want an allocator design from 1979.

It's about time glibc's custom streams[1] get standardized. Real talk, it's 2016 and we can't define custom FILE * types. This doesn't even need to expose the libc's FILE implementation; it can remain opaque just fine.

Language-wise, I'd like to see a pragma to covert an array of structs to a struct of arrays; i.e. declare

  #pragma struct SOA
  struct entity {
    uint32_t id;
    ...
  };

  struct entity all_entities[1000];

and have the compiler rewrite it to

  struct {
    uint32_t id[1000];
    ...[1000];
  } all_entities;  // where typeof(all_entites[N]) == typeof(struct entity)

à la Jai[2]. This isn't a particularly hard transformation to do by hand, but it's kind of tedious.

Standardizing M4 as a preprocessor would be a nice-to-have; i.e. you should be able to compile a .h.m4 or .c.m4 file directly. I don't know why GCC can't do this yet. It's a bit of a niche thing, but some of us are M4 addicts.

C is in a pretty good place otherwise. I like to think there are 3 known local optima in terms of programming languages: C, Lisp, and Forth. All of them are a little hard at first, but quickly become very easy to think in.

We'll probably never get any of this, which is a little depressing. I get the feeling the C committee doesn't really care that much about evolving the language any more. Modern, fast, safe(-ish), readable code is a GCC extension.

1. http://www.gnu.org/software/libc/manual/html_node/Custom-Str... 2. https://github.com/BSVino/JaiPrimer/blob/master/JaiPrimer.md...

comments