kdkeyser · HackerTrans

kdkeyser

no profile record

comments

4 năm trướcdiscuss

What kind of consistency guarantees does this offer?

5 năm trướcdiscuss

Did you consider using an B-epsilon-tree [1] ? This is a B-tree, but each node has a staging area for new writes. In practice, incoming writes get added to the staging area of the root node, and once a staging area is full, it gets propagated down. This results in write performance that is similar to append-only. For reads, you get the performance of a regular B-tree. Sounds like a good match for your use case.

[1] http://supertech.csail.mit.edu/papers/BenderFaJa15.pdf

5 năm trướcdiscuss

Any background info/links to this? The Wyze forum comes up empty, only some discussions on WebRTC being complex to add.

5 năm trướcdiscuss

This would be interesting to integrate into https://github.com/openmiko/openmiko which is a firmware for the T20 based ip-cameras.

Right now, I am not aware of any cheap ip camera that can stream its H264 video to a regular web browser, with sub 500 ms latency. All manufacturers seem to have moved to an app, I guess they can show an RTSP stream in that way.

Older ip cameras had MJPEG which you could view in the browser, but that is really inefficient w.r.t. bandwidth.

6 năm trướcdiscuss

Haxl is actually an example of "the more impressive and more useful" approach: it is an abstraction embedded within a Haskell library, that allows you to express your "intent" in simple, standard Haskell code, decoupled from the way this intent will be reached (i.e. the execution is determined/optimized by the library)

Haskell allows this to be done in a way that makes if very hard for the user to break this abstraction (the type system will prevent you from doing this), while still allowing the full language power of Haskell to be used.

6 năm trướcdiscuss

This indeed seems to be the main/only point of criticism in this post that is valid: however, it is not that Haskell has no "pragmatic" libraries to get stuff done (e.g. WAI/Warp, Yesod, Servant, ... are top notch, practical libraries if you are writing network/HTTP services). They do seem to drown in the sea of libraries/blog posts that are focused on the academic/abstract stuff. The end result does not feel consistent, and reminds me of the horrors we had with the early C++ metaprogramming efforts: it looks cool, but you end up fighting the language and produce unreadable code.

For Haskell to become a successful "industrial" language, I think most of the dependent-typing stuff should probably go (to Idris/Agda etc.), so that a clear and consistent Haskell subset can be defined.

The other arguments in the article are just weak. The rant about data not having a type, is missing the point. Sure, often you receive data that you need to inspect to know what it is. You can easily do this in Haskell (just label it with the UnknownData type), and have a function that inspects it and returns, depending on the contents, the right type). The big advantage is that you don't have to keep on doing this same check.

Types being the cause of difficulty in refactoring when business requirements change, is the opposite of my experience. In large dynamically typed codebases, being sure that a large refactor caught everything, is very hard / costly in test coverage. I have seen this go wrong many times. Having the compiler point out what you have missed, based on the types, is very helpful.

While I think the arguments are a bit weak, I do agree that it is at least unclear if Haskell is a sound choice as a production language at this time. Fighting against an ecosystem is not something you want to be doing while building your product. But in contrast to the author, I do think this is fixable, and see steps happening in the right direction (e.g. with IHP, but also with the efforts around the Haskell Foundation and the Haskell Language Server)

6 năm trướcdiscuss

The issue is that it hasn't translated into local production fabs. Which means that the semiconductor ecosystems in Belgium is still really small.

IMEC's research mainly benefits companies like Intel, Samsung, TSMC. The only relatively local link is ASML.

6 năm trướcdiscuss

ReScript (formerly called ReasonML), is an interesting option for web apps (especially the front-end part), if you are interested in Ocaml-like languages.

Language-wise, it is Ocaml with a different syntax layer, so all of the nice things about the language are still available.

6 năm trướcdiscuss

I second this. I dislike the liberal usage of metaprogramming, especially in the context of a code base that you have to work on with multiple people. It adds another layer of concepts/complexity/magic that everyone in the team has to learn.

With PPX, the barrier to use metaprogramming has become lower (especially compared to camlp4), resulting in most modern Ocaml codebases depending on it. But the ergonomics of the end-to-end software delivery have not really followed: the fact that PPX's break between compiler versions means that in practice, a compiler upgrade becomes a very costly thing to do. It is difficult to make the case for this in an industrial setting, where languages with almost religious backwards compatibility are common (Java, C++, even Go). For Ocaml, it is especially sad, because the core language is already quite powerful and expressive, lowering the need for metaprogramming (imo).

For scenarios where code generation is needed, I prefer a separate code generation step, which produces explicit "standard" code (e.g. like protoc)

6 năm trướcdiscuss

In your example, there is no relation between anything in Module1Interface and Module2Interface.

Probably closer would be (not sure if this is possible in Typescript):

    class CustomModule<S, T1 extends Module1Interface<S>, T2 extends Module2Interface<S> > {
      constructor(t1: T1, t2: T2) { ... }
    }

Meaning that, for example, within Module1Inteface, there is some function f1 that returns an S, and within Module2Interface, there is some function f2 that takes an S as argument.

This does become a bit tedious notation-wise, if possible at all. In Ocaml, this would look like:

  module CustomModule(M1: Module1)(M2: Module2 with type s = M1.s)

6 năm trướcdiscuss

I don't see how you come to that conclusion. Assigning a name to a requirement (~ an interface) is the most simple usage of the module. E.g. to define something like IComparable, you could write:

  module type Comparable = sig
    type t
    val compare : t -> t -> int
  end

6 năm trướcdiscuss

For me, the value of a module is that you can describe a set of multiple types and the functions that operate on these types, all in one place. In OO, there is essentially one type that is "special", the type of the class you define.

The power of the Ocaml module system is really in the functor, which the article only touches upon. You define a module and refer in its definition to types/functions of other modules that must be provided at the time you construct the module. Even better, you can define relations between the types and do type substitutions, e.g.: to construct this module, you need to give me 2 other modules, each with a specific set of functions, and for which type t1 of the first module, matches type t2 of the second module, while the actual type of t1/t2 does not matter.

See https://dev.realworldocaml.org/functors.html for more examples.

6 năm trướcdiscuss

Glass platters have been used a lot since the IBM "Deathstar" series. They are used in almost all 2.5 inch laptop drives.

6 năm trướcdiscuss

I am not sure that my message was completely clear: my point was that despite Go having an objectively old/primitive type system, compared to modern state of the art, that does not preclude it from being a successful "production" language. The value the type system brings is, in my experience, rarely the most important factor in the success/failure of a software project.

My second point was that the popularity of Go (or any language) does not mean that it is inherently a good language, where good can mean, productivity, defect rate, .... Adoption of a language depends on many things, I believe quality of the tools, availability of libraries, and good old marketing (the Google aura around Go has helped in adoption) are often more important. The effort to make an initial, crappy, proof-of-concept is in my experience often decisive for the choice of language. The cost of long term maintenance (where a good type system might bring value) is rarely considered.

For Go, it also hasn't hurt that some of the core developers where being paid by Google to work on it.

The metric of the "ability for mass amounts of programmers to program the computer", is indeed a very relevant one, and I think Go shines there. When reading Go code, I typically find it easy to understand what the goal of the code is. However, "to do the correct thing", is often less of a success: there are off-by-one error and corner cases that have bugs, and reimplementations of the same basic logic. If you have a lot of developers available, you simply have them fix these issues, and your Go project becomes a success.

My point is, that in this scenario, the language itself is of minor importance: success is determined by the fact that you have access to a large pool of relatively cheap labor of reasonable quality (Google or VC funded startups are perfect examples). A 20 or even 50% productivity gain by having a "better" language would simply not matter for the outcome, certainly not if it takes your labor longer to get up to speed.

So, in that context, compared to the alternatives, the quality of Go is relatively high. But I do think that, if during the design of Go they had taken a couple of basic concepts from ML like sum types, polymorphism, and a decent module system, they would have ended up with a language that would make the current Go look unproductive and poor quality, even in that same context. It wouldn't make a difference to Google, but it would make the life of many a developer a little bit more productive and joyful.

And yes, I do know who created Go, but arguments by authority carry little weight.

6 năm trướcdiscuss

I think what you witnessed is the fact that being an expert in programming language theory or category theory, does not make you a good software engineer.

Haskell is surprisingly well suited for back-end services (e.g REST). Warp (an HTTP server), Servant (a way to define your backend API), and Mio [1] (the Haskell network IO implementation) are amazingly performant and scalable.

Of course, that does not prevent you from messing up on the business logic level. I suspect that they lacked awareness of performance implications of their choices, this is not something Haskell will save you from. On the contrary, parts of the Haskell community do tend to go to over-abstraction, and I believe this does have a negative effect on people writing production software.

[1] https://dl.acm.org/doi/abs/10.1145/2503778.2503790

6 năm trướcdiscuss

I fail to see how else than "primitive" you can call the Go type system: it is roughly what you would get from a language designed in the 70's, and a lot of advances have since been made.

Whether or not these advances make a difference (positive or negative) in production is different question, but I'd rather not use "how much is a language used", as an inherent quality metric for a programming language. Otherwise, I fear we might all end up doing old-school Java, Javascript, Visual Basic and ABAP.

6 năm trướcdiscuss

I have not yet seen a storage system that is actually 100% S3 compatible. Either functionality is missing, and/or there are corner cases where the behavior is just slightly different. It is also a complicated API, that often looks like it is a reflection of some internal engineering choices that were made (e.g. the object metadata structure AWS uses probably is the reason why the list call you mentioned behaves the way it does on AWS S3)

However, in this case, MinIO makes the deliberate choice to deviate from AWS S3 behavior, on a very common call (listing objects). I do agree that this has the risk to break applications in non-obvious ways, e.g. the call succeeds, bu t your application behavior differs, compared to running against AWS S3, due to the missing directory entry.

I also find their argumentation in the ticket a bit worrisome, calling the AWS S3 behavior a "blunder". I saw the same hubris when looking at their data path, where they choose to ignore battle-tested approaches like Paxos / Raft for distributed coordination, and instead build their own distributed lock algorithm and implementation: it does not seem to persist state, so resilience to power failure and server/process crashes might not be what you expect.

6 năm trướcdiscuss

My experience with using general purpose languages for things like configuration or build systems has not been great. You always seem to end up with building a bunch of abstractions on top of that language, which culminates in another in-house configuration/build DSL, for which no public info is available.

It is also very hard to prevent people from using the escape hatch of the general purpose language to "fix a problem quickly", while introducing unpredictable side effects.

Scons (using Python) and Gradle (using Groovy / Kotlin) are 2 cases where I have seen this go very wrong, very quickly. I now strongly prefer Maven, even with its warts and XML verbosity.

On the other hand, configuration languages also often suck, the abomination that is HCL (e.g used in Terraform) comes to mind (especially in its early days). Turns out it is really hard to make a good language, and a bad configuration language is even worse than using a general purpose language.

I do like Dhall though: sufficiently powerful so you never have to repeat yourself, while also preventing you from doing fancy stuff you will later regret, all with a very readable syntax, and the type system helps to catch typos / mistakes early. I now use it to generate all config files for tools that consume json/yaml, and often for other text based config files as well (where you can use it as a powerful type-safe template engine)

6 năm trướcdiscuss

I worked at a startup (Amplidata) where we build a distributed storage system in OCaml. It got acquired by Western Digital, which later sold it to Quantum. Afaik, there is still development going on using OCaml.

Couple of things I took away from the experience:

- After an initial ramp-up, most of the developers liked or even loved the language. Downside was the limited tooling/libraries (2010-2015, seems better now).

- Quality of the resulting product was quite high: bug rate was fairly low, especially once experience with the language increased. Turns out sum types, gadts and the module system (functors) are very powerful to express invariants and design composable logic.

- OCaml was successfully picked up by team members without a formal computer science background, or even without a lot of programming experience. Not that it turns everybody into a great programmer, but it seemed to coerce new hires into delivering a working feature, without a huge risk of breaking a lot of other code.

- It is still possible to write crappy / unmaintainable code. But in OCaml it is often as easy (or even easier) to write good code.

- If something did turn out to be broken, it was painful. Debugger support (GDB) was initially non-existent. Under heavy load, we triggered a couple of bugs in libraries, which was quite painful to fix. On the other hand, we also triggered bugs in malloc and in the Linux kernel, which were as painful, so it might have had more to do with the high-load scenario, than with OCaml.

- For really low level stuff, or very high-performance cases, we did need to use the escape hatch to call C code. Interfacing Ocaml with C was not the most pleasant (was before ctypes library). Debugging bugs at that boundary was extremely unpleasant. One trigger for this was the fact that multicore OCaml never materialized, which also damaged the "political position" of OCaml within the company.

- Management (especially after the acquisition) was at best indifferent, and at worst hostile towards it. OCaml was perceived as being difficult to hire for (or outsource). Using more "standard" languages like Java, C++, Python was seen as the safer bet. They tried and often failed (C++ turns out to be damn complex, and maintaining a large Python codebase turned into whack-a-mole, but for bugs.).

6 năm trướcdiscuss

MIPS is still alive in the IP camera world. There exist very cheap SoC's (e.g. the Ingenic T20 - http://www.ingenic.com.cn/en/?product/id/14.html ), tailored towards making cheap network camera's (~ €20 retail price for the full camera). I guess at that price point, the ARM license fee does become visible in the bill of materials.