lekker · HackerTrans

They are a little verbose, though it is likely not to cause problems here. I suppose that it may slow things down a very tiny bit. The author could possibly have been trying to future-proof it with a bit more verbosity in the face of allowing for other keys (execute-mode, execute-permissions) etc.

lekker

9 anni fadiscuss

GPU hardware on Amazon is quite pricey. I would be careful since it can get out of hand rather quickly.

lekker

9 anni fadiscuss

Oh, that looks really interesting. I'll definitely give this a bash

lekker

9 anni fadiscuss

It had 512MB of RAM, which I realise is a bit rediculous to run anything like this. I don't have a whole lot of budget for R&D so I try to keep things cheaper where possible.

lekker

9 anni fadiscuss

All right, thank you. I'll look into the other options you mentioned

lekker

9 anni fadiscuss

Currently we have a Postgres database and run manual queries against it. One-off hackjob turned production.

lekker

9 anni fadiscuss

Okay, thank you. I've been looking into Rabbit quite a bit. I like the stream processing aspect of Kafka, where you can introspectively peek into things as they happen rather than a straight message queue.

Some of our stuff is on the GCP, so have also been investigating Pub/Sub and DataFlow, which seem more viable since we don't have the up-front cost of hardware and maintenance but pay for resources used. If we don't use much, we should not pay much.

lekker

9 anni fadiscuss

All right, that makes sense. It is likely that we'll see a peak in the 10s of thousands at a time, if that. Probably more as we move into the future.

I want to avoid deploying a piece of software now that we'll need to swap out in a year since it does not provide what we need anymore, though I suppose that is a natural part of scaling.

What would you recommend as an alternative to Kafka that does adequate real-time processing? The real-time-ness of it is important for our use-case. Maybe even some kind of time-series database system could work.

lekker

9 anni fadiscuss

I've been eyeing Kafka for a long time now - tried to set it up on a smaller DO box and had memory issues. Is it worth using this for smaller operations (<100k requests per day)? The stream processing seems extremely useful for my use case. Is there something else that may be a better fit?

comments