jaysh · HackerTrans

jaysh

no profile record

comments

28 ngày trướcdiscuss

Off the top of my head:

- substantially better performance on the same hardware, even moreso for larger range queries (multiple days)

- no new query language to learn

- significantly more expressive as you said

- agents for scraping logs use way less CPU (I used to use grafana-agent which used about 80%, vector uses sub 5%)

- very intuitive to manage TTLs - I can keep some logs for 10 years, and some for 1 week based on the event in the JSON

- more compact storage, I didn't check scientifically but CH storage is better compressed at least 4-5x for us

- no running into maximum stream limits - struggled with these even on Grafana Cloud and didn't realise we silently lost a lot of logs

Honestly: why wouldn't you. Loki always felt like a mistake to me. A brand new query language, really counter-intuitive configuration, large ramp-up time for complex queries, lots of arguing about labels/cardinality etc. It all goes away when you drop it. I think logging should not be exotic or behave in unexpected ways.

28 ngày trướcdiscuss

You only need the plugin: https://clickhouse.com/docs/observability/grafana - then you get basically everything natively.

28 ngày trướcdiscuss

Just replied to that question! Let me know if you have other questions.

28 ngày trướcdiscuss

Not really, ClickHouse is super forgiving so you can do something like:

    CREATE TABLE default.events (
      `timestamp` DateTime
      `event` String -- e.g. 'product.updated' or empty/null
      `message` -- human readable message
      `raw` -- the raw message - this is very useful when pushing logs that aren't JSON - you just let the `event` be null and dump the entire message here
    )
    ENGINE = MergeTree
    PARTITION BY toDate(timestamp)
    ORDER BY (timestamp, event)
    TTL timestamp + toIntervalMonth(6)

ClickHouse is extremely performant even in the cases of e.g.: SELECT count(*) FROM `events` WHERE `raw` LIKE '%hello world%'

Of course, the more columns you splat out (e.g. like correlation_id, user_id, order_id, etc) the better you can index and expect those queries to perform but in general I don't bother outside the obvious core domain ones (exampled above), the performance is so good that unindexed queries are significantly faster than indexed queries in Loki. I have reached the point where I JSON extract on-the-fly for the WHERE clause with very large queries with no meaningful performance issues.

28 ngày trướcdiscuss

Still via Grafana. I ran it side-by-side with Loki and despite trying to optimise Loki and using ClickHouse out of the box - it really was shocking how much faster ClickHouse was for every single query (e.g. in the last 12 hours give my the frequency of logs with a particular JSON event or even "find this log entry, then join back and find the number of times a different entry appears within the same correlation_id)

29 ngày trướcdiscuss

ClickHouse replacing Loki finally made our observability stack feel 'right'. It really is a powerhouse for logs and general analytical queries.

6 năm trướcdiscuss

That's part of it, but not the entire solution.

http://highscalability.com/blog/2013/7/8/the-architecture-tw...

> Outliers, those with huge follower lists, are becoming a common case. Sending a tweet from a user with a lot of followers, that is with a large fanout, can be slow. Twitter tries to do it under 5 seconds, but it doesn’t always work, especially when celebrities tweet and tweet each other, which is happening more and more. One of the consequences is replies can arrive before the original tweet is received. Twitter is changing from doing all the work on writes to doing more work on reads for high value users.

6 năm trướcdiscuss

It's worth noting that YouTube is optimising for the opposite: they want to prevent artificial view count inflation. Therefore, this behaviour (the view count not increasing when the videos are automatically accessed) would make sense.

If you're interested, an interesting measure for difficulty is how much companies charge to "buy youtube views" (e.g, $16 per 1000 views is ~8x the cost you'd pay for typical CAPTCHA solving).

7 năm trướcdiscuss

Spotify are not enabling audio piracy by much because they don't provide an audio stream for songs via their API (beyond a short sample).

7 năm trướcdiscuss

Looks slick! I've used Google Calendar to achieve something similar in the past but this looks better.

Minor suggestion would be to remove the unlimited text messages from the plans. It's harder to change later when this takes off, is ripe for abuse, and (to me) makes it feel unsustainable: everyone know unlimited is not actually unlimited. I'd reserve that for for the "call us" enterprise pricing options.

7 năm trướcdiscuss

There's one airline in particular that I dislike simply due multiple bad experiences. I vocalise this significantly. I still book their flights if it's the cheapest: I'd argue to myself that (a) flying is a transient experience and (b) I'll use the saving for a 'free' lounge pass, taxi to the airport, etc.

I wouldn't assume that just because people vocalise brand loyalty, means that they stick with it when it's time to open their wallets.

7 năm trướcdiscuss

Landed in Sydney a few weeks ago. First visit so I ordered an Uber (as I've used it a lot, trust it, etc.).

After we were picked up, the driver helpfully informed us if we'd ordered the same ride via Ola (he drives both), it'd have been ~50% cheaper (as a new user).

Naturally, I installed it and now I'll switch between Uber and m Ola. This then expands to Bolt, Grab, and so on, and you realise it's virtually the same experience with different branding and the sole differentiating factor is price.

I definitely buy the convenience aspect: it's way more comfortable to pull out your phone and order a ride on a platform you're already familiar with. My trouble is the barrier is so low to install another app (and new joiner incentives are financially motivating) that this leads to virtually no brand loyalty.

Perhaps in a different world, where there are no up significant upfront financial incentives and perhaps drivers were exclusive to a platform (i.e. to pass their rigorous screening processes is a sign of quality), that'd certainly buy some loyalty from me.

7 năm trướcdiscuss

It's easy to stick with something when it's working for you:

> If I need to go somewhere, or get home, Uber has never let me down.

The real test of brand loyalty it staying with them after a few bad experiences.

7 năm trướcdiscuss

Not OP, but in the same boat, and currently travel regularly. I always almost all of them installed (e.g. Uber/Lyft/Ola/Bolt/Grab etc.) and pick the cheapest. For now, there's the added incentive that you never pay full-fare for your first ride, too, so new competition is great.

7 năm trướcdiscuss

Thank you so much for your incredible public service. I wish I could offer more than words. Lots of organisations discuss their values, but when push comes to shove, I don't believe many would even consider, let alone commit resources to, something that has such positive externalities.

7 năm trướcdiscuss

I regularly see your comment (or ones for similar services) on these types of posts.

I assume the strategy is working as you keep replying, but have you considered starting posting yourself, as a measure of "you can independently verify how quickly we spotted this outage"?

(For what it's worth, for the alert component you can do it yourself for free by subscribing to individual status pages (assuming they have the option) to the email-to-Slack integration.)

7 năm trướcdiscuss

As long as your (production) config files are not checked in with the code, then little difference. The challenge then is: where do you store them?

Environment variables had the early upside of being first class in any virtually any deployment strategy - from Bash wrappers, supervisord to Docker and Kubernetes - so it's pretty much plug and play. For example, Docker has config files, but they're mounted to a specific place in your container, so that means your application is now married to that location.

7 năm trướcdiscuss

Route* not right. As an example, you can send traffic destined for 142.0.0.0/8 via the SSH connection, and leave everything else untouched. Any application connecting to that IP range will be automatically tunnelled via SSH.

7 năm trướcdiscuss

The key difference is you can right full or partial traffic (by subnet) over SSH with applications not being aware - individual applications do not require SOCKS support.

7 năm trướcdiscuss

If you're sticking with recurring, you'll still need to keep the data for some time after the payments stop. The PR will be disastrous for you if someone passed away, and therefore the payments stop, and then all of their data is lost (worse: now imagine consequential financial loss from the loss of their data). I don't know whether you could be held liable for that or not.