state_machine

9 years agodiscuss

FWIW, CockroachDB doesn't claim to beat CAP: https://www.cockroachlabs.com/blog/limits-of-the-cap-theorem

9 years agodiscuss

Yeah, that's fair. If I have to take LIRR, I'll generally use Atlantic and switch at Jamaica if needed -- anything to avoid Penn.

state_machine

9 years agodiscuss

it takes longer to load a train with assigned seating -- that's why say, Amtrak might do it, but commuter rail trains, that at peak run minutes apart and want to minimize dwell-times at highly contended platforms, optimize for loading speed.

state_machine

9 years agodiscuss

I've only done much less hellish Grand Central version, and I think Metro North publishes track assignments much earlier.

That said though, just imagine how much worse it would be if that mass rush was between platforms, fighting opposing traffic, instead of just from a (mostly) open waiting area.

state_machine

9 years agodiscuss

I can see why the MTA wouldn't want you telling people to go to a track before it is actually assigned -- if the train ends up on a different track, now you need to get all those people back up, off that platform an on to the new one, clogging stairs that that people who actually wanted that platform might be trying to use too. Obviously it would be nice to assign tracks earlier, so that people can head straight to the right platform, but sending people to potentially the wrong platform seems even worse.

EDIT: Penn is extremely platform/track constrained -- NJ Transit, LIRR and Amtrak are all sharing a fixed number of platforms, some of which are too short.

To maximize platform utilization, they have to wait until the last minute to finalize track assignments -- if you reserve one too early and the train ends up late, you're wasting an empty platform. Once you send a horde of people to a platform, moving them to a different one is a challenge (stairs/bottlenecks, communication, etc).

state_machine

9 years agodiscuss

I did not realize the physiological effects would be as severe:

> The fraction of carbon dioxide is growing: It just crossed 400 parts per million, and high-end estimates extrapolating from current trends suggest it will hit 1,000 ppm by 2100. At that concentration, compared to the air we breathe now, human cognitive ability declines by 21 percent.

state_machine

9 years agodiscuss

In case anyone is interested in even more of the technical specifics, the original design RFC might be interesting too: https://github.com/cockroachdb/cockroach/blob/master/docs/RF...

state_machine

9 years agodiscuss

One of the challenges for us in implementing something like LISTEN/NOTIFY comes from our distributed nature: since a table is likely broken up across many nodes, you somehow need to aggregate changes from all of them back into a single change feed wherever the listener is, and in such a way that it doesn't create a single point of failure.

state_machine

9 years agodiscuss

Yes, change feeds (and triggers) are on the roadmap (though not yet in active development).

state_machine

9 years agodiscuss

We're still working on integration with other monitoring systems, but the one we've tested the most and documented is prometheus: https://www.cockroachlabs.com/docs/monitor-cockroachdb-with-...

Additionally, you can get some of the same status info on the dashboard using the `node status` command (https://www.cockroachlabs.com/docs/view-node-details.html).

state_machine

9 years agodiscuss

Whoops, sorry about that. If you're looking for more on how it works (rocksdb, raft, distributed transactions across multiple raft groups, etc), you might find the design doc interesting: https://github.com/cockroachdb/cockroach/blob/master/docs/de...

state_machine

9 years agodiscuss

[cockroachdb employee]

Short answer: no.

Long answer: at their closest earth and mars are about 54m km apart, at the furthest it's over 400, with an average of around 225m km, so theoretical latency is varies between 4 and 24 minutes.

CockroachDB uses synchronous replication via raft, and that latency would cause problems as would some other setting like our window sizes and their interaction with timeouts.

state_machine

10 years agodiscuss

The 14 issues tagged 1.7.1:

  net: retry DNS lookups before failure?
  io: endless loop in MultiReader in Go 1.7
  path/filepath: EvalSymlinks is broken for relative paths on Windows
  net/http/httputil: Proxy terminates HTTP/2 stream before reading response body.
  hash/crc32: wrong output for unaligned input on s390x
  cmd/compile: incorrect assignment to uint64 via pointer converted to *uint16 (new in 1.7)
  doc: deprecation message for Transport.CancelRequest is not correct Documentation
  compress/zlib: Writer appears to ignore underlying writer errors at times.
  net: NATs client can't connect to server when client built with go1.7: "dial tcp: no suitable address found"
  doc: go1.7 release notes include typo for TLSConfig.NextProtos Documentation
  reflect: ChanOf makes "han" types instead of "chan" types
  x/mobile: Binding go mobile framework on iOS 9 with golang1.7rc6 crash when call debug.FreeOSMemory()
  net/http: nil pointer dereference in closeConnIfStillIdle
  website: retina favicon Suggested

state_machine

10 years agodiscuss

Personally I value (lack of) variance much more than absolute duration: I can easily plan around a either a 19 or 22 minute commute as long as that's consistent -- it's the needing to allow for 20-40 (looking at you, MUNI) that made planning hard, and that I'll pay a premium to avoid.

state_machine

10 years agodiscuss

The previous post in this series toured the `io` package: https://medium.com/@benbjohnson/go-walkthrough-io-package-8a...

state_machine

10 years agodiscuss

[engineer at cockroach]

How high you can set it depends on your access patterns -- there is some overhead to iterating though MVCC revisions during reads, in addition to the on-disk space you mention.

If your workload involves frequent writes to the same rows, GCing some of those revisions sooner would have a greater impact, whereas if you have a write-light workload, or if your writes are spread over rows such that a given row doesn't doesn't see frequent repeated updates, then you could probably use much a higher GC threshold with minimal overhead.

state_machine

10 years agodiscuss

[engineer at cockroach]

Basically, yes, all keys the KV, including the index entries, include a timestamp suffix.

Repeated edits to the same value will accumulate MVCC revisions until they are GCed, and those will be iterated over during access, so, as you point out, a use-case like yours would likely benefit from shorter a GC cutoff and GC _is_ configurable in cockroach.

state_machine

10 years agodiscuss

More details on the program: https://www.edf.org/climate/methanemaps and the other cities they've mapped: https://www.edf.org/climate/methanemaps/city-snapshots

state_machine

11 years agodiscuss

More info on the issue: http://www.openssh.com/txt/release-7.1p2

"experimental support for resuming SSH-connections (roaming) ... could be tricked by a malicious server into leaking ... private client user keys."

state_machine

Submissions

Yields Are Up, Latencies Are Down: Goroutine Scheduling in CockroachDB

How we rebuilt bulk operations to preserve a history of changes

Real-Time Holiday Consumer Spending Dashboard

Wind Energy Without Turbines

When Paris Closed a Major Road to Cars, Half Its Traffic Just Disappeared

Go 1.7.1 Released

Are Cities Too Complicated?

New data visualisation shows the beating heart of London

The morbid and mortal toll of sprawl

California is about to find out what a truly radical climate policy looks like

New Yorkers Will Pay $56 a Month to Trim a Minute Off Their Commute

NHTSA and FMCSA Propose Speed Limiters

Dubai's Hyperloop Dreams Are Shockingly Sensible

User Studies: Schools Add Washing Machines, See Attendance Improve

The Go Gopher – A Character Study [Renee French at GopherCon 2016]

How Marconi Gave Us the Wireless World

Go 1.7 is released

Go 1.7 Release Notes

Tour of Go's encoding Package

Go Walkthrough: bytes and strings packages

comments