HackerTrans
TopNewTrendsCommentsPastAskShowJobs

rhymer

no profile record

comments

rhymer
·vor 8 Monaten·discuss
Right, this is known as the inverse variance weighting https://en.wikipedia.org/wiki/Inverse-variance_weighting.
rhymer
·vor 2 Jahren·discuss
You're right, thanks for pointing that out. I missed adding the reference: Kammler DW. A First Course in Fourier Analysis. 2nd ed. Cambridge University Press; 2008. Figure 1.19. This visualization of the relationships between time and frequency domains is a good illustration of Pontryagin duality applied to signal processing
rhymer
·vor 2 Jahren·discuss
This tool is fantastic! I was able to generate a Fourier-Poisson cube [0] in about 10 minutes, and the UI is incredibly intuitive.

The focus on commutative diagrams, rather than a free-form canvas, is a brilliant design choice that keeps it clean and easy to use. I wish I'd had this during my thesis; it would have saved so much time.

[0] https://q.uiver.app/#q=WzAsOCxbMCwxLCJnIFxcdGV4dHsgb24gfVxcb...
rhymer
·vor 2 Jahren·discuss
Be careful, the weight of Algorithm A by Efraimidis and Spirakis cannot be interpreted as the inclusion probability, and thus cannot be used in survey sampling to construct the Horvitz–Thompson estimator.

See "Remarks on some misconceptions about unequal probability sampling without replacement" by Yves Tillé.

Quoted from Tillé's conclusion: "There is a multitude of correct and fast method of sampling... there is no reason to use an incorrect method like weighted random sampling where we do not control the inclusion probabilities"

It's not clear to me how easy it is to implement the "multitude of corect and fast methods" in SQL, though. Would love to see some reference implementation.
rhymer
·vor 2 Jahren·discuss
If the bus arrives on time, the arrival time would be [tau, 2 * tau, ..., N * tau].

One way to simulate "random" arrival time is to draw uniform points in the interval [0, N * tau].

It turns out the inter-arrival time generated this way is approximately exponential:

1. the difference of consecutive ordered uniformly distribution random variables follows a Beta(1, N) distribution [1].

2. As N goes to infinity, N * Beta(1, N) converges to Exponential(1) [2].

3. Since we scale the rand() by N * tau, the inter arrival time will follow an Exponential(1 / tau) distribution (as N goes to infinity), which has an expected value of tau [3].

Edit: I just realized that the author did mention this simulation is only an approximation in the side note.

[1] https://en.wikipedia.org/wiki/Order_statistic#The_joint_dist...

[2] https://en.wikipedia.org/wiki/Beta_distribution#Special_and_...

[3] https://en.wikipedia.org/wiki/Exponential_distribution#Relat...
rhymer
·vor 2 Jahren·discuss
Relevance: from the cited paper, the variance of the median estimator is proportional to 1/(n * f^2), where n is the sample size and f is the density at median.

Two observations: 1. With sufficiently large n, you can control your variance to an acceptable level. 2. The factor f is outside of your control. If your problem has a small density around the median, then you'll need to throw more samples to compensate for it.

I think your concern is about #2: you can always construct a pathological distribution to make the sampling-based approach unattainable. The paper provided guidance on what to expect when you apply the sampling-based approach.
rhymer
·vor 2 Jahren·discuss
Asymptotic properties of quantile estimators are widely studied [1]. The key is to have a sufficiently large sample size.

[1] Bahadur, R. R. (1966). A note on quantiles in large samples. Annals of Mathematical Statistics, 37, 577–580.
rhymer
·vor 2 Jahren·discuss
My takeaway is to avoid mixing the frequentist and Bayesian approaches. Choose one method: either follow the frequentist approach and avoid early data analysis, or use the Bayesian approach to compute posterior probabilities once data are available. Mixing the two without expertise can lead to errors.