HackerTrans
TopNewTrendsCommentsPastAskShowJobs

rhymer

no profile record

comments

rhymer
·8 mesi fa·discuss
Right, this is known as the inverse variance weighting https://en.wikipedia.org/wiki/Inverse-variance_weighting.
rhymer
·2 anni fa·discuss
You're right, thanks for pointing that out. I missed adding the reference: Kammler DW. A First Course in Fourier Analysis. 2nd ed. Cambridge University Press; 2008. Figure 1.19. This visualization of the relationships between time and frequency domains is a good illustration of Pontryagin duality applied to signal processing
rhymer
·2 anni fa·discuss
This tool is fantastic! I was able to generate a Fourier-Poisson cube [0] in about 10 minutes, and the UI is incredibly intuitive.

The focus on commutative diagrams, rather than a free-form canvas, is a brilliant design choice that keeps it clean and easy to use. I wish I'd had this during my thesis; it would have saved so much time.

[0] https://q.uiver.app/#q=WzAsOCxbMCwxLCJnIFxcdGV4dHsgb24gfVxcb...
rhymer
·2 anni fa·discuss
Be careful, the weight of Algorithm A by Efraimidis and Spirakis cannot be interpreted as the inclusion probability, and thus cannot be used in survey sampling to construct the Horvitz–Thompson estimator.

See "Remarks on some misconceptions about unequal probability sampling without replacement" by Yves Tillé.

Quoted from Tillé's conclusion: "There is a multitude of correct and fast method of sampling... there is no reason to use an incorrect method like weighted random sampling where we do not control the inclusion probabilities"

It's not clear to me how easy it is to implement the "multitude of corect and fast methods" in SQL, though. Would love to see some reference implementation.
rhymer
·2 anni fa·discuss
If the bus arrives on time, the arrival time would be [tau, 2 * tau, ..., N * tau].

One way to simulate "random" arrival time is to draw uniform points in the interval [0, N * tau].

It turns out the inter-arrival time generated this way is approximately exponential:

1. the difference of consecutive ordered uniformly distribution random variables follows a Beta(1, N) distribution [1].

2. As N goes to infinity, N * Beta(1, N) converges to Exponential(1) [2].

3. Since we scale the rand() by N * tau, the inter arrival time will follow an Exponential(1 / tau) distribution (as N goes to infinity), which has an expected value of tau [3].

Edit: I just realized that the author did mention this simulation is only an approximation in the side note.

[1] https://en.wikipedia.org/wiki/Order_statistic#The_joint_dist...

[2] https://en.wikipedia.org/wiki/Beta_distribution#Special_and_...

[3] https://en.wikipedia.org/wiki/Exponential_distribution#Relat...
rhymer
·2 anni fa·discuss
Relevance: from the cited paper, the variance of the median estimator is proportional to 1/(n * f^2), where n is the sample size and f is the density at median.

Two observations: 1. With sufficiently large n, you can control your variance to an acceptable level. 2. The factor f is outside of your control. If your problem has a small density around the median, then you'll need to throw more samples to compensate for it.

I think your concern is about #2: you can always construct a pathological distribution to make the sampling-based approach unattainable. The paper provided guidance on what to expect when you apply the sampling-based approach.
rhymer
·2 anni fa·discuss
Asymptotic properties of quantile estimators are widely studied [1]. The key is to have a sufficiently large sample size.

[1] Bahadur, R. R. (1966). A note on quantiles in large samples. Annals of Mathematical Statistics, 37, 577–580.
rhymer
·2 anni fa·discuss
My takeaway is to avoid mixing the frequentist and Bayesian approaches. Choose one method: either follow the frequentist approach and avoid early data analysis, or use the Bayesian approach to compute posterior probabilities once data are available. Mixing the two without expertise can lead to errors.
rhymer
·2 anni fa·discuss
I'm grateful for leetcode. Despite my background in electrical engineering, where I specialize in statistical signal processing, I never had the opportunity to delve into algorithm or data structure courses during my college years.

In my field, understanding signals and systems, probability, optimization, and numerical computing is very important.

Leetcode, in comparison, offers a more confined scope, making preparation more manageable and systematic, ultimately helping me break into software engineering.
rhymer
·3 anni fa·discuss
1/ I think you are referring to pushforward measure (https://en.wikipedia.org/wiki/Pushforward_measure): the random variable "pushes" the probability measure to its codomain. 2/ pdf requires a stronger condition: the pushforward measure needs to be absolutely continuous with respect to the sigma-finite measure (usually the Lebegue measure) on the codomain.
rhymer
·3 anni fa·discuss
Agree, this is great! I wonder if there're some ML equivalent sites that present topics in a modular way?
rhymer
·3 anni fa·discuss
Maybe "Probability via Expectation" by Peter Whittle https://link.springer.com/book/10.1007/978-1-4612-0509-8 or "Infinite Dimensional Analysis" by Charalambos Aliprantis and Kim Border https://link.springer.com/book/10.1007/3-540-29587-9?