Rendered at 09:27:28 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
wodenokoto 5 hours ago [-]
I don't understand the argument that COUNT(DISTINCT ...) cannot be parallelized, but you can count distinct values in parallel using COUNT(*) GROUP BY ...
But then again the article is being accused of heavy LLM. If you notice that COUNT(DISTINCT ...) doesn't parallelize and you ask an LLM to argue why, you get easily get an article without a good thread.
muth02446 13 hours ago [-]
If you are satisfied with an approximation of "DISTINCT", there is a facinating
probabilistic algorithm by Flajolet and Martin
But then again the article is being accused of heavy LLM. If you notice that COUNT(DISTINCT ...) doesn't parallelize and you ask an LLM to argue why, you get easily get an article without a good thread.
https://en.wikipedia.org/wiki/Flajolet%E2%80%93Martin_algori...
which fits on 10 lines and does not require sorting. Improved versions of it are LogLog and HyperLogLog.
https://www.sketchingbigdata.org/