CORDIC is more accurate, but takes as many iterations as you have bits of precision in your angle. Another demo called Warp in this contest used pipelined CORDIC to do atan2 on every pixel to create a tunnel, which is super impressive.
Yeah, it is exact in this specific circumstance. But yes, it's exactly the same trick; I also enjoyed that video in my Youtube recommender feed last week!
Yes, it has to in fact. If you have zero context to attend to in a transformer, and you try to predict the first token, you effectively are multiplying a zero-vector by the attention head, making all tokens equally likely in the final softmax (unless the lm_head has a bias, but at least in GPT it does not).
So the <|beginning of text|> token, with no context before it, learns to predict the first-token-in-a-document distribution. That's not quite the same as predicting nothing at all.
For finding interesting Ethereum addresses on the GPU (uses SHA3 instead of MD5) there's a tool called profanity.
Amusingly the tool was archived by the author years ago, and just last year a stupid bug in the random initialization code was found (32 bytes of state initialized with ~31 bits of entropy), leading to ~millions of $ in stolen ETH by exploiters.
I haven't looked at the project but would assume that several people generated their wallets by using this tool. The attackers were thus able to generate keys for existing wallets by simply bruteforcing the measly 31 bits that were used in the tool's random initialization.
> If anyone has an idea why the TIA’s designers used LFSRs for this stuff, I’d love to hear about it.
I'm guessing because a counter would have ripple carry (i.e. five extra gate delays when rolling over from 111111 to 000000) or need extra gates for carry lookahead and an LFSR is constant-delay.
The latency notwithstanding, you can implement a counter with an LFSR that hits every unique value in a 2^n-1 sequence with less transistors than a standard counter.
I have to admit I stole the idea from this 256-byte demo (after it wiped the floor with the one I made at a demo party): http://www.pouet.net/prod.php?which=5327
https://www.youtube.com/watch?v=K9mu3getxhU&t=780s