I've also noticed Google having indexing issues over the past ~year:
Some popular models on Hugging Face never appear in the results, but the sub-pages (discussion, files, quants, etc.) do.
Some Reddit pages show up only in their auto-translated form, and in a language Google has no reason to think I speak. (Maybe there's some deduplication to keep machine translations out of the results, but it's misfiring and discarding the original instead?)
Reddit auto translation is horrible. It’s an extremely frustrating feeling, starting to read something in your language believing it’s local, until you reach a weird phrase and realise it’s translated English.
It’s also clearly confusing users, as you get replies in a random language, obviously made by people who read an auto translation and thought they were continuing the conversation in their native language.
I will google for something in French when I don't find the results I want in English. Sometimes google will return links to English threads (that I've already seen and decided were worthless!) auto-tranalated to French. As if that were any help at all..
The issues with auto-translated Reddit pages unfortunately also happens with Kagi. I am not sure if this is just because Kagi uses Google's search index or if Reddit publishes the translated title as metadata.
I think at least for Google there are some browser extensions that can remove these results.
The Reddit issue is also something that really annoys me and i wish kagi would find some way to counter it. Whenever I search for administrational things I do so in one of three languages, German, French or English depending on which context this issue arises in. And I would really prefer to only get answers that are relevant to that country. It's simply not useful for me to find answers about social security issues in the US when I'm searching for them in French.
I think it's catnip for programmers, myself included. (See also: boids, path traced renderers, fluid simulations, old fashioned "generative"/plotter art, etc. - stuff with cool visual output)
Boids, Game of Life, Genetic Algorithms, Pixel Shaders...
All so satisfying to play with.
One of my favorites was when I was sure I was right about the Monty Hall problem, so I decided to write a simulator, and my fingers typed the code... and then my brain had to read it, and realize I was wrong. It was hilarious. I knew how to code the solution better than I could reason about it. I didn't even need to run the program.
What's crazy is that it only does the easy stuff (planting and watering). What we need is a robot to do the hard stuff (in my home-gamer opinion: pest control and weeding; maybe picking is most relevant for commercial agriculture).
Not sure if it comes out of the box, but it can also do simple pest control and weeding. Mechancical stomping plants at the wrong position or spraying with chemicals.
Harvesting would be fine for me to do by hand, because that is indeed he really hard part, especially with mixed crops.
Speaking personally, I've never broken/damaged a phone. Since the Pixel 1 started requiring removal of the screen in order to swap the battery, 100% of my phone replacements have been because the battery isn't good anymore. (Granted, I would've gotten a new phone eventually regardless, when the old one stopped receiving security updates.)
Yeah, burying this on page 8 is a bit suspect imo (the eval datasets are listed on page 3, so if you were familiar with them you would have a hint then).
The distillation of a student that predicts "anchor layers" and then acts as a backbone for classification is perfectly cool on its own; no need to stretch the title/abstract so much.
agreed re: title/abstract stretching. good work stands on its own without needing hype. "we found a nifty way to distill llama-70b using a much smaller student transformer model; the key is using intermediate activation layers in a compressed representation" would be about as effective at selling it while being more immediately approachable IMO
I agree - the results on the finetunes are not very surprising. The trained-from-scratch ResNets (Figure 2 and Section 3.2.1) are definitely more interesting, though somewhat limited in scope.
In any case, my impression is that this is not immediately more useful than a LoRA (and is probably not intended to be), but is maybe an avenue for further research.
I don't think its that surprising actually. And I think the paper in general completely oversells the idea.
The ResNet results hold from scratch because strict local constraints (e.g., 3x3 convolutions) force the emergence of fundamental signal-processing features (Gabor/Laplacian filters) regardless of the dataset. The architecture itself enforces the subspace.
The Transformer/ViT results rely on fine-tunes because of permutation symmetry. If you trained two ViTs from scratch, "Attention Head 4" in Model A might be functionally identical to "Head 7" in Model B, but mathematically orthogonal.
Because the authors' method (SVD) lacks a neuron-alignment step, scratch-trained ViTs would not look aligned. They had to use pre-trained models to ensure the weights shared a coordinate system. Effectively, I think that they proved that CNNs converge due to it's arch, but for Transformers, they mostly just confirmed that fine-tuning doesn't drift far from the parent model.
I think its very surprising, although I would like the paper to show more experiments (they already have a lot, i know).
The ViT models are never really trained from scratch - they are always finetuned as they require large amounts of data to converge nicely. The pretraining just provides a nice initialization. Why would one expect two ViT's finetuned on two different things - image and text classification end up in the same subspace as they show? I think this is groundbreaking.
I don't really agree with the drift far from the parent model idea. I think they drift pretty far in terms of their norms. Even the small LoRA adapters drift pretty far from the base model.
You’ve explained this in plain and simple language far more directly than the linked study. Score yet another point for the theory that academic papers are deliberately written to be obtuse to laypeople rather than striving for accessibility.
Vote for the Party that promises academic grants for people that write 1k character long forum posts for the laypeople instead of other experts of the field.
I don't think the parent post is complaining that academics are writing proposals (e.g as opposed to people with common sense).
Instead, it seems to me that he is complaining that academics are writing proposals and papers to impress funding committees and journal editors, and to some extend to increase their own clout among their peers. Instead of writing to communicate clearly and honestly to their peers, or occasionally to laymen.
And this critique is likely not aimed at academics so much as the systems and incentives of academia. This is partially on the parties managing grants (caring much more about impact and visibility than actually moving science forwards, which means everyone is scrounging for or lying about low hanging fruit). It is partially on those who set (or rather maintain) the culture at academic institutions of gathering clout by getting 'impactful' publications. And those who manage journals also share blame, by trying to defend their moat, very much hamming up "high impact", and aggressively rent-seeking.
Yes, thank you, exactly. It’s a culture and systems issue. Thank you for clarifying a post I wrote in the early morning while waiting for my baby to fall back to sleep!
Well, if you take the screenshot+absolutely positioned invisible anchors approach that Claude eventually settled on (in half an hour), you could probably do it much quicker. Of course having knowledge of the solution like that is kind of cheating.
Interesting - these models are all trained to do pixel-level(ish) measurement now, for bounding boxes and such. I wonder if you could railroad it into being accurate with the right prompt.
What models are good at this? I have tried passing images to models and asking them for coordinates for specific features, then overlaid dots on those points and passed that image back to the model so it has a perception of how far out it was. It had a tendency to be consistently off by a fixed amount without getting closer.
I don't doubt that it is possible eventually, but I haven't had much luck.
Something that seemed to assist was drawing a multi coloured transparent chequerboard, if the AI knows the position of the grid colours it can pick out some relative information from the grid.
I've found Qwen3-VL to be fairly accurate at detection (though it doesn't always catch every instance). Note that it gives answers as per-mille-ages, as if the image was 1000x1000 regardless of actual resolution or aspect ratio.
I have also not had luck with any kind of iterative/guess-and-check approach. I assume the models are all trained to one-shot this kind of thing and struggle to generalize to what are effectively relative measurements.
Feels like the "right" approach would be to have it write some code to measure how far off the elements are in the original vs recreated image, and then iterate using the numerical output of the program...
Some popular models on Hugging Face never appear in the results, but the sub-pages (discussion, files, quants, etc.) do.
Some Reddit pages show up only in their auto-translated form, and in a language Google has no reason to think I speak. (Maybe there's some deduplication to keep machine translations out of the results, but it's misfiring and discarding the original instead?)
reply