I'm confused about ARC-AGI. I thought the point of it was that you train a foundational model. Then you test it against ARC-AGI to figure out how well it reasons. Here and in some of the other reasoning papers, they are training on ARC-AGI. How much sense does that make in practice?

ARC-AGI allows (and encourages) training on their training set. Their evaluation setup is rigorous enough to avoid leaking between training and testing (public and private).

It's the classic breakup story, one party is just done and want to cut contact, but the other party is still hopeful and wants to find a way to restore the relationship. The EU is not independent minded in the same way that China and Russia are. That's the problem with them. Their leaders don't want to act independently from the US, because the European wealthy and politically connected classes consider themselves transatlantic, and they want to keep enjoying close ties to the US, even as the US pulls away.

Yep, I thought they might have some secret sauce in terms of training techniques, but that doesn't seem to be the case.

"Christian nationalist" is a term that on the one hand, insults Christians, and on the other hand flatters these power hungry grifters. Why would you use such a term?

> they also do not dehumanize large swaths of the population

They dehumanize and kill everyone equally.

I think the West (US especially) has a great system of government figured out, and I wouldn't try to say any group attempting to break that is better than another. It's not disingenuous to say that both are authoritarians who kill millions.

Please don't be childish. I have no love for communism and I get that you probably have some personal grudge against communism because of what your family has gone through. But the way you are expressing your hatred of communism is diminishing the evil committed by the NAZIs and the suffering of their victims, and that's just not right. And it's also not winning you any friends and you'd be better served if you showed solidarity with victims of other evil regimes, instead of trying to up them with your personal victim-hood.

The above commenter's point was actually drawing a distinction between these, and "one-upping" by saying there was a difference, or worse one, out of these extremist ideologies.

I disagreed and said that they are both equally horrid.

Expressing equal hatred for people and systems that kill countless millions through starvation, gulags/concentration camps, and disappearing them, is not childish. I have a ton of solidarity with victims of other evil regimes.

AlexNet was only released in 2011. The progress made in just 14 years has been insane. So while I do agree that we are approaching a "back to the drawing board" era, calling the past 14 years a "failure" is just not right.

If NVIDIA is blocked in China and China keeps developing AI models of comparable quality using home grown chips, it puts into question the dominance of NVIDIA in the market. If NVIDIA is allowed to continue operating in China, it doesn't matter much if China actually uses NVIDIA or Huawei, as long as there is some plausible deniability that China is using NVIDIA powered clusters for their strongest models.

There are a few "optimization startups". But in this context I find it a bit ironic that pretty much everyone is working with the same architecture, and the same hardware for the most part, so actually there isn't really that much demand for bespoke optimizations.

Those that are serious are paying through the nose for their engineers to work on these optimizations. Your competitor working on "the same hardware" does not magically make your MFU go up.

And when you have enough spending to account for 1%+ of revenue for the AI hardware companies?

You can get the engineers from those very hardware companies to do bespoke optimizations for your specific high load use cases. That's something a startup would struggle to match.

> When I look around, I see hundreds of billions of dollars being spent on hardware – GPUs, data centers, and power stations. What I don’t see are people waving large checks at ML infrastructure engineers like me and my team.

That doesn't seem to be the case to me. I guess the author wants to do everything on his own terms and maybe companies aren't interested in that.

There's probably a bit more to it. It really only takes one company to bet on optimizing infrastructure, to the degree that the author suggests to undermine the entire house of cards being built on Nvidia GPUs currently, yet not one AI company is willing to take that bet?

The author could also be correct. Investors tend to be herd animals, and if you're not buying into the same tech as everyone else, your proposal is higher risk. It might very well be easier to say to an investor that you're going to buy a million Nvidia GPUs and stuff them in a datacenter in Texas like everyone else.

I'm interested in the one company that does take the bet on infrastruture optimization. If that works, then a lot of people are going to lose a lot of money really quickly.