something tells me all these bells and whistles around gpt are signs that scaling laws have plateaued, otherwise OpenAI et al. would focus more on improving model quality.
Maybe GPT-4 is the 1080p of LLMs: Noticeably better than 720p and 480p models, and not bad enough to warrant additional improvements.
Sure, 4K, 8K, ... are technologically available, but for the majority of use cases, 1080p is enough. Similarly, even though o1 and other models are technically feasible, for most cases the current models are enough.
In fact, GPT-4 is more than enough for 80% of tasks (text summarization, Apple (un)Intelligence, writing emails, tool use, etc.)—small models (<32B) are perfectly fine for those tasks (and they keep getting better too.)
Yes, surely they only have one type of Software Engineer and they all know how to improve model quality.
Alternatively, does it not seem more likely that they have different product groups? Surely the folks working on ChatGPT are an entirely different beast than the folks working in model development?
Yes, surely a sarcastic reductio ad absurdum of what was was said will inspire dialogue.
I think the GP's point is that their investing in new distribution channels could mean ROI in models has diminished significantly. Incidentally, I disagree with GP that's what this means-- this is another investment in brand awareness, AND data for multi-modal/audio. They might have gotten to 1080p for text chat but definitely not for voice chat.
Nothing more absurd than your response. OpenAI has a large engineering staff, it’s foolish to say they are all working on advancing models. The folks working on ChatGPT are going to continue working on ChatGPT. Let’s not even forget that O1 just got released recently.
Nothing I said was absurd in response to making an unsupported idea that model development has plateaued.
I don’t get this. Define focus and how is just improving model quality gonna allow OpenAI to survive, they need a mix of commercialization and model improvement. No $$, no gpus, no researchers, no improvements
thing tells me all these bells and whistles around gpt are signs that scaling laws have plateaued, otherwise OpenAI et al. would focus more on improving model quality.
o1-pro is that model. Expensive and slow, but significantly better at many tasks that involve CoT reasoning.
o1 is way better than gpt-4 imo, feel that many people just don't have complicated tasks/questions they have to do in their day to day. it's like a half jump between 3.5 and 4 to me
That's not been my experience, though I guess it depends on what you're using o1 for.
My experience is that o1 is extremely good at producing a series of logical steps for things. Ask it a simple question and it will write you what feels like an entire manual that you never asked for. For the most part I've stopped caring about integrating AI into software, but I could see o1 being good for writing prompts for another LLM. Beyond that, I have a hard time calling it better than GPT-4+.
lots of coding tasks, discussions about physics/QM. I find that it produces better quality answers than 4o, which often will have subtle but simple mistakes.
Even writing, where it is supposed to be worse than 4O, I feel that is does better/has a more solid understanding of provided documents.
Interesting, could you share an example of this where it provides something of value? I've tried asking a few different LLMs to explain renormalization group theory, and it always goes off the rails in five questions or less.
My guess is that the model got good enough to make its own bells and whistles — even the original 3.5 was good enough to make its own initial chat web UI.
I know it was that good, because I got it to do that for me… and then the UI kept getting better and the expensive models became the free default option and I stopped caring.
Maybe GPT-4 is the 1080p of LLMs: Noticeably better than 720p and 480p models, and not bad enough to warrant additional improvements.
Sure, 4K, 8K, ... are technologically available, but for the majority of use cases, 1080p is enough. Similarly, even though o1 and other models are technically feasible, for most cases the current models are enough.
In fact, GPT-4 is more than enough for 80% of tasks (text summarization, Apple (un)Intelligence, writing emails, tool use, etc.)—small models (<32B) are perfectly fine for those tasks (and they keep getting better too.)