It's like anything - The devil is in the implementation details.
I've found the most leverage with agent-based LLM solutions comes from including feedback from prior iterations.
For example, if you are trying to write a text-to-SQL bot you should actually attempt to parse any generated query using the desired provider and then pass any errors back into the prompt. You can also incorporate snippets of result sets (if successful parse) as another form of feedback. In my experience, this can make an incredible difference in performance. It's a LOT cheaper to just run the generated query against an actual SQL database (and try again as needed) than to get the whole thing right in one shot with fine tuning & RAG.
I do think the right approach is to empathize with the agent. How far could you get with that complex 12-table query if you did not have a way to run your SQL against the actual database and see how it performs? You can extend this thinking to any domain. You can write your own "guard rails provider" that is deterministic and easy to inspect (i.e. you can set breakpoints and view locals). This can even recursively utilize the LLM in a more targeted way.
I think utility-scale solid-state molecular sensing is so vastly underhyped that you don't even know what I mean. Wait—you're asking about the thing that everyone on Earth is talking about 24 hours a day ceaselessly and asking if it's overhyped? Sorry, I misread the "Ask HN" somehow as "Do you think utility-scale solid-state molecular sensing is so underhyped that time travellers would be baffled that only one startup on Earth, namely Molecular Reality Corporation, is explicitly working on it with any kind of consistent technical vision?"
Their implementation so far leaves something to be desired.
Agents enable iterative changes to the environment. In contrast, LLMs are just "one shot" outputs and cannot edit what they have already done.
I think this "iterative" paradigm will ultimately be successful because iterative improvement is also what humans do -- we are agents "storing" most of our "knowledge" in the "environment" and then making small iterative changes and improvements, with an overall goal in mind. This isvery much in line with the influential "extended mind" thesis https://en.wikipedia.org/wiki/Extended_mind_thesis
The bandwidth of working memory and attention for tasks, especially new ones, is extremely limited. So we offload much of our cognition - working memory especially - onto our environment: think about how you write an essay or do a math problem or write a program.
So far, Yes. Seems like we just re-wrapped the old school "Chat Bots" using the "AI" keyword. For now at least. LLMs do have their use and advantage but most tools currently are not living upto the hype.
I think there is potential, but at the moment it's
1. RAG your data
2. Magic
3. Agent business logic
4. $$$
where step 2 is very unclear.
Also agent architecture what is it? A basic FSM which in essence is a bunch of business logic/rules with LLM API calls, how do you make this reliable for transactions.
I've yet to see a decent example of a business process replaced which isn't a question answer scenario i.e. call centre type role.
I don't think they are overhyped. I think that it's easy to hook up an LLM to some functions and get impressive results, but I think to make a really good agent system there are some core pieces that need to exist around the 'agent' to enable sophisticated workloads that many are not actually building out.
I personally believe they are under-hyped, but I am struggling with tooling in the space. Feels early and barebones. Lately, I've been pushing what I can build with no-code agent platforms but all the ones I've tried are limited to DAGs, for one thing, plus a bunch of other trade-offs/limitations.
"AI" is over-hyped. Chatbots are rarely useful/necessary for what people want in a product/website so it's annoying to see them shoehorned in where nobody was asking for them. They are infuriating when they replace actual humans for support. For personal use, they can be helpful though and they make for amusing toys too.
For me, the biggest issue is that the things I'd really like AI to do for me are things I would never trust a third party with for privacy reasons. As local/offline AI chatbots become better I'll use them for more things.
No, I don't. While you're free to ask, I don't think the question is useful. This happens with each major technological advancement. If you plan to stay in tech, you may want to get used to it and reduce your inputs to preserve your sanity. The floodgates are open, and the new technology is here whether one likes it or not.
However, I'm limiting my tech news and social media intake and using the technologies solely for real-world problems[1]. In the near future, the new ChatGPT voice conversations will be stunningly good, and conversations will soon become duplex, not simplex.
1. You get out of LLMs what you put into them. I'll stand on my soapbox and state that "prompt engineering" is critical to getting useful information out of LLMs. "Garbage in, garbage out" as they say. One should also have a healthy dose of "trust, but verify" with LLMs, as well as with any other technologies (Eg. don't follow your turn-by-turn instructions into a harbor, don't take a nap while your Tesla self-drives, etc.)
I'm curious what the HN community thinks. That's why I asked. I'm not sure the assumptions behind your comment "if you plan to stay in tech..."
In any case, I'm a big fan of prompt engineering. A few days ago, I played around with prompt eng techniques to nudge ChatGPT to solve a math puzzle. In case you're interested: https://www.sabrina.dev/p/chatgpt-math-riddle
I guess I'd further state that not only are AI agents never going away, they will evolve forms in awe-inducing ways that are obvious to most of us: humanoid forms and enough intelligence that humans will form long-lasting intimate relationships with them (for better or worse).
The "Do you think $foo is overhyped?" type questions seem like passive-aggressive complaining to me. Same with "Do you think $this_community talks about $thing too much?"
Perhaps I misinterpreted your intent. It might have been helpful to provide more context upfront to better understand your perspective.
My assumption there was that each new major tech landmark comes with the same question (eg. virtualization, social media, crypto, generative AI, and the Internet itself). I couldn't figure out a way to word it without sounding saucy.
I don't see the usefulness of the question when it's the sort of technology that's going to move forward and evolve, regardless of what anyone things. I suppose we can steer it with questions to some extent, but the questions should feel constructive.
Let's try again:
1. What are your thoughts on AI agents?
2. Have you seen or gotten access to the new OpenAI voice conversations beta? If so, how does this change your perspective? It changed mine considerably.
> In case you're interested: https://www.sabrina.dev/p/chatgpt-math-riddle
I am interested! You've got some other good posts on your blog, as well. Subscribed via RSS.
Edit: Hmm, my feed reader allows me to add your site like it has a feed, but it doesn't seem to actually have an RSS or Atom feed?!
They're underhyped. There just aren't good products readily available for it yet. When there are, it will make AI assistants truly assistants. Not just chat and answer, but "hey, can you do this for me?".
Considering how confidently wrong almost every answer I've gotten from an LLM has been, the last thing I think should be happening right now is for those wrong answers to be affecting the physical world. People will be dying left and right in its current state.
That's because an LLM needs the relevant information to answer correctly. Agents can research your question by searching the web, and use the information gathered from there to answer.
It's crazy how many people are willing to just give control of their devices over to LLMs. Especially when they consistently get things wrong all the time. Like what the fuck?
I think perhaps their current usage isn't optimized yet. I'm not sold on people chatting with a model as the best UX. Maybe some abstraction on top of this interaction
I also don't particularly like the technology (morally/ethically) because of what it says about what our country and tech in particular prioritizes.
So huge disclaimer aside that I'm biased and find it disgusting...
Technically... we will see I suppose, I just haven't seen anything very useful personally yet. There are some cool automations I imagine you could come up with and it might (tiny possibility) have a market with the kinds of people that already find stuff like Siri/Hands free useful. Not sure you'd ever turn a profit, but building it into your platform as a large tech co. you might find some adopters.
On the other hand...I can totally see it ruining the internet and a bunch of other stuff I enjoy.
Maybe as like a fancy playwright or testing platform it could be useful, but I'm not a web programmer so I'm really not the right person to speculate on that.
I've found the most leverage with agent-based LLM solutions comes from including feedback from prior iterations.
For example, if you are trying to write a text-to-SQL bot you should actually attempt to parse any generated query using the desired provider and then pass any errors back into the prompt. You can also incorporate snippets of result sets (if successful parse) as another form of feedback. In my experience, this can make an incredible difference in performance. It's a LOT cheaper to just run the generated query against an actual SQL database (and try again as needed) than to get the whole thing right in one shot with fine tuning & RAG.
I do think the right approach is to empathize with the agent. How far could you get with that complex 12-table query if you did not have a way to run your SQL against the actual database and see how it performs? You can extend this thinking to any domain. You can write your own "guard rails provider" that is deterministic and easy to inspect (i.e. you can set breakpoints and view locals). This can even recursively utilize the LLM in a more targeted way.