This sounds a lot like my dream setup, We've been slowly building something along those lines. I've linked a video at the bottom that shows how we did something similar with an Obsidian plugin. Hit me up if you're interested in more details, we'd be happy to get an alpha user who gets it.

We've mostly had trouble explaining to ppl what exactly it is that we're building, which is fine, since we're mostly building for us, but still it seems like something like this would be the killer app for LLMs

Obsidian Canvas UI demo -> https://www.youtube.com/watch?v=1tDIXoXRziA

Also linking out Obsidian plugin repo in case someone wants to dive deeper into what we're about -> https://github.com/cloud-atlas-ai/obsidian-client

> I'd be more interested in a tool where I can "add data" to it by drag and drop or folder import, then I can just type whatever prompt and the app's RAG system pulls relevant data/previous

This is something very similar to what i'm planning to add next, so stick around.

Well here’s a somewhat limited version of your idea and really only helps mitigate the copy/paste effort with coding: https://github.com/backnotprop/prompt-tower

My original idea was a DnD interface that works at the os level as a HUD… and functions like your idea but that is not so simple to develop.

For me this interface is canvas-based first, node-based second, meaning sometimes I might not even use connections to get my desired result from LLM but i have the place and form for the result and i know how to find it. Connections here are not set in stone like in mind mapping software for example, it's a tool.