The user's last prompt can be sent with an idempotency key that changes each time the user initiates a new request. If that's the same, use the cache. If it's new, hit the LLM again.
Weird take. The id field in the SSE spec is there specifically so you can resume a stream. And that requires persistence/caching on the server side. Once you have those things, you're practically at the pubsub option that the article prefers.
Precisely. In fact SSE as a protocol was specifically designed to support resuming. It’s unfortunate that most APIs don’t support that but that’s not the fault of SSE.
Neither are the Zigtools folks. If you've ever run an open source project, you know that instead of running on money, they run on community goodwill. Having people take the project's creation, claim it as their own, and not comply with the license, are all damaging to people's motivation to contribute.
I genuinely believe more people violate permissive licenses than copyleft license. I have no data to back this up, but just look at how much people focused on if LLMs were violating the GPL by reproducing code covered by the GPL without reproducing the license. If LLMs violate the GPL, they violate all licenses besides ones that are effectively public domain.
This probably depends on country, but AFAIK in most of europe, even in public domain, the „you can’t pass another’s work as your own” part of copyright is still active and doesn’t expire.
This piques my interest, what is the legally required recognition of a derivative's parent work? Must I be able to list dependencies, or should I be able to verify whether a parent work is included in mine? What if my work is a second derivative of a work which I am unaware of, because the work in between improperly didn't recognise its parent? Am I legally responsible to investigate such cases?
I tried that once. Specifically because I wanted to see if we could leverage some sort of productivity enhancements.
I was using a local LLM around 4B to 14B, I tried Phi, Gemma, Qwen, and LLama. The idea was to prompt the LLM with the question, the answer key/rubric, and the student answer. The student answer at the end did some prompt caching to make it much faster.
It was okay but not good, there were a lot of things I tried:
* Endlessly messing with the prompt.
* A few examples of grading.
* Messing with the rubric to give more specific instructions.
* Average of K.
* Think step by step then give a grade.
It was janky and I'll throw it up to local LLMs at the time being somewhat too stupid for this to be reasonable. They basically didn't follow the rubric very well. Qwen in particular was very strict giving zeros regardless of the part marks described in the answer key as I recall.
I'm sure with the correct type of question and correct prompt and a good GPU it could work but it wasn't as trivially easy as I had thought at the time.
Having everything in my phone is a great convenience for me as a consumer. Pockets are small, and you only have a small number of them in any outfit.
But cloud services run in... the cloud. It's as big as you need it to be. My cloud service can have as many backing services as I want. I can switch them whenever I want. Consumers don't care.
"One model that can do everything for you" is a nice story for the hyper scalers because only companies of their size can pull that off. But I don't think the smartphone analogy holds. The convenience in that world is for the the developers of user-facing apps. Maybe some will want to use an everything model. But plenty will try something specialized. I expect the winner to be determined by which performs better. Developers aren't constrained by size or number of pockets.
It's partly about Netflix getting sued by someone claiming infringement, but also partly (maybe mostly) about Netflix maintaining their right to sue others for infringement.
The scenario looks like this:
* Be Netflix. Own some movie or series where the main elements (plot, characters, setting) were GenAI-created.
* See someone else using your plot/characters/setting in their own for-profit works.
* Try suing that someone else for copyright infringement.
* Get laughed out of court because the US Copyright Office has already said that GenAI is not copyrightable. [1]
In a legal case? You question the authors under oath, subpoena communications records, billing records, etc.
If there's even a hint that you used AI output in the work and you failed to disclose it to the US Copyright Office, they can cancel your registration.
A restriction on government (as the First Amendment language is phrased) is not the same thing as an individual right. There are plenty of cases where a restriction is in the law, but only a very limited set of entities has standing to sue to enforce it. You could imagine such a case with regard to the First Amendment if we didn't have the corporate personhood doctrine.
I have no idea what that has to do with my comment or the one I responded to. The freedom of the press is guaranteed by the first amendment. The NYT doesn’t suddenly lose “rights to it” if it’s not seen as a person.
“Corporate personhood” is irrelevant, in this comment chain, and is just a way to take a swipe at an ostensibly left-leaning org in order to turn this into a team sport.
reply