I expect pretty much the opposite to happen: it makes sense for languages, stacks and interfaces to become more amenable to interfacing with AI. If a machine can act more reliably by simplifying its inputs at a fraction of the cost of the equivalent human labour, the system has always adjusted to accommodate the machine.
The most obvious example of this already happening is in how function calling interfaces are defined for existing models. It's not hard to imagine that principle applied more generally, until human intervention to get a desired result is the exception rather than the rule as it is today.
I spent most of the past 2 years in "AI cope" mode and wouldn't consider myself a maximalist, but it's impossible not to see already from the nascent tooling we have that workflow automation is going to improve at a rapid and steady rate for the foreseeable future.
> it makes sense for languages, stacks and interfaces to become more amenable to interfacing with AI
The theoretical advance we're waiting for in LLMs is auditable determinism. Basically, the ability to take a set of prompts and have a model recreate what it did before.
At that point, the utility of human-readable computer languages sort of goes out the door. The AI prompts become the human-readable code, the model becomes the interpreter and it eventually, ideally, speaks directly to the CPUs' control units.
This is still years--possibly decades--away. But I agree that we'll see computer languages evolving towards auditability by non-programmers and reliabibility in parsing by AI.
> The theoretical advance we're waiting for in LLMs is auditable determinism.
Non-determinism in LLMs is currently a feature and introduced consciously. Even if it wasn't, you would have to lock yourself on a specific model, since any future update would necessarily be a possibly breaking change.
> At that point, the utility of human-readable computer languages sort of goes out the door.
Its utility is having a non-ambiguous language to describe your solution in and that you can audit for correctness. You'll never get this with an LLM because its very premise is using natural language, which is ambiguous.
> Non-determinism in LLMs is currently a feature and introduced consciously. Even if it wasn't, you would have to lock yourself on a specific model, since any future update would necessarily be a possibly breaking change
What I'm suggesting is a way to lock the model and then be able to have it revert to that state to re-interpret a set of prompts deterministically. When exploring, it can still branch non-deterministically. But once you've found a solution that works, you want the degrees of freedom to be limited.
> You'll never get this with an LLM because its very premise is using natural language, which is ambiguous
That's the point of locking the model. You need the prompts and the interpreter.
> That's the point of locking the model. You need the prompts and the interpreter.
This still doesn't seem to work for me:
- even after locking the LLM state you still need to understand how it processes your input, which is a task nobody has been able to do yet. Even worse, this can only happen after locking it, so it needs to be done for every project.
- the prompt is still ambiguous, so either you need to refine it to the point it becomes more similar to a programming language or you need an unlimited set of rules for how it should be disambiguated, which an auditor needs to learn. This makes the job of the auditor much harder and error prone.
> The theoretical advance we're waiting for in LLMs is auditable determinism
I think this is a manifestation of machine thinking - the majority of buyers and users of software rarely ask for or need this level of perfection. Noise is everywhere in the natural environment, and I expect it to be everywhere in the future of computing too.
You're missing the point, there are specific reasons why these stacks have grown in complexity - even if you introduce "API for AI interface" as a requirement, you still have to balance that with performance, reliability, interfacing with other systems, and providing all of the information necessary to debug when AI gets it wrong. All of the same things that humans need apply to AI - the claim for AI isn't that it deterministically solve every problem it can comprehend.
So now we're looking at a good several decades of us even getting our human interfacing systems to amend themselves to AI will still requiring all the current complexity they already have. The end result is more complexity not less.
Based on what I've seen so far, I'm thinking a timeline more like 5-10 years where anything involving at least frontend has all but evaporated. What value is there in having a giant app team grind for 2 years on the perfect Android app when a user can simply ask for the display they want, and 5 variants of it until they are happy, all in a couple of seconds while sitting in the back of a car. What happens to all the hundreds of UI frameworks when a system as a widespread as Android adopts a technology approach like this?
Backend is significantly murkier, there are many tasks it seems unlikely an AI will accomplish any time soon (my toy example so far is inventing and finalizing the next video compression standard). But a lot of the complexity in backend derives from supporting human teams with human styles of work, and only exists due to the steady cashflow generated by organizations extracting tremendous premiums to solve problems in their particular style. I have no good way to explain this - what value is a $500 accounting system backend if models get good enough at reliably spitting out bespoke $15 systems with infinite customizations in a few seconds for a non-developer user, and what of all the technologies whose maintenance was supported by the cashflows generated by that $500 system?
These don't sound like the kinds of problems programmers solve. Users don't want to customize their UI (well some do), they want a UI that is adapted to their needs. They only want to customize it when it doesn't meet their needs (for example, if a corporation tries to use addicting features or hide things they need to increase engagement).
Accounting software has to be validated, and part of the appeal is that it simplified and consolidates workflows across huge bureaucracies. I don't see how on earth you can just spit one out from a prompt and expect that to replace anything.
I work on a compression algorithm myself, and I've found AI of limited utility. It does help me translate things for interfacing between languages and it can sometimes help me try out ideas, but I have to write almost everything myself.
EDIT: It is true, that lower skilled jobs are going to change or reduce in quantity in the short term. To a certain degree there might be a Jevon's paradox in terms of code quantity that needs management.
Imagine companies churning out tons and tons of code that no one understands that behaves bizzarely. Maybe it will become a boutique thing for companies to have code that works properly and people will just accept broken user interfaces or whatever so long as there are workarounds.
I wonder if AI is going to reduce the amount of JS UIs. AI bots can navigate simple HTML forms much easier than crazy React code with 10 layers of divs for a single input. It's either that or people create APIs for everything and document how they are related and interact with documentation.
There will always be "real thinking" roles in software but the sheer pressure on salaries from the vastly increasing free labour pool will lead to an outcome a bit like embedded software development, where rates don't really match the skill level. I think the most obvious strategy for the time being is figuring out how to become a buyer of the services you understand rather than a badly crowded out seller
the IP rights holders have yet to bare their teeth. I don't think the outcome you suggest is clear at all, in fact I think if anything entirely the opposite is the most probable outcome. I've lost count of the number of technology epochs that at the time were either silently or explicitly dependent on ignoring the warez aspects while being blinded by the possibilities, Internet video, music and film all went through this phase. GPTs are just a new medium, and by the end of it royalties will in all likelihood still end up being paid to roughly the same set of folk as before
I quite like the idea of a future where the AI job holocaust largely never happened because license costs ate up most of the innovation benefit. It's just the kind of regressive greed that keeps the world ticking along and wouldn't be surprised if we ended up with something very close to this
As I recall it, there was a time when copyright infringement on YouTube was so prolific that the rightsholders essentially forced creation of the first watermarking system that worked at massive scale. I do wonder if any corners of research are currently studying the attribution problem with the specific lens of licensing as its motivation
Yeah that was the old Viacom vs Youtube days. Here is a great video if you have half an hour to spare: https://www.youtube.com/watch?v=qV2h_KGno9w . Pretty funny court case where it turns out viacom was violating their OWN copyright... set a massive precedent.
But one thing this reminds me of is the idea of a "trap street", something mapmakers used to do was put in false locations on their maps to prove that other mapmakers were copying them: https://en.wikipedia.org/wiki/Trap_street . I figure you could do something similarly adversarial with AI to pollute the public training data on the internet. IDK like adversarial attacks on image classifiers https://www.youtube.com/watch?v=AOZw1tgD8dA . With an LLM you could try to make them into a manchurian candidate.
An environment where royalties inflate the pricing of ChatGPT by orders of magnitude seems like an environment where hosted models would be at a big disadvantage against whatever you can manage to get running on a pile of Macs in your garage.
>I quite like the idea of a future where the AI job holocaust largely never happened because license costs ate up most of the innovation benefit.
Not quite realistic. You are talking about very huge benefits, in favor of which licenses will be abandoned. And who don't abandoned them... I mean you can look at the Amish settlements.
It supports AMD cpus because, if I understand correctly, AMD licenses x86 from Intel, so it shares the same bits needed to run openVINO as Intel’s cpus.
Go look at CPUs benchmarks on Phoronix; AMD Ryzen cpus regularly trounce Intel cpus using openVINO inference.
The frequency of AI cope posts appears set to keep increasing until the very last one of us is fired. The reality is that users simply don't care about any concern to be found discussed in the church of software engineering, they want an app with a pink button, and very shortly we will be a much higher friction means to achieve that than the sloppy text generator. It's the same force behind why you can't buy high quality furniture any more or consumer devices that survive more than a few years: the vast majority of people simply don't care, they want to click "Buy" at a price level they don't have to think about and get on with their lives, only now it applies to knowledge economy work and entire industries are trying their hardest to ignore the transition we're so painfully obviously already in.
Even were it not the case for disposable CRUD Android/web apps that represent the bread and butter for half the industry, the effect on the structure of popular media is alarming in ways I don't think anyone has a hope of understanding yet. I imagine kids coming up now will not be glued to their phones anywhere nearly like the current batch are, or if they are glued to something, perhaps it is a conversational agent prompting them through an earpiece or similar.
Hand-wringing about the quality of LLM-driven app development really misses the point of all of this. We're currently using an extremely novel technology to emulate aspects of our now-defunct technology (which I believe includes the web), in much the same way fax gateways at one time were a popular application for email.
More efficient hardware mappings will happen, and as a sibling comment says, power requirements will drop like a rock. Check out https://www.youtube.com/watch?v=7hz4cs-hGew for some idea of what that might eventually look like
UML and "as little overhead as possible" probably shouldn't appear in the same train of thought. I remember it from the very earliest Linux VPS providers, IIRC it only got semi-usable with some custom work (google "skas3 patch"), prior to which it depended on a single process calling ptrace() on all the usermode processes to implement the containerization. And there's another keyword that should never appear alongside low overhead in the same train of thought
It gets more interesting when you think about the impact on groups. Sending an image to a group is enough for all devices associated with that group to be identifiable from CloudFlare's side, who additionally see a giant chunk of unencrypted traffic from the same client addresses going to other web sites. Given Cloudflare's less-than-straight approach to sales, it is astonishing the words "secure" and "Signal" ever appear in the same sentence.
CloudFlare get to see a fuckton of metadata from private and group chats, enough to trace who originally sends a piece of media (identifiable from its file size), who reads it, when it is is read, who forwards it and to whom. It really doesn't matter that they can't see an image or video, knowing its size upfront or later (for example in response to a law enforcement request) is enough
> Given Cloudflare's less-than-straight approach to sales, it is astonishing the words "secure" and "Signal" ever appear in the same sentence.
This is an overly binary take. Security is all about threat models, and for most of us the threat model that Signal is solving is "mainstream for-profit apps snoop on the contents of my messages and use them to build an advertising profile". Most of us using it are not using Signal to skirt law enforcement, so our threat model does not include court orders and warrants.
Signal can and should append some noise to the images when encrypted (or better yet, pad them to a set file size as suggested by paulryanrogers in a sibling comment) to mitigate the risks of this attack for those who do have threat models that require it, but for the vast majority of us Signal is just as fit for purpose as we thought it was.
Hello, I'm an organizer for a system to coordinate multiple mutual aid networks, many of which are only organizing by Signal & Protonmail exclusively because they think they're secure and private.
People who are doing work to help people in ways the state tries to prevent (like giving people food) rely on this tech. These are the same groups who were able to mobilize so quickly to respond to the LA fires, but the Red Cross & police worked to shut down.
This impacts the people who are there for you when the state refuses to show up. This impacts the future version of you who needs it.
Most people aren't disabled, yet. Doesn't mean they don't need us building infrastructure for if/when they become disabled.
I’m thinking as well more “mundane” things as well, like red states with “charitable feeding” laws that in effect make it illegal to feed the homeless without large amounts of red tape.
But, truly, I think you’re right to highlight wars.
Someone should tell anyone who seeks confidentiality that no email is secure. Use Signal and enable the data retention (i.e., automatic message deletion) feature. By itself that is not perfectly secure, but it's a start.
The people involved are likely all using Protonmail. So that would mean TLS for the connection to Protonmail with E2EE for messages passing through Protonmail.
Not sure that encrypted email in general would be less secure than, say, Signal. Since Signal is an instant messenger on a phone it might actually be less secure[1].
Maybe not individual warrants (at least not warrants to do non-scalable collections like hardware bugs in one's phone - I.e. warrants that, most users, with high probability, are not subject to). But mass surveillance, e.g. NSA, even with 'mass warrants' (e.g. Verizon-FISA warrant), that everyone is subject to, is probably in most people's attacker model. I don't have a study handy, but it seems reasonable that most users use signal to protect against mass surveillance and signal advertises itself as being good for this.
Also Marlinspike and Whittaker are quite outspoken about mass surveillance.
If cloudflare can compile a big part of the "who chats with whom" graph, that is a system design defect.
I thought it was digits only but see there's always been the option to use an alphanumeric passphrase as the "PIN". That prevents brute-forcing for anyone that bothered to use one, right?
It was only digits initially (https://old.reddit.com/r/signal/comments/oc6ow4/so_a_four_di...), with nothing preventing very easy ones like "1234", but even after they fixed it they continued to call it a PIN and many people would just assume is a number ("number" is right in the acronym), and often a very short one. Most people didn't want to set a PIN at all, they'd been being nagged about setting one and then got nagged again and again to reenter it.
It was not clear to most people that their highly sensitive info was being uploaded to the cloud at all let alone that it was only protected by the PIN. I wouldn't be surprised if a lot of people picked something as simple as possible.
Their announcement post says "at least 4 digits, but they can also be longer or alphanumeric", though maybe the feature had launched before that was written? https://signal.org/blog/signal-pins/
> Signal can and should append some noise to the images when encrypted (or better yet, pad them to a set file size as suggested by paulryanrogers in a sibling comment) to mitigate the risks of this attack for those who do have threat models that require it
Adding padding to the image wouldn't do anything to stop this "attack". This is just watching which CF datacenters cache the attachment after it gets sent.
Right, my bad on the ambiguity—I was replying to the OP's concern about image sizes, not the attack in TFA:
> It really doesn't matter that they can't see an image or video, knowing its size upfront or later (for example in response to a law enforcement request) is enough
I think the threat model of enough signal users to matter is nation-state actors, and signal should be secure against those actors by default so that they may hide among the entire signal user population
>It gets more interesting when you think about the impact on groups. Sending an image to a group is enough for all devices associated with that group to be identifiable from CloudFlare's side,
Doesn't this open up the possibility to identify groups that have been infiltrated by spies or similar posers? If you use this method to kinda-sorta locate or identify all the users in your group and one or more of those users ends up being located in a region where you should have no active group members then you may have identified a mole in your network.
Just thinking out loud here since there's no one else home.
>If you use this method to kinda-sorta locate or identify all the users in your group and one or more of those users ends up being located in a region where you should have no active group members then you may have identified a mole in your network.
...unless they happen to be using a VPN for geo-unblocking reasons or whatever.
If you're in a group like this where people are seriously concerned about their location being discovered by governments or by their own contacts, anyone in that group who is not already on a VPN all the time is either ignorant or nuts.
yeah, the person you're referring to is confused because the Cloudflare HTTP service terminates TLS and presents a Cloudflare certificate, but that doesn't have anything to do at all with Signal's E2EE which is not based on HTTPS PKI
Last time I used Cloudflare I think their settings default to only "Origin SSL/TLS" (or whatever they call it), which wouldn't encrypt anything between Cloudflare and the origin, it would only encrypt data between Cloudflare and the end-user/browser.
But the Signal client encrypts images before sending them to the Signal server. If it padded out the images at that point, the images would all be indistinguishable from each other unless Cloudflare were actually able to break the encryption (which would completely undermine the entire security model).
Ah yes, I'm sorry, I mistook the context. If Signal encrypts the images E2E, you're right that it wouldn't matter what Cloudflare does, especially if padded.
TLS doesn’t matter for End-to-end encrypted stuff though, you could exchange the data over Telnet and it would still be secure. The content itself is already encrypted before being transmitted and can only be decrypted by the receiver.
AFAIK the attack described by OP only works if the attacker knows the (randomly generated) URL of the image, which probably means they have a Signal client that can decrypt the image already. So the secrecy of the content is not at issue. The question is whether some specific person has received the same image, and from where.
Part of his attack requires disabling the cache on his (sender) side so that he doesn’t pollute the cache. That implies that both sides of the conversation share the same URL, which means Cloudflare could assume two IP addresses requesting the same URL on the Signal attachment domain are participating in a shared conversation.
Yeah, that's a problem. It is leaking metadata, not content.
Ideally, the image should be padded, encrypted with a different key, and given a different URL for each user who is authorized to view it. But this would increase the client's burden significantly, especially in conversations that include more than two people.
