Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Corrupt Signal DB issue crashing App and losing data (github.com/signalapp)
119 points by nh2 on April 17, 2021 | hide | past | favorite | 75 comments


I've more or less given up on Signal and recently moved to Telegram. I'll still use it for actually private communications, but for 90% of my chats, all I need is "good enough" security which both iMessage and Telegram cover with a much better UX than Signal.

I can't believe in 2021 they still don't have a native Mac app nor proper chat backups, and the recent announcement of their bullshit crypto integration clearly shows where their priorities are.


> for 90% of my chats, all I need is "good enough" security which both iMessage and Telegram

True for iMessage, but not for Telegram. Telegram has great UX, but does not use end-to-end encryption.

(Telegram's "secret chats" do not count - they're not the default, nobody ever uses them, they only work on mobile, and use a questionable DIY encryption schema)


Not true for iMessage either. The addition of iCloud Backup (on by default, and not e2e) functions as a key escrow for iMessage, rendering iMessage also no longer end-to-end encrypted (as Apple receives a copy of the keys in the automatic nightly backup).

Turning off iCloud Backup doesn't fix it either, as your conversation partners back up their keys.


To add to the list of flaws with Telegram's secret chat implementation, you have to wait until the other user is online before you can start one.


Why don't they count? I use them. I don't really care if they only work on mobile. This questionable DIY scheme operates under the assumption that Moxie is the only one allowed to roll his own crypto: MTProto 2 is fine.

Everything is a convenience trade-off, all the time, for everyone.


E2E encrypted chats do work on Telegram Desktop if you’re on a Mac and using the native Cocoa build (not the Qt one).

I remain curious as to why this is, but I use them often enough when I find Signal a pain in the ass to deal with.


iMessage is not end to end encrypted either.


To be honest, for 99.9% of my use cases, I value the ability to easily log in to different devices with automatically synchronized one-to-one/group chat log more than true E2E encryption, granted that Telegram has promised to not analyze the data, and I decided to trust Durov. The other 0.1% is when I need to send credit card numbers, which I can also do within Telegram, using secret chat, which does have E2E encryption.


Have you considered Matrix / Element as the alternative?


I’ve seen my connections mitm in group conversations with telegram


Signal does have a native Mac app. I use it every day.


It's not, it's an Electron-based app (not bad by default). Even says so on their Github page: https://github.com/signalapp/Signal-Desktop

Unless you're using a non-official version?


It’s also a bit worse if you’re an M1 user. Last I checked it also runs under Rosetta in compatibility mode, which is annoying since it drains my battery more as a result.

I’ve been awaiting an M1 build for a bit now...


I didn't realize you were using the term "native" to refer to compiled versus interpreted.

The electron binary is native code, of course, or it wouldn't execute.


> The electron binary is native code, of course, or it wouldn't execute.

It isn't always; after all you can run Electron apps under Rosetta if there's no M1 version of it even though it is compiled but for Intel instead.

So, it is not as simple as "compile vs. interpreted" because there's emulation as well.

In this case, many folks don't consider Electron as native because it doesn't use native UI frameworks as it uses the web stack tech (JS/CSS/HTML/etc) to render. That's why Electron apps can run with same UI on all desktop platforms.

In other words, pure Swift/Cocoa UI apps are considered to be native macOS apps, even if its backend is built in any native language like Objective C/Swift/Rust/C etc.

However, same folks will not consider Electron apps as native even if it is using a backend built in any native language such as Objective C, Swift or Rust/C/C++.


By native I don't mean Electron. I don't want to run yet another browser in addition to my main one. Even WhatsApp is better at this level, at least you can open it in your existing browser.


It would be nice if signal would implement support for BackupAgent based client side encrypted backups and device to device transfers for Android, the current risk of data loss during device migrations is very high IMO if you aren't careful.

https://community.signalusers.org/t/support-native-android-b...


There's already a backup transfer feature in the last version.


It doesn't work automatically though when doing a normal android data transfer/migration.


Does the normal Android transfer process involve network transfers and backups, out of interest?

(it's possible that there are privacy design reasons to keep data within user control, even if it cedes some convenience)


Per the docs(https://developer.android.com/reference/android/app/backup/B...):

"The transport is for a device-to-device transfer. There is no third party or intermediate storage. The user's backup data is sent directly to another device over e.g., USB or WiFi."


Thanks!


I upgraded Signal on my Linux machine recently. It started by re-schema-ing its configuration, and then froze, ignoring all UI events as if it had a modal dialog box up. So I downgraded Signal, but the old version refused to use the re-schema-ed configuration.

Always copy your Signal configs before trying a new Signal version.

Element's current release has the same UI problem, ignoring all UI input. New messages appear and show on counters, but no UI events break through.

Both Signal and Element are Electron apps, run on Debian in a VM, with software rendering. Evidently something changed in Electron. Ideas welcome.


Sound like an easy fix on the side of Signal. First work on temp and when everything turned out successful, delete the old directory and move temp to new.


That would be fine if the files were not already messed up, or if the Electron app framework they used building Signal could be persuaded to actually UI.


Open a thread on the Signal forum.


> Evidently something changed in Electron. Ideas welcome.

If you're a dev, write PWA.


Does it matter? Signal messages are meant to disappear. They live only on your device and the developers would rather build in sketchy crypto payments than any kind of automated backup.


Yes it matters and no, they are not meant to disappear arbitrarily.


Anything that's not backed up is meant to disappear arbitrarily.


Three users report they've experienced a bug (the fourth just says Signal won't start). We only know some symptoms of the bug, and almost nothing about its cause.

One user guesses at a db size limit, but there is strong evidence that's not the case - including users with much larger data stores and the fact that media is stored externally to the database (as we would expect).

Without more information, the problem could have little to do with Signal. It may be something funky about Android when it's low on space, or a million other possibilities.

Put on your IT hats - there is nothing of significance here; it's just an unconfirmed, undiagnosed bug reported by three users.


How do I donate to Signal?



can you explain why my question was downvoted

thanks for link couldn't find it on their webpage


You could literally google this and get 10 answers.


[flagged]


There's a hacky way to back up messages sent on IOS, if you sync messages to desktop client, because it stores messages in a SQLite database. While encrypted, encryption key for the database is kept as plaintext in a JSON file. :)

On Windows:

1. Download "DB Browser for SQLite" https://sqlitebrowser.org/

2. Unpack it and run "DB Browser for SQLCipher.exe".

3. Open database, select "%APPDATA%\Signal\sql\db.sqlite".

4. In the following dialog, select "Raw key" type and enter the encryption key stored in "%APPDATA%\Signal\config.json" into Password field (prefix it with 0x).

5. Now you can work with the database, including exporting its contents to unencrypted SQLite, CSV and JSON.

The process should be similar on other platforms.


The Signal desktop client is a poster child for “lousy Electron app” and a sucky user experience. I don’t use Signal a lot and get maybe three or four messages a day in total. Every time I launch the desktop client it counts the messages one by one like a small child would.


Yes, I instantly get sad each time I fire it up. Aside being the lousiest of Electron apps, it has no security features at all. Just pair once with your phone and anyone gaining access to your computer can read your messages. Are the messages on my computer encrypted? I don't know. I wish I could set up a strong password.


While encrypted, encryption key for the database is kept as plaintext in a JSON file.

...which is kept right next to the encrypted DB and presumably has the same permissions, so what's the point?


A more secure delete, for one.


Not really considering SQLite databases can use WAL commits and it’s easy to view those previous commits.


SQLCipher encrypts the WAL.[1]

[1] https://www.zetetic.net/sqlcipher/design/


From your source, it uses the same key:

> Write Ahead Log Files - Using the new WAL mode (i.e. PRAGMA journal_mode = WAL;), page data stored in the WAL file is encrypted using the datbase key. Pages in the rollback journal are encrypted using the same key as the main database. Verification: create an encrypted database, start a transaction, make changes, and then inspect the -wal file using hexdump or a similar program.


So destroying the key makes both unrecoverable.


Yes, Except the key isn’t destroyed it’s in the json file next to the main db file. Meaning user’s deleted messages could still be recovered.

Even if the user went as far as deleting the json file, it could still possibly be recovered due to the way the FS ‘deletes’ files.


I think the other posters are saying it’s much easier to shred a small file than it is to shred a multiple GB database, although these days it doesn’t matter so much for a database that small. It’s super efficient for file systems though.


Yes. And I'm talking generally. I don't know what Signal does.


Thank you. I haven't been able to try it immediately as I don't normally use the desktop client, but that looks like a workaround that could be useful.


What's up with yours and ttty's comments?

> When will we be able to make real backups of Signal messages using the iOS app? Privacy is valuable, but important messages without ability to make backups is already very risky. Having data loss bugs is unforgivable.

> So when will we be able to make real backups of Signal messages using the iOS app? Privacy is valuable, but important messages without ability to make backups is already very risky given the inherent fallibility (and steal-ability) of mobile devices. Having data loss bugs in conjunction with that is delete-the-app-and-use-something-else unforgivable.

Very fishy.


I don't think Silhouette had anything to do with this. The other user has been posting junk content which is either autogenerated, glued together from other content, or both. The comment where they copied from Silhouette (https://news.ycombinator.com/item?id=26842183) also copied from https://github.com/signalapp/Signal-Android/issues/11160#iss..., and probably other places too. I asked them to stop (https://news.ycombinator.com/item?id=26837415) and they didn't, so I've banned the account (https://news.ycombinator.com/item?id=26846079).


ttty has been posting comments that seem to be auto-summarised versions of other sources (or even complete copies). dang told them to stop (https://news.ycombinator.com/item?id=26837415) but they haven't.

(Edit: and now they're downvoting me for pointing it out?)

(Edit 2: It's now getting ridiculous: https://news.ycombinator.com/threads?id=ttty)


Bad bots


Feels more like a human-templated response from a fiverr job or something.


Looks like an extractive algorithm, so I doubt it's a fiverr job.


I don't understand. You're asking why concerns about trying to prevent backups of a user's own data are relevant in a discussion about a bug that reportedly causes catastrophic data loss?


I think they're more wondering why two separate users are posting nearly word-for-word identical comments.


I don't know. When I wrote my comment, it was the second comment in the entire discussion, and it seemed like an important issue to raise. Someone upvoted. Someone replied with a workaround, and I replied to thank them. I haven't seen this other comment that some of you seem to be referring to, so I assume it was subsequently deleted, but whatever it was had nothing to do with me.

It would be nice if we could avoid silly conspiracy theories and meta discussions on HN, but if anyone really wants to check, the link to my profile is right above this comment. It's the one that has been an HN member for well over a decade. You can't see that most of my comments across a broad range of issues get a few upvotes, but you can see that very few go down overall and I average around 1000/year in karma, which I think is a reasonable benchmark for being a real person who is contributing constructively here.


The other user is apparently a bot.


I'm deleting my signal if a way to back up my messages on MY terms (i.e. to a file) isn't implemented in two months.

I'd rather give Facebook my messages than lose it because Signal devs think users don't deserve to own their messages.


That's interesting. In our data-mined world I've always found this a feature, and a refreshing one.

I'm exposed to this feature/shortcoming quite often since I use the desktop version only intermittently.

Observing that lost messages are lost _is reassuring to me_.

On the infrequent event that I need a record, I'll usually just screenshot it then and there.


I think it depends on the content. With family photos, I'd rather have them public then lose them. For the public, it's likely not that interesting, but for me they are invaluable.


The Android and desktop databases can be backed up. Just the iOS version blocks it.


Good for you that that's your usecase! I myself would like to have a secure backup of all my messages and attachments.



Who wants permanent persistence as a feature of secure comms?

Save messages or files that you find permanently important to store?


It should be obvious end to end encryption and ephemeral messages are for different threat models. And blocking iOS backup but allowing Android backup, desktop backup, and iOS history transfer is a really bad way to implement ephemeral messages.

You assume someone will know what they'll want later when they receive it. Saving several messages together is tedious in most apps. So is organizing saved messages.


Everyone that uses chats for serious business. I can't afford to loose conversations with my clients. For that I use only messaging systems that allow me to save chats: email, and then WhatsApp. Even Telegram has a problem, because even if messages are on the cloud, one could delete single messages or a whole conversation on your device as well.

Regarding screenshots and saving messages: not only it's a thing that you have to remember doing, but screenshots have (at least in my country) no legal value, since everyone can easily produce them with Photoshop. You can say the same thing for WhatsApp messages, but sure it's more difficult (or at least it appears more difficult to a judge that doesn't know anything about IT). Basically screenshots are never accepted as a proof for something, WhatsApp messages on your phone are.


And screenshots are not searchable


You don't know that a message is important at the time.


But the aim of Signal is default secure comms. It's supposed to easily wrap everything, from the absolutely banal to the fairly sensitive, in strong encryption in order to cut into the returns on performing mass surveillance.

The question just comes down to who generally wants their communication contents persistent until actively destroyed, as opposed to quasi-randomly wiped out by technical snafus.

If you want transient there are the time-limited messages, for instance...


I use signal to not be spied on by facebook or telegram or w/e. That is entirely orthogonal to having or not having backups.


Sometimes you forget or your own storage fails, so having a backup plan is good. Like another commenter said, to me it depends on what the info is. Although I suppose you can't quite "pick and choose" in these matters.


Anyone who realizes that if it ever left your device, it has to be presumed to exist forever.


(You all could reply with your thoughts, as well)


Yes, they'll be so sorry to lose your custom. Remind me what they get from you?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: