It's virtually impossible to read old iMessages and they take up tons of storage(keydiscussions.com)
keydiscussions.com
It's virtually impossible to read old iMessages and they take up tons of storage
https://keydiscussions.com/2021/05/28/apple-charges-for-storing-gigabytes-worth-of-old-imessages-you-cant-reliably-access/
275 comments
> "Apple's chat.db has an esoteric schema"
This is basically one taken out of microsoft's .doc format playbook.
At first I thought Apple was slow and just didn't have time to get to some sort of export for iMessage chats. Mail had imap, calendar had caldav, address book had carddav. The original mac stuff did other standards like AIM and jabber.
But over time I began to suspect, then later those internal emails exposed in the lawsuits made it clear - apple was holding everyone's messages hostage.
What's annoying is that I couldn't save my messages, I could only upgrade to a bigger iphone.
What's downright evil is all the people - I know several - who have lost messages + embedded photos/videos from loved ones who have died because there was no export.
This is basically one taken out of microsoft's .doc format playbook.
At first I thought Apple was slow and just didn't have time to get to some sort of export for iMessage chats. Mail had imap, calendar had caldav, address book had carddav. The original mac stuff did other standards like AIM and jabber.
But over time I began to suspect, then later those internal emails exposed in the lawsuits made it clear - apple was holding everyone's messages hostage.
What's annoying is that I couldn't save my messages, I could only upgrade to a bigger iphone.
What's downright evil is all the people - I know several - who have lost messages + embedded photos/videos from loved ones who have died because there was no export.
>But over time I began to suspect, then later those internal emails exposed in the lawsuits made it clear - apple was holding everyone's messages hostage.
In an SQLiteDB any programmer can learn to navigate, and any third party can build a tool to export them?
Sorry but the "internal emails" prove nothing about Messages. It's just a quick and dirty implementation that "works good enough" for reading the latest, but not as an archive.
In an SQLiteDB any programmer can learn to navigate, and any third party can build a tool to export them?
Sorry but the "internal emails" prove nothing about Messages. It's just a quick and dirty implementation that "works good enough" for reading the latest, but not as an archive.
Yeah I agree. The lack of a good schema/export function sounds to me like negligence not maliciousness. It’s a different issue from only allowing Apple-tech-users to use iMessage
> But over time I began to suspect, then later those internal emails exposed in the lawsuits made it clear - apple was holding everyone's messages hostage.
Any source for that? I couldn't find any direct evidence for your claim.
Any source for that? I couldn't find any direct evidence for your claim.
There are probably better links:
https://texasnewsexpress.com/index.php/04162021-1249/apple-r...
https://texasnewsexpress.com/index.php/04162021-1249/apple-r...
Thanks, that's interesting, I didn't know that. On the one hand, I shouldn't be surprised since vendor lock-in is the standard way tech companies try to gain corporate advantage. On the other hand, it's kind of depressing to learn once again that customers count very little and no matter what these companies say, they will always try to abuse us in every way the law allows.
Just took a look at your product, I'm very impressed and have added myself to the waitlist. I'm almost even more impressed by the domain name itself though - how on earth did you manage to bag that domain?!
Bought it at the right time from the right person :)
I presume the right time was 20-30 years ago?
domain first, then figured out the product?
Hmm.... Has Julian M. sold tele.com yet?
EDIT: No, apparently not. Weird; we all thought that was going to make him a multi-millionaire.
EDIT: No, apparently not. Weird; we all thought that was going to make him a multi-millionaire.
That alone must be worth some serious money.
GoDaddy estimates it at $25,000
Easily worth 6 or 7 figures to the right person.
I would not even raise an eyebrow if there were offers out there for 10x that.
Those estimates mean nothing. GoDaddy would list it at 250k+.
Christmas.com sold for over $3M. I would think texts.com would be worth a lot more than $250k.
I modified this script to pull my entire db into a text file.
https://github.com/michael-danello/iMessageWrapped
EDIT: It is absurd Apple doesn't provide better message organization and UX/UI.
https://github.com/michael-danello/iMessageWrapped
EDIT: It is absurd Apple doesn't provide better message organization and UX/UI.
The fact so many people on this site can throw something functional together without much effort is even more disappointing apple doesn't do anything about it.
iPhone is the only apple device I own but it crashes it can't go too far back into my messaging history for any one individual too far.
Usually once I scroll to ~5 months back with people I text regularly the iMessage app just straight crashes ... it's a little absurd. To top it off, every couple pages it gives a loading icon and introduces severe latency for whatever reason. There don't seem to be any apps in their App Store that let you search and lookup old messages either.
This is actively biting me right now. Are there any solutions for accessing the iMessage/chat.db database on my phone or via iCloud from a Linux system (Ubuntu in my case)?
Usually once I scroll to ~5 months back with people I text regularly the iMessage app just straight crashes ... it's a little absurd. To top it off, every couple pages it gives a loading icon and introduces severe latency for whatever reason. There don't seem to be any apps in their App Store that let you search and lookup old messages either.
This is actively biting me right now. Are there any solutions for accessing the iMessage/chat.db database on my phone or via iCloud from a Linux system (Ubuntu in my case)?
I’d also be interested in something that works with Ubuntu. I’d love it if you could give me a shout should you find something.
Looks cool, will give it a try.
A little feedback on the landing page: Almost turned away because I thought „Sign up with Google“ was the only choice and I missed the little „Prefer entering email instead?“ below. I just prefer not having a Google account ;)
Maybe the styling of that link could be adjusted so that it isn’t glanced over that easily.
A little feedback on the landing page: Almost turned away because I thought „Sign up with Google“ was the only choice and I missed the little „Prefer entering email instead?“ below. I just prefer not having a Google account ;)
Maybe the styling of that link could be adjusted so that it isn’t glanced over that easily.
Good call. We'll de-emphasize Google, it's just for convenience atm.
I don't mean to be pessimistic , but I wonder how you're going to be anything but reactionary to apples changes to the database. I've written a similar (much simpler) version of what you're proposing to sell- the database queries are hysterical.
I've been thinking about using the accessibility I framework to ETL the messages into Kinesis to facilitate event driven architecture. 'archived' Messages and attachments are stored as bsob.
Hell, I've even experimented with some goofy stuff, like FaceTime to VOIP- asterisk is just so flexible
I've been thinking about using the accessibility I framework to ETL the messages into Kinesis to facilitate event driven architecture. 'archived' Messages and attachments are stored as bsob.
Hell, I've even experimented with some goofy stuff, like FaceTime to VOIP- asterisk is just so flexible
Looking over your site, I'm curious about a few things:
- The FAQ says you'll eventually open source your SDK. I admire that as a software engineer, but from a business perspective, isn't this risky? It seems like--if you don't pass messages through an intermediary service--your value-add is mostly in this SDK + your design. All of these elements are copy-able. - The jobs page includes a backend engineer, so what will go through your backend, if not any message data?
- Are there legal concerns about reverse-engineering these APIs and using them? Some of your supported chat services have open APIs, but others seem intentionally closed. (And a couple parent companies seem downright hostile to 3rd party use.)
- The FAQ says you'll eventually open source your SDK. I admire that as a software engineer, but from a business perspective, isn't this risky? It seems like--if you don't pass messages through an intermediary service--your value-add is mostly in this SDK + your design. All of these elements are copy-able. - The jobs page includes a backend engineer, so what will go through your backend, if not any message data?
- Are there legal concerns about reverse-engineering these APIs and using them? Some of your supported chat services have open APIs, but others seem intentionally closed. (And a couple parent companies seem downright hostile to 3rd party use.)
Source: I built a third-party desktop client for iMessage at https://texts.com
Does the texts.com client solve the original problem? Does it make it easy to go back and view all those old text messages? I know the website says "from forever ago easily". But you didn't say that explicitly in your post.
I assume you're being modest and not "pushing" your solution. But if it simply solves the problem, maybe you should be shouting this out loud!
Does the texts.com client solve the original problem? Does it make it easy to go back and view all those old text messages? I know the website says "from forever ago easily". But you didn't say that explicitly in your post.
I assume you're being modest and not "pushing" your solution. But if it simply solves the problem, maybe you should be shouting this out loud!
It absolutely does, scrolling up shows older messages instantly.
Also, a "jump to date" feature (like Telegram) is planned (and super easy to build.)
Also, a "jump to date" feature (like Telegram) is planned (and super easy to build.)
A piece of feedback: it would be nice to know what the price is on your site. You say "We make money by charging you a monthly subscription," but there's no pricing.
Indeed, we’re in private beta atm experimenting with pricing and product. Don’t want to mislead or turn away people until we finalize it.
OK, so document that?
I've built an iMessage chat.db reader as well. To me the most astounding thing is that chat.db is unencrypted sqlite. Anyone who can see your home folder can read all your private chats or upload the chat.db to be read somewhere else.
It's impressively insecure.
It's impressively insecure.
macOS now has System Integrity Protection which requires manual Full Disk Access perms to read chat.db so any random app cannot touch it.
Which you end up applying to random things like ruby because Emacs.app uses a ruby-based launcher and anything less than Full Disk Access breaks dired.
So as a developer you need full disk access to run your programs? Oh my god, the surprise ;-)
Reading this after GP called it reverse engineering makes me feel he should've use different terminology.
Why would a lack of encryption make it not reverse engineering? There's a database schema, it was engineered* and used to build a database. They went from the live data back to the schema.
*For some definitions of "engineered", apparently.
*For some definitions of "engineered", apparently.
Very eager to see how your app works when it opens up to the public. Are there ways to provide assurances for your end-to-end encryption which may be digested by experts?
You also mention on your website that end-to-end only works when the platform "supports" it — does that include Windows with your currently always-on Mac solution?
You also mention on your website that end-to-end only works when the platform "supports" it — does that include Windows with your currently always-on Mac solution?
> Very eager to see how your app works when it opens up to the public. Are there ways to provide assurances for your end-to-end encryption which may be digested by experts?
We recommend using a simple MITM proxy or a firewall app like Little Snitch / LuLu to inspect the network traffic.
> You also mention on your website that end-to-end only works when the platform "supports" it — does that include Windows with your currently always-on Mac solution?
Yes – there's a peer-to-peer connection between the Windows and Mac devices.
We recommend using a simple MITM proxy or a firewall app like Little Snitch / LuLu to inspect the network traffic.
> You also mention on your website that end-to-end only works when the platform "supports" it — does that include Windows with your currently always-on Mac solution?
Yes – there's a peer-to-peer connection between the Windows and Mac devices.
I had done something too, but I also needed a Manifest.mbdb parser to reconstruct backup to disk, then process the messages and copy all the images around to make it work. I did it mostly to make a simple-to-read archive of chats before deleting them.
Did your work turn into some potentially sharable shell scripts, perhaps? Would be a useful tool to avoid having to store messages in iCloud.
I have not used it for a very long time, so I can't comment to how well it works.
That said, this is it: https://gist.github.com/brianv0/35f36a32366a2c34be8d
(You need an unencrypted backup directory to run it over)
Note: It looks like the flat directory structure of the backup is now 256 dirs prefixed by first two characters of file name. I'm not sure, but a small change to the name "sha_file" may be needed to just path.join that with the first two chars of the sha.
That said, this is it: https://gist.github.com/brianv0/35f36a32366a2c34be8d
(You need an unencrypted backup directory to run it over)
Note: It looks like the flat directory structure of the backup is now 256 dirs prefixed by first two characters of file name. I'm not sure, but a small change to the name "sha_file" may be needed to just path.join that with the first two chars of the sha.
couple of extra hints:
Python 2 but can be ported forward without much effort. Manifest.mbdb is now Manifest.plist (there's a sqlite thing now too). timestamp sql might not work so great, would need to check query.
Python 2 but can be ported forward without much effort. Manifest.mbdb is now Manifest.plist (there's a sqlite thing now too). timestamp sql might not work so great, would need to check query.
I also have a similar but older personal project that allows you to send iMessages over the web via an always on Mac.
https://github.com/CamHenlin/iMessageWebClient
I also have some other iMessage related git repos on my GitHub. Interesting stuff!
https://github.com/CamHenlin/iMessageWebClient
I also have some other iMessage related git repos on my GitHub. Interesting stuff!
Woah, that's a heroic effort. Can't wait to try it! Huge thanks :)
why the need to sign up with Google? I don't want Google to be associated with text messages.
Maybe they just added it, but there's also a "Prefer entering email instead?" link below, which I appreciated
Does what's app have and open api? Otherwise how did you integrate it?
Is texts.com a work around for sending live imessages on a PC?
Yes, with a catch: it requires an always on Mac (which can be in the cloud) or a jailbroken iOS device. We plan to make it work without the catch.
I've worked with 10+ messaging platforms so far and Apple's protocol is the most obfuscated and complicated to reverse engineer. Apple has invested millions of dollars to make the iMessage protocol super hard to reverse engineer. It's how they sell tons of iPhones after all:
> However, Craig Federighi, Apple’s Senior Vice President of Software Engineering and the executive in charge of iOS, feared that “iMessage on Android would simply serve to remove [an] obstacle to iPhone families giving their kids Android phones”. (https://www.androidpolice.com/2021/04/28/apple-admits-that-i...)
I've worked with 10+ messaging platforms so far and Apple's protocol is the most obfuscated and complicated to reverse engineer. Apple has invested millions of dollars to make the iMessage protocol super hard to reverse engineer. It's how they sell tons of iPhones after all:
> However, Craig Federighi, Apple’s Senior Vice President of Software Engineering and the executive in charge of iOS, feared that “iMessage on Android would simply serve to remove [an] obstacle to iPhone families giving their kids Android phones”. (https://www.androidpolice.com/2021/04/28/apple-admits-that-i...)
Looks like a super cool project. I'm always happy to see new client-side software for communication protocols written that aims to improve upon built-in! And iMessages in particular could certainly use it. With the demise of iTunes, it now feels like iMessages is probably one of the most crufty-but-heavily-utilized user facing pieces of software Apple puts out. Maybe they have some long term plans to refactor it ala iTunes but yeesh.
This bit did make me wonder though:
>We plan to make it work without the catch.
Out of curiosity, do really think that's realistic, or even desirable long term? iMessage is ultimately an Apple service that runs heavily on Apple's infrastructure, and is directly subsidized by sales of their highly vertically integrated hardware platforms. If it turns into a cat-and-mouse fight it seems like they're always going to have the eternal upper hand, which in turn seems like it'd make for a subpar user experience (ie., breaks randomly which for an instant messaging service would be pretty bad). Also seems like they might actually be motivated to respond rather than ignore it since it'd actually be directly leeching their infra if it will work on PCs/Android without a Mac/iDevice purchase in the equation (unlike hackintoshes for example, where whatever debate there is to be had about probably very low "lost sales" it doesn't actually cost them anything).
Obviously you've probably thought this all through, but seems like just requiring an old cheap Mac or old jb'd idevice and thus avoiding Apple might be an easier path. Or alternately just stick to offering a flat out better client with the iMessage bit being Mac-only. Will be interested to see how it goes though!
This bit did make me wonder though:
>We plan to make it work without the catch.
Out of curiosity, do really think that's realistic, or even desirable long term? iMessage is ultimately an Apple service that runs heavily on Apple's infrastructure, and is directly subsidized by sales of their highly vertically integrated hardware platforms. If it turns into a cat-and-mouse fight it seems like they're always going to have the eternal upper hand, which in turn seems like it'd make for a subpar user experience (ie., breaks randomly which for an instant messaging service would be pretty bad). Also seems like they might actually be motivated to respond rather than ignore it since it'd actually be directly leeching their infra if it will work on PCs/Android without a Mac/iDevice purchase in the equation (unlike hackintoshes for example, where whatever debate there is to be had about probably very low "lost sales" it doesn't actually cost them anything).
Obviously you've probably thought this all through, but seems like just requiring an old cheap Mac or old jb'd idevice and thus avoiding Apple might be an easier path. Or alternately just stick to offering a flat out better client with the iMessage bit being Mac-only. Will be interested to see how it goes though!
> one of the most crufty-but-heavily-utilized user facing pieces of software Apple puts out. Maybe they have some long term plans to refactor it ala iTunes but yeesh.
The Messages app on Mac was rewritten for Big Sur (last year). It gained some features but also some bugs in the process.
The Messages app on Mac was rewritten for Big Sur (last year). It gained some features but also some bugs in the process.
iMessage is backwards compatible with older devices. This means Apple can’t stop supporting old versions of the protocol, even as they are “cracked”.
Agreed on intentional obfuscation. It even has booby traps. https://neosmart.net/blog/2018/imessage-for-windows/
I know it's a long-shot, but do you plan on documenting the protocol?
Wasn’t it also that they had a great distributed way of doing messages but got sued to changes their implementation?
FaceTime was. Not sure if it happened to iMessage
> We plan to make it work without the catch.
Interesting, how do you plan to do that?
Interesting, how do you plan to do that?
> it requires an always on Mac (which can be in the cloud)
That’s surprising. How come?
That’s surprising. How come?
Likely because each iMessage receiving device has a private key to decrypt the messages, and senders encrypt the iMessage against all the keys in your key bag.
To access iCloud and or services you also need to have a device security key that is tied to genuine hardware.
To access iCloud and or services you also need to have a device security key that is tied to genuine hardware.
[deleted]
Anyone know of something similar for Android texts?
any way to get around the reg/invite wall to try it out?
Author here: the gist is that it's practically impossible to scroll to the beginning of an iMessage chat, and that's the only way to get to the beginning of a conversation. These old conversations take up several gigabytes of space, yet there's no good way to search them (because search for old messages is unreliable).
This aspect of the iMessage user experience has remained untouched for a decade, despite everyone's chats growing longer. Searching for particular iMessages sucks because the results miss the broader context of the conversation (a single message is not useful when you want to reread a conversation).
(Also just noting, as the post author I created a title better-suited to HN's 80 character limit.)
This aspect of the iMessage user experience has remained untouched for a decade, despite everyone's chats growing longer. Searching for particular iMessages sucks because the results miss the broader context of the conversation (a single message is not useful when you want to reread a conversation).
(Also just noting, as the post author I created a title better-suited to HN's 80 character limit.)
Have you tried emailing Craig Federighi about this? I emailed him a bunch of times about several things and he's a really nice and responsive person.
If you do, tell us about it here on HN.
If you do, tell us about it here on HN.
I actually did try emailing Tim Cook a few months ago, but I got bounce backs. I tried [email protected] and [email protected] but the mailer daemons said the addresses weren't registered. It sounds like a crazy thing to do, but I remembered how Steve Jobs would reply to users' emails and read an article where Tim said he set aside time to read users' emails.[0]
[0] - https://www.inc.com/business-insider/tim-cook-wakes-up-at-4-...
[0] - https://www.inc.com/business-insider/tim-cook-wakes-up-at-4-...
Craig's email is [redacted]. The guy is very chill. It's better you email a software guy about software.
Also, be sure to file a bug report / enhancement at https://www.apple.com/feedback/iphone.html
Also, be sure to file a bug report / enhancement at https://www.apple.com/feedback/iphone.html
- I’m never filing a bug report ever again on a bug filing system where I don’t get to see the progress in a transparent manner or that I might have to pay for a developer account to see that progress (assuming that’s an option)
- I’m not wasting my time emailing a top level company executive to inform of decades old most basic of bugs because if they don’t know of it then that’s even a bigger problem.
- I’m not wasting my time emailing a top level company executive to inform of decades old most basic of bugs because if they don’t know of it then that’s even a bigger problem.
Use https://feedbackassistant.apple.com/ instead, it has a mildly better chance of being read.
I’m somewhat surprised that acknowledging outside emails is a thing. I know unsolicited ideas/content/etc. is commonly rejected without review in other fields. Is this successful beyond identifying technical bugs?
It's [email protected]
Seems like a humble choice for an executive to not take <firstname>@corp.com.
I was under the impression that Steve Jobs’ was [email protected]
"steve" was a collision from the start.
I could imagine Steve Wozniak offering to use surnames only as a joke - because that wouldn't work either.
It would probably work better for the two Steves than it would for many others.
According to the below random site, there 218 Jobs and 10,602 Wozniaks. It claims 410,801 Steves.
http://howmanyofme.com/search/
http://howmanyofme.com/search/
Sorry my point was about [email protected] being the possible target of random job inquiries making the address unusable for Steve Jobs.
cook@apple is appealing somehow.
thanks cryptoz
Related, I emailed Tim Cook about iPhone sensor future plans, and a high-level exec of like 25+ years wrote back to me within a week. Apple is remarkably good at getting back to cold emails to top execs.
Interesting, perhaps I’ll cold email them about the pinned contact feature in iMessage being totally worthless.
If it just pinned the message thread to the top and didn’t change the icon to a giant circle that sometimes doesn’t show notifications, it’d be fine. I pinned my GF’s message thread and I ended up missing lots of messages and notifications from her and ended up unpinning her so the notifications would show up.
Not to mention the comically huge circle icon that is centered when everything else is left aligned.
If it just pinned the message thread to the top and didn’t change the icon to a giant circle that sometimes doesn’t show notifications, it’d be fine. I pinned my GF’s message thread and I ended up missing lots of messages and notifications from her and ended up unpinning her so the notifications would show up.
Not to mention the comically huge circle icon that is centered when everything else is left aligned.
Interesting, I love the way pinned threads are implemented, especially the big circles
I found pinned messages on Monday. I've now unpinned things.
It only made things worse :(
It only made things worse :(
Same here. Shocked that managed to get past an initial beta.
It defeats the entire purpose of pinning.
It defeats the entire purpose of pinning.
I concur with that.
I had a problem that appeared to be a corner case with Apple Pay. I wrote a paper letter to Tim Cook about it, and within a week got contacted by a staff person in his office. He worked exclusively with me to identify issues I had not considered, and followed up on all of the direct effects and side effects until they were resolved.
I had a problem that appeared to be a corner case with Apple Pay. I wrote a paper letter to Tim Cook about it, and within a week got contacted by a staff person in his office. He worked exclusively with me to identify issues I had not considered, and followed up on all of the direct effects and side effects until they were resolved.
I emailed Craig about several controversies and he was very quick to reply to me. We last talked about the rumor that Apple was giving iPhones trust scores.
Seconded, I wrote about an issue I encountered on facetime and I was sent many iPhones to test how widespread the problem was.
If you can remember a term, any term, that was written in the message, you can search & jump to it directly. For my girlfriend and I, it was the bar we went to on our first date. This feature only gives you small subsets (a couple dozen, maybe) messages near the query result, but then you can just search for the last message again and get the next “page,” so to speak. Of course, remembering such a term is hard, which is what I assume you mean by “search for old messages is unreliable,” but it’s at least a possible workaround.
In the article they say that they have a screenshot of their first several messages and have tried to search for the text in the screenshot.
Hm, yet I have no problem searching phrases that go back at least five years.. I would say the burden of proof is on the author here to extract the database from the phone and prove that the messages didn’t get deleted, lost in an upgrade, or otherwise cleaned up.
Please, please just read the article before posting. Please.
My sister was having a problem where she was taking all the iCloud storage for our family because she keeps sending videos to people but couldn't figure out a way to efficiently delete them. I had to write up a Python script (https://gist.github.com/saagarjha/615dee3c04e226b44828e910a5...) to scrape the iMessage database for large attachments, and then print out the text to messages around it so that you could search for it and jump to around that place in the chat history and then delete the files from there. It's kind of insane how hard it is to manage old messages :/ I feel like Apple didn't really prepare for Messages coming in from iCloud meaning that every device had every single message you've ever sent, and now it's really hard to get through them without reading the database manually…
I am convinced iMessages right now is a tool to keep your storage full, push people to buy higher storage iPhone, and pay iCloud Storage backup to increase their Services Revenue.
None of the problem listed in the blog post or what you point out are new. They have been there since day one. And it is not that they neglect it either like iTunes or Aperture. They added Memoji!
Which is why I am surprised a the popularity of iMessages in US ( and France ). I think it is mostly due to SMS usage before iMessages was a thing.
None of the problem listed in the blog post or what you point out are new. They have been there since day one. And it is not that they neglect it either like iTunes or Aperture. They added Memoji!
Which is why I am surprised a the popularity of iMessages in US ( and France ). I think it is mostly due to SMS usage before iMessages was a thing.
Never attribute to malice…
Never attribute to malice what you could attribute to a desire for profit.
"Wait, so this user story has negative business value? Let's put it in the icebox."
...that's what malicious people say! ;)
This is also why phones have absurdly high resolution poorly compressedt camera picture/video despite the optics not makit use of all the detail.
Did Settings > General > iPhone Storage > Review Large Attachments not work?
I’ll have to check that actually. Maybe I can give her a high-capacity iOS device and let her sign into that, and then delete everything from there.
Is every video considered a large attachment?
Pulled it up on my phone just to see what they considered "large", looks like 4MB is the cutoff (nothing smaller than that shows up for me). But nearly everything I'm seeing is a video or a photo taken with my real (not phone) camera. And a random PDF I apparently sent over iMessages.
However, deleting through this interface would be a pain. You select "Edit" and then each attachment is selected individually.
However, deleting through this interface would be a pain. You select "Edit" and then each attachment is selected individually.
It seems to just show you the largest set (50? 100?). If you delete them, then you will eventually show smaller ones once you get through enough of them.
Really wish there was a "delete all" or "select all".
Really wish there was a "delete all" or "select all".
Thanks. That makes sense. But I didn’t feel like deleting anything. Just my wife and I on the iCloud storage and there’s plenty of it at present. My photo collection that I’ve been cleaning up is a bigger offender right now than messages. Damn cats doing cute things.
When I am on the iMessage dialogue screen with a contact, I can usually click on their name, then Info, and it displays all the video/photo attachments sent in that chat. I believe I can then delete them from there. Is that useful? I guess if there's a number of photos you'd like to keep, but not the large videos, it's a more complicated issue.
Maybe this is some kind of anti forensics because this is so hard?
On your Mac, iMessages are stored in a SQLite database which you can inspect.
https://spin.atomicobject.com/2020/05/22/search-imessage-sql...
https://spin.atomicobject.com/2020/05/22/search-imessage-sql...
Can confirm. Shameless plug: I wrote a script to extract the pertinent information from the SQLite database into an HTML page a while back: https://github.com/kccqzy/imessage-db-extract/blob/master/im... It's open source abandonware though. Lost interest in keeping up with the schema updates because I no longer use iMessage.
That's an undocumented implementation detail. It might change without warning at any time. Are you seriously recommending this as a solution for users who want to be able to read past messages?
Works a lot better than doing nothing.
I’d imagine it’d be difficult for Apple to migrate to anything substantially different as it’d require rewriting a potentially multi GB file, the space for which many users would not have. It’s effectively a massive, distributed, database migration.
Database formats change all of the time. There's no technical challenge here that hasn't been solved before.
Your indignation is strange.
Why would you not recommend it when there's no alternative?
Why would you not recommend it when there's no alternative?
His hacker ethos is broken recently.
Lighten up a little, GP.
Lighten up a little, GP.
[deleted]
Seems relevant news for hackers.
That does work, but only for messages stored locally. If you've enabled Messages in iCloud, it won't reliably contain everything.
[deleted]
Same format as on iOS; you can/could copy the db out of an unencrypted iTunes dump.
And, yeah, it’s a total mess.
And, yeah, it’s a total mess.
I save them on the expectation that Apple will someday release a better tool for browsing them. Especially the texts between me and my wife, they are a story of our life from the very first one.
I was in a similar situation with email. In 2005 or so I was migrating to new computer when I found a directory of photos I forgot to back up. Rather than burn them to DVD or upload them to my domain, I decided to just email them to myself. I sent a handful of images, then replied to that same email with the next 10, and so on, until the five dozen or so images were uploaded in a single email thread.
When I went to open the email on my new machine, the loading indicator just spun and spun until finally my browser crashed. When viewing an email conversation, Gmail will display previews of all the emails in the chain, including their attachments, which was causing my browser to crash. Next I tried connecting my Google account to Apple Mail, but Gmail's SMTP server would time out before the attachments were able to download.
After a while I forgot about the photos all together, until I randomly stumbled across them while searching my Gmail account for some other files I had sent myself around that time. At this point it was 2014 or so, and my browser was very easily able to open the messages so I could download the photos, which of course at that point I had zero interest in.
When I went to open the email on my new machine, the loading indicator just spun and spun until finally my browser crashed. When viewing an email conversation, Gmail will display previews of all the emails in the chain, including their attachments, which was causing my browser to crash. Next I tried connecting my Google account to Apple Mail, but Gmail's SMTP server would time out before the attachments were able to download.
After a while I forgot about the photos all together, until I randomly stumbled across them while searching my Gmail account for some other files I had sent myself around that time. At this point it was 2014 or so, and my browser was very easily able to open the messages so I could download the photos, which of course at that point I had zero interest in.
I used gmail to subscribe to lkml at some point. Then switched back to my own email server. Forgot to unsubscribe on gmail. A few years later I deleted that folder. And couldn't open Gmail anymore for weeks, everything would just time out.
[deleted]
On a similar note, deleting old photos to save space on iCloud is very hard. I could mass-delete photos older than a certain date in the Mac Photos app (or whatever its called these days) using a smart album, but then that mass deletion would not sync to iCloud. Or, I could log into iCloud via a web page and select a load of photos (but not more than 1000 at a time) and delete them - but then it takes ages - like about 20 minutes. Why does it take so long to delete 1000 photos from the cloud? Its unfathomable. And then after deleting them you have to go and find them in the iCloud bin and delete them again - which also takes ages.
I can only conclude that they have made it deliberately very slow and painful to clear space on iCloud. Because they'd rather you paid for more space ...
I can only conclude that they have made it deliberately very slow and painful to clear space on iCloud. Because they'd rather you paid for more space ...
> I could mass-delete photos older than a certain date in the Mac Photos app (or whatever its called these days) using a smart album, but then that mass deletion would not sync to iCloud.
Why wouldn’t it sync? If I did this right now, my expectation is that it would sync. Is there a limitation in how this works?
Why wouldn’t it sync? If I did this right now, my expectation is that it would sync. Is there a limitation in how this works?
I expected it to, but it didn't. The photos got deleted locally but not on iCloud. Maybe it was a glitch? Thats what happened to me.
Sync for some reason hasn't been working for a decade.
It never gets fixed - very weird for a trillion dollar company.
Take a photo on you iPhone and it take days for i to show up in the photos app, it instantly shows up in iCloud accessed through the browser.
Same thing in various other directions, delete on web, and it takes days to disappear from other places. Makes zero sense.
Often if i just took some pictures i have to send them via airdrop instead of just opening Photos. Come on Apple!
Take a photo on you iPhone and it take days for i to show up in the photos app, it instantly shows up in iCloud accessed through the browser.
Same thing in various other directions, delete on web, and it takes days to disappear from other places. Makes zero sense.
Often if i just took some pictures i have to send them via airdrop instead of just opening Photos. Come on Apple!
We need a better communication standard. Apple made a killing selling their proprietary iMessage layer on top of SMS, but we could be actually making changes in this sector if we all used a standardized protocol. At least then we have a documented interface that we can choose to store however we want, or (perhaps wishful thinking) in a modular style. RCS is a great start, but I've already started to put my money on Matrix. While the latter won't ever see adoption by Google or Apple, it's a pretty convincing attempt at rich, decentralized communication.
Is it really "on top of SMS" though? The way I see, it's just a messaging app like WhatsApp, Telegram etc but they just combined it with the SMS app on their platform.
I am failing to see the additional value of combining them. If Google bought WhatsApp and combined it with the stock SMS app of Android, I don't think I would prefer that.
I am failing to see the additional value of combining them. If Google bought WhatsApp and combined it with the stock SMS app of Android, I don't think I would prefer that.
One underrated feature of iMessage is graceful degradation. If I'm unable to send/receive data for whatever reason, my message (and their replies) automatically fall back to SMS. There's been a few occasions where this was my only way of communicating important information, so I'm grateful for it.
Does this also apply when the recipient cannot receive data?
Yes. If the message is unable to be delivered to the recipient via iMessage, then it is sent as SMS instead. So for example if you try to send a message to someone and their phone is turned off or doesn’t have signal, it will eventually fail and then send as SMS.
This is all controlled via a setting in iOS iMessage settings “Send as SMS”, which I believe is turned on by default but I’m not sure. So your mileage may vary.
This is all controlled via a setting in iOS iMessage settings “Send as SMS”, which I believe is turned on by default but I’m not sure. So your mileage may vary.
The value to Apple was that if you tried to SMS someone, iPhone would check if the destination phone number was registered with Apple, and silently switch to iMessage via Apple instead.
Cynically, they intercepted millions of people's comms with hardly a murmer. Charitably they switched millions of people from country-dependent, pricey and limited SMS to internet messaging with read receipts, pictures, video, emoji, audio recordings without any hassle of logging in, changing apps, choosing clients, etc.
Cynically, they intercepted millions of people's comms with hardly a murmer. Charitably they switched millions of people from country-dependent, pricey and limited SMS to internet messaging with read receipts, pictures, video, emoji, audio recordings without any hassle of logging in, changing apps, choosing clients, etc.
> I am failing to see the additional value of combining them. If Google bought WhatsApp and combined it with the stock SMS app of Android, I don't think I would prefer that.
Well, it's not useful now, but imagine in the 00s when a minority of users had iPhones. Having two clients, one for SMS and one for iMessages, would have gone against the simplicity Apple was aiming for with respect to user experience. Combining it meant that users could get a better experience (than SMS/MMS at least) when communicating with other iMessage peoples, but not have to swap out clients to chat with others.
Well, it's not useful now, but imagine in the 00s when a minority of users had iPhones. Having two clients, one for SMS and one for iMessages, would have gone against the simplicity Apple was aiming for with respect to user experience. Combining it meant that users could get a better experience (than SMS/MMS at least) when communicating with other iMessage peoples, but not have to swap out clients to chat with others.
Personally as a college student, all my friends use iPhones still. I have always wanted to switch to andriod, but one of the things that really holds me back is the fact iMessage still feels leagues above how SMS and texting works on andriod. Many subtle features of Imessage really make a difference with the overall texting experience on the iPhone over the long term.
Most of my friends do not have whatsapp, and we are constantly sharing photos and video. Regular SMS can barely send pictures let alone videos, and I don't want to go through other means to simply shoot my friend a quick video of something.
Most of my friends do not have whatsapp, and we are constantly sharing photos and video. Regular SMS can barely send pictures let alone videos, and I don't want to go through other means to simply shoot my friend a quick video of something.
> iMessage still feels leagues above how SMS and texting works on andriod
iMessage is a messaging app that uses internet. It should be compared to WhatsApp, Telegram, Signal, WeChat, FB Messenger.
It doesn't make sense to compare iMessage to SMS. Because nobody uses SMS as their main communication method. The only people who ever use SMS are iPhone users when they text Android users.
All this confusion comes from the fact that Apple merged their messaging app with the SMS app. This creates the illusion that iMessaging isn't just a messaging app but it is actually something built on top of SMS. It's just a messaging app that is not cross-platform.
> I don't want to go through other means to simply shoot my friend a quick video of something.
By "other means", do you mean installing a new messaging app? I'm guessing this isn't about the effort of downloading an app but rather it's about getting used to a new app and convincing your friends to switch to it.
iMessage is a messaging app that uses internet. It should be compared to WhatsApp, Telegram, Signal, WeChat, FB Messenger.
It doesn't make sense to compare iMessage to SMS. Because nobody uses SMS as their main communication method. The only people who ever use SMS are iPhone users when they text Android users.
All this confusion comes from the fact that Apple merged their messaging app with the SMS app. This creates the illusion that iMessaging isn't just a messaging app but it is actually something built on top of SMS. It's just a messaging app that is not cross-platform.
> I don't want to go through other means to simply shoot my friend a quick video of something.
By "other means", do you mean installing a new messaging app? I'm guessing this isn't about the effort of downloading an app but rather it's about getting used to a new app and convincing your friends to switch to it.
“convincing your friends to switch to it” is the real issue. Or remembering which platform various friends use and keeping that straight. And when you have multiple friends who prefer different platforms, figuring out the intersection of messaging platforms that they all use. Or when you have a friend on Facebook messenger but you don’t know their phone number, or a friend who can’t have WeChat installed on their corporate phone, or any number of other situations.
And I think the biggest of those subtle features is the blue coloured bubble, isn’t it?
For me it's the fact that you can generally see the other person typing in an iMessage, while you can't in normal SMS.
That tells me whether the person is replying right now, so it's worth keeping my phone out, or busy and therefore won't be getting back to me for a while.
The time savings add up.
That tells me whether the person is replying right now, so it's worth keeping my phone out, or busy and therefore won't be getting back to me for a while.
The time savings add up.
No. iMessage has the low entry barrier of SMS, but has features like a real chat app. SMS is not an iMessage replacement. Instead you have to get everyone you message with to change to something like WhatsApp.
This blue coloured bubble thing I've never heard anyone outside of Silicon Valley engineers/media mention, but I've heard it extrapolated as a deciding factor to the rest of the world by those same people many times.
I don't think most people actually give a shit about the bubble colour.
I don't think most people actually give a shit about the bubble colour.
Google has this already - check out Messages by google. Behaves the same as iMessage
iMessage is a vestigial product from the days when people had unlimited data but also only 400 text messages a month. It made sense then. It doesn't make much sense now.
It still makes sense over SMS/MMS which are terrible protocols in comparison. Even if you get unlimited messaging now, group and media messages are still remarkably unreliable.
iMessage is not a protocol, it's an app. Nobody uses SMS because everybody uses a messaging app. It comes down to whether you choose to use a cross-platform one or an iOS-exclusive one.
I think it is less that we have more SMS allotment and more that nobody wants to actually use them anymore, because they're insecure / don't work on wifi / are slow / etc.
The additional value for Apple is that their platform had a billion+ users from the start.
Apple had nowhere near a billion customers when iMessage was launched.
There were a billion people you could send messages to, and receive messages from. That’s what got them of the ground.
A system where you can’t message half your friends because they bought their phone from a different manufacturer would have had more trouble getting traction.
A system where you can’t message half your friends because they bought their phone from a different manufacturer would have had more trouble getting traction.
I think that in America most people still use SMS, but in Europe barely anyone I know does, so RCS is basically a non-starter. It's all WhatsApp and Telegram (and some Signal). Like my kids sports teams coordinate everything through WhatsApp so I'm basically forced to use it. Would love to see an alternative, but getting everyone to switch would require a lot.
The problem with decentralized protocols is that centralized ones outcompete them in terms of consumer usage.
XMPP did really well for a while until Google and Facebook decided they didn't want to participate anymore.
It's a question of who comes first- Linux was first to market as a cloud OS, so it dominated. Email came first for long-form asynchronous messages, so it dominates.
I think people should get in the habit of deleting all their convos that are more than a month old. Most convos are 99.99% worthless, and it's very freeing to be rid of them.
One of my friends had her H1B cancelled and was given a 10 year ban from entering the US because of her text messages. When she crossed the border back into the US, the border guards requested her phone, and searched it and found messages to her ex-bf about marrying him for a green card. She was joking but it didn't matter and she had to fly back to her country of origin that she didn't live in since she was a kid. Since then, even though I don't have anything to worry about, as a habit I delete all my apps except iMessage and delete all my conversations there.
One of my friends had her H1B cancelled and was given a 10 year ban from entering the US because of her text messages. When she crossed the border back into the US, the border guards requested her phone, and searched it and found messages to her ex-bf about marrying him for a green card. She was joking but it didn't matter and she had to fly back to her country of origin that she didn't live in since she was a kid. Since then, even though I don't have anything to worry about, as a habit I delete all my apps except iMessage and delete all my conversations there.
If you have Messages set up on a Mac you can access your chat archives under ~/Library/Messages/Archive. Each conversation is stored by date in an .ichat file which can be re-opened in the Messages app.
https://appletoolbox.com/qa-where-are-imessage-files-stored/
https://appletoolbox.com/qa-where-are-imessage-files-stored/
That's a good hint.
It does have some important limitations.
1) I do clean installs when I update macOS, so the Library folder is created new each time. Which means that archived messages only go back to that point. (I browsed via Terminal and used the command line 'open' command to view them.)
I do retain old ~/Library folders and it's possible to use the same technique in finding messages there.
2) But, and here's the big limitation: at a quick glance, they're simply individual messages, by date. There's no way to follow entire threads. I suppose it would be possible to write your own program to organize these. But the point of the article is that Apple should make it easy. Your method isn't easy.
Edit: response to sibling comment. I'm still on Catalina and the above works. I have no idea about Big Sur.
It does have some important limitations.
1) I do clean installs when I update macOS, so the Library folder is created new each time. Which means that archived messages only go back to that point. (I browsed via Terminal and used the command line 'open' command to view them.)
I do retain old ~/Library folders and it's possible to use the same technique in finding messages there.
2) But, and here's the big limitation: at a quick glance, they're simply individual messages, by date. There's no way to follow entire threads. I suppose it would be possible to write your own program to organize these. But the point of the article is that Apple should make it easy. Your method isn't easy.
Edit: response to sibling comment. I'm still on Catalina and the above works. I have no idea about Big Sur.
No longer true in Big Sur -- or, at least, that folder is empty for me.
Searching Messages on my Mac is a joke. If type the name of a friend in the search field, it returns (after unreasonable delay) various results, mostly of texts about the person I'm searching for.
How does it not realize that it should first search recipient names and present these results first? A search for "Delilah" should not show a bunch of texts about Delilah, obscuring the one thread with my friend Delilah.
How does it not realize that it should first search recipient names and present these results first? A search for "Delilah" should not show a bunch of texts about Delilah, obscuring the one thread with my friend Delilah.
So, I just tried this in Messages on Big Sur. If I search for a name, I get a list of existing-conversations that the person with that name is in at the top of the results, then the list of text results within conversations.
Messages on Big Sur is (basically) the iOS Messages app ported over with Catalyst, so it has a lot of improvements compared to the highly-stale app it replaced.
Messages on Big Sur is (basically) the iOS Messages app ported over with Catalyst, so it has a lot of improvements compared to the highly-stale app it replaced.
Glad to hear it's been fixed in newer versions. Wish I didn't have to upgrade my entire OS to get the current version of Messages!
I think there’s a separate idiomatic autocomplete flow for what you’re trying to do. Try hitting “New Message”; typing your friend’s name; and selecting the result. The New Message window will transform into the existing-conversation window for that contact.
Not saying having it also work from search wouldn’t be good — but having this flow available is probably why the other issue hasn’t been fixed sooner.
Not saying having it also work from search wouldn’t be good — but having this flow available is probably why the other issue hasn’t been fixed sooner.
> ry hitting “New Message”; typing your friend’s name; and selecting the result. The New Message window will transform into the existing-conversation window for that contact.
I think this is how MS teams chat works too. It kind of makes sense, because it has a workflow for solving the "do you want your existing chat with Alice, or are you trying to start a new group chat with Alice & Bob" .
I think this is how MS teams chat works too. It kind of makes sense, because it has a workflow for solving the "do you want your existing chat with Alice, or are you trying to start a new group chat with Alice & Bob" .
It does make sense if you're looking to send a new message. In my case, I'm typically doing a search so I can find information/photos from that thread. In this context, it's not so intuitive, and it never occurred to me to even try.
Never thought to do this — thanks!
I've found the only way to read super old imessages is to go to the sqlite db they're stored in on a mac and look directly in the db...
I didn't know that was possible! Is there a guide available to locate/browse the messages?
If I have an idea of what I’m looking for, I just use search. It was useless until a few years ago but has gotten dramatically better recently.
The scroll-load-scroll-load cycle does suck, though.
The scroll-load-scroll-load cycle does suck, though.
Where’s that? I’ll totally play with that if I can !
If Apple is focusing on “services” now, as numerous tech/biz pundits have talking about for quite some time now, it doesn’t show in their tech and software offerings, except for the gloss in the presentations and the painfully obvious exaggerated wordings.
With every iOS and Mac OS release I’ve been hoping they’d fix iMessage, iCloud, App Store search, app discovery, extension ecosystem for browsers, or my iPhone taking those extra 7-8 phantom GBs (..iPhone Storage/Others) that everybody guesses could be some cache or logs or something magically else and speculates maybe one day it’ll go away but nobody knows for sure etc.
But no! It seems Apple has been saving these things so that after 10 years they get to say on stage “Now you can search apps like never before. We have named this patented revolutionary app discovery as iDiscover™. As simple as searching for apps!“
Or after another 5 years something like “Now you can read your iMessages in a more readable manner and it will truly sync now. Meet TrueSync™“ when their engineers will have a breakthrough after 17 years of backbreaking research.
I have always wondered how the engineers look in the mirror after hearing those pathetic stage ridiculousness.
With every iOS and Mac OS release I’ve been hoping they’d fix iMessage, iCloud, App Store search, app discovery, extension ecosystem for browsers, or my iPhone taking those extra 7-8 phantom GBs (..iPhone Storage/Others) that everybody guesses could be some cache or logs or something magically else and speculates maybe one day it’ll go away but nobody knows for sure etc.
But no! It seems Apple has been saving these things so that after 10 years they get to say on stage “Now you can search apps like never before. We have named this patented revolutionary app discovery as iDiscover™. As simple as searching for apps!“
Or after another 5 years something like “Now you can read your iMessages in a more readable manner and it will truly sync now. Meet TrueSync™“ when their engineers will have a breakthrough after 17 years of backbreaking research.
I have always wondered how the engineers look in the mirror after hearing those pathetic stage ridiculousness.
It seems like a lot of iCloud storage-using services are conveniently designed to just keep eating up more and more space. Take photos for example: most people only need to archive old photos and rarely access them, but such a use case isn't really supported by iCloud photos. You either keep your entire collection in the cloud and in a state of fast (expensive) retrieval, or not at all.
For iMessages every photo or video you sent seems to take up separate storage in the iMessage portion of iCloud, and it's difficult to delete because you have to scroll through and select them.
For iMessages every photo or video you sent seems to take up separate storage in the iMessage portion of iCloud, and it's difficult to delete because you have to scroll through and select them.
Can you catch downscale them with applescript/automator on a Mac?
Dropping in here to say that as a programmer I'm always super impressed with how easy it is to go back many years in WhatsApp via find and quickly scroll around it. Always makes me wonder if I'm doing something wrong in my client-server app which struggles at the tens of thousands scale and if I'd be better off re-architecting to replicate all data to the client and just using the insanely powerful CPUs that are available there. I'm sure if I did it in my high scale, highly mutable (esp access control), high user count application replicating all that state reliably would also become a nightmare, whereas whatsapp has a pretty clear replication model built in.
I wish I could have an ms outlook like interface for the long term storage of the messages. Folder per contact or group and excellent search, sort and filtering. Also could be stored offline or synced up with a mobile app.
2 apps I use for similar non iMessage archival are - x1 - for file and email search https://www.x1.com/products/x1-search/
- mailstore https://www.mailstore.com/en/products/mailstore-home/
2 apps I use for similar non iMessage archival are - x1 - for file and email search https://www.x1.com/products/x1-search/
- mailstore https://www.mailstore.com/en/products/mailstore-home/
Nah - it's already hard enough to manage email + Slack, and I don't want yet another workaround to manage / catalog / filter incoming messages to me.
No one asked for managing texts like we manage files.
No one asked for managing texts like we manage files.
They are priceless catalogs of entire relationships. It feels flimsy, or maybe flippant, to not be able to properly search and sort or archive.
Maybe not managed like files, maybe you’re right, but it seems like if we can make trillion dollar tech companies we should be able to completely solve this particular issue.
I’d frame texts as one of the most intimate expressions of dialog and journaling between people and families and friends. Just seem odd that this isn’t better.
Maybe not managed like files, maybe you’re right, but it seems like if we can make trillion dollar tech companies we should be able to completely solve this particular issue.
I’d frame texts as one of the most intimate expressions of dialog and journaling between people and families and friends. Just seem odd that this isn’t better.
I have had a bug out on this since 2019 - FB7336938 if anybody inside apple is looking for ammo to make improvements to this
Flexibits developed an app to do this search called Chatology (https://flexibits.com/chatology) but they've discontinued it since Big Sur came out. Works pretty well on my Mojave machine.
It unfortunately doesn't work well if you've enabled Messages in iCloud [1]. You can only search the messages that're in your device's local cache. Which isn't actively cleared much, that I've seen, but it does mean that it's not very helpful for any messages that predate the current install of your OS.
I'd assume this is why they discontinued it, rather than adapt it to the new from-iOS Messages in Big Sur.
[1] https://support.apple.com/en-us/HT208532
I'd assume this is why they discontinued it, rather than adapt it to the new from-iOS Messages in Big Sur.
[1] https://support.apple.com/en-us/HT208532
If it’s discontinued and they don’t charge for it I hope they consider open sourcing the solution so others can try to make it functional.
Can’t you use something like iMazing (I’m not a paid shill just a satisfied user and there are many similar other products from competitors) and do an offline dump from the backup?
Another vote for iMazing. It's great.
Reading from a backup works marvelously. I have done it.
Is there a good option for people who have offloaded some of their Messages to iCloud?
Article mentions this: “...there’s a whole slew of third party tools that slurp up your iMessages, and Apple should care about this, because having a strong demand for apps that process users’ messages should be seen as a privacy failure by the company.”
Many comments here are suggesting you can look at the DB on a Mac. What about people who have Windows computers? Is there a way to extract that information?
I also have a similar gripe with the browser. How can I sync my Safari tabs that are open on my iPhone with my PC?
I also have a similar gripe with the browser. How can I sync my Safari tabs that are open on my iPhone with my PC?
You need to buy a Mac for that, it's all part of their plan.
I would recommend switching iOS browser to be able to sync your browsing across vendors
I would recommend switching iOS browser to be able to sync your browsing across vendors
The complete iMessage history in your backups is also not e2e encrypted, so all of it is readable by Apple. They regularly turn this information over to US federal authorities without a warrant (so, no probable cause) too.
Having such a huge trove of history of all of your communications saved that can be accessed by the police or a rogue Apple sysadmin at any time is something that I feel should even make the law-abiding nervous.
Having such a huge trove of history of all of your communications saved that can be accessed by the police or a rogue Apple sysadmin at any time is something that I feel should even make the law-abiding nervous.
Messages are not included in backups if you use Messages in iCloud as the backup option. Messages in iCloud also offers end-to-end encryption.
The encryption key for “Messages in iCloud” (a cross-device sync service) gets backed up in the iCloud Backup (which is, as discussed, not e2e and thus readable by Apple), however, backdooring the e2e nature of Messages in iCloud by escrowing the key to Apple in the backup.
If MIC is enabled, iCloud Backup backs up the key, breaking the e2e. If MIC is disabled, iCloud Backup backs up the plaintext message history, breaking the e2e. This is well documented in the Apple KB article 202303 on the topic.
It’s no longer an accident at this point.
https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...
https://support.apple.com/en-us/HT202303
> Messages in iCloud also uses end-to-end encryption. If you have iCloud Backup turned on, your backup includes a copy of the key protecting your Messages. This ensures you can recover your Messages if you lose access to iCloud Keychain and your trusted devices. When you turn off iCloud Backup, a new key is generated on your device to protect future messages and isn't stored by Apple.
It’s written in a misleading style, but if you read it carefully it is apparent what is happening.
Turning off iCloud Backup is insufficient, as each conversation is backed up twice: once from each side. Your conversation partners’ devices default iCloud Backup to on.
Apple has the ability to read almost every single iMessage sent or received, on a 12-24 hour lag (iCloud Backups happen overnight), from data already on their servers, without ever touching a customer phone. All of the encryption is a handwavey distraction.
If MIC is enabled, iCloud Backup backs up the key, breaking the e2e. If MIC is disabled, iCloud Backup backs up the plaintext message history, breaking the e2e. This is well documented in the Apple KB article 202303 on the topic.
It’s no longer an accident at this point.
https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...
https://support.apple.com/en-us/HT202303
> Messages in iCloud also uses end-to-end encryption. If you have iCloud Backup turned on, your backup includes a copy of the key protecting your Messages. This ensures you can recover your Messages if you lose access to iCloud Keychain and your trusted devices. When you turn off iCloud Backup, a new key is generated on your device to protect future messages and isn't stored by Apple.
It’s written in a misleading style, but if you read it carefully it is apparent what is happening.
Turning off iCloud Backup is insufficient, as each conversation is backed up twice: once from each side. Your conversation partners’ devices default iCloud Backup to on.
Apple has the ability to read almost every single iMessage sent or received, on a 12-24 hour lag (iCloud Backups happen overnight), from data already on their servers, without ever touching a customer phone. All of the encryption is a handwavey distraction.
arguably the worst part about imessage and scrolling is that even if you do manage to scroll up and find a previous message, you are automatically scrolled down to the bottom if a new message is received in that chat (on iphone)
You can touch the middle of the top bar and it will automatically scroll to the top of the current conversation. Works in a lot of apps too
yes it is really bad, the UIs for PMs was better 10 years ago, thinking about ICQ, Adium, MSN Messenger and the like. You could export history as file and easily scroll and search them
Google Voice has the same problem. The search function is useless. It does not return results that should match, and it does return results that shouldn't. Scrolling back through just a few months worth of messages can take 10-20 minutes. I wish I knew why it was so awful.
Are those messages queryable in Gmail?
No, I am referring to Google's SMS platform. It interfaces to nearly all other carriers, and is accessible through the "Voice" app or the web page at https://voice.google.com
Tell Voice to send your messages to email, and they'll be searchable from there. Use a query to ensure that you're searching text messages from Voice based on from address or subject. It won't help with searching messages that you've sent, unfortunately, not will it help with messages from before you turned the option on; I've had it on forever as I used to use it for desktop notifications.
One of the options without using third party tools and sdripts:
1. Use macbook.
2. Open imessage and scroll 1 year back.
3. Export to pdf.
4. Ensure the pdf contains 1 year of chatting.
5. Delete all the messages that have just been exported.
6. Repeat above for another year.
In the end you will have a pretty looking archive of imessage.
Even an old 2013 macbook with 8gb RAM is capable of accomplishing this task.
1. Use macbook.
2. Open imessage and scroll 1 year back.
3. Export to pdf.
4. Ensure the pdf contains 1 year of chatting.
5. Delete all the messages that have just been exported.
6. Repeat above for another year.
In the end you will have a pretty looking archive of imessage.
Even an old 2013 macbook with 8gb RAM is capable of accomplishing this task.
I've had this sort of experience with many systems, which is why I decided to use .txt files as the base data store for my current project.
It is indexed in a relational database, and it is presented in a Web 1.1 frontend, but it all flows back through the text files, and they are always available for download, optionally in zip or tar.gz archives.
It is indexed in a relational database, and it is presented in a Web 1.1 frontend, but it all flows back through the text files, and they are always available for download, optionally in zip or tar.gz archives.
Ironically SQLite is excellent choice in this situation.
SQLite is great, and I use it for caching and for providing data summaries... but it is not near-universally compatible and accessible, human-readable, the way text-files are.
About 10 years ago, I wrote an app for reading, searching, and exporting iPhone texts based on the sqlite files in a backup.
I don't think it works any longer on the latest iOS, but I open sourced the code if anyone wants to pick it up:
http://www.allyourtexts.com/
I don't think it works any longer on the latest iOS, but I open sourced the code if anyone wants to pick it up:
http://www.allyourtexts.com/
It doesn't now, but I can confirm that it was an excellent piece of software before. Thank you!
I want a chat app that is: E2EE and allows me to change phone numbers.
I started migrating people to Signal when that whole WhatsApp ToS hullaballoo happened, but now I need to change my phone number and apparently this is not supported in Signal, which is just crazy to me.
I started migrating people to Signal when that whole WhatsApp ToS hullaballoo happened, but now I need to change my phone number and apparently this is not supported in Signal, which is just crazy to me.
Signal is also guilty of the huge-and-useless database of old messages. Turning your chat history into text is not easy.
> A tale of wasted space and Services revenue
With photos, I guess this could end up being around 1GB. Maybe this will push you off the free plan to the $1/mo plan, but I don't see it being that big of a needle-mover for services revenue.
With photos, I guess this could end up being around 1GB. Maybe this will push you off the free plan to the $1/mo plan, but I don't see it being that big of a needle-mover for services revenue.
There are 1 billion iPhones in current, active use.
A dollar from each of those is 1 billion dollars.
That's a nice bonus, no matter how much the needle moves.
https://appleinsider.com/articles/21/01/27/there-are-1b-ipho...
That's a nice bonus, no matter how much the needle moves.
https://appleinsider.com/articles/21/01/27/there-are-1b-ipho...
You have a point that it adds up. I just suspect that your photos eclipses the size of your texts.
I made the mistake of turning on automatic photo back up (I have no idea how to bulk copy photos off the iphone without a mac, I've given them up as lost honestly, most of them are random crap anyway) and now my icloud account is completely full and I keep getting told I need to send Apple money.
I definitely prefer my current phone that lets me just rsync ~/Pictures.
I definitely prefer my current phone that lets me just rsync ~/Pictures.
I paid $0.99 for photosync. It syncs to many cloud providers, and (most importantly to me) to ssh/sftp.
You can set it to sync whenever your phone is inside a particular location boundary (e.g. at home, where there is wifi), so it works transparently with zero ports open to the internet, and no firewall-bypassing cloud services.
You can set it to sync whenever your phone is inside a particular location boundary (e.g. at home, where there is wifi), so it works transparently with zero ports open to the internet, and no firewall-bypassing cloud services.
Not that I'm suggesting you should send Apple money: but it's like £0.79 per month for 50gb - less than £10/year. For me at least: that's pretty cheap compared to other services.
My sister has a chat with 15 GB worth of attachments. Lots of videos :(
I have an awful problem with iMessage, that also speaks IMHO about the lack of care behind it. I upgraded an iPad Mini 1st. Gen (iOS 9.3.6) to an iPad Mini 5st. Gen (iOS at the time of exit I think), restoring from a backup.
iMessage never worked since the upgrade. I can only send messages, I see them once, but once I close the app and open it again, I don't see anything else. There's absolutely no way to reset the db of an Apple-included app unless I do a full format of the device, which after my full custom-configuration, I don't want to do. It sucks.
iMessage never worked since the upgrade. I can only send messages, I see them once, but once I close the app and open it again, I don't see anything else. There's absolutely no way to reset the db of an Apple-included app unless I do a full format of the device, which after my full custom-configuration, I don't want to do. It sucks.
Apps that are not have a good export story should be considered as keeping your content hostage. Either treat these as ephemeral, syndicate the content into them or switch. Your content in their app is lock in.
Who uses iMessage? I mean, I never seen one using it seriously. All my friends and family members uses WhatsApp (that has more or less the same problems, but at least you can export a conversation in a text file, or now import it in Telegram), and Telegram is nowadays used by all the young people.
This is probably due to the fact that in my country the majority of people has Android phones, and thus even who has an iPhone uses the same app that uses with people with Android to chat with other Apple users.
This is probably due to the fact that in my country the majority of people has Android phones, and thus even who has an iPhone uses the same app that uses with people with Android to chat with other Apple users.
The AI has trouble knowing what is important in the conversation. It seems like such difficulty creates an incentive for a user to ask someone something again instead of looking it up in history (although this also may be a signal for AI that something in the history is important).
Other reason may be that it is used by services - they want as long history as possible to find out what kind of person you are in case you are being investigated.
Anyway these artificial obstacles most often have sinister motives.
Other reason may be that it is used by services - they want as long history as possible to find out what kind of person you are in case you are being investigated.
Anyway these artificial obstacles most often have sinister motives.
Imagine buying a top of the line smartphone in 2021 and it comes with 64GB of storage. Truly Apple has done wonders with their marketing that people buy their phones.
The cynic in me says this is all part of the plan. Sure, we'll give you seemingly impossibly large amounts of cloud storage for free, learn to never delete anything that you ever talk about in your entire life.
Ten years down the road, "oh, looks like you need to start paying us monthly for more cloud storage!"
I kinda wonder how many Gmail users now pay Google for more storage space just to cover their email archive... And how many will in another ten years.
Ten years down the road, "oh, looks like you need to start paying us monthly for more cloud storage!"
I kinda wonder how many Gmail users now pay Google for more storage space just to cover their email archive... And how many will in another ten years.
> I kinda wonder how many Gmail users now pay Google for more storage space just to cover their email archive...
Gmail’s storage space has consistently grown over the years to keep up with the average user’s demand.
Google don’t want to do anything — like charging for email storage — that would disincentivize their mass of sensors^H users from feeding them ever more data.
(If you’re an outlier with way more storage than usual, they’ll happily charge you, knowing that the data point you represent is mostly going to get pruned out of their models anyway.)
Gmail’s storage space has consistently grown over the years to keep up with the average user’s demand.
Google don’t want to do anything — like charging for email storage — that would disincentivize their mass of sensors^H users from feeding them ever more data.
(If you’re an outlier with way more storage than usual, they’ll happily charge you, knowing that the data point you represent is mostly going to get pruned out of their models anyway.)
This was previously true. Free storage at Google hasn't increased from 15 GB since 2013. (I have a few promotional bonuses that bring me up to about 19 GB, but it's been that way for... over half a decade without increase.) Personally, my Google storage is mostly full, even though I haven't used Google Drive or Gmail since 2016, and have removed much of my personal content since then.
Also, now they're actually effectively decreasing everyone's storage. Previously, Google Drive document types and Google Photos didn't count towards that 15 GB limit. They will as of June 1st, 2021: https://blog.google/products/photos/storage-policy-update/
So, while cloud storage usage has exploded over the past eight years, Google's storage limits have not only not expanded to meet it, they've actually begun to tighten the belt.
Also, now they're actually effectively decreasing everyone's storage. Previously, Google Drive document types and Google Photos didn't count towards that 15 GB limit. They will as of June 1st, 2021: https://blog.google/products/photos/storage-policy-update/
So, while cloud storage usage has exploded over the past eight years, Google's storage limits have not only not expanded to meet it, they've actually begun to tighten the belt.
They don’t need that, they can just start saving pics in HEIC at 4MB per pic, and backups start exploding.
I use Gimp and generally obtain the same quality with a 450KB JPEG, sometimes 50KB depending on the contents. Images can be remarkably small. They should try increasing the compression ratio until a change is noticeable, but do they care?
I use Gimp and generally obtain the same quality with a 450KB JPEG, sometimes 50KB depending on the contents. Images can be remarkably small. They should try increasing the compression ratio until a change is noticeable, but do they care?
> They should try increasing the compression ratio until a change is noticeable, but do they care?
That fine for final/production images but it's a terrible idea for your originals, or really anything you might want to edit beyond cropping).
That fine for final/production images but it's a terrible idea for your originals, or really anything you might want to edit beyond cropping).
Wish this could be done to Facebook Messenger as well. I have some old conversations spanning over five years that would be nice to export.
Not sure if that's available outside the EU as well, but over here you can export an archive with all data they have on you including messages and attached media. I'm sure they leave out things but I use it to backup my chats.
FB Messenger does have data takeout, it comes in some kind of JSON format. Be prepared for some surprisingly massive file downloads though -- my chat history alone was over 50gb!
I recently had to restore from an iCloud backup that contains all my messages (which I never and will never delete) and it took over 24 hours to restore them.
The restore daemon actually downloads every single image and attachment you’ve ever sent or received one file at a time.
Not that I can scroll back that far, but you never know when you need to search for something.
The restore daemon actually downloads every single image and attachment you’ve ever sent or received one file at a time.
Not that I can scroll back that far, but you never know when you need to search for something.
The comparison to Photos in TFA is apt. Apple could copy the most relevant subset of features, in particular the Years/Months/Days/All navigation views, and it would dramatically improve the utility of iMessage for old messages.
Improving the abysmal search result quality is probably more important, though.
I'm not sure if it even works anymore or if there's a better option, but I've used PhoneView <https://www.ecamm.com/mac/phoneview/> in the past to make a PDF archive of my iMessages.
I've used this as well, mostly with success, but this last year I tried and it kept failing and I haven't had the time to revisit it.
It may be a problem with PhoneView, but I'm not convinced that it is. Also, due to the size of the backup, I do have to do it to an external drive.
It may be a problem with PhoneView, but I'm not convinced that it is. Also, due to the size of the backup, I do have to do it to an external drive.
If you have them archived as iChat files on a Mac (Library/Messages/Archive), you could try Past: https://zeezide.de/en/products/past/
Have a look at this project https://airmessage.org/blog/airmessage-is-now-open-source and you'll like find ways for raw reading of your chat databases..
my biggest problem with iOS storage is that if you pay to store a photo in iCloud and then use Messages to send it to 5 friends, you are now paying to store it in iCloud 6x. And of course it’s totally unfair that there is no priced option between 200GB and 1TB. Same with Google Photos.
Kinda dying to know if they dedup on the server? Cause technically speaking if everyone was on Apple then that 1 photo becomes 12 not to mention it getting send to another layer of people.
Yeah exactly. It’s nuts.
I've used a mess of find and rm to clear up space on my local drive, but afaik they'll still take up space in iCloud storage -- I agree that there should be a way to more efficiently discard them, and that automatically deleting old ones is not the right thing to do
Tangential: Telegram has a great search feature. It’s super fast, allows to visit messages by date and in group chats you can even see the list of people who feature in the keyword search results. The UX is great, and all chat apps should just imitate it as a start.
Try Google Messages. How do I export them? There's no chat.db. It's impossible scrape, unless you wait(5) and scrollTo() etc. I've found no good solution for extracting/backing up Google Messages (outside of Google's dominion).
You can use this to read the iMessage binary storage file and convert it to JSON [0]
[0] https://github.com/matrix-hacks/ichat2json
[0] https://github.com/matrix-hacks/ichat2json
The contention that iMessage doesn't do a "space save" mode on device is wrong.
I have 75GB of messages in iCloud, I see 25GB on my 500GB Mac and 11GB on my 64GB iPhone.
I have 75GB of messages in iCloud, I see 25GB on my 500GB Mac and 11GB on my 64GB iPhone.
https://imazing.com/ Is worth a look if you want to backup export old messages.
I'm just commenting because I also find iMessages to be a completely neglected app considering how central it is to so many people's communications.
You'd be surprised how few engineers work on apple projects relative to their size, and how apple underpays compared to other big tech companies like FB or Google unless your working on the current hot project.
It’s tedious, but all conversations can be reviewed by size and manually deleted. Far from ideal, but not quite limited to automatic Apple tools.
Why would anyone keep using iMessage under these circumstances? My chat messages take up less than 1% of my phone storage.
I regularly delete all messages and call entries. Could not care less about reading any of them more than once.
> Texts has integrations for all major messaging platforms...
Any plans to include phone calls, voice mails, and email too?
Future perfect:
RSS reader-like UI. Top level is Contacts. Including groups (w/ aliases). Drill down to see missives. Show read, unread, return receipt, etc. Sparklines representing activity.
Any plans to include phone calls, voice mails, and email too?
Future perfect:
RSS reader-like UI. Top level is Contacts. Including groups (w/ aliases). Drill down to see missives. Show read, unread, return receipt, etc. Sparklines representing activity.
iMsgs, esp vids --> GB --> new phone + icloud $$
Yeah, can second this.
After years of herding imessages i went ahead and deleted everything older than a year.
i encourage othes to consider doing the same. its been great for me.
i encourage othes to consider doing the same. its been great for me.
Why are we so hung up on keeping texts? (I'm guilty of this hang-up too.) I didn't archive my AIM messages. I don't record my phone conversations.
The article mentions 2 examples in the 1st paragraph. Maybe not common, but to each their own.
because it feels free.
just in case.
i hope we all learn what good digital hygiene is.
just in case.
i hope we all learn what good digital hygiene is.
They’re not for the user they’re for the FBI and CIA
[deleted]
Once you have the schema figured out, it's dead easy to build a third-party client that works better than the official one. Even search works great with a simple LIKE query but Apple re-indexes all messages leading to your CPU going over 1000%: https://twitter.com/KrauseFx/status/1396433852126670852
Source: I built a third-party desktop client for iMessage at https://texts.com and reverse engineered the complete sqlite structure.