Applebot(support.apple.com)
support.apple.com
Applebot
https://support.apple.com/en-us/HT204683
314 comments
> It was then modified to do it's own DNS resolution and caching, fond memories...
Unlike other languages, Go bypasses system's DNS cache, and goes directly to the DNS server, which is a root cause of many problems.
Unlike other languages, Go bypasses system's DNS cache, and goes directly to the DNS server, which is a root cause of many problems.
Yeah, I've never had to implement my own DNS cache for a language before...
If you're on a system with cgo available, you can use `GODEBUG=netdns=cgo` to avoid making direct DNS requests.
This is the default on MacOS, so if it was running on four Mac Pro's I wouldn't expect it to be the root cause.
If you're on a system with cgo available, you can use `GODEBUG=netdns=cgo` to avoid making direct DNS requests.
This is the default on MacOS, so if it was running on four Mac Pro's I wouldn't expect it to be the root cause.
It's possible that wasn't the default setting on Macs back then. I don't know that cgo would be a good choice either, if you're resolving a ton of domains at once. Early versions of Go would create new threads if a goroutine made a cgo call, and an existing thread was not available. I remember this required us to throttle concurrent dial calls, otherwise we'd end up with thousands of threads, and eventually bring the crawler to a halt.
To make DNS resolution really scale, we ended up moving all the DNS caching and resolution directly into Go. Not sure that's how you'd do it today, I'm sure Go has changed a lot. Building your own DNS resolver is actually not so hard with Go, the following were really useful:
https://idea.popcount.org/2013-11-28-how-to-resolve-a-millio...
https://github.com/miekg/dns
To make DNS resolution really scale, we ended up moving all the DNS caching and resolution directly into Go. Not sure that's how you'd do it today, I'm sure Go has changed a lot. Building your own DNS resolver is actually not so hard with Go, the following were really useful:
https://idea.popcount.org/2013-11-28-how-to-resolve-a-millio...
https://github.com/miekg/dns
This is true but a little misleading. On Windows Go uses GetAddrInfo and DNSQuery which does the right thing. But on Linux there are two options: netgo and netcgo -- a pure Go implementation that doesn't know about NSS, and a C wrapper that uses NSS.
Since netgo is faster, by default Go will try its best to determine if it must use netcgo by parsing /etc/nsswitch.conf, looking at the tld, reading env variables, etc..
If you're building the code you can force it to use netcgo by adding the netcgo build tag.
If you're an administrator the least intrusive method I think would be setting LOCALDOMAIN to something or '' if you can't think of anything which will force it to use NSS.
Since netgo is faster, by default Go will try its best to determine if it must use netcgo by parsing /etc/nsswitch.conf, looking at the tld, reading env variables, etc..
If you're building the code you can force it to use netcgo by adding the netcgo build tag.
If you're an administrator the least intrusive method I think would be setting LOCALDOMAIN to something or '' if you can't think of anything which will force it to use NSS.
And Java.
As I understand it, Go and Java are both trying to avoid FFI and calling out to system libs for name resolution.
I tend to always offer a local caching resolver available over a socket.
As I understand it, Go and Java are both trying to avoid FFI and calling out to system libs for name resolution.
I tend to always offer a local caching resolver available over a socket.
>Up until the release of iOS 9, Applebot ran entirely on four Mac Pro's in an office. Those four Mac Pro's could crawl close to 1B web pages a day.
The scale of web stuff sometimes surprises me. 1B web pages sounds like just about the daily web output of humanity? How can you handle this with 4 (fast) computers?
The scale of web stuff sometimes surprises me. 1B web pages sounds like just about the daily web output of humanity? How can you handle this with 4 (fast) computers?
Computers are very fast. We just tend to not notice because today's software is obese.
Yes, let's all run separate web browsers as the application and run our own JavaScript inside our browser. Who cares if there's 5 other "apps" doing exactly the same!
Insanity.
Insanity.
Multiple tabs/browser windows is similar and generally not an issue.
I think they were referring more to apps like Slack and other similar JS browser/ JS based apps which run separate from the browser. Maybe I'm being generous? Slack is certainly itself a beast.
Yes this is precisely what I meant. eg. Electron apps, Slack, VS Code, Skype etc. etc. ad nauseum
[deleted]
Doesn't it depend on a lot of things? For example you can only do head requests to see if a page changed since a given timestamp. If not then there is no need to process it.
For anybody wondering how:
The HTTP HEAD method requests the headers that would be returned if the HEAD request's URL was instead requested with the HTTP GET method. For example, if a URL might produce a large download, a HEAD request could read its Content-Length header to check the filesize without actually downloading the file.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HE...
The HTTP HEAD method requests the headers that would be returned if the HEAD request's URL was instead requested with the HTTP GET method. For example, if a URL might produce a large download, a HEAD request could read its Content-Length header to check the filesize without actually downloading the file.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HE...
Typically you wouldn’t bother with a HEAD request, you’d do a conditional GET request.
When you request a page, the response includes metadata, usually including a Last-Modified timestamp and often including an ETag (entity tag). Then when you make subsequent requests, you can include these in If-Modified-Since and If-None-Match request headers.
If the resource hasn’t changed, then the server responds with 304 Not Modified instead of sending the resource all over again. If the resource has changed, then the server sends it straight away.
Doing it this way means that in the case where the resource has changed, you make one request instead of two, and it also avoids a race condition where the resource changes between the HEAD and the GET requests.
When you request a page, the response includes metadata, usually including a Last-Modified timestamp and often including an ETag (entity tag). Then when you make subsequent requests, you can include these in If-Modified-Since and If-None-Match request headers.
If the resource hasn’t changed, then the server responds with 304 Not Modified instead of sending the resource all over again. If the resource has changed, then the server sends it straight away.
Doing it this way means that in the case where the resource has changed, you make one request instead of two, and it also avoids a race condition where the resource changes between the HEAD and the GET requests.
Do a lot of random pages return etags? I've only ever seen them in the AWS docs for boto3
nginx sends it by default for static files (example: hacker news [0]), I assume other web servers do too.
[0] https://news.ycombinator.com/y18.gif
[0] https://news.ycombinator.com/y18.gif
I am particular curious about data storage.
Does it use a traditional relational database or another existing database-like product? Or is built from scratch just sitting on top of a file system.
Does it use a traditional relational database or another existing database-like product? Or is built from scratch just sitting on top of a file system.
Nope, you don't really need a database. What you need for fast, scalable web crawling is more like key-value storage: a really fast layer (something like RocksDB on SSD) for metadata about URL's, and another layer that can be very slow for storing crawled pages (like Hadoop or Cassandra). In reality, writing directly to Hadoop/Cassandra was too slow (because it was in a remote data center) so it was easier to just write to RAID arrays over Thunderbolt, and sync the data periodically as a separate step.
Interesting stuff. I've used libcurl to crawl at that kind of pace, is the parsing/indexing separate from that count per day? Also interested in how you dealt with DNS and/or rate limiting
I've done similar at smaller scale. Instead of messing with underlying DNS or other caching in our code we just dropped a tuned dnsmasq as the resolver in front. The crawler had a separate worker to fill hosts so it was mostly hot when the crawler was asking.
In my case I was just fetching the home page of all known domain names. First issue I'd noticed was ensuring DNS requests were asynchronous. I wanted to rate limit fetching per /28 IPv4 to respect the hosts getting crawled, but couldn't really do that without knowing the IP beforehand (and keeping the crawler busy) so ended up creating a queue based on IP. I used libunbound. Found that some subnets have hundreds of thousands of sites and although the crawl starts quickly, you end up rate limited on those.
Also interested at the higher end of the scale about how 'hard'/polite you should be with authoritative nameservers as some of them can rate limit also.
Also interested at the higher end of the scale about how 'hard'/polite you should be with authoritative nameservers as some of them can rate limit also.
Roughly estimating, each Mac Pro could crawl around 3k pages per second.
[deleted]
Which is not possible
Say the average web page is 100kb, and assuming gigabit connection in the office, then that's about a thousand pages per second. If the office switch is on 10gbit that would work out to 4000p/s naively counting. But we're in the same order of magnitude for the speed even on gbit, and we're not accounting for gzip, and the actual average page size might be a bit lower too.
Everything was on 10gigE. The average page size was around 17KB gzipped. Everything's a careful balance between CPU, memory, storage, and message throughput between machines.
Apple's corporate network also had incredible bandwidth to the Internet at large. Not sure why, but I assumed it was because their earliest data centers actually ran in office buildings in the vicinity of 1 Infinite Loop.
Apple's corporate network also had incredible bandwidth to the Internet at large. Not sure why, but I assumed it was because their earliest data centers actually ran in office buildings in the vicinity of 1 Infinite Loop.
The average is a lot closer to 25KB IIRC, gzipped
Why not?
Can you share some more details about the current state? Is it still written in Go?
No idea, it's been years since I last worked on it. It was also not the only Go service written at Apple (90% of cloud services at Apple were written in Java), though it may have been the first one used in production.
And I sit here kind of shocked that Apple would use Java for anything, backend or not. I thought Apple had a strong preference for owning its own tech stacks, whether that be ObjC/WebObjects or later Swift...
I think WebObjects was supporting Java even before it came to Apple from Next. In the early days, many of Apple's services built with WebObjects even ran on Sun server hardware, and XServe's. But nowadays it's all commodity Linux hardware, like you would find in any data center.
Not sure why you'd be shocked, it's a solid language for enterprise services like Apple offers, and their other languages - C/C++, Objective-C, Swift - aren't very kind for web services.
Great use case for Go though, especially its concurrency features for web crawlers. I reckon Scala could work too, although it's a lot more complicated / clever.
Great use case for Go though, especially its concurrency features for web crawlers. I reckon Scala could work too, although it's a lot more complicated / clever.
Out of curiosity, why would C or C++ not be good for web services?
I would guess because the input sanitizing requirement is harder for the web; having a stackoverflow when running locally requires the attacker to execute locally -- having a use-after-free from port 80 would be a much wider audience
Some Apple services were written in C/C++. One downside is it's very hard to source engineers across the company who can then work on that code, or for those engineers to go work on other teams.
WebObjects has been fully Java since version 5 was released in 2001:
https://en.wikipedia.org/wiki/WebObjects#WOWODC
Apple's server stack has been primarily Java for about 20 years.
Apple's server stack has been primarily Java for about 20 years.
Apple had a very odd obsession with Java right after the NeXT purchase. WebObjects got converted and they tried to do a Java Cocoa. Both were worse than the original.
At the cocoa heads user group I heard that ruby is very popular for their services more recently.
Apple employs the founder of the Netty project, who has given plenty of open talks about Apple’s use of Netty (which implies Java services). Same is true for Cassandra.
Can you talk more about the specific? What kind of parsers did you guys use? How about storage? How often did you update pages?
You should check out Manning's "Introduction to Information Retrieval", it has far more detail about web crawler architecture than I can write in a post, and served as a blueprint for much of Applebot's early design decisions.
Nice, thanks for the recommendation!
The book is freely available online at https://nlp.stanford.edu/IR-book/information-retrieval-book....
The book is freely available online at https://nlp.stanford.edu/IR-book/information-retrieval-book....
With 1b pages per day I guess you needed 1gbit/s connections on each of those machines? Especially if they also wrote back to centralized storage.
I guess there are not many places where you can easily get 4GB/s sustained throughput from a single office (especially with proxy servers and firewalls in front of it). Is that standard at Apple or did the infrastructure team get involved to provide that kind of bandwidth?
I guess there are not many places where you can easily get 4GB/s sustained throughput from a single office (especially with proxy servers and firewalls in front of it). Is that standard at Apple or did the infrastructure team get involved to provide that kind of bandwidth?
Do you have a timeline of how AppleBot has evolved?
Was that including the ability to render js driven asynchronously loaded pages, including subsequent XHR requests? If so, it's beyond impressive.
Thanks for sharing mate. That is amazing insights!
Why did you leave Apple?
Sorry to be pedantic, but your misuse of apostrophes in an otherwise perfect text annoys me.
All three uses of "it's" should be "its".
And I would just write "Mac Pros" instead of Mac Pro's".
All three uses of "it's" should be "its".
And I would just write "Mac Pros" instead of Mac Pro's".
The context for this is that Apple is rumored to be starting their own search engine
https://appleinsider.com/articles/20/08/27/apple-may-launch-...
https://appleinsider.com/articles/20/08/27/apple-may-launch-...
I know Apple's walled garden approach isn't for everybody, but I'd personally love to see them make a search engine that isn't built around Ads. The strategy also is consistent with their pursuit of vertical integration.
I use DuckDuckGo for search, and do trust them with privacy. But if Apple did a half-decent job at nailing the search experience, I'd switch.
I use DuckDuckGo for search, and do trust them with privacy. But if Apple did a half-decent job at nailing the search experience, I'd switch.
> I'd personally love to see them make a search engine that isn't built around Ads
The ads that Apple sells on the app store contribute $2 billion in revenue -- I don't think they'd build a search engine that isn't built around ads.
The ads that Apple sells on the app store contribute $2 billion in revenue -- I don't think they'd build a search engine that isn't built around ads.
Which is pretty staggering, given that the ads only appear on search results within the App Store. The scale is amazing.
I find it unlikely Apple is going to compromise their their privacy first reputation to make an advertising platform which is competitive with Google's. I wouldn't be surprised to see advertising if Apple releases search, but it will likely be context (what you searched for) driven reliance rather than based on deep tracking. That would make it far less appealing to the skeezier advertisers. I'd expect something similar to what you see in News.
I can't see how Apple would even do the kind of search result pollution Google does now. In fact, I suspect the only reason Apple is launching advertising is because they see an opportunity to provide a better experience—namely one where you see organic results immediately.
I can't see how Apple would even do the kind of search result pollution Google does now. In fact, I suspect the only reason Apple is launching advertising is because they see an opportunity to provide a better experience—namely one where you see organic results immediately.
Yes, and they've also been caught selling iTunes user data.
People thinking Apple aren't going to copy Google's revenue model to some degree are delusional.
People thinking Apple aren't going to copy Google's revenue model to some degree are delusional.
If you’re referring to this lawsuit [0], then the plaintiffs provided no evidence and the court dismissed the case with prejudice.
[0] https://9to5mac.com/2019/11/26/accusing-apple-of-selling-cus...
[0] https://9to5mac.com/2019/11/26/accusing-apple-of-selling-cus...
I think it's too soon to assume that Apple's search engine would be accessible directly from the web, rather than just being a backend for search functionality from within iOS. After all, they don't host any Apple Maps equivalent to Google Maps on the web.
Actually they do offer maps for web, but the way you get to it is via DuckDuckGo. Example:
https://duckduckgo.com/?q=San+Francisco&t=h_&ia=web&iaxm=map...
https://duckduckgo.com/?q=San+Francisco&t=h_&ia=web&iaxm=map...
Is this an exclusive DuckDuckGo and Apple partnership, or are there other websites that also use Apple Maps?
If you have an Apple Developer subscription, you should be able to use Apple Maps using Mapkit on any website - https://developer.apple.com/maps/web/
When I do that, I can't get directions, though. It refers me to Bing, GMaps, HERE (the Nokia one), and OpenStreetMap.
A search engine that optimizes for good content instead of SEO-spam and ads would be a very welcome addition. Google seems to have given up on fighting malicious/spammy SEO (I'd expect them to identify and downrank spammy tricks like what the recipe websites do).
I'm in favour of competition for Google, but I can't be the only one who genuinely has no issue using Google? I see people complaining on HN quite a lot but genuinely the only thing I have - in years - struggled to find was the correct eBPF documentation but even then it only took a bit of nudging.
I think there are types of searches it does very poorly on, but also different people have different expectations. In my experience, if something is a product for sale, the first several pages are always places to buy it, which is never what I’m looking for, I already have online shops I like and saw the item there, or am searching from inside a store. I want reviews, forum experience, or even just pictures. for example, actual pictures of a bike rack installed on a bike like mine, not just the manufacturer’s steel floating in a white void pics.
In addition, google has become very aggressive with disregarding words. So a search like (a fake example, not tried the search, not a real place) Jim’s tasty pizza Albuquerque 90s where I’m searching for where a restaurant I remember as a child was located, and has maybe 5 real results from a local history Wordpress blog 15 years ago and Facebook posts, Google will just drop everything specific and give you results for Albuquerque pizza, and then mainly show ads for national chains up top. So I need quotes in the search pretty much always.
In addition, google has become very aggressive with disregarding words. So a search like (a fake example, not tried the search, not a real place) Jim’s tasty pizza Albuquerque 90s where I’m searching for where a restaurant I remember as a child was located, and has maybe 5 real results from a local history Wordpress blog 15 years ago and Facebook posts, Google will just drop everything specific and give you results for Albuquerque pizza, and then mainly show ads for national chains up top. So I need quotes in the search pretty much always.
If you go into the advanced search options you can tell Google to explicitly search on what you type. I use it so often for the reasons you point out (because nowadays in their infinite wisdom they will helpfully ignore quotes too) it occurs to me I should either find an extension or script to just have it enabled by default.
I use Google and Google Maps plenty, though I'm one of the ones who might complain about their [lack of] privacy practices.
I use it for certain types of searches, and not others. I'm conscious of the fact that what I'm typing and searching, and subsequently clicking through will be piled on to the stack Google thinks is me. So it's usually tech searches, real estate/apartments, street view (property or exploring).
Health matters, personal matters, etc I tend to keep out of their search box.
Though more and more I tend to do a lot of cross-searching if privacy is not a concern.
I use it for certain types of searches, and not others. I'm conscious of the fact that what I'm typing and searching, and subsequently clicking through will be piled on to the stack Google thinks is me. So it's usually tech searches, real estate/apartments, street view (property or exploring).
Health matters, personal matters, etc I tend to keep out of their search box.
Though more and more I tend to do a lot of cross-searching if privacy is not a concern.
Google is fine for me (I use it as fallback when DDG fails) for technical results but for anything else it's pretty bad (recipes are a typical example). It's not just the fault of Google to be honest - the internet itself has turned into a cesspool - however Google would be in the best position to curate that and discourage that behavior by penalizing the offenders' ranking.
Google ads don’t affect what search ranks. Those are two different entities. And google realllllly wants to get rid of search spam. But it’s insanely profitable to rank, so people try really hard and often beat google.
It's debatable whether Google Ads affect Google search ranking (it's impossible to prove either way considering how many factors affect ranking), but regardless of whether they currently influence rankings, there might be reluctance to change that due to Google's business model.
Google's job as a search engine is to provide relevant and quality content. Ads are spam and noise, so if Google does want to provide quality content they should be penalizing ads just like they would penalize keyword spam and similar unpleasant SEO tactics. Given their business model however it is not in their best interests to do so, so it's very unlikely that Google will do it.
Another company that does not make their money on advertising has a better chance of doing this.
Google's job as a search engine is to provide relevant and quality content. Ads are spam and noise, so if Google does want to provide quality content they should be penalizing ads just like they would penalize keyword spam and similar unpleasant SEO tactics. Given their business model however it is not in their best interests to do so, so it's very unlikely that Google will do it.
Another company that does not make their money on advertising has a better chance of doing this.
Ads do get penalized for being a bad fit: they get a lower quality score and cost more.
Meanwhile, ads are distinct from search. The #1 result refers to the first result past the ads.
Meanwhile, ads are distinct from search. The #1 result refers to the first result past the ads.
> Ads do get penalized for being a bad fit: they get a lower quality score and cost more.
My point is that as a user, ads, regardless of how "good" they are, are not what I am looking for when I click on a search result (I am talking about website ads and not ads on the results page itself) and thus a search engine that downranks pages with ads in favor of those without them would be beneficial for me.
It would be against Google's interest to implement that (as they'd also be fighting against their own ad network) but a different company not involved in the advertising business (and making high enough margins on other products) has a higher chance of delivering this successfully.
My point is that as a user, ads, regardless of how "good" they are, are not what I am looking for when I click on a search result (I am talking about website ads and not ads on the results page itself) and thus a search engine that downranks pages with ads in favor of those without them would be beneficial for me.
It would be against Google's interest to implement that (as they'd also be fighting against their own ad network) but a different company not involved in the advertising business (and making high enough margins on other products) has a higher chance of delivering this successfully.
Millionshort is a good search engine for avoiding spam sites.
It would seem silly not to include ads based on the text string the user searches. Even without tracking/personalization those are insanely valuable.
I would love to see a decent search engine.
Google search has gone down the toilet. Most Google search results are ads or SEO optimized junk. It was funny once when I was searching for a medical term, all articles on first page were clearly SEO optimized and all articles were very similar from word usage, sentence construction, verbiage, like written by same person. I am surprised that Google doesn't filter out the results using some sort of similarity analysis to offer variety in results. I don't want to see all links giving exactly the same info.
Google search has gone down the toilet. Most Google search results are ads or SEO optimized junk. It was funny once when I was searching for a medical term, all articles on first page were clearly SEO optimized and all articles were very similar from word usage, sentence construction, verbiage, like written by same person. I am surprised that Google doesn't filter out the results using some sort of similarity analysis to offer variety in results. I don't want to see all links giving exactly the same info.
Google is vulnerable. Never thought I'd be able to say that.
And the same rumours has been going on for years, and I seriously doubt that.
Reason is that Apple already has a Search Engine. It is called Siri. It is what you get when you ask Siri Questions. What I think Apple hopes to achieve is that get 80% of what you want, From may be Recipes, Simple Answers like Exchange Rate, Nutrients, Sport Scores... etc away from Google.
Letting you to Google the absurd, hard, questions that has less Data value.
All while collecting roughly $10 per user for Default Search Engine Placement in Safari.
And remember the Privacy stand Apple has giving less Data to Google.
Basically Apple is squeezing Google in every single direction. And pitching them against Azure and AWS for Cloud Services for discount.
Reason is that Apple already has a Search Engine. It is called Siri. It is what you get when you ask Siri Questions. What I think Apple hopes to achieve is that get 80% of what you want, From may be Recipes, Simple Answers like Exchange Rate, Nutrients, Sport Scores... etc away from Google.
Letting you to Google the absurd, hard, questions that has less Data value.
All while collecting roughly $10 per user for Default Search Engine Placement in Safari.
And remember the Privacy stand Apple has giving less Data to Google.
Basically Apple is squeezing Google in every single direction. And pitching them against Azure and AWS for Cloud Services for discount.
I can't believe this hasn't happened yet honestly. Apple Maps has been out for 7 years now. Also a bit troubling if it gains traction, considering Apple has been a lot more heavy handed in keeping content they don't approve of out from their ecosystem.
I’m confused as to why it’s even happening now. A good index costs a fortune to build and maintain.
How does this benefit Apple? They’re a hardware company.
How does this benefit Apple? They’re a hardware company.
No, Apple is a 'full-stack' tech company. People buy their products both for their hardware and software. If they were just a hardware company, why would they bother making macOS, iOS, iMessage, Safari etc?
To sell more hardware, obviously. You wouldn’t buy a mac without macOS.
How does a search engine sell more hardware?
How does a search engine sell more hardware?
Haven’t you heard people whine about how Apple is so much about privacy but they take money from Google to have it as the default search?
They don’t have Google as the default search because of the money, they have it because they think it is the best. Even despite the privacy problems and the control this grants Google.
How does a maps service sell more hardware? Because using their own maps service Apple can completely control the experience.
Now maps was hard and search is much harder still so we may very well never see this released. But even then they can use this as leverage while negotiating with Google, both for money and for control.
They don’t have Google as the default search because of the money, they have it because they think it is the best. Even despite the privacy problems and the control this grants Google.
How does a maps service sell more hardware? Because using their own maps service Apple can completely control the experience.
Now maps was hard and search is much harder still so we may very well never see this released. But even then they can use this as leverage while negotiating with Google, both for money and for control.
> How does a search engine sell more hardware?
Because your claim about them being a hardware company is incorrect.
They market themselves as a privacy focused company. Privacy is all about software and services, not hardware. Search is something they could add to their portfolio to strengthen their privacy stance.
Because your claim about them being a hardware company is incorrect.
They market themselves as a privacy focused company. Privacy is all about software and services, not hardware. Search is something they could add to their portfolio to strengthen their privacy stance.
I would think that they can leverage a custom search engine in more flexible ways than Google to improve UX.
Also, Apples main revenue isn’t hardware but services IIRC. They’re not a hardware company.
Also, Apples main revenue isn’t hardware but services IIRC. They’re not a hardware company.
You do not recall correctly.
Services are ca. 18% of Apple’s revenue, after years of huge spending and tons of dedicated effort to grow this segment. The fact that it is as high as a fifth is itself a monumental feat.
Hardware sales of the iPhone alone, excluding all computers, AirPods, and iPads, is over 40%.
Services are ca. 18% of Apple’s revenue, after years of huge spending and tons of dedicated effort to grow this segment. The fact that it is as high as a fifth is itself a monumental feat.
Hardware sales of the iPhone alone, excluding all computers, AirPods, and iPads, is over 40%.
The same reason why having a maps product benefits them. They want to have complete end-to-end control over a user's experience on their device, well beyond just the hardware. Having their biggest competitor dictate what happens whenever someone opens the web browser on an iPhone is a massive hole.
An index of enough of the web to keep 90% of users happy can be built for ~$10k of storage.
The last 10% quickly gets very expensive tho.
The last 10% quickly gets very expensive tho.
Hard drives are cheap enough that storing the index itself is the least expensive part of building it. Engineer time to make that index usable for a search engine is that much money every week.
One of the selling points of apple devices is getting away from google spyware. Every bit of spyware they can prevent the user from being exposed to is better.
Apple doesn’t care at all about the most common types of spyware. Nearly every app in the iOS App Store contains tons of spyware.
Apple’s view is that you consented to this sort of unaccountable invisible tracking by dozens of third parties (against whom you have zero recourse), when you agreed to the TOS of the App Store.
The data is aggregated, sold, and resold again.
https://techcrunch.com/2020/07/09/data-brokers-tracking/
Apple’s view is that you consented to this sort of unaccountable invisible tracking by dozens of third parties (against whom you have zero recourse), when you agreed to the TOS of the App Store.
The data is aggregated, sold, and resold again.
https://techcrunch.com/2020/07/09/data-brokers-tracking/
I imagine the release of Apple Maps may have convinced them to go slow with this kind of thing.
Exactly this
There's nothing slow about this, it's already launched. I don't understand why people don't get this -- it even says so at the top of this page: "Products like Siri and Spotlight Suggestions use Applebot"
I don't know why anyone would think that Apple would make a web page search engine like Google. Apple has already launched the Apple version in your phones integrated into the software you use.
I don't know why anyone would think that Apple would make a web page search engine like Google. Apple has already launched the Apple version in your phones integrated into the software you use.
They are going slow, though.
That they use their own search for this is not a big headlining marketing feature so they aren’t really evaluated on it.
I do think that’s a direct consequence of their experience with Maps, though I guess their hand was more forced on Maps and they had to tell the world about it.
However, using Siri and search suggestion they can, bit by bit, replace search results from other sources with their own without telling anyone about it. It’s a more careful approach.
That they use their own search for this is not a big headlining marketing feature so they aren’t really evaluated on it.
I do think that’s a direct consequence of their experience with Maps, though I guess their hand was more forced on Maps and they had to tell the world about it.
However, using Siri and search suggestion they can, bit by bit, replace search results from other sources with their own without telling anyone about it. It’s a more careful approach.
Well they've gone to the effort to build a search index and they have sufficient market power to push millions of user onto whichever search engine they'd like. It's essentially leaving money on the table not to take the final step of popping a web frontend on it and making it the Safari default.
It already is the Safari default. As you type your search, it gives you a Siri Suggestion. When you type a query into Spotlight, you use it.
Apple isn't an ad-supported business. It's not going to look like Google. It looks the way it's already released.
Apple isn't an ad-supported business. It's not going to look like Google. It looks the way it's already released.
Maybe because they are desperate for a different search provider than Google?
I don't use Siri or any assistant, so I don't seem to use what I'm being averaged into using. Majority opinion here is that Siri is seriously lacking behind the other offerings. Apple won't be oblivious to this. If it is solely due to their back-end search engine, then making that a public search engine website won't do Apple any favors if the results are truly that bad.
I don't use Siri or any assistant, so I don't seem to use what I'm being averaged into using. Majority opinion here is that Siri is seriously lacking behind the other offerings. Apple won't be oblivious to this. If it is solely due to their back-end search engine, then making that a public search engine website won't do Apple any favors if the results are truly that bad.
Voice assisted search probably won't give you much feedback. With a web based search engine, users provide feedback by ignoring some results and clicking others. Therefore releasing a search engine could help them improving it and deliver results with Siri that people would've clicked on otherwise.
Why would they be desperate for using another search engine when Google pays them $8 billion+ a year?
Because Google makes enough money on ads from Google search to spend $8 billion on making it the default search on iOS devices
Apple is mostly a focused company. There is no money in search except advertising and user tracking. Apple has already completely failed at advertising once.
Yes I realize that the App Store has ads and it is a stain that should not have ever happened.
Yes I realize that the App Store has ads and it is a stain that should not have ever happened.
In all seriousness, Apple may do certain things well, but cloud-anything just doesn't appear to be in its DNA. Maps, mail, iCloud, Timemachine, etc. Pretty much every service I can think of is laden with bugs, quirks, actual data loss risk, or is slow enough to be unusable. I'm not even remotely surprised that there is no "Apple search" yet.
They also run the App Store, Push Notifications, iAd, Apple News, Apple Music, Apple TV, Siri, Apple Pay and some of the largest systems for photo sharing, file storage, subscriptions, retail billing and more in the world. Saying Apple is bad at cloud services in general is incorrect.
The CDNs are fine, push and subscriptions are mercurial at best and you have no choice but to use, and Siri is an example of Apple's cloud weakness.
Mercurial is certainly a step above the antics of Loki, so count your blessings!
Ssssssssmokin'!
They do have plenty of high-volume services e.g. Siri, Siri Suggestions, Maps, Online Store, Apple Music that haven't had any major issues. And even iCloud is substantially better now than in the early days.
Most of your issues seemed like the old, rehashed ones from nearly a decade ago.
Most of your issues seemed like the old, rehashed ones from nearly a decade ago.
Their cloud-based Notes App is simply killer. Sure, it’s a simple app but I have had a flawless experience on it.
Photos is pretty great too, but I’m not sure how it compares to other photo services.
Photos is pretty great too, but I’m not sure how it compares to other photo services.
And yet they forced you to upgrade all your operating systems an all devices you own with the last Notes update (was it iOS 13?) or you'd lose data.
Maps is a search problem and Apple Maps has gotten quite good. It’s my go to now after a decade of Google maps.
Apple Maps is my goto because Google Maps just hates showing street names for some reason. Zoom in so a street takes up 100% of your screen and it still won't display the name many times.
Obviously that's not the only reason, haha, it's gotten surprisingly good a few years ago. Public transit and cycling is better in Google (though nothing beats CityMapper for transit if it supports your city). But for walking and driving Apple Maps is my winner.
Obviously that's not the only reason, haha, it's gotten surprisingly good a few years ago. Public transit and cycling is better in Google (though nothing beats CityMapper for transit if it supports your city). But for walking and driving Apple Maps is my winner.
It still lacks cycling, that is usually the reason fo me to open google maps on my iphone.
As of the next iOS release that will no longer be the case
As long as you live in New York City, the San Francisco Bay Area, or Shanghai and Beijing.
That's coming in iOS 14, evidently.
So I'm using iOS 14 and just tried to pull it up. It has a message stating "Cycling directions are not yet available in [my city]"
A quick google search to see if it was simply because of beta or there is a limited roll out, produced this:
>New York City, the San Francisco Bay Area, and Shanghai and Beijing in China.
A quick google search to see if it was simply because of beta or there is a limited roll out, produced this:
>New York City, the San Francisco Bay Area, and Shanghai and Beijing in China.
Look for the arrival of cycling in Apple maps upon the release of iOS 14 in the upcoming weeks
[deleted]
Just got an iphone last month and apple maps has worked perfectly every time for me. I'm not exactly a maps power user but I haven't seen an issue.
I find maps much more pleasant looking than Google Maps, and that's a big deal for me.
Lack of maps.apple.com is still problem for use it for primary map service.
define "good", the POI database is still woefully lacking outside the US.
You’re 2 for 4. Time Machine and Mail (separate from iCloud) are not cloud services.
I thought the same thing about Time Machine. It works really well with a local laptop that I plug in from time to time for this to work. However, is it considered Time Machine when an iDevice backs itself up to the cloud?
Mail is definitely a local app, even if they did screw the pooch pretty badly with that gmail bug they had a few years ago.
Mail is definitely a local app, even if they did screw the pooch pretty badly with that gmail bug they had a few years ago.
“However, is it considered Time Machine when an iDevice backs itself up to the cloud?”
No. Strictly speaking, iCloud doesn’t even do backups; it syncs data. Difference is that, with backups, you can get a copy back if you accidentally throw away a file. You typically also have multiple ‘old’ copies.
iWork apps (1) kind-of have backups in the form of versions that get synced to the cloud (https://support.apple.com/en-us/HT205411), but I think that’s implemented independently from iCloud’s syncing (You can use it to have multiple versions of a document locally, too)
(1) Third-party apps can use this, too (https://developer.apple.com/design/human-interface-guideline...), but I wouldn’t know how commonly this is used
No. Strictly speaking, iCloud doesn’t even do backups; it syncs data. Difference is that, with backups, you can get a copy back if you accidentally throw away a file. You typically also have multiple ‘old’ copies.
iWork apps (1) kind-of have backups in the form of versions that get synced to the cloud (https://support.apple.com/en-us/HT205411), but I think that’s implemented independently from iCloud’s syncing (You can use it to have multiple versions of a document locally, too)
(1) Third-party apps can use this, too (https://developer.apple.com/design/human-interface-guideline...), but I wouldn’t know how commonly this is used
> No. Strictly speaking, iCloud doesn’t even do backups; it syncs data
There’s literally a switch called “iCloud Backup” in Settings though. This isn’t a per-app data sync, it’s the full device restore that you can use if your phone gets stolen, dropped in the ocean, etc.
Although you should consider not using it for privacy reasons and running backups to a local machine instead, since last we heard Apple has the encryption keys and will happily hand the entire contents your phone to law enforcement given a warrant.
There’s literally a switch called “iCloud Backup” in Settings though. This isn’t a per-app data sync, it’s the full device restore that you can use if your phone gets stolen, dropped in the ocean, etc.
Although you should consider not using it for privacy reasons and running backups to a local machine instead, since last we heard Apple has the encryption keys and will happily hand the entire contents your phone to law enforcement given a warrant.
They have components that run in the cloud, and unfortunately, when I wrote "cloud services" I probably should have written "services that leverage the cloud."
I don't think Time Machine does, everything I've seen is local to the machine or across your local network.
As far as I am aware, neither Time Machine nor Mail have any components that "run in the cloud".
I have a vague sense that you're right, that the services team just doesn't perform as well as other teams.
However, Apple operates many services at scales few companies ever reach, and doesn't seem to have outages much more often than those other few companies. So they're clearly doing a lot of things incredibly well, in the if-things-work-you-don't-notice sense.
They don't seem to emphasize their cloud services like Google does, and Apple charges for theirs at a much lower level than Google does, but I'm starting to think my vague intuition is wrong.
However, Apple operates many services at scales few companies ever reach, and doesn't seem to have outages much more often than those other few companies. So they're clearly doing a lot of things incredibly well, in the if-things-work-you-don't-notice sense.
They don't seem to emphasize their cloud services like Google does, and Apple charges for theirs at a much lower level than Google does, but I'm starting to think my vague intuition is wrong.
My impression from Twitter “new job” announcements in the last 6 months is that Apple has seriously ramped up its kubernetes staff.
iCloud Mail and Photos have been nigh flawless for me for over 8 years now.
Also of note, Apple Maps in the iOS 14 beta just started accepting ratings and photos at map locations, rather than dumping you out to Yelp reviews. I'm happy with this change.
https://9to5mac.com/2020/08/26/apple-maps-reviews-inhouse/
https://9to5mac.com/2020/08/26/apple-maps-reviews-inhouse/
Google's results are already pretty clean. Search results are heavily filtered (e.g. copyright, adult content) so I don't think Apple would filter more. If anything, filtering more than Google could deteriorate results.
But the SEO market would certainly change if webmasters suddenly have to appeal to more than just Google to be successful.
But the SEO market would certainly change if webmasters suddenly have to appeal to more than just Google to be successful.
They have a heading titled "About Search Rankings". Seems like a good indicator for a search engine to me.
Apple built their own search engine over 5 years ago, under the Siri / Spotlight umbrella. When people talk about Apple building their own search engine, they generally seem to expect a website dedicated primarily to web page results, but under the covers what powers Apple's Spotlight results is basically a search engine.
The big question would be what Apple would gain from a dedicated website for search results. Would people really switch to it from Google? Why would it be a better delivery mechanism for search results than Spotlight? Not sure the answers to these questions has changed much, from 5 years ago to today.
The big question would be what Apple would gain from a dedicated website for search results. Would people really switch to it from Google? Why would it be a better delivery mechanism for search results than Spotlight? Not sure the answers to these questions has changed much, from 5 years ago to today.
They have a lock on mobile devices for the western world. They can by default pry away search revenue from Google at a large scale. They have the cash reserves to see this done properly. They'd be crazy not to come onto Google turf. Their hardware lines are sagging revenue growth wise. Service revenue is their biggest growth area. I have no idea why they didn't do this before.
> They have a lock on mobile devices for the western world.
They don't though. There are only a couple countries where they're more popular than Android, and only barely there. There's no country where they have 60% market share.
https://deviceatlas.com/blog/android-v-ios-market-share
They don't though. There are only a couple countries where they're more popular than Android, and only barely there. There's no country where they have 60% market share.
https://deviceatlas.com/blog/android-v-ios-market-share
Or just like Iran's nuclear program, they can extract more concessions if they are permanently 1-2 years away from deploying a search engine. Google's spot as the default search in Safari already earns Apple billions per year.
It might look like that for people inside the walled garden but I suggest applying for an exit visa to have a look around the great wide world out there where you'll quickly find out that it is a whole lot more diverse than the picture painted inside the walls. Not only is it more diverse, those rumours spread about the constant onslaught of viruses on those poor creatures on the other side of that big, safe wall turn out to be untrue, most people seem to never have even seen one. Even stranger things will start to become clear, like those supposedly poor and oppressed outsiders needing far less money to get around the world than those inside the walls while at the same time having are larger choice of methods to navigate it. It wouldn't surprise me if you decided to stay, just don't forget to plan this carefully so your identity does not get stuck on the inside after which you won't be able to message your friends any more. It is a bit tricky but it is doable, many others have gone that way before.
Google pays Apple a reported $8 billion a year to be the primary search engine on iOS.
Apple’s hardware sells were also up slightly in every category year over year.
https://9to5mac.com/2020/07/30/apple-q3-2020-earnings/
Apple’s hardware sells were also up slightly in every category year over year.
https://9to5mac.com/2020/07/30/apple-q3-2020-earnings/
Presumably Google are making quite a bit more on that traffic. Apple might just be eyeing those additional profits.
Through advertising? How would that be any better than what we have now.
It might not, I'm just trying to think of their motives.
Alternatively: Would people really switch back to Google if Apple changed the default search engine on iOS?
Spotlight does result rankings.
Let's see if Apple is the first one to nail it. I love using DDG but their results are not close to the same quality as Google for me, especially with localized results and unspecific requests. And DDG is backed by Microsoft's Bing which invested billions into this. Though having run a few websites, Google's crawler easily crawls 10x the amount of all other crawlers combined if they don't detect page slowdowns. I never understood why Bing was so conservative when it comes to crawling.
I believe if anyone can build a search engine that can truly deliver the same quality as Google, the whole market could change. Google is dependent on search ad revenue like nothing else, if they lose that monopoly it could really change the company.
I believe if anyone can build a search engine that can truly deliver the same quality as Google, the whole market could change. Google is dependent on search ad revenue like nothing else, if they lose that monopoly it could really change the company.
my first reaction , was oh god, please dont! considering how worst siri is, i dont want that.. but after reading other comments like native ad tracking etc etc it could be just fine.
But google pays Apple in billions to keep its search as default, what could be the key motivation for Apple to build its own engine? its a highly saturated market and we have Duck Duck Go for privacy based , bing for alternative and heck, even wolfram alpha for computational! what Apple can improve?
But google pays Apple in billions to keep its search as default, what could be the key motivation for Apple to build its own engine? its a highly saturated market and we have Duck Duck Go for privacy based , bing for alternative and heck, even wolfram alpha for computational! what Apple can improve?
Based on usage of DDG, bing and wolfram alpha, I would think Apple can improve a great deal. I'm not saying it can necessarily achieve that, but I do think there is a lot of space to improve (indexing, ranking, UX)
That would be nice.
I remember when I rolled my eyes at Apple making a Maps product. I thought it was a fruitless, dumb idea.
I prefer Apple Maps to Google Maps now... don't underestimate Apple's ability to plant a flag and move it inch by inch each year.
Any specific reason why?
In my personal experience Google Maps has better navigation data, better place information and overall much better maps. The user contributed content makes the platform such a pleasure to use even if you are in remote parts of the world.
In my personal experience Google Maps has better navigation data, better place information and overall much better maps. The user contributed content makes the platform such a pleasure to use even if you are in remote parts of the world.
> better place information
Probably 75% of my Google Maps usage is to check business hours or grab a phone number; they've really got that down. Apple is catching up, though!
Probably 75% of my Google Maps usage is to check business hours or grab a phone number; they've really got that down. Apple is catching up, though!
Yeah, I really liked when Pixel integrated the phone number bit into dialer itself. I can start typing a business name and it just pulls in from Google maps.
The iPhone I have now is a work phone, so don't use maps a lot, should see if it has improved.
The iPhone I have now is a work phone, so don't use maps a lot, should see if it has improved.
I mostly use siri to call a business and it works every time for me.
Totally. In apple's own back yard, SF, business hours are too often totally out of whack, that you have to confirm in Google Maps each time.
And I am biased towards Apple Maps for privacy and usability reasons
And I am biased towards Apple Maps for privacy and usability reasons
I'm pretty sure Apple Maps is using Yelp's data for business hours -- those seemed to be mostly reliable until the pandemic. Now, not so much. I don't know if Google is doing better on that point.
I hate Google maps when it shuts itself off with a 'journey complete' status while I'm still looking for the corner or the building that I was supposed to arrive at.
On my iPhone I think Apple Maps has a vastly better navigation UI. Part of that is its first-party ability to turn the phone screen on when issuing instructions for upcoming maneuvers, which I much prefer to having the screen on constantly for a multi-hour trip (not to mention working well with Siri). Part of it is the overall aesthetic of the map features. I really like how it highlights traffic lights along your route to let you easily count how many lights are remaining until you need to turn.
That said, even here in the Bay Area I tend to do a sanity check of the route on Google Maps first if I’m taking an unfamiliar route. Just a few months ago a section of I-80 was closed just south of SF and Apple Maps had no clue! I believe it was a fairly last-minute schedule change for some planned construction due to vastly reduced traffic during the lockdowns.
That said, even here in the Bay Area I tend to do a sanity check of the route on Google Maps first if I’m taking an unfamiliar route. Just a few months ago a section of I-80 was closed just south of SF and Apple Maps had no clue! I believe it was a fairly last-minute schedule change for some planned construction due to vastly reduced traffic during the lockdowns.
For me having those stop signs and signals show up in navigation is big plus for Apple maps. I have a pixel phone but if I'm driving with my spouse I always use her apple maps
One great thing with the Apple Maps UI is that when using CarPlay it gives you a 3D view of the street. So you're really seeing buildings on the left and right, which has helped me not overshoot some destinations because I can 'see' the building in the map, and then irl.
> which I much prefer to having the screen on constantly for a multi-hour trip
I’ve found myself swinging in the other direction. I tend to glance at Waze for my speed instead of my own speedometer, but I’ve been driving a mixed bag of cars lately.
I’ve found myself swinging in the other direction. I tend to glance at Waze for my speed instead of my own speedometer, but I’ve been driving a mixed bag of cars lately.
I’ve used Google Maps since it first came out on the nexus one. I recently made the switch to Apple Maps. And it’s like stepping into a new world.
There’s the maps layout and where I want to go and my route. That’s it. Google Maps has ads. Switching between Apple Maps and Google Maps really shows how the Google product has friction and information overload. We are talking about a Maps app. Less is better unless I ask for it. Apple has nailed the color scheme, layout, and information density. If I need more information that Apple Maps doesn’t provide, I double check google maps. On a long trip, I will check routes from both apps. Apple Maps and google maps show the same route and same time and same traffic congestions for me. I rely more and more on Apple Maps now.
There’s the maps layout and where I want to go and my route. That’s it. Google Maps has ads. Switching between Apple Maps and Google Maps really shows how the Google product has friction and information overload. We are talking about a Maps app. Less is better unless I ask for it. Apple has nailed the color scheme, layout, and information density. If I need more information that Apple Maps doesn’t provide, I double check google maps. On a long trip, I will check routes from both apps. Apple Maps and google maps show the same route and same time and same traffic congestions for me. I rely more and more on Apple Maps now.
[deleted]
The main time I find myself using Apple Maps is when visiting China. Google Maps is blocked so the place data isn't as great, and the local apps are in Chinese, which I'm illiterate in.
I similarly find Apple Pay great for traveling, and easier to use than a credit card.
I similarly find Apple Pay great for traveling, and easier to use than a credit card.
Same. I have no idea how I would navigate in China without an iPhone.
All the other English language map applications are straight up unusable.
Have you tried it lately? It gets better with each iteration, and the street photos are way better where they exist. The satellite and aerial photos are also better in most places that I travel too.
Also, the Google Maps app has gotten much more cluttered and complex than in past versions. They are still the gold standard map, but they regress over time.
Also, the Google Maps app has gotten much more cluttered and complex than in past versions. They are still the gold standard map, but they regress over time.
Yup, Google Maps has better data.
But, on my no-longer-bleeding-edge iPhone, the turn-by-turn directions suck. It's constantly telling me to turn onto a street I just passed, or rerouting to find a way to get me back to the street that I'm already on. The worst part is that I can't trust it even when it's right. If it says "in 300 feet turn left," I have to desperately look at the street names because it's quite possible that I actually have to turn in 30 feet.
It was such a relief when I realized that it's not my GPS hardware, Apple Maps gives directions perfectly. And the map data is way better than it used to be. Not as good as Google Maps, but good enough.
But, on my no-longer-bleeding-edge iPhone, the turn-by-turn directions suck. It's constantly telling me to turn onto a street I just passed, or rerouting to find a way to get me back to the street that I'm already on. The worst part is that I can't trust it even when it's right. If it says "in 300 feet turn left," I have to desperately look at the street names because it's quite possible that I actually have to turn in 30 feet.
It was such a relief when I realized that it's not my GPS hardware, Apple Maps gives directions perfectly. And the map data is way better than it used to be. Not as good as Google Maps, but good enough.
Not OP, but the Apple Maps + iPhone + Apple Watch combo is now my favorite way to drive with turn-by-turn directions. That alone won me over.
The little nudge on the watch is a nice addition
I also find that Apple Maps respects local mapping conventions. For example in New Zealand Google treats all state highways as equivalent and uses the same weight and colour line for all of them, but Apple Maps follows the NZ convention and assigns appropriate weights to the minor roads.
For those not familiar "state highway" simply means the central government agency pays for upkeep, rather than indicating any particular road quality or status.
For those not familiar "state highway" simply means the central government agency pays for upkeep, rather than indicating any particular road quality or status.
Much much better voice directions; with Google, you have to check your screen often; with Apple, you just follow the voice
Google maps likes to wait until you 5m away from a turn before telling you you must cross 4 lanes to turn right
Not the parent commenter, but while I still use Google Maps over Apple Maps, I constantly have issues with the former.
I've put in business addresses or searched the business themselves, and then had it route me to a residential block behind the business on a completely different street, sometimes multiple blocks away from it. This is even after confirming the exact address is being shown and everything.
I only really use Google Maps because it's what I've been using, and I kind of like that its UI is a bit simpler with less animations and things than Apple Maps.
I've put in business addresses or searched the business themselves, and then had it route me to a residential block behind the business on a completely different street, sometimes multiple blocks away from it. This is even after confirming the exact address is being shown and everything.
I only really use Google Maps because it's what I've been using, and I kind of like that its UI is a bit simpler with less animations and things than Apple Maps.
Not OP, but I'm in the same boat. Agreed Google Maps' data is better. Their routing is sometimes better. But I still prefer Apple Maps in most cases. For me, it comes down to a few UX things:
* iPhone + Apple Watch combo is amazing to use w/ Apple Maps. Audible & haptic alerts on your wrist change how one interacts with the turn-by-turn directions, and it's an improvement.
* The fact that iPhone nav is available on the lock screen is a huge help. Doesn't require full unlock to get to the map.
* Apple Maps just seems to handle the accelerometer in the iPhone better. I found that Google Maps never knows what direction I'm facing/walking, and it jumps all over the place all the time. Apple Maps is just smoother and better to use.
The only thing that's missing in Apple Maps is bicycle directions, but apparently that's coming in iOS 14, which is great.
* iPhone + Apple Watch combo is amazing to use w/ Apple Maps. Audible & haptic alerts on your wrist change how one interacts with the turn-by-turn directions, and it's an improvement.
* The fact that iPhone nav is available on the lock screen is a huge help. Doesn't require full unlock to get to the map.
* Apple Maps just seems to handle the accelerometer in the iPhone better. I found that Google Maps never knows what direction I'm facing/walking, and it jumps all over the place all the time. Apple Maps is just smoother and better to use.
The only thing that's missing in Apple Maps is bicycle directions, but apparently that's coming in iOS 14, which is great.
Judging from the responses here, some of the reasons seem to be because Apple Maps is much more vertically integrated with iOS, and that doesn't seem like a good thing to me. Sure, you may get a better experience, but that's more likely because Apple is keeping Google Maps from getting as good than because it's a better solution, period. I have no love for Google but that's not a good model, and I feel like this kind of thing is making Spotify's case for Apple being a monopoly much stronger.
You don't need particularly tight integration with the operating system to make a good maps app; you'd want access to CarPlay, which Google Maps has, and you'd probably want a good sharing extension, which I don't think Google Maps has but could if Google cared about making better iOS apps than they seem to. Google is rather notorious for being very, very slow to adopt new iOS features in their apps, and it's definitely not because Apple is making it difficult for them to do so.
The history here is also at least worth a passing mention -- the iOS Maps app originally used Google mapping services. The reason Apple built their own back end is because Google refused to give them access to vector-based maps and turn-by-turn navigation unless Apple shared more user data with Google than they were willing to provide.
The history here is also at least worth a passing mention -- the iOS Maps app originally used Google mapping services. The reason Apple built their own back end is because Google refused to give them access to vector-based maps and turn-by-turn navigation unless Apple shared more user data with Google than they were willing to provide.
>but that's more likely because Apple is keeping Google Maps from getting as good than because it's a better solution,
I would say Waze is a solid counter-argument.
I would say Waze is a solid counter-argument.
Apple Maps has the unique and critical advantage of being operable while the phone is locked or sleeping.
I live in a bilingual country, GMaps decides location and street names at random, there is no way to pick which language to use.
I speak one of the languages, not the other. Guess which language Google decides is the correct one?
I speak one of the languages, not the other. Guess which language Google decides is the correct one?
In my experience, Apple Maps has been more up to date with recent changes to roads here in Norway. There was a new tunnel here recently, and it was updated in Apple Maps when it opened, while with Google Maps it took a while.
I'm guessing that it depends a lot on the area. Google Maps is still probably better in most places, but Apple Maps is catching up and may be better in some populated places. It's impressive considering the head-start Google has.
Apple should add an option of OpenStreetMaps overlay or something. OSM has been better than Google Maps in mapping paths in the forest and such here.
I'm guessing that it depends a lot on the area. Google Maps is still probably better in most places, but Apple Maps is catching up and may be better in some populated places. It's impressive considering the head-start Google has.
Apple should add an option of OpenStreetMaps overlay or something. OSM has been better than Google Maps in mapping paths in the forest and such here.
My main reasons for using Apple Maps are:
1 - It actually runs smoothly on my iPhone 11. Google Maps stutters and yanks. Yes, Google cannot build a performing maps app for the fastest phone in the world.
2 - It is a map. Not a yellow pages of coffee places, not an Instagram with amateur photos of locations.
1 - It actually runs smoothly on my iPhone 11. Google Maps stutters and yanks. Yes, Google cannot build a performing maps app for the fastest phone in the world.
2 - It is a map. Not a yellow pages of coffee places, not an Instagram with amateur photos of locations.
I don't have unlimited data, and I have found Apple maps is much lighter on data use than Google. I have also found that even if I don't have a solid connection Apple maps seems to still be capable of giving me directions quickly, whereas on the other hand with Google I may never get directions.
Its really small, but for me its traffic lights. It's SOOOO much easier to be navigating and say, turn in two lights, instead of "turn in the next three streets". Especially on busy roads when you need to be in a certain lane to make that left or right hand turn.
Google maps added this to their desktop app [1] and maybe their iOS app, but apple has had this for years.
Also Apple maps offers saved places in an offline, non-synced way. Google requires access to all my location data to save places (like web bookmarks) and that's just too privacy invasive for me.
[1]: https://arstechnica.com/gadgets/2020/08/google-maps-starts-w...
Google maps added this to their desktop app [1] and maybe their iOS app, but apple has had this for years.
Also Apple maps offers saved places in an offline, non-synced way. Google requires access to all my location data to save places (like web bookmarks) and that's just too privacy invasive for me.
[1]: https://arstechnica.com/gadgets/2020/08/google-maps-starts-w...
Even a few years ago I preferred the way a few-block area was represented on Apple Maps vs Google Maps. Apple Maps seemed to be much better at showing relevant landmarks and business names, where Google Maps would randomly select some businesses that happened to have an address there, but often no signage, etc.
Both applications would select the same number of businesses to show (obviously not showing ALL names due to density restrictions on the screen and clutter), but the Google ones were just so painfully arbitrary, and Apple's building skeletons seemed a lot more relevant/recognizable.
Both applications would select the same number of businesses to show (obviously not showing ALL names due to density restrictions on the screen and clutter), but the Google ones were just so painfully arbitrary, and Apple's building skeletons seemed a lot more relevant/recognizable.
My house has not been in Apple Map's index for approaching three years now. It's caused _all kinds_ of problems. Apple seems to drop the ball on the details that are needed to really get a product to be quality.
EDIT: I looked back in my emails and Twitter DMs and it's been 25 months at this point. So "approaching three years" is not accurate.
EDIT: I looked back in my emails and Twitter DMs and it's been 25 months at this point. So "approaching three years" is not accurate.
Can you add it?
Nope. You can report it's missing if you own an Apple device. I have done that (many times). I have also talked with Apple support on Twitter many times, and they even assigned a dedicated support person who called me several times. I facetimed with him and showed him how my complex, plus several of the nearby complexes, are not in Apple Maps but are in Google Maps. That was ... over a year ago. They still haven't fixed it. I finally gave up.
Whenever someone needs to come to my house I need to ask them to use Google Maps. The challenge from that comes from less techy people who don't know the difference. Many people just know their phone has a "maps app". I will admit, experiencing that more closely (how less technical people interact with their phones that is) was interesting fwiw.
Whenever someone needs to come to my house I need to ask them to use Google Maps. The challenge from that comes from less techy people who don't know the difference. Many people just know their phone has a "maps app". I will admit, experiencing that more closely (how less technical people interact with their phones that is) was interesting fwiw.
Have you tried reporting it? My parents recently moved into a new house built one year prior and I reported it a few days after they moved in. It took about 9 months, but they finally added the road in and populated ALL the addresses on the street, not just the one I reported.
Google maps still does not have the roads or street numbers listed. Just wanted to throw that out there.
Edit: Also want to add that it has been two years since my parents moved in. Went in to Waze Map Editor and added the road myself, so their turnaround time was instant. Apple Maps took 9 months, like I said, but Google Maps still has no info on the street despite having all the houses in clear satellite view.
Edit: Also want to add that it has been two years since my parents moved in. Went in to Waze Map Editor and added the road myself, so their turnaround time was instant. Apple Maps took 9 months, like I said, but Google Maps still has no info on the street despite having all the houses in clear satellite view.
I replied with my interactions with Apple here: https://news.ycombinator.com/item?id=24338389
It did take Google a few months to add my complex. I'm not sure what an appropriate turn around time would be on something like this. But I do think it's less than "years".
It did take Google a few months to add my complex. I'm not sure what an appropriate turn around time would be on something like this. But I do think it's less than "years".
Just read your reply. That's very frustrating to read -- hopefully the Apple employees come to their senses and add the building soon. It's also weird that they called you through FaceTime and still did nothing. Either way, sorry to hear about all the frustration you've been through.
What's the point of reporting anything to the Apple Maps team? I've reported stuff for a few years, and it hasn't changed nor have they had the basic courtesy to acknowledge the report.
It's not like it's some street address thing, they are missing a whole town.
It's not like it's some street address thing, they are missing a whole town.
I've gotten changes made by reporting stuff to the Apple Maps team. I remember years ago, my company's business location didn't show up correctly. I reported it, and within a week it showed up incorrectly in an entirely different way!
I’ve reported issues with POIs in the UK via the in-app features (“Edit Location” and “Report an issue”) and they were fixed within a few days.
In general, Google still has better POI data than Apple Maps in the UK, but the gap has closed a lot in recent years.
In general, Google still has better POI data than Apple Maps in the UK, but the gap has closed a lot in recent years.
try adding it in openstreetmap if it's not there already.
Google Maps is making it easier on them by seemingly becoming worse and worse, in my opinion. The newest act of telling you it's changing your route if you don't click something is so idiotic it borders on malice. So going 70mph down the expressway, I need to take my eyes off the road to read the message and click a button asking it not to take action.
Further, it repeatedly picks a hard route. There's one park I go to that's easier to drive to using one few mile stretch of highway. Google insists it's faster to take these back, poorly lit, windy 2 lane roads. Maybe it is. But it turns a safe easy 3 turn trip into a nerve-wracking 10 turn trip. And it will, as noted above, keep trying to reroute me if I just ignore it and go the easy way.
Further, it repeatedly picks a hard route. There's one park I go to that's easier to drive to using one few mile stretch of highway. Google insists it's faster to take these back, poorly lit, windy 2 lane roads. Maybe it is. But it turns a safe easy 3 turn trip into a nerve-wracking 10 turn trip. And it will, as noted above, keep trying to reroute me if I just ignore it and go the easy way.
Noticed that recently too - it insisted on taking stupid backroads and what not, instead of straight highway and no traffic. It seems to be getting dumber, or something.
Another thing google maps always does to me is telling me what the next turn is when its too late and I can't make it anymore. Using google maps with audio only is impossible.
I tried AppleMaps for the first time like 4-5 years ago. Feature-wise it wasn't where it is now but what annoyed me the most is that the GPS on map lagged FAR (1-2 seconds) behind my real position. More than once I missed a turn because of that, especially when you are driving in an urban area with a lot of smaller streets.
Fast-forward to 2020 - in the meantime I used Google Maps with no issues at all. A friend of mine suggested that I should try the latest version and maybe I would change my mind. So I used it for a day - same urban area - and it's still lagging behind - mind you: at this time I had a new iPhone and a new carrier. For me this makes the App unusable. I don't want to plan ahead 500 - 1000m (Germany) while using navigation.
I'm really curious, does someone have the same problem?
Fast-forward to 2020 - in the meantime I used Google Maps with no issues at all. A friend of mine suggested that I should try the latest version and maybe I would change my mind. So I used it for a day - same urban area - and it's still lagging behind - mind you: at this time I had a new iPhone and a new carrier. For me this makes the App unusable. I don't want to plan ahead 500 - 1000m (Germany) while using navigation.
I'm really curious, does someone have the same problem?
Absolutely this. Not sure if it's 1-2 seconds but the delay is enough to make it unusable. I've missed turns and exits because of it. Google Maps seems to have some predictive ability that allows it to appear to be "real-time".
I used to say that but lately apple maps seems to be wrong a lot - back to how it was when it first started, probably a temporary regression (I hope).
Yesterday for example it told me to do a U turn in a (sort of) one way street in the middle of the city, and not the first time its been dumb in the last few weeks.
Yesterday for example it told me to do a U turn in a (sort of) one way street in the middle of the city, and not the first time its been dumb in the last few weeks.
I think Apple Maps has finally reached "competent maps app" status, but still has miles to go to catch up to Google Maps.
Apple Maps is way better than Google for the wandering pedestrian. Google hides so many restaurants and bars that don’t pay them for prominent display. Apple Maps shows every place near my location. Obviously hasn’t mattered recently, but I always go to Apple naps for nearby browsing.
Google Maps's heuristics for displaying POIs are nothing short of bizarre. I've seen it decide that weed dispensaries are the most important thing to display in an entire city. It's also really aggressive about displaying home businesses on the map, even if they aren't open to the public.
Does anyone know for sure if there is a way to pay to get your business to show up? It seems so unnatural it must be an advert. While zoomed out on my city it just shows icons for hospitals and pizza hut.
> Google hides so many restaurants and bars that don’t pay them for prominent display
Is there public info on the pricing for sponsored POIs, e.g. is it auction-based, for a period of time? Does the listing get dropped from the map after the business stops paying? How does Google get around the FTC requirement to label sponsored content?
Is there public info on the pricing for sponsored POIs, e.g. is it auction-based, for a period of time? Does the listing get dropped from the map after the business stops paying? How does Google get around the FTC requirement to label sponsored content?
I too prefer Apple Maps, but just for driving. Google Maps has way better coverage of public transportation in cities.
For instance Apple Maps doesn't have U-Bahn support in Munich, which makes it unusable for most people.
For instance Apple Maps doesn't have U-Bahn support in Munich, which makes it unusable for most people.
Only if the Map's Data works for you. From a Worldwide perspective, Google Map's Data is still thousand times better than Apple Map.
Correct
Meanwhile Bing maps manages to be the most frustrating experience of any current online maps.
Meanwhile Bing maps manages to be the most frustrating experience of any current online maps.
Is there any reason this is getting traction on HN? Applebot has been confirmed for at least 5 years[1][2].
[1] https://appleinsider.com/articles/15/05/06/apple-details-new...
[2] https://appleinsider.com/articles/15/05/06/apple-challenges-...
[1] https://appleinsider.com/articles/15/05/06/apple-details-new...
[2] https://appleinsider.com/articles/15/05/06/apple-challenges-...
I think people are assuming that this confirms a rumor that Apple is launching a search engine. Note that the linked page mentions “Apple Search”
The page was updated in July with much more detail.
A random older archive from last year:
https://web.archive.org/web/20190429220555/https://support.a...
What seems to be new:
- "About search rankings" section listing its purpose
- Expanded bot identification and verification information
https://web.archive.org/web/20190429220555/https://support.a...
What seems to be new:
- "About search rankings" section listing its purpose
- Expanded bot identification and verification information
The pívot podcast made a prediction that Apple will launch a search engine. Might be overlap in audience of hacker news and podcast
For me Applebot activity has increased by at least 500% YOY.
Google needs the competition. Search it's getting more and more laden with ads, and I have found the kind of specific technical searches it used to excel at are increasingly on page 2 after articles and videos and product listings.
hell, it ain't just ads, it's outright malware at this point. i had to look up some shit about my printer, and hoo boy, searching for that churns up even more of a cesspool than i get when trying to search specialized windows topics. those don't turn up anything useful most of the time, but it's at least benign--a wall of posts on official MS forums where the top answer is some "top community contributor" yeeting out a "my friend, have you tried rebooting and reinstalling drivers?" copy-pasta response to any conceivable question
the printer stuff is something else. i put some fault on canon here for pushing me to unofficial sites because their official shit requires you to download a fucking exe to deliver a PDF manual (cmon, wtf, how can your docs delivery pipeline be THAT bad), but the unofficial stuff is some ninth circle of hell
the first was some bog-standard infini-redirect ad fraud crap, but the second was uh... well, i was impressed to the degree that the indian tech chop shop "YOU ARE INFECT DIAL 1-800-LEGIT-MICROSOFT TO CLEAN COMPUTER" site was able to shut down a fully up-to-date copy of firefox, somehow eating all my other tabs, grabbing hold of my mouse pointer in strange ways when the window was in focus AND preventing the windows "close window" button from killing it.
task manager was still able to execute it but damn, that was some ARTISTRY put into making the fraud site as clingy as possible--puts 90s-era malware sites to shame, and this shit kicks around on right on the first page of google search results for a printer manual query, not some shady-ass porn site you'd drunkenly click through to via 3 levels of ever-sketchier ads
people ranted and raved about goods of questionable quality gaming amazon reviews, but eh--honestly, most of those i've bought were fine, if not artisanal high-end shit--they just engaged in a lot of dubious "get free shit for a good review" growth hacking tactics that are A-OKAY THATS JUST HOW YA DO BUSINESS when it's some SF startup pulling the same crap in a different form. it's weird how seemingly nobody gives a fuck that a lot of google searches will boost things that are far worse--offering purestrain fraud on a platter is fine, selling mediocre flash drives and kitchen gadgets with inflated reviews is THE HIGHEST OF CRIMES
the printer stuff is something else. i put some fault on canon here for pushing me to unofficial sites because their official shit requires you to download a fucking exe to deliver a PDF manual (cmon, wtf, how can your docs delivery pipeline be THAT bad), but the unofficial stuff is some ninth circle of hell
the first was some bog-standard infini-redirect ad fraud crap, but the second was uh... well, i was impressed to the degree that the indian tech chop shop "YOU ARE INFECT DIAL 1-800-LEGIT-MICROSOFT TO CLEAN COMPUTER" site was able to shut down a fully up-to-date copy of firefox, somehow eating all my other tabs, grabbing hold of my mouse pointer in strange ways when the window was in focus AND preventing the windows "close window" button from killing it.
task manager was still able to execute it but damn, that was some ARTISTRY put into making the fraud site as clingy as possible--puts 90s-era malware sites to shame, and this shit kicks around on right on the first page of google search results for a printer manual query, not some shady-ass porn site you'd drunkenly click through to via 3 levels of ever-sketchier ads
people ranted and raved about goods of questionable quality gaming amazon reviews, but eh--honestly, most of those i've bought were fine, if not artisanal high-end shit--they just engaged in a lot of dubious "get free shit for a good review" growth hacking tactics that are A-OKAY THATS JUST HOW YA DO BUSINESS when it's some SF startup pulling the same crap in a different form. it's weird how seemingly nobody gives a fuck that a lot of google searches will boost things that are far worse--offering purestrain fraud on a platter is fine, selling mediocre flash drives and kitchen gadgets with inflated reviews is THE HIGHEST OF CRIMES
Diversity would definitely be positive, the web is too important to have it indexed and sorted from one perspective only. Would prefer half a dozen engines and perhaps all enabled to provide results to metas who could further offer more diversity.
You can use/host searx (https://searx.space/) or yacy (https://yacy.net/) to go one step further.
Yacy is about as useful as searching the internet with grep. You can search "Facebook" and the first 20 pages are foreign spam blogs that have the word facebook on them.
way back in 00s, Google felt exactly like grepping the whole internet, including niche blogs, minus the spam (for the most part).
Somewhere in between, Google stopped caring, and we have what we have today.
Somewhere in between, Google stopped caring, and we have what we have today.
They're definitely not great but they're a good place to start.
And Searx is only useful in English or so it seems.
I made the switch to a Searx instance from Google about 3 weeks ago and it's been great for me (now I just get Google's results in Searx with added privacy).
Dark mode to save my eyes, setting my own personal preferences for which search engines to use for results, Cached results link and proxied links, a files search engine...
The only difference is that I wasn't already heavily invested in other Google products. e.g. Gmail, Lots of saved Maps POI, etc.
For someone who is heavily invested, I'm not sure if the switch to Searx is worth the lack of convenience.
Dark mode to save my eyes, setting my own personal preferences for which search engines to use for results, Cached results link and proxied links, a files search engine...
The only difference is that I wasn't already heavily invested in other Google products. e.g. Gmail, Lots of saved Maps POI, etc.
For someone who is heavily invested, I'm not sure if the switch to Searx is worth the lack of convenience.
There's these alternatives also, all privacy focused but mostly reliant on other engines for data: https://www.eu-startups.com/2020/09/5-european-google-compet...
Part of that, I think, is just that the SEO industry has matured so much any random person can quickly climb the rankings by short-circuiting Google with basic SEO, taking over space which should've gone to more quality content instead.
Yes. Apple hardware is a business model but the search engine business is a hefty one.
Why would it be any better than Bing?
I’m hearing a lot of theories in this thread about Apple building a search product.
Yes, that’s possible, but Applebot is used in a lot of ways today that are wholly unrelated to search. (If search consists of crawling, indexing, ranking, retrieval, frontend, etc., Applebot only does the crawling part.)
Applebot is used for generating attachment previews in iMessage (eg. Send someone a URL — the preview is from Applebot crawling it). From the docs, it sounds like it’s also used for similar previews in Siri.
Yes, that’s possible, but Applebot is used in a lot of ways today that are wholly unrelated to search. (If search consists of crawling, indexing, ranking, retrieval, frontend, etc., Applebot only does the crawling part.)
Applebot is used for generating attachment previews in iMessage (eg. Send someone a URL — the preview is from Applebot crawling it). From the docs, it sounds like it’s also used for similar previews in Siri.
Applebot was built for crawling web pages, to be used for search results in Spotlight and Siri. That user agent might also be used for attachment previews, but the original intent of Applebot was for search indexing.
Pretty neat, likely the Applebot preview functionality that let us build Rapbits on iMessage: https://rapbits.com/video/ad.mp4
> Applebot is used for generating attachment previews in iMessage (eg. Send someone a URL — the preview is from Applebot crawling it)
Is this true? I would have assumed that would be done locally on device. The preview image, for example, is what's designated by the Open Graph[0] og:image tag.
[0]: https://ogp.me
Is this true? I would have assumed that would be done locally on device. The preview image, for example, is what's designated by the Open Graph[0] og:image tag.
[0]: https://ogp.me
Crawling and caching is pretty common for any service that offers unfurled previews to avoid sending extra traffic to sites that may not be able to handle it, and in many cases I'd imagine retrieving the pre-extracted cached version to be faster.
Apple is 100% building a search engine.
But do you remember when Apple launched their maps? A decade later and it's still not nearly as good as Google maps. But it's "good enough". I imagine that's what they're going for with this new search service
But do you remember when Apple launched their maps? A decade later and it's still not nearly as good as Google maps. But it's "good enough". I imagine that's what they're going for with this new search service
I'm totally fine with good enough if it keeps Google in check
The quality of the product is irrelvant. It will have usage as long as it is the default on Apple devices.
The data that Apple previously paid TomTom and others to use was certainly subpar compared to Google's data.
However, Apple started collecting their own data in 2015, and rolling out their own maps in 2018.
>Apple is filling its map with so many [details] that Google now looks empty in comparison and all of these details create the impression that Apple hasn’t just closed the gap with Google—but has, in many ways, exceeded it.
https://www.justinobeirne.com/new-apple-maps
Apple's map data now covers the US and its territories and data for the UK and Ireland are currently in testing.
https://www.justinobeirne.com/new-apple-maps-expansion-9
However, Apple started collecting their own data in 2015, and rolling out their own maps in 2018.
>Apple is filling its map with so many [details] that Google now looks empty in comparison and all of these details create the impression that Apple hasn’t just closed the gap with Google—but has, in many ways, exceeded it.
https://www.justinobeirne.com/new-apple-maps
Apple's map data now covers the US and its territories and data for the UK and Ireland are currently in testing.
https://www.justinobeirne.com/new-apple-maps-expansion-9
Think the summary from what I've read the past few days
- It's established applebot is already used for Siri, but there is no general web search
- Much commentary diverts to how poor Apple maps (is?) was.
- Google pay apple a handsome amount of money to be the default search engine on Apple devices
- If Apple were no longer to accept that payment, about 80% of people would still use Google anyway, given a choice [0]
- Googles high earnings per search means they can offer more than any other search engine, like Bing, DDG et al
- People would generally like more competition in the search space and Apple has the means to do it
- Anecdotally, there isn't a drop in replacement for a search default other than Google that would meet the quality requirements for end users. In this case, the only other alternative would be an Apple search engine.
- As per recent articles, there does seem to be more activity from Apple wrt applebot crawling and hiring
[0] https://spreadprivacy.com/search-preference-menu-research/
- It's established applebot is already used for Siri, but there is no general web search
- Much commentary diverts to how poor Apple maps (is?) was.
- Google pay apple a handsome amount of money to be the default search engine on Apple devices
- If Apple were no longer to accept that payment, about 80% of people would still use Google anyway, given a choice [0]
- Googles high earnings per search means they can offer more than any other search engine, like Bing, DDG et al
- People would generally like more competition in the search space and Apple has the means to do it
- Anecdotally, there isn't a drop in replacement for a search default other than Google that would meet the quality requirements for end users. In this case, the only other alternative would be an Apple search engine.
- As per recent articles, there does seem to be more activity from Apple wrt applebot crawling and hiring
[0] https://spreadprivacy.com/search-preference-menu-research/
If they get a choice with no silent default yes, but if the default is set for them and they have to go into preferences to change it the results are almost completely reversed.
This is why Apple Maps is so heavily used on iOS. Most people don't know or don't care. In the first year after the launch of Apple Maps it hit over 65% of maps usage on iPhones in the UK, even though back then the UK maps were awful.
This is why Apple Maps is so heavily used on iOS. Most people don't know or don't care. In the first year after the launch of Apple Maps it hit over 65% of maps usage on iPhones in the UK, even though back then the UK maps were awful.
This reminded me of the fact that all of 17.0.0.0/8 was assigned to Apple.
I find it even more surprising that Ford owns 19.0.0.0/8.
That's crazy. So Apple owns 1/256 of all IP Addresses?
Yes, and a few other companies/agencies enjoy the same privilege: https://en.wikipedia.org/wiki/List_of_assigned_/8_IPv4_addre...
AT&T and Apple make sense but why is Ford on that list. And how did Comcast get theirs in 2005? I guess if you just ask very nicely, anything can happen.
Remember: ARPANet was originally just a DOD research project to connect some universities and military sites. Back when it was opened to the public you could get a .com domain name for free just by asking. Same deal with IP addresses: in the very early days you just had to ask and you could have your own /8 (assuming you were a university, medium-to-large company, or government entity).
There has never been any point in clawing back the large class A assignments. If that were done it wouldn't even give us an extra year of IPv4 address assignments before we exhaust the pool again... not that there is any legal mechanism to take them back anyway.
The solution is IPv6. Despite the slow rollout it is finally happening now that even the regional RIRs have started to run out of addresses.
There has never been any point in clawing back the large class A assignments. If that were done it wouldn't even give us an extra year of IPv4 address assignments before we exhaust the pool again... not that there is any legal mechanism to take them back anyway.
The solution is IPv6. Despite the slow rollout it is finally happening now that even the regional RIRs have started to run out of addresses.
> Aggregated user engagement with search results
My bad experience is generally rooted in the above criteria. It's particularly problematic on news sites. For example, if I click a news article with certain political association, I keep seeing only news from that political side. This creates an echo chamber which makes uninformed believe that 'whole world' is thinking that way. It definitely make sense to increase more engagement, but might not be totally appropriate in all instances. Curious to see how it works in live (if the apple search does become a reality)
My bad experience is generally rooted in the above criteria. It's particularly problematic on news sites. For example, if I click a news article with certain political association, I keep seeing only news from that political side. This creates an echo chamber which makes uninformed believe that 'whole world' is thinking that way. It definitely make sense to increase more engagement, but might not be totally appropriate in all instances. Curious to see how it works in live (if the apple search does become a reality)
The only serious study I know of on the information bubble says rather the opposite is true: https://www.pnas.org/content/117/6/2761
Thanks for the reference. It's good to know that there is research backing SNS (social network sites) exposes us to more diverse news contrary to popular belief. Size of the data set seems to be big enough to rely on the results.
I'm wondering if the same holds true when that experiment is conducted in isolation. For example, if we take any particular news site, if a user is repeatedly visiting the same news site, then my hypothesis is that the kind of news one sees will converge onto a specific political view point. It might vary based on each news site - google news vs apple news based on how they are optimizing for users.
In reality, the isolated experiment might be moot as users do end up on news site from SNS which ultimately leads to recommendation algos picking up diverse signals to show diverse results. Given that, whether the algo itself is diverse in isolation or not wouldn't matter as the environment that feeds it is diverse. Does it make sense?
I'm wondering if the same holds true when that experiment is conducted in isolation. For example, if we take any particular news site, if a user is repeatedly visiting the same news site, then my hypothesis is that the kind of news one sees will converge onto a specific political view point. It might vary based on each news site - google news vs apple news based on how they are optimizing for users.
In reality, the isolated experiment might be moot as users do end up on news site from SNS which ultimately leads to recommendation algos picking up diverse signals to show diverse results. Given that, whether the algo itself is diverse in isolation or not wouldn't matter as the environment that feeds it is diverse. Does it make sense?
Good questions. I suspect diverse news may not actually result in diverse perspective anyway. I wouldn't be surprised significant fraction of people simply don't change their opinion, even while consuming news from diverse source. So it may be doubly moot in the end.
On a related note, since there are video podcasts, I've always wondered why they haven't been positioned as an alternative/competitor to YouTube by now.
A lot of YouTube channels and types of content could be replaced by video podcasts, and be more convenient for viewers.
As for monetization, Apple wouldn't even have to build an ads platform; content creators could embed ads themselves into the content, as they currently do for audio podcasts.
A lot of YouTube channels and types of content could be replaced by video podcasts, and be more convenient for viewers.
As for monetization, Apple wouldn't even have to build an ads platform; content creators could embed ads themselves into the content, as they currently do for audio podcasts.
This isn't anything new. Apple has had this bot as far back as 2015. https://searchengineland.com/apple-confirms-their-web-crawle...
Everyone appears to be nervous because of the supposed search engine. I believe this was part of their initiative to improve Siri results; when the service was even worse.
Everyone appears to be nervous because of the supposed search engine. I believe this was part of their initiative to improve Siri results; when the service was even worse.
If robots instructions don't mention Applebot but do mention Googlebot, the Apple robot will follow Googlebot instructions.
Is that pretty much normal?
Is that pretty much normal?
I don't see this crawling my company's site much, but it's there (67 crawls over the past month) https://i.judge.sh/well-off/Dash/chrome_yquafZlF2U.png
I wish Apple would acquire Mozilla, use it to improve Safari and then create a privacy first search engine.
The most FOSS hostile company in the world acquiring the last properly open source browser does not sound good.
Being open source will be irrelevant if nobody uses it anymore.
https://gs.statcounter.com/browser-market-share/desktop/worl...
https://gs.statcounter.com/browser-market-share/desktop/worl...
Then they would completely stop Firefox development in favor of Safari.
I'd be hoping they do something like Edge/Chromium and replace Webkit with Gecko along with everything else that's better in Firefox.
Apple never accepts GPL, not even in App Store: https://news.ycombinator.com/item?id=3488833
I had this same wish yesterday. I know it isn't the ideal scenario, but there are definitely some net positives that could come out of it in the form of keeping Mozilla's lights on for a long time to come and massive improvements to areas of Safari that (to me) are worse than competitors such as dev tools and extensions.
[deleted]
Given that the App Store search has one of the worst search algorithms in a major search engine that I know of, I am not holding my breath for an Apple search product...on their hand, when Apple does get something right, they hit it out of the park.
I keep hearing that but when I dug a bit into it to do some ASO, the impression I got is that it's simply not easy to game in name of Search Engine Optimisation because the consensus is that the search engine only looks into the App Title, subtitle, description and keywords and the ranking is affected only by the App's performance. SEO people seem annoyed by Apple that they cannot get their wallpapers app for the "stocks" keyword, so I have a very little sympathy. When they complain that App Store Search is dumb, they essentially mean that If I want to rank up my wallpaper app "stocks" and figure a way to people use it every day. They have all kinds of tricks for Google Play but the ASO tutorials are rather dull, recommending stuff like picking an appropriate title and doing A/B tests on the keywords.
As a user, I tend to be quite happy with the search results in the App Store, I don't remember any time where I could not find the thing I am looking for.
As a user, I tend to be quite happy with the search results in the App Store, I don't remember any time where I could not find the thing I am looking for.
I suspect this is all about Siri. They went from being top of the game in personal assistants to being a second stringer. Apple can't compete with Google Assistant without the data set and analytics. Personal assistants and web search are two sides of the same coin.
There are reports that right now Applebot is being used to scrape the content used by Siri. If youre already scraping the web, indexing it and building search and analytics services on top of it for Siri, user facing search is just a short step to take.
There are reports that right now Applebot is being used to scrape the content used by Siri. If youre already scraping the web, indexing it and building search and analytics services on top of it for Siri, user facing search is just a short step to take.
Here is a discussion from last week:
https://news.ycombinator.com/item?id=24290613
This is one of the most exciting developments in the history of the internet.
Apple has always sided with privacy and while DDG toots it, they’re too small to build an independent service.
I am thrilled, excited and perhaps surprised why it took so long for the most used website for large majority of the people in the world to see some competition.
Please don’t fuck this up, Apple.
Apple has always sided with privacy and while DDG toots it, they’re too small to build an independent service.
I am thrilled, excited and perhaps surprised why it took so long for the most used website for large majority of the people in the world to see some competition.
Please don’t fuck this up, Apple.
Since this was posted on "support.apple.com", I assumed this to be about a robot doing customer support.
I think Applebot is nothing new (apart from the name). It's been years now since I found in my logs traces of crawlers from Apple IPs with an iOS 8 Safari-like user-agent. Maybe, well, since iOS 8 at least.
Edit: see here @jd20 thread for insights and confirming this.
Edit: see here @jd20 thread for insights and confirming this.
Any idea what Apple is using this for?
The bottom of the page says "Apple Search may take the following into account when ranking web search results:", is this another confirmation that Apple is developing a search engine?
The bottom of the page says "Apple Search may take the following into account when ranking web search results:", is this another confirmation that Apple is developing a search engine?
"Products like Siri and Spotlight Suggestions use Applebot."
Apple already has a search engine, called Parsec, that's used when you search in spotlight.
[deleted]
[deleted]
I'm keeping an eye on siri.com - right now it redirects to a sort-of-holding page on apple.com... but I can totally envision that URL becoming the home of their new search engine.
I am mega curious how does Apple circumnavigate problem of getting flagged for crawling website. Doesn't CDN cloudfare eventually flag them?
If proxies, what is their source for proxies.
If proxies, what is their source for proxies.
Likely whitelisted as a 'good bot' in Cloudflare. If a bot isn't well known, doesn't respect robots.txt or crawls too fast then much more likely to get flagged, but applebot are none of those.
I was scared while writing crawlers to various sites since I don't know how far it's legal, but Apple says look, this is my crawler.
[deleted]
>Customizing robot.txt rules
Shouldn't it be "robots.txt"? It even says so below.
Shouldn't it be "robots.txt"? It even says so below.
So when is this going to replace Google as default search engine in Safari?
[deleted]
They're going to fail, yet pretend that they succeeded. Examples: Siri, Maps, Home.
Siri and maps work really well for me.
Have you compared them to Google counterparts? There's really no comparison.
I have used both and they both 100% cover my use cases. Gmaps could be 1000x better but it doesn't matter because there is nothing more I need that isn't in Apple maps.
Apple can't built software for shit. They just want ad dollars. What does Apple care about providing search? They don't.
I'm going to slow all traffic with "Applebot" in the user agent by 30%.
Might do that same for all iOS traffic.
Might do that same for all iOS traffic.
What would be your desired outcome from this?
My comment was made in jest.
But.
If Apple wants to embrace the openness of the web that they benefit from, I feel they should not cripple it on their mobile devices.
But.
If Apple wants to embrace the openness of the web that they benefit from, I feel they should not cripple it on their mobile devices.
Of course to then complain that Apple and their ‘monopoly’ doesn’t get them the exposure they imagine their users are demanding
Run slow loris directly on your server.
- Applebot was originally written in Go (and uncovered a user agent bug on redirects, revealing it's Go origins to the world, which Russ Cox fixed the next day).
- Up until the release of iOS 9, Applebot ran entirely on four Mac Pro's in an office. Those four Mac Pro's could crawl close to 1B web pages a day.
- In it's first week of existence, it nearly took Apple's internal DNS servers offline. It was then modified to do it's own DNS resolution and caching, fond memories...
Source: I worked on the original version.