Ask HN: Who here would be interested in a free daily updated GeoIP database?
127 comments
Have you considered simply contributing to the service at http://www.hostip.info/ ? They seem to be doing what you describe, and have been around long enough to have reasonably good information (in my experience)— I'm sure they would make use of any additional data, or hacking on their backend, which you might provide.
This would be very welcome. Also having SQL load files ready to go would be something that could be very easy to become dependant on and start paying for more frequent updates.
On a related note, what's the best way to host big databases for free? As a release on Github?
Very interested, but worried that if it's free, it won't last or be maintained, etc.
But in specific answer to this question, if it's really public and free, why not see about adding it to Amazon's public datasets [1]? S3 is fast, durable, available, and I've enjoyed these other data sets.
[1] http://aws.amazon.com/datasets/
But in specific answer to this question, if it's really public and free, why not see about adding it to Amazon's public datasets [1]? S3 is fast, durable, available, and I've enjoyed these other data sets.
[1] http://aws.amazon.com/datasets/
I would be interested, personally and I believe many other developers would also be interested :-)
How/where do you get more accurate data than MaxMind does? And under what license do you have access to this data, and under what license would you be able to offer it to others? If all the legal stuff looks right, and the data is indeed more accurate, then of course there's going to be interest in it. I think it would take a while for word to spread and links to propagate, but eventually you'd see quite a bit of sustained traffic on that project. IP-to-location data is a very common need.
My only experience with MaxMind was the trouble we had when we got a new IPv4 assignment from ARIN. I filled out the form on their website the first week, our address was in the WhoIS records for our IPs, and our BGP announcements made our location obvious, yet they had our IP's (obviously from ARIN) showing up as originating from South Africa for months.
Seems like anyone who can look at BGP tables and whois information can get more current and accurate data than MaxMind.
Seems like anyone who can look at BGP tables and whois information can get more current and accurate data than MaxMind.
Probably a lot of sources, depends how well connected you are.
The biggest one is just routing information and rDNS hosts which give away a lot of the information required anyway. If you were really thrifty you would make agreements with services (or use services you run yourself) that have users supply their location information and use that to construct a more granular database. I suspect this already happens to a degree, some of the fake information I give out has ended up in at least one IP address database.
I strongly considered doing this at one point but lacked the connections to pull it off properly (not enough sources of data I could get without approaching random people, basically). A secondary concept would just be to crowd source it and have the information freely available, but I was never sure how to deal with people poisoning the database or just honestly supplying conflicting information.
The biggest one is just routing information and rDNS hosts which give away a lot of the information required anyway. If you were really thrifty you would make agreements with services (or use services you run yourself) that have users supply their location information and use that to construct a more granular database. I suspect this already happens to a degree, some of the fake information I give out has ended up in at least one IP address database.
I strongly considered doing this at one point but lacked the connections to pull it off properly (not enough sources of data I could get without approaching random people, basically). A secondary concept would just be to crowd source it and have the information freely available, but I was never sure how to deal with people poisoning the database or just honestly supplying conflicting information.
I have long thought that we could do better than maxmind. I have a website with tens of millions of users all over the world that could help you build this database is your want to lead the effort. I imagine showing users what country we think they are in and they correct it if it is wrong.
I imagine browser language settings would be a useful indicator too. Not perfect obviously, but highly useful given enough data.
That, too. We could use a number of different indicators to help figure it out.
So it would be a 'simple' table dump ? Updated daily by adding only the changes ?
I would be interested. Although a service like ip-api.com is not expensive ($8 per month or so if commercial use).
The structure of the data seems very simple: ip address range plus a location, I guess. So maybe just dump as a simple csv text file ? Always easy to process.
You have to make sure the legal side is covered: how do you get your changes ?
I would be interested. Although a service like ip-api.com is not expensive ($8 per month or so if commercial use).
The structure of the data seems very simple: ip address range plus a location, I guess. So maybe just dump as a simple csv text file ? Always easy to process.
You have to make sure the legal side is covered: how do you get your changes ?
I'm not sure I want a free alternative. I want an inexpensive option that I'm pretty sure won't get acquired and shutdown, but also has reasonable enough revenue to remain in business, occasionally blessing me with new features.
A free service like locationkit (seen here a few days ago IIRC) looks OK, and I'm reasonably sure it's supported by revenues for other services from the parent company. However, even that feels like it might get shut down if it becomes too big a burden.
Something with a good understandable license that I can pay a reasonable amount ($5-$20/year, for most low-mid volume use cases) would be great, as long as I can really use it for commercial purposes. I had some high hopes for Yahoo services a few years back, but they seemed determined to plague everything with confusing or commercial-hostile licensing.
A free service like locationkit (seen here a few days ago IIRC) looks OK, and I'm reasonably sure it's supported by revenues for other services from the parent company. However, even that feels like it might get shut down if it becomes too big a burden.
Something with a good understandable license that I can pay a reasonable amount ($5-$20/year, for most low-mid volume use cases) would be great, as long as I can really use it for commercial purposes. I had some high hopes for Yahoo services a few years back, but they seemed determined to plague everything with confusing or commercial-hostile licensing.
That sounds a lot like some of MaxMind's products.
I think it is. I've not tested their quality, but saw their pricing - buy credits then they get used up per API call, similar to Twilio's per minute deductions from a credit. If the OP has a way to best MaxMind's quality,and can charge, that'd be good. But the OP was talking about a 'free' model, which I'm not really interested in.
> $5-$20/year
That seems to be way too low.
$20/month+ could make sense, but less than that should be simply free (limited edition).
Dealing with payments is just too much overhead.
That seems to be way too low.
$20/month+ could make sense, but less than that should be simply free (limited edition).
Dealing with payments is just too much overhead.
maxmind already does something like this, although you're just buying credits. twilio does the same thing. Other mail services i've used do credits as well. For my projects/demos/etc - $20/year is where I'm at with twilio (which has led to clients using it and spending hundreds or thousands more, minimum).
With maxmind, $10/year might be all that I use (considering 1000 calls might only be $1 charge).
With maxmind, $10/year might be all that I use (considering 1000 calls might only be $1 charge).
Ahh - Amazon Services model: charge on a metered basis. You would make almost nothing on small customers, but if some of your customers grow significantly - that's where your real revenue would come in (seamlessly for you and your customers).
Should work too.
Should work too.
You should also try selling that strategy to investors, while not being Bezos, or find another job to support your venture :-)
I agree, paying a small fee for it sounds good. However it might gain a lot of popularity and support if there is also a free version, e.g. for non-commercial purposes only and updated only once a year or so.
I would personally not want to pay up front without knowing anything about this database from a company (or private person) that only asks 5 bucks a year for it.
I would personally not want to pay up front without knowing anything about this database from a company (or private person) that only asks 5 bucks a year for it.
Tons of devs are interested
Would LOVE this, possibly in json? Id even consider hosting it for you.
I would be interested. We pay for Maxmind and still get a lot of inaccurate data.
I think a lot of people would be interested. I'm also curious of the implementation details. Open source? :p
To echo others, aim for low-priced and not free. That'll give you some operating revenue to avoid it going the way of the dodo.
To echo others, aim for low-priced and not free. That'll give you some operating revenue to avoid it going the way of the dodo.
[deleted]
I would be very interested in such a thing. Where are you getting your data from? Does it work for IPv6 too?
Hmm. Can you determine such things using BGP? I guess that wouldn't be very accurate, though...
In the EU, the sales tax now (I think, since this year) must be paid to the country, the buyer comes from (before, the sellers country was decisive). Also the height of the tax depends on this. So, at least in Europe there are plenty of small companies that could need such a thing, at least as long they don't have a payment-provider that handles the sales-tax for them.
I myself, could also be interested, but I think, I will leave that to my payment-provider ... (ugly, EU!)
Edit: Ok, I should not, but why the downvote, please? When I wrote something wrong, than please explain!
I myself, could also be interested, but I think, I will leave that to my payment-provider ... (ugly, EU!)
Edit: Ok, I should not, but why the downvote, please? When I wrote something wrong, than please explain!
You almost certainly don't want to be using IP geolocation to compute sales tax, rather some kind of billing address explicitly provided by the customer.
In that case, the customer could cheat. I am not totally sure about the current discussion on that, but the problem is, that you definitively can not rely on the address given of the customer, because in this case the customer will always give a country with the lowest sales tax. Also you must provide proofs for the location. The IP address is IMHO (and not only my opinion) better and will be more likely be recognized as proof instead of relying solely on some customer input.
But, you (because you are so sure about the best method) by sure know, that the laws are explicitly for non-physical goods, where no delivery-address must be provided, right?
Here is a link, how one company is doing it (forget about government information, they seldom go so much in details!): https://rachelandrew.co.uk/archives/2014/12/08/consensus-and...
They use the IP address to validate the user given address and for later proof.
BTW: Downvoting all the time does not help the discussion. I try to discuss this topic seriously and I was bringing some very relevant topic here, and am constantly downvoted. (It is not my fault, that the EU came up with such laws, and provide no useful guide how to implement it!)
But, you (because you are so sure about the best method) by sure know, that the laws are explicitly for non-physical goods, where no delivery-address must be provided, right?
Here is a link, how one company is doing it (forget about government information, they seldom go so much in details!): https://rachelandrew.co.uk/archives/2014/12/08/consensus-and...
They use the IP address to validate the user given address and for later proof.
BTW: Downvoting all the time does not help the discussion. I try to discuss this topic seriously and I was bringing some very relevant topic here, and am constantly downvoted. (It is not my fault, that the EU came up with such laws, and provide no useful guide how to implement it!)
In Denmark, at least, billing address is standard for non-physical goods as well. It's validated via the payment method, typically credit card or bank transfer. It's possible to cheat this, too, but harder than just making up a fake address. Afaik none of the main online sellers in DK are relying on IP address currently, in part because accuracy is not good enough, and in part because people traveling across borders is common (many residents of Malmö, for example, have Danish IP addresses during the day and Swedish IP addresses at night).
You can not validate all payment methods by country (or do you want to limit the methods provided?). Also nobody hinders you from creating a bank account in a different EU country.
That "all" are doing it, does not guarantee, that one time a judge or tax controller will come and argue with you that you have no proofs at all. In this case, you are really bust, because the new law also says, that you are accountable for the tax to all the countries tax authorities. Much fun, when you get citations from different EU countries.
(Don't say it is not realistic: There are cases already public, where British shops got claims from remote EU countries, because of tax differences of <10 Euro).
And as long the EU does provide laws that are hard to implement in real life (only in the minds of the politicians they are simple) and nobody says how to implement them, it is better to collect all proof you can get.
That "all" are doing it, does not guarantee, that one time a judge or tax controller will come and argue with you that you have no proofs at all. In this case, you are really bust, because the new law also says, that you are accountable for the tax to all the countries tax authorities. Much fun, when you get citations from different EU countries.
(Don't say it is not realistic: There are cases already public, where British shops got claims from remote EU countries, because of tax differences of <10 Euro).
And as long the EU does provide laws that are hard to implement in real life (only in the minds of the politicians they are simple) and nobody says how to implement them, it is better to collect all proof you can get.
A parallel poster posted an official link from the British Government:
https://www.gov.uk/government/publications/vat-supplying-dig...
There is clearly stated, that IP information is one of the proofs accepted for the country determination.
So, maybe next time, please first ask before judge.
https://www.gov.uk/government/publications/vat-supplying-dig...
There is clearly stated, that IP information is one of the proofs accepted for the country determination.
So, maybe next time, please first ask before judge.
[deleted]
I'll echo the "low cost, not free" thought. Stuff takes money to maintain. I don't want to build a service that depends on an unsustainable business model.
Maxmind is low cost, not free, and has a pretty good track record. What would you want from an alternative?
I would want:
- Easier updates. Now it is a minor hassle to update my MySQL database table from CVS. I need to do it again soon.
- Availability in different formats. For my personal needs, I would love to download a MySQL MyISAM table.
- More accurate data, including "ambiguous country" for employees of multinational firms.
- Related open source tools through github.
(Disclaimer: I work for MaxMind)
As far as formats go, if anyone is thinking of putting together a custom database with information keyed on IP address or IP ranges, I'd encourage you to look into our MaxMind DB format:
http://maxmind.github.io/MaxMind-DB/
There are open source file readers available in many languages: https://github.com/maxmind/?utf8=%E2%9C%93&query=MaxMind-DB
There's also an open source writer https://metacpan.org/pod/MaxMind::DB::Writer
Our officially supported libraries are actively maintained and pull requests are always welcome.
I'm actually about to begin writing a blog post on how you can create your own custom databases. I'll share it on HN once it's ready.
As far as formats go, if anyone is thinking of putting together a custom database with information keyed on IP address or IP ranges, I'd encourage you to look into our MaxMind DB format:
http://maxmind.github.io/MaxMind-DB/
There are open source file readers available in many languages: https://github.com/maxmind/?utf8=%E2%9C%93&query=MaxMind-DB
There's also an open source writer https://metacpan.org/pod/MaxMind::DB::Writer
Our officially supported libraries are actively maintained and pull requests are always welcome.
I'm actually about to begin writing a blog post on how you can create your own custom databases. I'll share it on HN once it's ready.
I don't work for MaxMind and I'd recommend it too.
A couple years ago I worked on a project that involved processing the "Internet Census of 2012" data.
The MaxMind DB format was a fast, efficient way to store simple data keyed on IPs and subnets.
A couple years ago I worked on a project that involved processing the "Internet Census of 2012" data.
The MaxMind DB format was a fast, efficient way to store simple data keyed on IPs and subnets.
Can you please confirm that the free versions are not 'good enough'? http://dev.maxmind.com/geoip/geoip2/geolite2/
If you want to compare the accuracy of different products, you can do so on this page:
https://www.maxmind.com/en/geoip2-city-database-accuracy
To get the actual product comparisons, you'll need to enter a country in the input box. For instance:
https://www.maxmind.com/en/geoip2-city-database-accuracy?cou...
You can try refining the distances to see how the different products compare. If you fiddle with it a bit, you'll see that the results do vary from country to country. So, "good enough" will depend on some combination of your tolerance for error + which countries you care about.
https://www.maxmind.com/en/geoip2-city-database-accuracy
To get the actual product comparisons, you'll need to enter a country in the input box. For instance:
https://www.maxmind.com/en/geoip2-city-database-accuracy?cou...
You can try refining the distances to see how the different products compare. If you fiddle with it a bit, you'll see that the results do vary from country to country. So, "good enough" will depend on some combination of your tolerance for error + which countries you care about.
Awesome, thanks!
we have used the free version of MaxMind for several years and upgraded to the paid version two years ago (the additional coverage provided by the paid version was the driver for us). I can highly recommend MaxMind.
That would be amazing. I'd love a datadump that could be accessed on my own servers instead of having to hit an API. The format of the datadump doesn't really matter to me, as long as there's documentation on how to read it.
Most geo data providers (like MaxMind) provide datafile downloads so you don't have to use an API.
Right, but those aren't usually free ;) On the other hand there are plenty of free (limited) APIs.
They do have a free version which is just a bit less accurate than the commercial version.
http://dev.maxmind.com/geoip/geoip2/downloadable/
http://dev.maxmind.com/geoip/geoip2/downloadable/
Oh, I didn't know that. How much less accurate? Will it accurately determine country of origin?
Yes, it will
[deleted]
Note that you could build a simple API with Google App Engine:
https://cloud.google.com/appengine/docs/python/#Request_Head...
and it would likely be free if the number of requests was below a certain threshold per month.
https://cloud.google.com/appengine/docs/python/#Request_Head...
and it would likely be free if the number of requests was below a certain threshold per month.
Id be interested, especially if it used same file format as Maxmind's binary dat file ... so it can be a drop in replacement.
Yes, one initial download and then subsequent deltas on a regular basis. Will be willing to pay. I know of plenty of Fortune 500 companies will be willing to pay for it.
On a related note, would anyone in the community be interested in an inexpensive service that differentiates dynamic from static IPs? Dynamic IPs would be accompanied by an estimate of lease duration.
I wouldn't want this to exist because it discriminates my dynamic-looking IP address from server IP addresses (even though mine is statically assigned to my Internet subscription) and I have enough trouble hosting things at home already.
But I understand that's not really an argument: you might say you don't want nukes to exist because they hurt people. I just wanted to mention it.
That said, maybe. Usually an rdns lookup does the trick already though (a4-8e-ef-78.dynamic.ziggo.nl or cpe-12-34-56-78.rr.com or 80-100-131-150.adsl.xs4all.nl is clear enough).
But I understand that's not really an argument: you might say you don't want nukes to exist because they hurt people. I just wanted to mention it.
That said, maybe. Usually an rdns lookup does the trick already though (a4-8e-ef-78.dynamic.ziggo.nl or cpe-12-34-56-78.rr.com or 80-100-131-150.adsl.xs4all.nl is clear enough).
Yeah rDNS is helpful, but as you pointed out with your home IP, many dynamic IPs (those that are rotated by an ISP) are actually granted very long leases. The service I'm proposing would make an additional effort to estimate the duration of dynamic leases.
Yes, clear enough if you are Dutch ;-) But what about other countries ? Singapore, India, Brasil.......
I'd be interested if it included cities. If there was a javascript plugin to automatically help contribute (is that possible), I'd add it to some of my web pages. CSV is fine.
Mozilla runs an open/free geolocation service: https://location.services.mozilla.com/
https://mozilla-ichnaea.readthedocs.org/en/latest/config.htm...
"The geoip section contains settings related to the maxmind GeoIP database."
If I don't get it wrong, this service relies on MaxMind's db for GeoIP.
The Mozilla Location Service does use MaxMind for GeoIP, but the service's primary use is to map Wi-Fi SSIDs to longitude/latitude position. Mozilla has its own Wi-Fi database continually fed new data from the Mozilla Stumbler app (on the Google Play Store and F-Droid markets) and Firefox for Android. The Mozilla database also has a partnership with commercial Wi-Fi company Combain and the OpenCellID project for fallback results.
https://blog.mozilla.org/services/2015/03/06/combain-deal-he...
(I work at Mozilla and I used to work on the Location project. :)
https://blog.mozilla.org/services/2015/03/06/combain-deal-he...
(I work at Mozilla and I used to work on the Location project. :)
I pitched the idea of Mozilla creating a free GeoIP database by collecting (opt-in) traceroutes from Firefox users to some Mozilla servers. The traceroutes could then be overlaid to create a physical map of the Internet. I know other projects have mapped the internet, but this approach would work from the endpoints in, not from some centralized crawler.
There seems to be heavy interest in a low-cost service, even free if it was sustainable. Just curious if anyone has seen innovative pricing around services, similar to how open-source software made expensive software much cheaper or free, where the vendor made the main "service/data" much less expensive but monetized in a different way.
So what I'm envisioning are services designed for zero profit. Prices are indexed to AWS/Azure/Google prices; as they fall, the service's prices falls. Customers who need predictable pricing can buy long term contracts, as they do on AWS. Customers who use the service more, pay more, because the service offers utility-based pricing. The goal is to make the service as cheap as possible and maintained by the community. The source code and all data, except private data (e.g., API keys), is open and editable by the community.
Think it would be an interesting way to deliver services and data, just not sure how it turns into a business (since the API/service portion is designed to be zero profit).
Think it would be an interesting way to deliver services and data, just not sure how it turns into a business (since the API/service portion is designed to be zero profit).
I use GeoIP fairly regularly. How I would set this up:
- Free version: only updated once [every 6 months / a year] and non-commercial use only
- Paid version: $9.99 for a year of free updates
- Updates come both in diff and in full version (for the commercial version)
- Use a common format, or use multiple formats. People love csv, but more serious applications probably use an existing format (I see maxmind a lot in this thread).
- Perhaps an API with heavy ratelimiting for people who need only a very occasional lookup (for very occasional use, I use ip2location.com/demo currently; for more serious stuff I download a database).
Having a free version with a nice license might also get you included in a lot of open source software, which increases name visibility and makes people write parsers for your format of choice.
A minor note on payment: accept Paypal and/or Bitcoin, or I (and many other Europeans) can never pay you. These credit card-only services are impossible to order from.
Idea for the diff: if you have a binary format, perhaps use this: https://www.chromium.org/developers/design-documents/softwar...
- Free version: only updated once [every 6 months / a year] and non-commercial use only
- Paid version: $9.99 for a year of free updates
- Updates come both in diff and in full version (for the commercial version)
- Use a common format, or use multiple formats. People love csv, but more serious applications probably use an existing format (I see maxmind a lot in this thread).
- Perhaps an API with heavy ratelimiting for people who need only a very occasional lookup (for very occasional use, I use ip2location.com/demo currently; for more serious stuff I download a database).
Having a free version with a nice license might also get you included in a lot of open source software, which increases name visibility and makes people write parsers for your format of choice.
A minor note on payment: accept Paypal and/or Bitcoin, or I (and many other Europeans) can never pay you. These credit card-only services are impossible to order from.
Idea for the diff: if you have a binary format, perhaps use this: https://www.chromium.org/developers/design-documents/softwar...
I actually use ip2location's demo page as well. I put together my own api that grabs ip to location data from the 4-5 sources that provide the information for free, and then look for similarities. So if a (city, region, country) is present 3/5 times, it defaults to that. If they're not any similarities, it will default to maxmind geoip2's demo (if it hasn't already used up the daily free lookups). And then cache it all of course. Works well for the limited amount of lookups I need.
I haven't thought about making this a for-profit project yet. Maybe a paid API in the future, but I really want to offer an accurate & free option that's up to date.
Re: CC's, I'm also based in Europe (The Netherlands) and also hate services that only allow credit cards.
Thanks for the link regarding diffs, I'll definitely give it a read.
Re: CC's, I'm also based in Europe (The Netherlands) and also hate services that only allow credit cards.
Thanks for the link regarding diffs, I'll definitely give it a read.
Contra to the other posters requests for paypal,
I refuse to have a paypal account, and recent changes to how they process payments mean I can't just pay with a cc without having an account.
If you are accepting payments, it might be a good idea to have more than one payment processor.
Especially if paypals notoriously badly timed account suspensions might force a existential crisis for your business.
I refuse to have a paypal account, and recent changes to how they process payments mean I can't just pay with a cc without having an account.
If you are accepting payments, it might be a good idea to have more than one payment processor.
Especially if paypals notoriously badly timed account suspensions might force a existential crisis for your business.
Why is paying with credit cards hard for Europeans?
I can only comment for Finland, but at least here it is not so common for people to have company credit cards. If you want to buy something for the company you then either need to find out the person who has the card or use your personal credit card. Both options are bit cumbersome.
Getting the person who has the card to enter the necessary card details to a web site is not always convenient. On the other hand passing the card number, cvc etc to many people in the company has certain risks associated with it. Using personal credit card is painful, especially with monthly fees, since you need to do all the paperwork to get money from company.
Usually the credit card invoices are not enough for the bookkeeping. You need to have more detailed record about what was purchased and about the VAT included in the price. With shared company credit cards somebody then ends up chasing the actual receipts, which happen to be in somebody's inbox.
In most cases it is just so much easier to handle things with invoicing. The invoices are handled electronically and usually companies have systems in place which allow the accounting department to just pass the incoming invoice to whoever for approval.
A quite good solution this would be a virtual credit card service where somebody could assign "virtual cards" to whoever on the company and put specific monthly/yearly limits on those. It would be pretty nice to add the collection of receipts to this one as well. So the person responsible for specific virtual card, would also upload the invoices that match to the payments made.
Getting the person who has the card to enter the necessary card details to a web site is not always convenient. On the other hand passing the card number, cvc etc to many people in the company has certain risks associated with it. Using personal credit card is painful, especially with monthly fees, since you need to do all the paperwork to get money from company.
Usually the credit card invoices are not enough for the bookkeeping. You need to have more detailed record about what was purchased and about the VAT included in the price. With shared company credit cards somebody then ends up chasing the actual receipts, which happen to be in somebody's inbox.
In most cases it is just so much easier to handle things with invoicing. The invoices are handled electronically and usually companies have systems in place which allow the accounting department to just pass the incoming invoice to whoever for approval.
A quite good solution this would be a virtual credit card service where somebody could assign "virtual cards" to whoever on the company and put specific monthly/yearly limits on those. It would be pretty nice to add the collection of receipts to this one as well. So the person responsible for specific virtual card, would also upload the invoices that match to the payments made.
Because the majority don't have/want credit cards.
I'm European and I'm yet to find a credit card field where I can't put my debit card number. I'm not sure I understand what you're saying.
Debit card number is not your IBAN, right? Because that's the only number on my bank card and it never works (because all forms want your "credit card" number, I thought).
Oh, no, it's not your IBAN. It's a number that looks like a credit card number, but the card it points to is actually a debit card.
What kind of credit card form doesn't allow debit cards? I've never seen one and been shopping all over Europe, both online and in person.
Oh, you mean you mostly use non-VISA/MC debit cards like we do in Canada? (we have Interac)
how do you pay for things online?
In Belgium something called MrCash is popular, The Netherlands has iDeal supported by pretty much all banks. Other than that: bank transfers, direct debit, or third-party things like PayPal (or in my case, Bitcoin).
Can you provide some contact info?
Would like to help adding changes, I personally find Maxmind inaccurate as well.
Shoot me an email at [email protected] and I'll provide you with updates
Can you provide some contact information?
I would like to contribute as well, I find Maxmind extremely unreliable when it comes to Home ISPs.
Shoot me an email at [email protected] and I'll provide you with updates
I've been building exactly this:
http://browserlocation.co/
I have a JS library coming also for use when the w3c geolocation fails. The data is open and downloadable with cities etc (CSV right now). There's a mailing list for updates, and anyone can contribute just by opening the website in a browser!
http://browserlocation.co/
I have a JS library coming also for use when the w3c geolocation fails. The data is open and downloadable with cities etc (CSV right now). There's a mailing list for updates, and anyone can contribute just by opening the website in a browser!
It shouldn't be free, it should be reasonably priced so you don't run out of money and can keep this thing running for years to come.
There's huge demand for it - just search for geoip on Github. Some more pointers are here [0],[1]. So many projects, but most of them work with Maxmind data, which is the problem. Inconsistencies in the data they provide makes it painful to make performant fallback solutions. Performance across geolocations and lack thereof in "exotic" geos is another one. It seems none of these services is deployed across the globe. There's a need for a blazing fast, reliable service with granular, comprehensive data, that is performing well for millions of requests across the globe, be it for mobile or web traffic.
I found a great base[2] and I think Go is a perfect language for this kind of service.
Here are the free checkers I tested the other day, the IP is a good example for incomplete data:
There's huge demand for it - just search for geoip on Github. Some more pointers are here [0],[1]. So many projects, but most of them work with Maxmind data, which is the problem. Inconsistencies in the data they provide makes it painful to make performant fallback solutions. Performance across geolocations and lack thereof in "exotic" geos is another one. It seems none of these services is deployed across the globe. There's a need for a blazing fast, reliable service with granular, comprehensive data, that is performing well for millions of requests across the globe, be it for mobile or web traffic.
I found a great base[2] and I think Go is a perfect language for this kind of service.
Here are the free checkers I tested the other day, the IP is a good example for incomplete data:
checkers = [
'http://freegeoip.net/json/103.1.29.57',
'http://geoip.nekudo.com/api/103.1.29.57',
'http://ip-api.com/json/103.1.29.57',
'http://ipinfo.io/103.1.29.57/json',
'http://www.telize.com/geoip/103.1.29.57'
]
ip-api guy here.
> It seems none of these services is deployed across the globe
Our free service has servers in USA, EU and Asia
> There's a need for a blazing fast, reliable service with granular, comprehensive data, that is performing well for millions of requests across the globe, be it for mobile or web traffic.
We have more than 2 billion requests per month!
> It seems none of these services is deployed across the globe
Our free service has servers in USA, EU and Asia
> There's a need for a blazing fast, reliable service with granular, comprehensive data, that is performing well for millions of requests across the globe, be it for mobile or web traffic.
We have more than 2 billion requests per month!
honest opinion: I hate geoip. I think judging your visitors from their geolocation is more and more becoming a problem. It goes against the fundamental value of the Internet being a global network. I don't want Netflix to sell a sub-standard product in some countries because some licensing foo. I want everyone on this planet to have access to the same Internet.
If I could I would redesign internet routing in a way that the webpage doesn't know where a request comes from.
If I could I would redesign internet routing in a way that the webpage doesn't know where a request comes from.
Sadly, with the whole European VAT requirements the guidelines are as follows:
https://www.gov.uk/government/publications/vat-supplying-dig...
"the business making the supply must obtain and keep 2 pieces of non-contradictory information to support and evidence the member state where the customer is normally located."
One of those being: "the Internet Protocol (IP) address of the device used by the customer"
So sign up for something while abroad and you could get caught out.
https://www.gov.uk/government/publications/vat-supplying-dig...
"the business making the supply must obtain and keep 2 pieces of non-contradictory information to support and evidence the member state where the customer is normally located."
One of those being: "the Internet Protocol (IP) address of the device used by the customer"
So sign up for something while abroad and you could get caught out.
GeoIP can be used for good. For example it can be used to set defaults for various forms and choices. Why should you have to scroll past about half the world's countries just to select yours? The most likely airport I want is the closest one. Same for store locators.
Estimating my taxes, shipping charges and delivery date are great too. Even sorting products and choices by what those in my area prefer is great too.
The geoip doesn't need to be street level accurate, but to the nearest ~30 miles can make the online experience much more pleasant and sensible.
Estimating my taxes, shipping charges and delivery date are great too. Even sorting products and choices by what those in my area prefer is great too.
The geoip doesn't need to be street level accurate, but to the nearest ~30 miles can make the online experience much more pleasant and sensible.
> I don't want Netflix to sell a sub-standard product in some countries because some licensing foo.
"some licensing foo" is a great way of eliding that in some countries, the service would be illegal. The alternative is that the product could not exist at all, if there is no way to negotiate a contract for it.
You can certainly argue that products should not exist if that would involve contradicting some abstract ideal for society, or that it's good for things on the internet to blatantly violate mundane real-world laws until real-world governments become irrelevant. Respectable people hold both of those positions. But they're not very widely-accepted positions.
"some licensing foo" is a great way of eliding that in some countries, the service would be illegal. The alternative is that the product could not exist at all, if there is no way to negotiate a contract for it.
You can certainly argue that products should not exist if that would involve contradicting some abstract ideal for society, or that it's good for things on the internet to blatantly violate mundane real-world laws until real-world governments become irrelevant. Respectable people hold both of those positions. But they're not very widely-accepted positions.
Presume a hypothetical world where everybody's traffic is onion-routed, or at least VPNed. Streaming-content licensing would still happen in such a world—the location-enforcement would just have to happen by other means, like account phone-number verification.
That still involves Netflix selling a sub-standard product in some countries, no?
I'm not being pedantic about the argument, I just don't see how there's a benefit to any fundamental values if we still allow this delta, but we're picky about what technical means are used to make it happen. I could see an argument that geo-IP databases are often wrong and that's a problem, but that seems like a very different argument.
I'm not being pedantic about the argument, I just don't see how there's a benefit to any fundamental values if we still allow this delta, but we're picky about what technical means are used to make it happen. I could see an argument that geo-IP databases are often wrong and that's a problem, but that seems like a very different argument.
> If I could I would redesign internet routing in a way that the webpage doesn't know where a request comes from.
The internet already has that - it's called spoofing a packet.
But in all seriousness - this is an extremely bad idea. Having a unknown source would make it extremely easy for horrible people to do bad things to good people.
The internet already has that - it's called spoofing a packet.
But in all seriousness - this is an extremely bad idea. Having a unknown source would make it extremely easy for horrible people to do bad things to good people.
or pay 5 bucks for a vpn.
> I want everyone on this planet to have access to the same Internet.
Really? Do you want everyone on this planet to get taxed equally too? Because if he answer is 'yes', then you're strategy isn't working.
Really? Do you want everyone on this planet to get taxed equally too? Because if he answer is 'yes', then you're strategy isn't working.
I have a domain name you could use localipy.com. Just tell me your nameservers.
I like the generosity here but no one in their right mind should consider this kind of offer unless the domain was transferred to them. There's too much risk involved and things that can go wrong unless a domain is 100% owned by you.
Right. I have so many names and I never put them to good use :S
I have a domain I've not done much with that might be perfect for this geoip.io
Let me know if interested or if we could maybe partner on this
Let me know if interested or if we could maybe partner on this
You seem to be solving a very sought after problem which also makes it so much more difficult. Where do you plan to source this much info?
Can somebody make me understand why ipv4 assignments along with their requests are not made public?
Can somebody make me understand why ipv4 assignments along with their requests are not made public?
> Can somebody make me understand why ipv4 assignments along with their requests are not made public?
What do you mean? With any of the RIR's you can get it down to a specific company. With ARIN, you're supposed to be able to get it down to a particular customer. What more information do you want?
What do you mean? With any of the RIR's you can get it down to a specific company. With ARIN, you're supposed to be able to get it down to a particular customer. What more information do you want?
I'd be interested. I'm not sure daily updates add much value, how often do these things change? If you have a way to provide more accuracy though, that is very interesting.
I'd definitely be interested, but I'd also be worried that you could really sustain interest in doing this for a decade.
hell yes!
A slight tangent but good feedback to OP regardless:
I use MaxMind at work but because we're on the RoR stack, we have to download and commit the massive 30MB databases into our repository in order for it to be used on Heroku. We use https://github.com/yhirose/maxminddb which seems to be the only good option available that reads the new "mmdb" format. It's a complete pain to have to commit such massive DBs for it to work and would prefer for Rails to fetch the database upon initialization, cache it for at most 2 days and redownload if it hasn't already been restarted by Heroku.
If you can provide a solution to that, it would give us enough value for me to pay a few dollars per month for it.
I use MaxMind at work but because we're on the RoR stack, we have to download and commit the massive 30MB databases into our repository in order for it to be used on Heroku. We use https://github.com/yhirose/maxminddb which seems to be the only good option available that reads the new "mmdb" format. It's a complete pain to have to commit such massive DBs for it to work and would prefer for Rails to fetch the database upon initialization, cache it for at most 2 days and redownload if it hasn't already been restarted by Heroku.
If you can provide a solution to that, it would give us enough value for me to pay a few dollars per month for it.
I'm in a similar boat, but just have a license key on the machine (committed) and use the maxmind updater to fetch our licensed db.
It happens once per provision, and a cron job to keep it up to date.
It happens once per provision, and a cron job to keep it up to date.
We do millions of lookups per day. We do not use Maxmind.
Mind to elaborate a little bit on this?
What do you use instead of Maxmind?
I love a good free service where the data quality is improved. I have played around with maxmind's service a bit but the data for India is far from accurate. I have been hunting for a local database (no service) since
I run a IP lookup service http://ipof.in where I want the service subsumed in. This is a free service for anybody who uses it. It supports XML/JSON and plain text.
Location to Timezone would be of value to me. No IP involved, want to map a Google Places / Location City value ~ to a timezone. Would pay monthly fee.
We offer this at http://geocod.io :)
I took a look thanks. Pricing seems good, however we have a very international audience. We need to get timezones for geocodes (preferably Google Place IDs) from all over Asia and the Middle East.
How good is your global timezone data? Is it possible to map Google Places (from their places autocomplete form) to a timezone?
How good is your global timezone data? Is it possible to map Google Places (from their places autocomplete form) to a timezone?
[deleted]
Me! Once or twice a year I need that stuff on different occasions and each time I find it really annoying.
Two considerations that help avoid the IP issue, but require user permission and so have a big obvious disadvantage:
1) We can prompt mobile users to enable location/gps and get their location that way.
2) We can prompt modern browsers to allow location services and get their location that way.
This is if a user wants to have their location 'auto-set' instead of via pulldowns.
3) A hard problem to solve is - mapping a unified set of city names to timezones, and updating the timezones as per Daylight Savings Time and general timezone updates.
Anyone have references for 3) ?
1) We can prompt mobile users to enable location/gps and get their location that way.
2) We can prompt modern browsers to allow location services and get their location that way.
This is if a user wants to have their location 'auto-set' instead of via pulldowns.
3) A hard problem to solve is - mapping a unified set of city names to timezones, and updating the timezones as per Daylight Savings Time and general timezone updates.
Anyone have references for 3) ?
Would it be possible to run some kind of GeoIP DB over a P2P/DHT network?
Also see http://hostip.info
I prefer csv. I care more about the content it provided, such as location , city, longitude , latitude， isp. In my company, we need a accurate database very much.And now we use IP2LOCATION-LITE-DB11.CSV and ipip.net .
Thank you all for your feedback, I will continue working on this and make a new post once this project goes live.
I started hacking away yesterday at making an alternative database that seems to be more accurate, but it's still in the works. Who here would be interested in a free alternative, including and excluding cities, updated on a daily basis, and if so, which format's would you like to see?