A collection of free datasets from Microsoft Research(msropendata.com)
msropendata.com
A collection of free datasets from Microsoft Research
https://msropendata.com/
46 comments
SQL Server ships something called Adventure Works since over a decade - same idea as Northwind but larger to also demo and play around with Analytivs and BI.
Adventure Works, pffft... I’ll take Scott any day of the week.
Also available in other flavours
https://github.com/pthom/northwind_psql
https://github.com/pthom/northwind_psql
The Northwinds sample database predates SQL Server. I remember using it with Microsoft Access 1.0 back in 1993...
Heh, you probably read my post and thought "darn kids" to yourself, eh?
This page has problems loading the datasets. Is it overloaded or just broken?
Does anyone happen to know what licenses the various datasets are under?
Does anyone happen to know what licenses the various datasets are under?
Can confirm that I'm having the same issues, could be overloaded indeed.
Turn off your ad blocker. Fixed it for me.
Thanks!
[deleted]
How did you find that fix??! Just Curious..
Ctrl+shift+i in Chrome, Console, you may see something like "blocked by client".
[deleted]
Well if you are looking for a huge dataset, I would suggest stack overflow.
https://archive.org/details/stackexchange
I remember I wrote a node script to download all question and dump into my Postgres instance. It was fun, Postgres with an index could able to fetch results super fast
How many answers for this question?
How many questions are unanswered?
With gin index, I could do a free text search as well.
https://archive.org/details/stackexchange
I remember I wrote a node script to download all question and dump into my Postgres instance. It was fun, Postgres with an index could able to fetch results super fast
How many answers for this question?
How many questions are unanswered?
With gin index, I could do a free text search as well.
Wow, 230GB celebrity photo dataset caught me off guard.
Oh great!
On a similar note, does anyone know of a similar type of dataset/source (I'm expecting smaller) that's available for commercial use?
On a similar note, does anyone know of a similar type of dataset/source (I'm expecting smaller) that's available for commercial use?
Is it the one used in this paper?
https://youtu.be/XOxxPcy5Gr4
https://youtu.be/XOxxPcy5Gr4
I thought for SURE this would be a rickroll video. Pleasantly surprised.
This took me down memory lane to what could be done in the 1990's with a regular PC/Amiga with one of those special framebuffer cards that could render a frame at a time to a Digibeta D1 video recorder for your typical acid house pop video:
https://www.youtube.com/watch?v=TPFAYIr8z2I
https://www.youtube.com/watch?v=TPFAYIr8z2I
There's also a 2.3 GB dataset of celebrity photos from the web.
[deleted]
They don't make it easy to find out what the Microsoft Research License Agreement is, when they put it behind a login wall on a site that doesn't work with ad-block. But this appears to be a copy of it: http://www.cs.toronto.edu/~aditya/langid/msrla.txt
These license terms are a trap. Nobody should ever accept them.
You can't use the data for commercial purposes, which is harsh enough on its own -- it means the data should only be interesting to people who plan to be academics permanently, or who don't plan to succeed at anything they do with it.
You can't even open-source anything you make out of it, because open source software does not restrict the purpose for which it's used. You can't combine it with data under any Creative Commons license except CC-By or CC-0, because its restrictions are definitely not Creative Commons compatible.
The truly insulting part is that you have to give an unrestricted license back to Microsoft for everything you make out of this data. You can't benefit from what you make out of the data, but Microsoft can, with no limitations.
Whatever you make from this data, you can't benefit from it. Most other people can't benefit from it. Microsoft can benefit from it, and if they want to, they can just take it without crediting you.
This is like an artist being asked to do something "for the exposure", by someone who will just take it and give you no exposure.
These license terms are a trap. Nobody should ever accept them.
You can't use the data for commercial purposes, which is harsh enough on its own -- it means the data should only be interesting to people who plan to be academics permanently, or who don't plan to succeed at anything they do with it.
You can't even open-source anything you make out of it, because open source software does not restrict the purpose for which it's used. You can't combine it with data under any Creative Commons license except CC-By or CC-0, because its restrictions are definitely not Creative Commons compatible.
The truly insulting part is that you have to give an unrestricted license back to Microsoft for everything you make out of this data. You can't benefit from what you make out of the data, but Microsoft can, with no limitations.
Whatever you make from this data, you can't benefit from it. Most other people can't benefit from it. Microsoft can benefit from it, and if they want to, they can just take it without crediting you.
This is like an artist being asked to do something "for the exposure", by someone who will just take it and give you no exposure.
I thought you are overreacting until I reached this part : " 3. That Microsoft is granted back, without any restrictions or limitations, a non-exclusive, perpetual, irrevocable, royalty-free, assignable and sub-licensable license, to reproduce, publicly perform or display, install, use, modify, post, distribute, make and have made, sell and transfer your modifications to and/or derivative works of the Data, including but not limited to the results and analysis of any tests that are run by or conducted by you with or for the Data, for any purpose."
"Not limited" - so basically any software processing it.
"Not limited" - so basically any software processing it.
It's a different license. Here's a direct link to the Microsoft Research Data License Agreement (PDF): https://msrodr-api.azurewebsites.net//licenses/2f933be3-284d...
Disclaimer: I don't know if this affects your concerns, but it's at least the right license. Disclaimer 2: I work for Microsoft, but not for MSR, and don't know anything specific about this.
Disclaimer: I don't know if this affects your concerns, but it's at least the right license. Disclaimer 2: I work for Microsoft, but not for MSR, and don't know anything specific about this.
Thanks for the link. This version of the license has the same terms and then several more, including:
- If we accidentally gave you people's personal identifying information, it's your job to destroy it
- Your license to us is perpetual but our license to you terminates after 2 years (?!)
- The data is confidential and you may not talk about it for 5 years (OMG WTF)
- If we accidentally gave you people's personal identifying information, it's your job to destroy it
- Your license to us is perpetual but our license to you terminates after 2 years (?!)
- The data is confidential and you may not talk about it for 5 years (OMG WTF)
"We're all in on open source." [1]
Thanks for taking the effort to check. :) I believe this kind of attitude from MS is pretty problematic for Github.
[1] https://www.cnbc.com/2018/06/04/microsoft-ceo-satya-nadella-...
[1] https://www.cnbc.com/2018/06/04/microsoft-ceo-satya-nadella-...
While I find it awful that this particular department of Microsoft is offering "free data" with poisonous terms, I see no reason to believe it has anything to do with GitHub. Microsoft is a big place.
[deleted]
"You can't use the data for commercial purposes, which is harsh enough on its own -- it means the data should only be interesting to people who plan to be academics permanently, or who don't plan to succeed at anything they do with it."
It'd also be useful for someone farting around with machine learning who isn't anywhere close to being at a point of shipping anything to anyone else.
What you can benefit from is what you learn while using the data. Then you start a new project using what you've learned, and using different data.
It'd also be useful for someone farting around with machine learning who isn't anywhere close to being at a point of shipping anything to anyone else.
What you can benefit from is what you learn while using the data. Then you start a new project using what you've learned, and using different data.
It'd also be useful for someone farting around with machine learning who isn't anywhere close to being at a point of shipping anything to anyone else.
This is the only use I can see for this data under those license terms. And even then, you are still technically granting Microsoft a super-license to do whatever they want with whatever you come up with, and probably also not allowed to upload it to Github.
This is the only use I can see for this data under those license terms. And even then, you are still technically granting Microsoft a super-license to do whatever they want with whatever you come up with, and probably also not allowed to upload it to Github.
Why would you plan to fail? You could just use different data from the start that doesn't require you to sign an NDA.
It's not like MSR is providing a rare commodity.
It's not like MSR is providing a rare commodity.
"Why would you plan to fail?"
Because it isn't failure if your only goal is to learn.
Because it isn't failure if your only goal is to learn.
[deleted]
Absolutely nothing on this website works for me. Search gives an infinite spinner. Categories gives a blank page. What gives?
For me it was uBlocker Origin.
Are you sure that the culprit here it's uBlock Origin itself and not your configuration/filter lists?
I'm asking this because i've experienced similar problems with some pages and in most if not all cases it was the latter.
I'm asking this because i've experienced similar problems with some pages and in most if not all cases it was the latter.
Do we really need more sample data sets
Yes.
I can't really fathom the snark here. Is providing this a bad thing?
I can't really fathom the snark here. Is providing this a bad thing?
Well. In this case it happens to be a bad thing, because you have to agree to a poisonous NDA to download this data.
Providing actual free data would have been great though.
Providing actual free data would have been great though.
I think the commenter above you missed that it's not "sample data", but curated datasets for machine learning tasks, which we most certainly need more of.
Not sure if it still does or not, as I'm in a different DB engine entirely these days, but I kind of grinned with a wee bit of nostalgia checking out this site thinking of that. I cut my teeth on that data set.
Edit: Son of a gun they still offer it https://docs.microsoft.com/en-us/dotnet/framework/data/adone...