It’s not common, but it does happen. Andy Weir, author of “The Martian” and “Project Hail Mary”, originally gave his work away for free online on his website. He only self-published to Kindle (for the lowest possible price setting, 99 cents) because some of his fans didn’t know how or didn’t want to manually install his home-rolled ePubs on their devices, and begged him for the Amazon/Kindle distribution.
The US federal government is in yet another shutdown right now, so how would some sub-agency even know if there were an unplanned outage, who would report it, and who would try to fix it?
Ancestry has a somewhat smaller copy of the BIRLS database online, covering just the years 1850-2010 [1], and it seems to have been published on their website in 2011:
https://www.ancestry.com/search/collections/2441/
Our data set from the VA contains data through mid-2020, and was turned over to us in 2022 after undergoing extensive double-checking by the agency, including through non-public VA sources, to confirm the veterans really were all deceased. There's a paper showing the agency's methodology on our site, which we FOIAed from them.
There are a significant number of deceased veterans whose data is *not* included in the BIRLS database, because they (or their family/heirs) simply did not have any contact with the VA concerning benefits in or after the 1970s, which is when the database was first starting to be built. That is, their files almost always still exist on a warehouse shelf somewhere, but they weren't active any time in the past fifty years so they didn't get pulled and indexed into the database. You can still make a FOIA request to the agency asking for one of those files, but the VA will have a lower chance of successfully finding the file, and it usually will take longer for you to get a response.
[1] 1850 is very likely an approximation. While there are certainly deceased veterans listed in the BIRLS database who had birthdates or deathdates in the mid nineteenth century and/or service in the late nineteenth century, they are relatively few. Many of them are actually veterans with likely birthdates or deathdates in the twentieth century whose data seems to have been initially recorded by the VA with a two digit year of birth or death or enlistment/entry, and then assigned to the wrong two digit prefix, causing an incorrect four digit year of birth or death or year of entry/enlistment into service.
In other words, the VA's historic data is very messy and is a great example of an actual Y2K issue.
Actually, “op and crew” were the Plaintiffs (well, really the Petitioners, to be pedantic) who already sued the VA for this database in federal court (SDNY), and won that multi-year lawsuit, and even won our attorneys fees too. If you had checked our website, you’d see we even posted the court papers online for free, from both sides — and the judge’s order in our favor, of course.
Yes, a terrible fire, although there are efforts ongoing to restore some of those files, even reading the data from charred papers and edges with newer technology.
However, these particular files (benefits claims files, or C-Files) are a different type of file and never burned. Better yet, they often have some parts of the veteran’s OMPF that were copied *into* the C-File, to establish eligibility for those benefits — copies that were made before the fire! In other words, these files could serve as partial backups…
We love MuckRock! And we made the original FOIA request to the VA for this dataset via MuckRock’s platform. You can see the actual screenshot in the “Reclaiming These Records” legal papers section. They also get a shoutout in our colophon for indirectly inspiring the FOIA-by-fax-via-web method, although I believe their site uses e-mail, including interfacing with agency FOIA portals when possible.
Indeed. This is ABSOLUTELY not the first time we’ve dealt with a government agency (at the local, state, or federal levels) providing a copy of a public dataset to Ancestry.com and not to the general public. Our taxpayer-funded data keeps ending up solely behind a $300/year paywall. It’s not fair.
(Also, the stripped-down version of BIRLS that has been on the Ancestry website for a while now is much smaller and older.)
Yes, that’s on purpose. SSNs of *deceased* people are public, not private. They are never reused. They are available under FOIA from other sources as well, such as the SSDMF.
The processes for getting these very particular records (C-Files, as opposed to something like an OMPF or other better known military records) has been horrendously broken for years. They were almost completely inaccessible from this specific agency (the VBA, inside the VA) their entire existence. Only 5% of the files have been turned over to NARA, even for records that are very old.
And even now, the “processes” to get the records, as defined by a 58+-year-old law (FOIA) are not really being followed. An agency refusing to process any FOIA requests except by fax (!) is insane, in this day and age. But more specifically, it’s against the law. A letter AND an e-mail are supposed to work. Hence our use of a fax API on this website…
Furthermore, the “requirement” that a FOIA requester must hand-sign the paperwork is absolutely made up by this agency. Hence our signature widget on this website…
Point being, if they’re going to shamelessly ignore or misinterpret the federal law, we are going to just jump through those hoops and say no, we want the files, please do your jobs.
The VA worked to confirm that everyone in this dataset is deceased, in order to satisfy the judge’s order, and produced an internal document about how they did it — which we then FOIAed and posted online too. (It’s up on the site, next to the legal paperwork.) The veterans and their SSNs are believed to have been deceased prior to mid-2020, checked by the VA’s internal datasets as well as public data sets such as the SSDMF. And SSNs of deceased people are *not private*, since they are never reused. The Social Security Administration also makes copies of all deceased peoples’ original SS-5 applications available to the public under FOIA.
Thanks. The original data set, as provided by the VA, has all sorts of data errors and oddities in it. The major ones involving surnames include the inconsistent use of apostrophes in names like O’BRIEN, often written as O BRIEN, and/or vice versa — or the inconsistent formatting of MC and MAC names like MCMAHON as MC MAHON, and/or vice versa. There are also some names where the VA includes an errant dash, not meant to be a hyphen, and other mistakes, as well.
So we try our best to help a user find the veteran even with the dirty data we have. For example, there is code here (using a common NPM package) to convert a user’s potential typed accent marks to a non-accented version of the same letter. In compound surnames we will also break up the surname on a space or a hyphen and search both parts, but not if a surname part is three letters or fewer. It’s imperfect but we have to work with the data we’ve got and can’t and shouldn’t normalize or clean the underlying file.
“The second issue is return-to-office policies. Though Times management has told the bargaining unit that it has no plans to change the current hybrid-work model, Goran Svorcan-Merola, a software engineer who is on the Tech Guild’s bargaining committee, said that management wants to reserve the right to eventually have tech employees in the office five days a week. “It’s not the standard around the tech industry,” he said.”
More of this RTO bullshit from upper management? For a tech team?! Ugh. Solidarity with the Tech Guild.
Have they turned those squishy-sounding different views into horrifically inhumane absolute laws that affect every female person in the state, passing through the state, or considering taking a job (or keeping an existing job) in the state?
And yet the elected leaders of Arkansas purposely rejected a bill that would have allowed these children -- excuse me, these edge cases -- to be allowed abortions if they were raped. They voted the bill down eighteen months ago.
Words mean things, and the four-week fetus of a raped eleven-year-old is not a “baby”. Medical care to terminate that pregnancy and that fetus is literally medical care — and is illegal in Arkansas.
Arkansas currently has a 100% total abortion ban. They do not even allow abortion at any week if you’re raped, if you’re a child who has been raped, or if you’re a victim of incest. No woman in her right mind would ever move there, and this CTO is female.
There's an established organization called Open Collective that can let you support various open source software projects, with the organization acting as a legal and fiscal sponsor. It's not a 501c3 but it is a 501c6.
You can also support some groups that use open data philosophies or methods but aren't necessarily or primarily building software, like ArchiveTeam:
https://opencollective.com/archiveteam