The People's Code(code.gov)
code.gov
The People's Code
https://code.gov/#/
199 comments
I'm impressed with how quickly they responded to [3], but unfortunately the answer was exactly what I expected on this one.
> We fought the law and the law won unfortunately. We're hoping to get more leeway on this in the near future from the legal team. Stay tuned!
Lawyers. The answer is always lawyers.
> We fought the law and the law won unfortunately. We're hoping to get more leeway on this in the near future from the legal team. Stay tuned!
Lawyers. The answer is always lawyers.
> Lawyers. The answer is always lawyers.
The law is just software for people.
The law is just software for people.
Getting a quote from The Clash from the US Government, that's actually pretty awesome.
The Clash?
"I Fought the Law" is a song written by Sonny Curtis of the Crickets and popularized by a remake by the Bobby Fuller Four, which went on to become a top-ten hit for the band in 1966 and was also recorded by the Clash in 1979.
Yes indeed you're correct. Another nice theory of mine ruined by actual evidence. I'm still hoping that the guy making the comment got it from The Clash though, just cause I'm biased towards them.
The past.
Breaking rocks in the hot Sun
I really hope this sort of promotion of innovating in the tech sector remains growing.. with the current political climate, I doubt many sectors (besides military/finance) will see governmental spending growth in coming years.
If you're a US citizen, vote for someone that will keep it going. Some have promised to undo a lot that the current administration has done, and this might be part of that.
I'm not aware of any free code from Trump so far, btw.
Also, when his campaign has tried to code, they fail:
http://qz.com/762424/trumps-campaign-donation-website-used-o...
"A programmer named Shu Uesugi, an engineer at a California company called EdSurge, discovered a major flaw with the way Trump’s website was using jQuery. Instead of downloading the open-source code from GitHub and running it off a server they controlled, the developers who built Trump’s website simply ran the code off GitHub directly, Uesugi found.
While the code’s location might seem like a minor detail, running it off GitHub meant that the person who controlled the code on GitHub could change the code at his whim, and those changes would take hold on Trump’s website. Since GitHub is for open-source projects, it also meant that any user could submit a request to modify the code and impact Trump’s website, if the change was approved by the plug-in’s author, a developer in Lisbon named Igor Escobar."
Then Igor tweeted about how he could have modified it: https://twitter.com/igorescobar/status/766367306662440960?re...
I'm not aware of any free code from Trump so far, btw.
Also, when his campaign has tried to code, they fail:
http://qz.com/762424/trumps-campaign-donation-website-used-o...
"A programmer named Shu Uesugi, an engineer at a California company called EdSurge, discovered a major flaw with the way Trump’s website was using jQuery. Instead of downloading the open-source code from GitHub and running it off a server they controlled, the developers who built Trump’s website simply ran the code off GitHub directly, Uesugi found.
While the code’s location might seem like a minor detail, running it off GitHub meant that the person who controlled the code on GitHub could change the code at his whim, and those changes would take hold on Trump’s website. Since GitHub is for open-source projects, it also meant that any user could submit a request to modify the code and impact Trump’s website, if the change was approved by the plug-in’s author, a developer in Lisbon named Igor Escobar."
Then Igor tweeted about how he could have modified it: https://twitter.com/igorescobar/status/766367306662440960?re...
> Since GitHub is for open-source projects, it also meant that any user could submit a request to modify the code and impact Trump’s website, if the change was approved by the plug-in’s author, a developer in Lisbon named Igor Escobar."
That's how it works for any open source project you use, regardless of where it's been hosted. Unless you review the entire codebase (as well as all changes made in new releases), you're trusting the maintainers' judgement.
That's how it works for any open source project you use, regardless of where it's been hosted. Unless you review the entire codebase (as well as all changes made in new releases), you're trusting the maintainers' judgement.
That's not exactly the same as choosing to use something that is hosted on someone else's server, which they could then subsequently modify now that you are using it in a very high-profile project.
Of course, judging candidates' by the code quality of their campaign websites is a rather obscure and somewhat useless pastime.
Of course, judging candidates' by the code quality of their campaign websites is a rather obscure and somewhat useless pastime.
True (on both counts). And yet it's common practice to include libs, fonts, and other bits from third party sites that the dev has no control over
It's a stupid and lazy practice. It's common because most web developers aren't exactly highly trained specialists who know what they're doing.
I agree that it's still bad practice. My main objection was to the statement "Since GitHub is for open-source projects, it also meant that any user could submit a request to modify the code and impact Trump’s website". If don't trust the maintainers' judgement in merging PRs, hosting it yourself isn't a solution (short of reviewing the entire project yourself).
I think the concern here is that the maintainer could subsequently merge a malicious PR knowing who was using the library from GitHub. That wouldn't be an issue if that group was hosting a version themselves (before the maintainer might find out who was using it).
I agree it's overblown (based on the article).
Was it sourced from a particular SHA or a "latest" link?
Was it sourced from a particular SHA or a "latest" link?
Oddly enough, it appears to have linked to the project's Github Pages site: 'https://igorescobar.github.io/jQuery-Mask-Plugin/js/jquery.m... (see https://web.archive.org/web/20160817080309/https://secure.do... for the original page).
This still wasn't a good idea, but for a different reason - it's relying on the demo at https://igorescobar.github.io/jQuery-Mask-Plugin/ continuing to exist, and continuing to host the plugin at that same path.
This still wasn't a good idea, but for a different reason - it's relying on the demo at https://igorescobar.github.io/jQuery-Mask-Plugin/ continuing to exist, and continuing to host the plugin at that same path.
Full disclosure, this mistake was not made by the campaign directly, but rather by Revv.co, who is the payments software provider (not processor, DJT is using Stripe).
I admire the work that 18F is doing, and live in the DC area, even contracted for the government early in my career -- but I can never go back. I just fundamentally disagree with the security clearance apparatus is in place and never want to subject myself to that again. Especially when purely private companies pay more and don't care what you do in your personal time.
Working for the government felt like being a child again.
Working for the government felt like being a child again.
Thank you! I didn't have the words to express it but that is it precisely. It is infantalizing in a way that makes ball pits and arcades in startup culture seem tame
What was so bad about the security clearance aspect?
They want to know deeply personal details about your life, show up at your neighbour's and friends doors to question them, have explicit control over where you can travel on your vacation, deeply care about what you do in your free time, and mishandled all of it -- all in the vague guise of preventing blackmail and sabotage.
The blackmail and sabotage argument, as near as I can tell, is a farce because our enemies and "friends" are engaged in massive corporate espionage. If it were a real concern to them than they would use thier powers to require the same level of scrutiny on the private sector.
In addition to that they say be truthful so you don't have anything to hide to prevent possible blackmail by foreign powers, but in reality it's the truthful ones who don't make it through the process.
I've come to the conclusion that it's really just about having an excuse deny employment to "people not like us."
The blackmail and sabotage argument, as near as I can tell, is a farce because our enemies and "friends" are engaged in massive corporate espionage. If it were a real concern to them than they would use thier powers to require the same level of scrutiny on the private sector.
In addition to that they say be truthful so you don't have anything to hide to prevent possible blackmail by foreign powers, but in reality it's the truthful ones who don't make it through the process.
I've come to the conclusion that it's really just about having an excuse deny employment to "people not like us."
Its pretty backwards if you think about it, if the places you go or people you know or mistakes you make, can easily make you lose your security clearance and therefore your career could be ruined, it makes you much easier to blackmail than under a system that was more laissez-faire
I think the idea is more like if you are able to pass those clearances , you are less likely to find yourself in a future position that would compromise them. Similar to the belief that a high credit score shows that you're more likely to pay your bills and not abuse your credit - because you haven't so far.
Hmmm... If that were true we'd have lots of empirical evidence, so I'd like to see some link to that.
> we'd have lots of empirical evidence
That assumes there would be many attempts at this kind of blackmail, and that any attempts that did occur were discovered and documented.
> I'd like to see some link to that.
Even if the conditions above were satisfied (which I find highly unlikely), this assumes that we (the public) have access to the documentation.
That assumes there would be many attempts at this kind of blackmail, and that any attempts that did occur were discovered and documented.
> I'd like to see some link to that.
Even if the conditions above were satisfied (which I find highly unlikely), this assumes that we (the public) have access to the documentation.
Man, and I though Snow Crash's portrayal of the feds was wildly exaggerated.
Guess it was significantly less exaggerated that I thought.
Guess it was significantly less exaggerated that I thought.
> show up at your neighbour's and friends doors to question them
I think there's a lot of people that would like the associated intrigue that would come with this, it may be a cheaper way of hiring good developers that have a bit of a vanity streak.
I think there's a lot of people that would like the associated intrigue that would come with this, it may be a cheaper way of hiring good developers that have a bit of a vanity streak.
Then after collecting your deeply personal information, they let it leak through poor security practices. This has happend to me and to other members of my family: https://en.wikipedia.org/wiki/Office_of_Personnel_Management...
The explicit and everpresent awareness that you are owned by your employer in your off hours, for a limited sub-selection of possible activities.
In other words, you are not your productivity, you are required to live their moral code in your off hours.
In other words, you are not your productivity, you are required to live their moral code in your off hours.
Ironically, the Federal Government still ends up paying the price despite all of their efforts. I know around 10 top tier programmers who use drugs of some sort, have left of mainstream politics or just don't fit the "government worker" bill, but are no less patriotic and competent people. The loss of these people from the upper echelons of the state tech system is almost palpable. The CIA director was hacked by a bunch of teenagers from Eastern Europe, for Christ's sake!
And then on top of that, even with all the screening they do, they still stop the Snowdens from infiltrating, still can't stop foreign intelligence, still can't get a sysadmin worth their salt to keep our future President's clandestine email server safe from the public's prying eyes!
And then on top of that, even with all the screening they do, they still stop the Snowdens from infiltrating, still can't stop foreign intelligence, still can't get a sysadmin worth their salt to keep our future President's clandestine email server safe from the public's prying eyes!
That is...most jobs. Unless I'm not understanding your meaning. Many jobs will fire you for, say, getting a DUI or getting arrested for drug possession or something. And many employers do drug testing. But realistically, a "limited sub-selection of possible activities" is just about everything everyone else can do, except maybe go to Cuba or smoke pot, the latter of which will get one fired at many non-government places.
Is drug testing always legal in the US? Can drug use on weekends get you fired for any job?
I lived in several European countries, and as far as I know there's no way an employer can force you to take a drug test, with the exception of people handling heavy machinery. If you are on drugs while you're at your job this will of course have consequences, but a drug test that would reveal drug use in your free time is nothing the employer should be allowed to do or even care about.
As an example, the rules in the UK clearly say that you need consent and that it has to be required by the nature of the job, which should exclude most (if not all) office jobs. [1]
[1] https://www.gov.uk/monitoring-work-workers-rights/drug-testi...
I lived in several European countries, and as far as I know there's no way an employer can force you to take a drug test, with the exception of people handling heavy machinery. If you are on drugs while you're at your job this will of course have consequences, but a drug test that would reveal drug use in your free time is nothing the employer should be allowed to do or even care about.
As an example, the rules in the UK clearly say that you need consent and that it has to be required by the nature of the job, which should exclude most (if not all) office jobs. [1]
[1] https://www.gov.uk/monitoring-work-workers-rights/drug-testi...
You can be fired for almost any reason in the US. There are only a few exceptions such as race, religion, sex or for reporting violations of some laws.
So, yes, in almost any job you could be required to take a drug test and be fired if you refuse.
So, yes, in almost any job you could be required to take a drug test and be fired if you refuse.
Smoking weed is not going to get you fired anywhere, except for the worst jobs in the worst places. Any place that does regular drug testing and/or cares one teeny tiny itsy bitsy bit about employees smoking weed when they're not working is guaranteed to be a terrible place to work at, for that and any number of other reasons. There is not a single reputable company in the entire tech industry that does this.
Uh, yeah, "in the tech industry" being the huge caveat here. Most HN readers might work in tech, but most people in general don't.
> Smoking weed is not going to get you fired anywhere, except for the worst jobs in the worst places.
Do you have numbers to back this up? Because almost every place I've worked at has had this policy. I think it's a terrible policy and should be illegal, but it's been pretty standard everywhere. Maybe it's not common in SV, but everywhere else it seems to be the norm, IME.
Do you have numbers to back this up? Because almost every place I've worked at has had this policy. I think it's a terrible policy and should be illegal, but it's been pretty standard everywhere. Maybe it's not common in SV, but everywhere else it seems to be the norm, IME.
It has never been an issue for me as I don't partake, but anecdotally, I've worked at 5-10 mid range (100-200) firms in Virginia and Illinois and while I have been provided many free beers at work, if anyone suggested a drug testing program they would have been laughed out of the building.
I even worked for a company that ran drug testing programs for the government and they did not drug test.
I even worked for a company that ran drug testing programs for the government and they did not drug test.
That is definitely not true:
http://money.cnn.com/2015/06/15/news/companies/dish-employee...
http://money.cnn.com/2015/06/15/news/companies/dish-employee...
I think you just proved his point.
http://cable.tmcnet.com/topics/cable/articles/2013/01/07/321...
"Out of 617 ratings on glassdoor.com, fully 246 called themselves “very dissatisfied.” The number of employees considering themselves “very satisfied” was just 48. Some comments even suggest that executive meddling may be involved in the “very satisfied” scores, with one commenter saying “Joe Clayton (CEO) put us up to upgrading our score.” Another commenter called their time with Dish “...like a prison sentence.”
http://cable.tmcnet.com/topics/cable/articles/2013/01/07/321...
"Out of 617 ratings on glassdoor.com, fully 246 called themselves “very dissatisfied.” The number of employees considering themselves “very satisfied” was just 48. Some comments even suggest that executive meddling may be involved in the “very satisfied” scores, with one commenter saying “Joe Clayton (CEO) put us up to upgrading our score.” Another commenter called their time with Dish “...like a prison sentence.”
A few people are a little disgruntled that the government gave all of the information in our security clearance applications/re-ups to China. That's my beef, I'm sure other folks have other things to say.
[deleted]
How about open source code for some makes and models of voting machine?
The US Veterans' Administration health records software system is in the public domain. https://en.wikipedia.org/wiki/VistA#Licensing_and_disseminat... But it's not listed here. (It's also kind of complex. "wget; tar x; ./configure ; make" probably won't get you a running instance. Still.
The US Veterans' Administration health records software system is in the public domain. https://en.wikipedia.org/wiki/VistA#Licensing_and_disseminat... But it's not listed here. (It's also kind of complex. "wget; tar x; ./configure ; make" probably won't get you a running instance. Still.
Voting machines are products produced by private firms. Getting them to publish the source code is about as likely as getting Apple to publish the source for iOS.
Actually, with monopsony it's rather simple:
"If you don't publish the code, then we will have to use paper ballots."
"If you don't publish the code, then we will have to use paper ballots."
No effort towards innovation would then go into that sector. Ever again.
I'm not sure if this it's a popular decision, but I don't really think the private sector should be innovating with our electoral system. Integrity is of utmost importance, and voting machines have been proven vulnerable in the past. I don't think the private sector has appropriate motivation to really get voting right.
The difficulty is the purse strings to pay for election infrastructure comes from politicians. Private firms have funds for product advocacy without advance funding from government, unlike open source solutions. So the entire field is distorted from day 0 when it comes to proprietary vs open source.
A smarter thing to do would be to create a selection pool defined as any registered voter over the age of 75; and from that pool they choose their election technology board. And that board procures the election system based on the budget the legislature gives them. That is, the budget is the budget only, it doesn't say anything about what technology or brand is used, the only attachment is that funds are used for the state election system. If someone over 75 can't understand how our election system works, we're screwed, and in fact we've seen how voting systems have failed, have altered the outcome of elections, and because they had no paper trail, and no meaningful way to do recounts or audits there's no recourse.
A smarter thing to do would be to create a selection pool defined as any registered voter over the age of 75; and from that pool they choose their election technology board. And that board procures the election system based on the budget the legislature gives them. That is, the budget is the budget only, it doesn't say anything about what technology or brand is used, the only attachment is that funds are used for the state election system. If someone over 75 can't understand how our election system works, we're screwed, and in fact we've seen how voting systems have failed, have altered the outcome of elections, and because they had no paper trail, and no meaningful way to do recounts or audits there's no recourse.
The question is about who you trust and how you make sure that all the actors are being fair.
Would you feel better if the government designed the machines? A lot of people would probably feel worse.
Right now, you have each state's board of elections making decisions - they get observed by both parties so that they can all sign off on what's happening. Idk. You can never eliminate the possibility of fraud. I just think the likelihood of all of these actors colluding is really low... I mean congress can't even pass a budget.
Would you feel better if the government designed the machines? A lot of people would probably feel worse.
Right now, you have each state's board of elections making decisions - they get observed by both parties so that they can all sign off on what's happening. Idk. You can never eliminate the possibility of fraud. I just think the likelihood of all of these actors colluding is really low... I mean congress can't even pass a budget.
Integrity being of the utmost importance is probably the #1 reason why the people put in power by elections should not be in charge of running those elections.
I'm not saying I prefer Diebold, but I certainly prefer it if the only other option is the House/Senate $PARTY Caucus.
I'm not saying I prefer Diebold, but I certainly prefer it if the only other option is the House/Senate $PARTY Caucus.
I, as a taxpayer/legislator, can't contract a private company to (for a token fee) write and support a codebase that they will (as part of the contract) open source? The people win, as we get open source and hopefully transparent code. The company wins, as they're paid for their work (as they should be).
I fully expect that this would be more expensive than closed source.
(I'm not convinced that we should be using electronic voting at all, but that's a different debate.)
I fully expect that this would be more expensive than closed source.
(I'm not convinced that we should be using electronic voting at all, but that's a different debate.)
Well, ostensibly, given enough time, we'd end up with a single code base, where the modular add-ons are really just about user interface, similar to how every county in the U.S. does their own ballot layout. In the old days, pen and paper could be considered to be a single code base. If you get the basic thing to do right, it's a choice, not a necessity, to have different code bases that do the same damn thing.
So eventually it would be less expensive than closed source, to implement and maintain. Basically wealthier and more ideologically committed states would subsidize that reference implementation, and the rest of the country, should they adopt that open source reference, would get to use the code more cheaply.
From a global perspective, there's a pretty decent chance a huge chunk of the code is already done - i.e. it could use the Linux kernel with a bunch of its more extraneous modules disabled. A handful of math libraries. A handful of UI libraries. And then a code signing mechanism from stem to stern. There's a lot more to it than that, but code wise, a bunch of it might already be written, the work is not innovating ducks but getting them in rows, and documenting it in a way that it's reproducible and ideally only boots up when it's exactly conforming to the spec.
Closed source with multiple competing companies nationally by definition means many different code bases all in different maintenance states. It really is wheel reinvention over and over again.
So eventually it would be less expensive than closed source, to implement and maintain. Basically wealthier and more ideologically committed states would subsidize that reference implementation, and the rest of the country, should they adopt that open source reference, would get to use the code more cheaply.
From a global perspective, there's a pretty decent chance a huge chunk of the code is already done - i.e. it could use the Linux kernel with a bunch of its more extraneous modules disabled. A handful of math libraries. A handful of UI libraries. And then a code signing mechanism from stem to stern. There's a lot more to it than that, but code wise, a bunch of it might already be written, the work is not innovating ducks but getting them in rows, and documenting it in a way that it's reproducible and ideally only boots up when it's exactly conforming to the spec.
Closed source with multiple competing companies nationally by definition means many different code bases all in different maintenance states. It really is wheel reinvention over and over again.
We should be using electronic voting to tabulate and aggregate the data, but at the most local level personally I think electronic but with two paper receipts (one to the voter with who they voted for, one to the state w/o) is the best option.
How much Innovation is there? Its a voting booth, not an iphone.
It should be a solid device with open-source code deep enough to validate counting.
Hmm. No effort, ever again, in open source products? I wonder what RMS, Linus Torvalds, and the BSD folks have to say about that?
Seriously, the open source movement has generated some pretty good businesses. Maybe the case can be made that electoral equipment is an exception to that.
But asserting the proposition doesn't prove it.
Seriously, the open source movement has generated some pretty good businesses. Maybe the case can be made that electoral equipment is an exception to that.
But asserting the proposition doesn't prove it.
>No effort towards innovation would then go into that sector. Ever again.
Good. You're talking about something that needs to handle a small handful of registers perfectly and produce a paper audit trail. Innovation is not only not needed, it should be actively suppressed due to the risk to the security of the electoral process.
Good. You're talking about something that needs to handle a small handful of registers perfectly and produce a paper audit trail. Innovation is not only not needed, it should be actively suppressed due to the risk to the security of the electoral process.
Linux seems to be doing fine.
That used to be true. The market is being disrupted by new open source models. Cost of election tech should drop precipitously.
That would not improve things.
A voting system must be able to be checked by any citizen at any moment.
Most of the citizen can't:
- understand a complex code.
- ensure the machine is running this code, easily, conveniently or quickly.
- ensure the machine is not, or will not be hacked, by software or hardware mean.
- ensure the program does calculate the votes correctly, with no bug.
- do any of this in an autonomous and spontaneous way required to make a check healthy.
A voting system must be able to be checked by any citizen at any moment.
Most of the citizen can't:
- understand a complex code.
- ensure the machine is running this code, easily, conveniently or quickly.
- ensure the machine is not, or will not be hacked, by software or hardware mean.
- ensure the program does calculate the votes correctly, with no bug.
- do any of this in an autonomous and spontaneous way required to make a check healthy.
Don't let perfect be the enemy of good. Opening up code in voting machine would be leagues better than leaving it closed source. At least other experts in the field could verify it.
Its also not like every citizen today can or understand all of the vote counting procedures with paper ballots, even when just constrained to their ballot.
Its also not like every citizen today can or understand all of the vote counting procedures with paper ballots, even when just constrained to their ballot.
Oh yes, I don't discourage it.
I just think that you can't be satisfy with it. Open source voting machine are still an abomination for the democratic process.
I just think that you can't be satisfy with it. Open source voting machine are still an abomination for the democratic process.
remind me again how the open-source part makes anything worse than a standard voting machine?
I think it's an electronic voting machine in general. There must be a paper ballot trail. The absolute best setup for voting is the scan-tron style paper ballot. Immediate confirmation, quick and continuous vote tallies, errors are very rare and it includes a paper trail.
It does not. Read the entire thread before being offended.
Most people can't understand the code in openssl. So we shouldn't use it? Let's just make all cryptography closed source then. The average person doesn't even know what an elliptic function is.
The point is that there is a large amount of people that do. They check. Not every citizen needs to check, but it is harder for there to be an error or to hoodwink someone if there are more eyes on the code. Essentially why opensource cryptography is good too.
Essentially, it can easily be audited.
The point is that there is a large amount of people that do. They check. Not every citizen needs to check, but it is harder for there to be an error or to hoodwink someone if there are more eyes on the code. Essentially why opensource cryptography is good too.
Essentially, it can easily be audited.
Crypto and democracy can't be compared.
You can delegate advanced science, you should not delegate democracy, because it's the root of all the rest.
You can delegate advanced science, you should not delegate democracy, because it's the root of all the rest.
Open source voting software isn't delegating democracy, it is auditing the tool that is being used.
Good point, @godelski
I've been thinking about this in the past few days, and decided to make a petition about it.
https://wh.gov/iervT
tl;dr lets develop public-domain voting systems so we can audit them.
https://wh.gov/iervT
tl;dr lets develop public-domain voting systems so we can audit them.
> The US Veterans' Administration health records software system is in the public domain
This was programmed by humans?
http://code.osehra.org/gitweb/?p=VistA-M.git;a=blob_plain;f=...
http://code.osehra.org/gitweb/?p=VistA-M.git;a=tree;f=Packag...
Was released after obfuscating?
This was programmed by humans?
http://code.osehra.org/gitweb/?p=VistA-M.git;a=blob_plain;f=...
http://code.osehra.org/gitweb/?p=VistA-M.git;a=tree;f=Packag...
Was released after obfuscating?
https://github.com/samilliken/openDCIM/search?utf8=%E2%9C%93...
https://github.com/USEPA/E-Enterprise-Portal/search?utf8=%E2...
Looks like the US Government is oblivious to the risk of PHP Object Injection.
https://paragonie.com/blog/2016/04/securely-implementing-de-...
It looks like they're publishing code that, in particular, is vulnerable to CVE-2015-2171.
https://github.com/samilliken/openDCIM/blob/d3e137294179e392...
https://github.com/USEPA/E-Enterprise-Portal/search?utf8=%E2...
Looks like the US Government is oblivious to the risk of PHP Object Injection.
https://paragonie.com/blog/2016/04/securely-implementing-de-...
It looks like they're publishing code that, in particular, is vulnerable to CVE-2015-2171.
https://github.com/samilliken/openDCIM/blob/d3e137294179e392...
You should send in a pull request, or file a bug report in the repo.
I'm not on their payroll, and I'd already fixed the CVE upstream over a year ago, so no.
You took the time to go search through those repos, and then took the time to come here and point out your findings.
You could have easily just copy pasted what you wrote here into an Issue on those repos.
It would have helped out your government, made these projects a tad bit better, and taught a few developers something they'll remember in the future when they write more software for the government.
It's part of the "spirit" if you will. Let's support these efforts - they're long overdue and difficult enough to get done for those few trying.
You could have easily just copy pasted what you wrote here into an Issue on those repos.
It would have helped out your government, made these projects a tad bit better, and taught a few developers something they'll remember in the future when they write more software for the government.
It's part of the "spirit" if you will. Let's support these efforts - they're long overdue and difficult enough to get done for those few trying.
Reporting issues to a vendor takes a hell of a lot of effort and time compared to idly posting them in a comment thread.
This is like those remarks along the lines of "it only takes 5 minutes to submit a patch to the documentation". No, it doesn't - it takes 5 minutes to write it and create a PR, and usually two hours to defend it from the maintainer's criticisms.
This is like those remarks along the lines of "it only takes 5 minutes to submit a patch to the documentation". No, it doesn't - it takes 5 minutes to write it and create a PR, and usually two hours to defend it from the maintainer's criticisms.
>usually two hours to defend it from the maintainer's criticisms
Isn't that what we want, though? Discussion and review, particularly with the people who are most familiar with the project (the maintainers) is what makes software better, not worse. If you're not willing to do that then why comment on issues you find in open source projects at all, other than to stroke your own ego?
Isn't that what we want, though? Discussion and review, particularly with the people who are most familiar with the project (the maintainers) is what makes software better, not worse. If you're not willing to do that then why comment on issues you find in open source projects at all, other than to stroke your own ego?
No, it's not. Legitimate discussion and review, sure - but that doesn't take two hours. The problem is with maintainers arguing over small changes, trying to trivialize them, finding reasons to reject them, asking contributors to do more work first... It's pretty much impossible to just leave a contribution and move on.
This is something I encounter constantly. It's why I've stopped bothering submitting documentation fixes in most cases, and now resort to writing and publishing my own documentation. It makes one-off contributions to open-source unreasonably costly and non-viable. I know many others who feel the same.
If maintainers want more contributions to their open-source projects, they need to start recognizing that the contributor's time and skills are valuable as well - that means not second-guessing contributions unless there's a concrete reason to do so, and making an effort to write a comprehensive response to a contribution (especially if it's a rejection). And most of all, being clear about this policy.
It's easy to say "why don't you submit it to the project?" from the sidelines, but the reality is that 'formal' contributions nearly always come with an expectation of dedicating your time and attention towards them on a more-or-less ongoing basis. Many people cannot afford that, and as such publish them elsewhere.
Changing that starts with maintainers, not with contributors.
This is something I encounter constantly. It's why I've stopped bothering submitting documentation fixes in most cases, and now resort to writing and publishing my own documentation. It makes one-off contributions to open-source unreasonably costly and non-viable. I know many others who feel the same.
If maintainers want more contributions to their open-source projects, they need to start recognizing that the contributor's time and skills are valuable as well - that means not second-guessing contributions unless there's a concrete reason to do so, and making an effort to write a comprehensive response to a contribution (especially if it's a rejection). And most of all, being clear about this policy.
It's easy to say "why don't you submit it to the project?" from the sidelines, but the reality is that 'formal' contributions nearly always come with an expectation of dedicating your time and attention towards them on a more-or-less ongoing basis. Many people cannot afford that, and as such publish them elsewhere.
Changing that starts with maintainers, not with contributors.
> Isn't that what we want, though?
Generally. But on the other hand, no, this isn't a monoculture, where everyone wants the same exact thing.
> Discussion and review, particularly with the people who are most familiar with the project (the maintainers) is what makes software better, not worse.
Tell that to vim: https://github.com/vim/vim/issues/638
Tell that to node.js: https://github.com/nodejs/node/issues/5798
Tell that to WordPress: https://core.trac.wordpress.org/ticket/21022 https://core.trac.wordpress.org/ticket/25052 etc.
Despite all of the time that industry experts have contributed towards identifying and discussing security problems, these projects have severe inertia in the security realm.
How much more time do you want us to carve out of our lives for projects that won't fix their problems?
> If you're not willing to do that then why comment on issues you find in open source projects at all, other than to stroke your own ego?
I don't like the way this is framed. It's a hidden false dichotomy.
Generally. But on the other hand, no, this isn't a monoculture, where everyone wants the same exact thing.
> Discussion and review, particularly with the people who are most familiar with the project (the maintainers) is what makes software better, not worse.
Tell that to vim: https://github.com/vim/vim/issues/638
Tell that to node.js: https://github.com/nodejs/node/issues/5798
Tell that to WordPress: https://core.trac.wordpress.org/ticket/21022 https://core.trac.wordpress.org/ticket/25052 etc.
Despite all of the time that industry experts have contributed towards identifying and discussing security problems, these projects have severe inertia in the security realm.
How much more time do you want us to carve out of our lives for projects that won't fix their problems?
> If you're not willing to do that then why comment on issues you find in open source projects at all, other than to stroke your own ego?
I don't like the way this is framed. It's a hidden false dichotomy.
I don't disagree with your or joepie91_'s points in that many (maybe most) open source project maintainers are perhaps more guarded and difficult in accepting contributions than they should be.
That said, even if you submit a pull request or an issue ticket and the maintainer doesn't follow up, or is making the issue unnecessarily complicated to resolve formally and merge into the main branch, at least the record of your findings is there attached to the repo as a rejected pull request or a closed issue ticket, and that can help someone who comes along after the fact who maybe experienced the same issue you did and can benefit from your comments. Maybe that same person who comes along later will have the time to fight for having the changes merged, and can use your comments as further evidence that the issue is worth reconsidering.
I stand by the idea that finding the issue, but posting it on an unrelated forum (like HN) instead of attaching it to the project itself serves no purpose than to tell others who are probably not even users of the open source code "look at this problem I found/solved, aren't I great?"
That said, even if you submit a pull request or an issue ticket and the maintainer doesn't follow up, or is making the issue unnecessarily complicated to resolve formally and merge into the main branch, at least the record of your findings is there attached to the repo as a rejected pull request or a closed issue ticket, and that can help someone who comes along after the fact who maybe experienced the same issue you did and can benefit from your comments. Maybe that same person who comes along later will have the time to fight for having the changes merged, and can use your comments as further evidence that the issue is worth reconsidering.
I stand by the idea that finding the issue, but posting it on an unrelated forum (like HN) instead of attaching it to the project itself serves no purpose than to tell others who are probably not even users of the open source code "look at this problem I found/solved, aren't I great?"
> That said, even if you submit a pull request or an issue ticket and the maintainer doesn't follow up, or is making the issue unnecessarily complicated to resolve formally and merge into the main branch, at least the record of your findings is there attached to the repo as a rejected pull request or a closed issue ticket, and that can help someone who comes along after the fact who maybe experienced the same issue you did and can benefit from your comments. Maybe that same person who comes along later will have the time to fight for having the changes merged, and can use your comments as further evidence that the issue is worth reconsidering.
In my experience, this is a continuous source of frustration and anxiety. Not responding doesn't stop the notifications, and on top of that it also makes the contributor look bad by ignoring follow-ups. It's much easier and less frustrating to just not file anything in the first place.
> I stand by the idea that finding the issue, but posting it on an unrelated forum (like HN) instead of attaching it to the project itself serves no purpose than to tell others who are probably not even users of the open source code "look at this problem I found/solved, aren't I great?"
This is just flat-out bullshit, and frankly says more about you and your assumptions than about CiPHPer (who, for the record, is somebody I know personally). Have you considered that maybe CiPHPer is just tired of arguing with vendors and getting nowhere, and has now resorted to just publicizing their fuckups in the hope that maybe that will change something?
To be clear: nobody owes any vendor anything. While I personally at least attempt to disclose a vulnerability privately before going public, there is absolutely no requirement to do this. It is first and foremost the vendor's responsibility to keep their software and its dependencies up-to-date, not that of the public. Be happy that it's being reported at all, rather than just quietly sold on the black market.
The reality is that vendors have been grossly negligent of security for the past several decades, and still continue to do so, using all kinds of shit excuses about how much time and money it costs to fix something they shouldn't have fucked up in the first place. Don't be surprised when security researchers get tired of that shit, and just start dumping vulnerabilities publicly when there are indications that the vendor is negligent (like is the case here).
Again: the change here is to come from the vendors, not from the security researchers.
In my experience, this is a continuous source of frustration and anxiety. Not responding doesn't stop the notifications, and on top of that it also makes the contributor look bad by ignoring follow-ups. It's much easier and less frustrating to just not file anything in the first place.
> I stand by the idea that finding the issue, but posting it on an unrelated forum (like HN) instead of attaching it to the project itself serves no purpose than to tell others who are probably not even users of the open source code "look at this problem I found/solved, aren't I great?"
This is just flat-out bullshit, and frankly says more about you and your assumptions than about CiPHPer (who, for the record, is somebody I know personally). Have you considered that maybe CiPHPer is just tired of arguing with vendors and getting nowhere, and has now resorted to just publicizing their fuckups in the hope that maybe that will change something?
To be clear: nobody owes any vendor anything. While I personally at least attempt to disclose a vulnerability privately before going public, there is absolutely no requirement to do this. It is first and foremost the vendor's responsibility to keep their software and its dependencies up-to-date, not that of the public. Be happy that it's being reported at all, rather than just quietly sold on the black market.
The reality is that vendors have been grossly negligent of security for the past several decades, and still continue to do so, using all kinds of shit excuses about how much time and money it costs to fix something they shouldn't have fucked up in the first place. Don't be surprised when security researchers get tired of that shit, and just start dumping vulnerabilities publicly when there are indications that the vendor is negligent (like is the case here).
Again: the change here is to come from the vendors, not from the security researchers.
In this case: I did, and it was immediately closed without any change to the code, documentation, process, or culture.
> In this case: I did, and it was immediately closed without any change to the code, documentation, process, or culture.
You left out the part where they told you this perceived issue did not impact their codebase, due to how they were using the upstream project you cited.[1]
> While this is an issue it does not affect us as we aren't using session cookies with slim
You then started to rant about how they don't update their dependencies "quick enough".
[1] https://github.com/samilliken/openDCIM/issues/837
You left out the part where they told you this perceived issue did not impact their codebase, due to how they were using the upstream project you cited.[1]
> While this is an issue it does not affect us as we aren't using session cookies with slim
You then started to rant about how they don't update their dependencies "quick enough".
[1] https://github.com/samilliken/openDCIM/issues/837
> You left out the part where they told you this perceived issue did not impact their codebase, due to how they were using the upstream project you cited.
This is completely irrelevant. Vulnerable dependencies are vulnerable dependencies, and trying to avoid updates because you're "not affected" is a really good way to get owned.
This is for the exact same reason that "yes, there's an XSS vulnerability in the admin panel, but that doesn't matter because it requires an admin login" is invalid. At some point, somebody is going to combine multiple "unexploitable" vulnerabilities or exploitation paths that you overlooked, and successfully compromise your system.
A vulnerable dependency means that you need to update that dependency, full stop. No exceptions.
This is completely irrelevant. Vulnerable dependencies are vulnerable dependencies, and trying to avoid updates because you're "not affected" is a really good way to get owned.
This is for the exact same reason that "yes, there's an XSS vulnerability in the admin panel, but that doesn't matter because it requires an admin login" is invalid. At some point, somebody is going to combine multiple "unexploitable" vulnerabilities or exploitation paths that you overlooked, and successfully compromise your system.
A vulnerable dependency means that you need to update that dependency, full stop. No exceptions.
Whether it's relevant or not is up for the project maintainer to decide. All anyone can do is point out what they perceive as a potential problem, and then let the others take it from there.
CiPHPerCoder didn't exactly do that. He came out guns ablaze from the start, using bolded text, italics and inflammatory phrasing - basically just stopping short of calling the project maintainer a complete idiot.
It's not surprising CiPHPerCoder got the reaction he did - it is, however, surprising he decided to do all this under his company's name.
It's almost like CiPHPerCoder is personally offended that some joe random developer hasn't heard about some obscure CVE CiPHPerCoder was involved with, or that they didn't handle it like he would like. "Do you know who I am!?"
The initial issue was he came here and ranted about it, instead of pointing it out to the projects. Eventually he "gave in" and reported it to a single project, but took this holier-than-thou tone and was aggressive the entire time. That's ridiculous.
CiPHPerCoder didn't exactly do that. He came out guns ablaze from the start, using bolded text, italics and inflammatory phrasing - basically just stopping short of calling the project maintainer a complete idiot.
It's not surprising CiPHPerCoder got the reaction he did - it is, however, surprising he decided to do all this under his company's name.
It's almost like CiPHPerCoder is personally offended that some joe random developer hasn't heard about some obscure CVE CiPHPerCoder was involved with, or that they didn't handle it like he would like. "Do you know who I am!?"
The initial issue was he came here and ranted about it, instead of pointing it out to the projects. Eventually he "gave in" and reported it to a single project, but took this holier-than-thou tone and was aggressive the entire time. That's ridiculous.
I'd say you did your due diligence then.
[deleted]
[deleted]
I have no intention of getting into an argument with you but wow, you must be having a bad day. I wonder if you would have gotten a different response to your report if you'd taken a different tone with the people trying to understand the issue itself.
> if you'd taken a different tone
I've been thinking about this a lot lately....I think "why didn't you just use a different tone" is very similar to how people that grew up in environments where everything went right when they say to people struggling "why don't you just do x,y,z". The problem is, they don't know how.
I've been thinking about this a lot lately....I think "why didn't you just use a different tone" is very similar to how people that grew up in environments where everything went right when they say to people struggling "why don't you just do x,y,z". The problem is, they don't know how.
I agree, but with a little twist: the problem is not that the latter party doesn't know how and the former party has the answers, maybe it's that the former party doesn't know how either.
In this particular instance, the actual problem is notification. OP said he fixed the bug in his original code, and one of his downstreams should fix the bug too.
Fair enough.
But then things took a turn towards "you should do the work", "civic duty", and what not. To which OP replied he's not going to fix it in a code base he's not compensated for.
The actual problem in this scenario here isn't OP's willingness to donate his time, it's that the right person at the downstream needs to be notified (and in turn acknowledge the problem). And in the world of open source, this is a common issue which is impacted by multiple notification avenues, developer continuity, and general infosec policy.
But instead of any one of us creating an issue saying "Hey dudes, this was fixed in the upstream -- here's a copy-and-paste pointing to the fixed and respective vulnerable code", it's a pile-on with holier-than-thou mantras regarding why OP should do the free work of reading the source code, testing a fix, and creating a patch.
In this particular instance, the actual problem is notification. OP said he fixed the bug in his original code, and one of his downstreams should fix the bug too.
Fair enough.
But then things took a turn towards "you should do the work", "civic duty", and what not. To which OP replied he's not going to fix it in a code base he's not compensated for.
The actual problem in this scenario here isn't OP's willingness to donate his time, it's that the right person at the downstream needs to be notified (and in turn acknowledge the problem). And in the world of open source, this is a common issue which is impacted by multiple notification avenues, developer continuity, and general infosec policy.
But instead of any one of us creating an issue saying "Hey dudes, this was fixed in the upstream -- here's a copy-and-paste pointing to the fixed and respective vulnerable code", it's a pile-on with holier-than-thou mantras regarding why OP should do the free work of reading the source code, testing a fix, and creating a patch.
[deleted]
This is a great idea. I feel like the big picture isn't open source code for government, but open source APIs for interacting with the government. In the medium-term future, I think we should push for a system where all government documents that ought to be a matter of public record are managed in a publicly readable versioned source repository like, like a federal git server.
Other things that I'd like to see in publicly visible git repositories:
- federal laws and regulations (particularly regulations, standards, etc.).
- procedures (rules for federal processes)
- all documents for making requests of the federal government, with mechanisms for getting those requests to the right place.
Other things that I'd like to see in publicly visible git repositories:
- federal laws and regulations (particularly regulations, standards, etc.).
- procedures (rules for federal processes)
- all documents for making requests of the federal government, with mechanisms for getting those requests to the right place.
Absolutely![0]
[0] I know this doesn't contribute to the conversation and is frowned upon, but I had to exclaim when I read this.
[0] I know this doesn't contribute to the conversation and is frowned upon, but I had to exclaim when I read this.
Why the subsidy to Google? Sending tracking data to Google isn't appropriate for a government site.
<script>
/* i='GTM-M9L9Q5' */
/* ... */
src='https://www.googletagmanager.com/gtm.js?id='+i
/* ... */
</script>
<!-- ... -->
<noscript>
<iframe src="https://www.googletagmanager.com/ns.html
?id=GTM-M9L9Q5>m_auth=GTM-M9L9Q5"
height="0" width="0"
style="display:none;visibility:hidden">
</iframe>
</noscript>
Just a wild guess here, but I bet they're using Google Analytics for the same reason everyone else uses Google Analytics--to see what their audience is looking at.
> Also, requiring Javascript in a single page application is a terrible design for this kind of site.
Good luck making a single page application without Javascript! It is all in 25 requests and 668KB, not the lightest site, but that's actually less data than the homepage of Google is today. It's also properly setting expires headers so future page loads are very quick.
> Also, requiring Javascript in a single page application is a terrible design for this kind of site.
Good luck making a single page application without Javascript! It is all in 25 requests and 668KB, not the lightest site, but that's actually less data than the homepage of Google is today. It's also properly setting expires headers so future page loads are very quick.
I know why they want analytics; I'm asking why they are giving out live tracking data to a single private company. This free data is effectively a subsidy.
> Good luck making a single page application without Javascript
The bad design decision was making it a single page application in the first place. This should be a static site, or very lightweight (and easily cached) framework.
> 25 requests and 668KB
That's an obscene amount of bloat. This should be 3-5 requests not counting images, and 668kB is an order of magnitude too big.
> less data than the homepage of Google is today
Comparing against another massively over-engineered, bloated web page isn't particularly useful.
> expires headers
That's great, but it doesn't affect the initial download size or the memory/CPU usage in the user agent.
https://vimeo.com/147806338
> Good luck making a single page application without Javascript
The bad design decision was making it a single page application in the first place. This should be a static site, or very lightweight (and easily cached) framework.
> 25 requests and 668KB
That's an obscene amount of bloat. This should be 3-5 requests not counting images, and 668kB is an order of magnitude too big.
> less data than the homepage of Google is today
Comparing against another massively over-engineered, bloated web page isn't particularly useful.
> expires headers
That's great, but it doesn't affect the initial download size or the memory/CPU usage in the user agent.
https://vimeo.com/147806338
> I know why they want analytics; I'm asking why they are giving out live tracking data to a single private company. This free data is effectively a subsidy.
How is it a subsidy? They are getting the analytical data they want and are not having to pay anything for it. They could spend tons of time and money to build an analytics system, but that sounds like a waste of tax dollars. You'll also notice that their code is hosted on GitHub, is that a subsidy too?
How is it a subsidy? They are getting the analytical data they want and are not having to pay anything for it. They could spend tons of time and money to build an analytics system, but that sounds like a waste of tax dollars. You'll also notice that their code is hosted on GitHub, is that a subsidy too?
That's not what subsidy means.
Mobile browsers don't always keep things until the server says they expire. That being said a frequent user will probably end up saving data in the long run.
Looks like the US government makes widescale use of Google analytics, so it's hardly inappropriate: https://analytics.usa.gov/
Pointing out more instances of a problem doesn't make the problem "hardly inappropriate"; it indicates the problem is more widespread.
I hear they use Ford vehicles and Boeing airplanes as well. Highly inappropriate for the US Government to use products from some of the US's biggest companies!
/s
/s
Analytics makes websites better, and Piwik is not easy or cheap to use at scale. Sharing web traffic data with Google Analytics (especially with IPs anonymized) is a pretty small issue IMO, especially when the benefits are you get good data like what's on analytics.usa.gov.
(Disclosure: I work on analytics.usa.gov.)
(Disclosure: I work on analytics.usa.gov.)
> (especially with IPs anonymized)
While I'm sure you have good intentions in trying to anonymize the collected data, you should know that the claims made by Google at [1] are highly misleading. Their claim is that they strip the last octet of the IP address, but that isn't nearly enough cooking to call the data "anonymous".
Assuming Google actually does this and isn't simply lying about saving the full IP (which has to be sent to them, as it is obviously in the SRC field of the IP header), this means they are only binning IPs into groups of 256. They only need at most 8 bits of identifying data other than the IP to uniquely correlate individuals within each IP group.
More than 8 bits of unique ID are available, given that various types of data that being tracked[2] about the user agent and (presumably) IP geolocation.
Also, Google is keeping enough of the IP to lookup the ASN.
> I work on analytics.usa.gov.
So if you're actually interested in claiming that "The program does not track individuals, and anonymizes the IP addresses of visitors"[2], then you really shouldn't be using an analytics service that - in spite of their claims of anonymizing" some data - obvious is tracking individuals and saving the most interesting parts of their IP address.
> Sharing web traffic data with Google Analytics [...] is a pretty small issue IMO,
Building profiles of everything we read or do online is probably the largest problem we will have going forward into the future. Data doesn't go away, and the problem compounds when the data can be correlated with other types of data. If you think this isn't a problem, then you really need to study what is possible with pattern-of-life analysis[4].
[1] https://support.google.com/analytics/answer/2763052?hl=en
[2] https://analytics.usa.gov/data/
[3] https://analytics.usa.gov/#explanation
[4] https://en.wikipedia.org/wiki/Pattern-of-life_analysis
While I'm sure you have good intentions in trying to anonymize the collected data, you should know that the claims made by Google at [1] are highly misleading. Their claim is that they strip the last octet of the IP address, but that isn't nearly enough cooking to call the data "anonymous".
Assuming Google actually does this and isn't simply lying about saving the full IP (which has to be sent to them, as it is obviously in the SRC field of the IP header), this means they are only binning IPs into groups of 256. They only need at most 8 bits of identifying data other than the IP to uniquely correlate individuals within each IP group.
More than 8 bits of unique ID are available, given that various types of data that being tracked[2] about the user agent and (presumably) IP geolocation.
Also, Google is keeping enough of the IP to lookup the ASN.
> I work on analytics.usa.gov.
So if you're actually interested in claiming that "The program does not track individuals, and anonymizes the IP addresses of visitors"[2], then you really shouldn't be using an analytics service that - in spite of their claims of anonymizing" some data - obvious is tracking individuals and saving the most interesting parts of their IP address.
> Sharing web traffic data with Google Analytics [...] is a pretty small issue IMO,
Building profiles of everything we read or do online is probably the largest problem we will have going forward into the future. Data doesn't go away, and the problem compounds when the data can be correlated with other types of data. If you think this isn't a problem, then you really need to study what is possible with pattern-of-life analysis[4].
[1] https://support.google.com/analytics/answer/2763052?hl=en
[2] https://analytics.usa.gov/data/
[3] https://analytics.usa.gov/#explanation
[4] https://en.wikipedia.org/wiki/Pattern-of-life_analysis
From https://analytics.usa.gov/#explanation it sounds like it just happens to use Google Analytics at the moment, but could switch in the future.
"The Federal Source Code Policy is designed to support reuse and public access to custom-developed Federal source code. It requires new custom-developed source code developed specifically by or for the Federal Government to be made available for sharing and re-use across all Federal agencies. It also includes an Open Source Pilot Program that requires agencies to release at least 20% of new custom-developed Federal source code to the public."
That's really nice, but I think the 20% is too little. It should be at least 50%. They could still keep a critical low percentage secret.
That's really nice, but I think the 20% is too little. It should be at least 50%. They could still keep a critical low percentage secret.
I agree that 20% is too little, considering that works by the federal government are supposed to be in automatic public domain[0].
[0] https://en.wikipedia.org/wiki/Copyright_status_of_work_by_th...
[0] https://en.wikipedia.org/wiki/Copyright_status_of_work_by_th...
You make a great point. Can anyone explain why software is exempt from this?
They are not exercising a copyright claim when they don't release things, they're just not releasing them. Public domain only helps you if you have access to the work.
In addition to the point already made that the government has no obligation to release anything, code written by contractors is generally copyright by the contractors.
The 20% number is part of the initial 3-year pilot program, not necessarily the ultimate goal. That's also the mandatory minimum, with agencies "strongly encouraged" to release as much as possible.
See the policy [0], linked from the launch announcement [1].
[0] https://www.whitehouse.gov/sites/default/files/omb/memoranda...
[1] https://www.whitehouse.gov/blog/2016/08/08/peoples-code
See the policy [0], linked from the launch announcement [1].
[0] https://www.whitehouse.gov/sites/default/files/omb/memoranda...
[1] https://www.whitehouse.gov/blog/2016/08/08/peoples-code
20% is a LOT! Consider the amount of work it takes to open-source things. It's a huge step forward.
The amount of work is kind of moot. If it's tax dollars funding its development don't you think it should all be open sourced?
The work to open source something is often extra work. So it would effectively increase the cost or, keeping costs fixed, reduce the impact of the software on it initial target...those who it was developed for.
I also imagine that in many cases the effort to open source some project would be much greater than the pay off. There are many custom software solutions developed that are of almost no value outside of the small domain specific target audience.
I also imagine that in many cases the effort to open source some project would be much greater than the pay off. There are many custom software solutions developed that are of almost no value outside of the small domain specific target audience.
What do you think takes the most effort to make these projects open source? If this were the policy for all new projects, why would that be difficult?
There is a lot of internal resistance to open sourcing code. Lots of people believe that their code is their secret sauce - and why would you have highly paid consultants create code just so the next department can take it?
There's also security concerns to take into account - which I know someone will say is a good thing, because of transparency and so forth. But just realize that it's not a trivial amount of work to clean up a code-base and display it to the public all of the time.
And if the policy were open-source by default, the unspoken policy for project managers would be "automatically submitting a request to be closed source"
It's not perfect, but I know that 20% is a lot, and I think anyone who's worked in the sector would agree.
There's also security concerns to take into account - which I know someone will say is a good thing, because of transparency and so forth. But just realize that it's not a trivial amount of work to clean up a code-base and display it to the public all of the time.
And if the policy were open-source by default, the unspoken policy for project managers would be "automatically submitting a request to be closed source"
It's not perfect, but I know that 20% is a lot, and I think anyone who's worked in the sector would agree.
Those reasons, which surely place some non-negligible burden on the teams involved, do not seem to be so overwhelming as to ultimately dictate that 80% of projects are not made open source.
I'm definitely aware of the difficulties involved in open sourcing significant amounts of your work. The bigger burden is less, in my opinion, about cleaning up your code base. If you're committing potential security vulnerabilities into your code that are then tracked by your version control system – and you're a Federal agency – that's already a problem that's just going to be exacerbated by making it public; making that code private doesn't make the problem disappear.
The real meaty problems that all open source projects share is people: people like me who come in and overwhelm the project with support requests. I just opened four issues tonight just for this website, in the span of several minutes. (Sorry team!) If you already have poor project management practices in place, or your team is too small, this can quickly overload you.
Of course, with a vibrant community around your project, even the social and management problems could become trivial with time. Look at especially great examples like Hoodie. Given the number of technical people who have left cushy, high-paying jobs to serve in 18F, USDS, and the other alphabet agencies of late, I have to imagine the rallying cry to support truly useful code by compassionate people will be significant enough to justify the upfront expense here.
Not only is 20% not perfect, I don't think it's enough. I want 100%, and I think it's a fair request as a taxpayer, even if there is some burden. This country has fought two world wars and gone to the moon. We can always do better.
I'm definitely aware of the difficulties involved in open sourcing significant amounts of your work. The bigger burden is less, in my opinion, about cleaning up your code base. If you're committing potential security vulnerabilities into your code that are then tracked by your version control system – and you're a Federal agency – that's already a problem that's just going to be exacerbated by making it public; making that code private doesn't make the problem disappear.
The real meaty problems that all open source projects share is people: people like me who come in and overwhelm the project with support requests. I just opened four issues tonight just for this website, in the span of several minutes. (Sorry team!) If you already have poor project management practices in place, or your team is too small, this can quickly overload you.
Of course, with a vibrant community around your project, even the social and management problems could become trivial with time. Look at especially great examples like Hoodie. Given the number of technical people who have left cushy, high-paying jobs to serve in 18F, USDS, and the other alphabet agencies of late, I have to imagine the rallying cry to support truly useful code by compassionate people will be significant enough to justify the upfront expense here.
Not only is 20% not perfect, I don't think it's enough. I want 100%, and I think it's a fair request as a taxpayer, even if there is some burden. This country has fought two world wars and gone to the moon. We can always do better.
> But just realize that it's not a trivial amount of work to clean up a code-base and display it to the public all of the time.
Are you saying the federal government is running insecure code in production? I'm shocked. /s
Are you saying the federal government is running insecure code in production? I'm shocked. /s
But isn't everything made by the government in the public domain?
Assuming that's true, private companies contracted by the government may not be under that same requirement (think military specs and designs). The output as a whole may be public domain (I know what an F22 is, and the parts to build one, but the specs to those parts are proprietary) but the operations, and binaries,(specs and requirements to build the individual pieces) to produce that output may remain private.
Public domain just means it can't be copyrighted. It doesn't necessarily mean it has to be distributed to the public.
you mean like NSA toolkits?
serge2k(1)
New projects with new companies is easy. The problem is open source isn't a requirement for old projects, retroactively adding it is hard.
The companies hired would need to be new companies to contacting, with higher costs. Don't forget, government contracts are a lowest bidder war. Going full open source immediately might expose code that built using proprietary technology from a different division of a company. Therefore open source from scratch will have a high startup cost.
The existing projects can be small, open source, but more importantly these projects begin to build a foundation others can build on. The library of code created to support the framework of these projects will eventually be used as the basis for larger projects.
Wrapping back to the 80% this allows the existing, proprietary code reuse. While new, it's useless without its is proprietary components. This code, if released with proprietary components (and that's a big if since the binary would then be openly distributed) may not qualify as open source. One step further, public free binaries can be reverse engineered (the legality of which isn't something I'm well versed in). Which may not be in a companies current best long-term interest.
Tldr; The largest effort to being 100% open source is the library of code built on must also be open source or commonly distributed. Starting at 0 the current goal is 20%.
Provide costs don't rise too much expect 30% within 2 to 3 years (I'm being optimistic) or the next major government project timeline.
The companies hired would need to be new companies to contacting, with higher costs. Don't forget, government contracts are a lowest bidder war. Going full open source immediately might expose code that built using proprietary technology from a different division of a company. Therefore open source from scratch will have a high startup cost.
The existing projects can be small, open source, but more importantly these projects begin to build a foundation others can build on. The library of code created to support the framework of these projects will eventually be used as the basis for larger projects.
Wrapping back to the 80% this allows the existing, proprietary code reuse. While new, it's useless without its is proprietary components. This code, if released with proprietary components (and that's a big if since the binary would then be openly distributed) may not qualify as open source. One step further, public free binaries can be reverse engineered (the legality of which isn't something I'm well versed in). Which may not be in a companies current best long-term interest.
Tldr; The largest effort to being 100% open source is the library of code built on must also be open source or commonly distributed. Starting at 0 the current goal is 20%.
Provide costs don't rise too much expect 30% within 2 to 3 years (I'm being optimistic) or the next major government project timeline.
I can only imagine the nightmare attempting to explain to non-tech folks why it's good to give away the code they are writing.
We give away the laws we write. In fact, the government provides an awful lot of services. If the taxpayer paid for it, the taxpayer should certainly be able to own it. I've never personally found it difficult to explain that to non-technical people.
I think even a 5 year old can understand "if I paid for something, it's mine".
That's why I own a SR-71 Blackbird and my own naval destroyer... come on guys, grow up.
Doesn't 20% seem a bit pedantic? I feel like if you throw a requirement like that into a large bureaucracy only bull shit will come out. Like an "Open source web interface" where only the CSS is posted...
[deleted]
Took a look at some of the code.
I see one for Gov of Commerce:
The Commerce.gov API is under active, but not public, development. As such, API code is not currently made available publically. This Github repository will be used to collect and respond to feedback regarding the API and engage with developers interested in using the API.
---
I don't believe that's the correct usage of GitHub. What's the point if you're not going to showcase the code?
I see one for Gov of Commerce:
The Commerce.gov API is under active, but not public, development. As such, API code is not currently made available publically. This Github repository will be used to collect and respond to feedback regarding the API and engage with developers interested in using the API.
---
I don't believe that's the correct usage of GitHub. What's the point if you're not going to showcase the code?
They said what they think the point is in what you quoted: "to collect and respond to feedback..."
I can see why you might not like it, or not agree with it, but it's not like they didn't say it!
I can see why you might not like it, or not agree with it, but it's not like they didn't say it!
My feedback would be a FOIA request for commerce.gov's API code, and a PR to add it to their repo.
You can't make an FOIA request for code. You can only make requests for agency records, and in the definitions section of the bill, computer code is explicitly listed as something that isn't an agency record.
"computer software, including source code, object code, and listings of source and object codes, regardless of medium are not agency records"
"computer software, including source code, object code, and listings of source and object codes, regardless of medium are not agency records"
That sounds like something that you might be able to get your congressperson interested in changing, if you pitched it the right way.
Works created by the federal government are in the public domain, per section 105 of the copyright code. So you'd be right I can't use a FOIA request to obtain the code, but if obtained through other methods, I believe I'm in the clear for hosting it on Github.
With the exception of the recent 18F/USDS/etc groups acting as contractors to the rest of the government, the USG doesn't produce much code. Code is produced for the government by contractors, and that's not in the public domain. Here's a decent discussion of why the government doesn't regularly produce its own code, or much of anything:
http://politics.stackexchange.com/a/7742
http://politics.stackexchange.com/a/7742
Basically, you're trying to figure out some way to piss off the techies and managers you want to convince to be more open? Even if FOIA worked on code, you really aren't helping your cause..
> you're trying to figure out some way to piss off the techies and managers
I'm not trying to piss anyone off, but to be honest, I don't really care if I do. I'm simply trying to pry government open into being more transparent through any means necessary.
> you want to convince to be more open
No. Not in the slightest. My opinion doesn't matter anywhere in government, so I'm left with the tools at my disposal.
> Even if FOIA worked on code, you really aren't helping your cause..
When the carrot doesn't work, one must use the stick.
I'm not trying to piss anyone off, but to be honest, I don't really care if I do. I'm simply trying to pry government open into being more transparent through any means necessary.
> you want to convince to be more open
No. Not in the slightest. My opinion doesn't matter anywhere in government, so I'm left with the tools at my disposal.
> Even if FOIA worked on code, you really aren't helping your cause..
When the carrot doesn't work, one must use the stick.
I'd give them a pass since they do put a lot of other code up. It makes sense not to fragment the community, and more to the point I don't think keeping a precisely one-to-one relationship between repositories with code and issue trackers is that important as long as it leans heavily in the code direction.
Otherwise yeah, I'd say they should host their own damn issue tracker.
Otherwise yeah, I'd say they should host their own damn issue tracker.
This is awesome! At work we're trying to help with this at the municiapl level, it's hard so it's really great to see it coming from the Fed. Shout out to Becky Sweger[1] from 18F who has been instrumental in helping move things forward, talking about open data and open code.
[1] https://twitter.com/bendystraw
[1] https://twitter.com/bendystraw
The one project that i would love to see opensource is the FBI Sentinel project.
A lot of countries have broken infrastructure for law enforcement. Just like USAID, it would probably be more efficient to give software that can be adopted.
The sentinel project captures millions of man hours of wasted... and ultimately successful product development - all focused towards law enforcement collaboration. It would be good to have that.
A lot of countries have broken infrastructure for law enforcement. Just like USAID, it would probably be more efficient to give software that can be adopted.
The sentinel project captures millions of man hours of wasted... and ultimately successful product development - all focused towards law enforcement collaboration. It would be good to have that.
If there's one thing that deserves a federal massive budget, it's this. If laws and regulations can be understood easily by software, we would remove so many problems the government creates. Complexity would be easier to manage, but also transparency would inherently breed simplicity.
Just for an example, HNR Block and other companies built on doing your taxes for you lobby the government to prevent simplification of the tax code, because they profit off its complexity.
The same applies to almost all law. The complexity does not exist without a reason - there is almost always some entity actively funding the continued obfuscation of law for profit.
So don't expect government to ever actively get simpler, because that would only benefit the outsiders rather than incumbents in the industries such laws affect, and those outsiders never have the capital to compete in lobbying and bribery.
The same applies to almost all law. The complexity does not exist without a reason - there is almost always some entity actively funding the continued obfuscation of law for profit.
So don't expect government to ever actively get simpler, because that would only benefit the outsiders rather than incumbents in the industries such laws affect, and those outsiders never have the capital to compete in lobbying and bribery.
Exactly - there are tax code companies you've probably never even heard of like Avalara entering the Unicorn range. The kicker: all they do is calculate sales tax! Actually a cool company, but one that should never exist.
The bigger influence on the tax code is the litany of deductions, which are lobbied for by the wealthy.
The value of a $100,000 deduction is worth $30,000 to a wealthy person, but probably around $1 for H&R block.
edit: fixed
The value of a $100,000 deduction is worth $30,000 to a wealthy person, but probably around $1 for H&R block.
edit: fixed
It's worth more like $30,000. You only save the tax on the deduction, not the whole thing.
(and some deductions will of course be worth less than the tax; a $100,000 donation to the Red Cross costs $100,000 and doesn't have any financial return, so the $30,000 tax benefit there costs $70,000)
(and some deductions will of course be worth less than the tax; a $100,000 donation to the Red Cross costs $100,000 and doesn't have any financial return, so the $30,000 tax benefit there costs $70,000)
Most of the code looks fairly dull and single-purpose. Except NASA's code, as you may expect. Trick and OpenMCT both look at least interesting, and OpenMCT in particular could be used as the basis for many an interesting web-based project.
I've already seen OpenMCT integrated with Kerbal Space Program. It looks like a great platform for real-time sensor data visualization. I'd love to hook it into a drone and violate some FAA regulations :)
OpenMCT instantly caught my attention and seems like a solid foundation. Cool stuff, but I am not surprised since it is NASA/JPL.
In case any code.gov website people are looking: there's an unfortunate bug: when the site is first loading on a slow network, you can see all the agency names and click them, but they all show text like "No repositories found." This made me think that this was a brand-new project that hadn't done anything yet.
Also, the little department logo images shouldn't have alt-text. That alt text of the department name overlays the actual text of the department name and just makes it even less accessible when the image isn't there.
(I suppose the lesson for web development in general is to test on a slow connection.)
Also, the little department logo images shouldn't have alt-text. That alt text of the department name overlays the actual text of the department name and just makes it even less accessible when the image isn't there.
(I suppose the lesson for web development in general is to test on a slow connection.)
Why does this simple landing page have to be over 600K and require Javascript?
I wonder how they will determine which open source projects are included at code.gov. For instance, I contribute to a few projects that are used within government for determining the economic impacts of fiscal policy, and the code is in the public domain. I'd love for them to be included at code.gov, but I'm not sure whether they meet the criteria.
The general guidelines are in the policy [0], linked from the launch announcement [1], but each agency's CIO is responsible for the particulars at their org. If it's not a new project post-August it won't contribute to the 20% so it might not be reviewed for release without someone suggesting it specifically, I would guess.
[0] https://www.whitehouse.gov/sites/default/files/omb/memoranda...
[1] https://www.whitehouse.gov/blog/2016/08/08/peoples-code
[0] https://www.whitehouse.gov/sites/default/files/omb/memoranda...
[1] https://www.whitehouse.gov/blog/2016/08/08/peoples-code
is this page 508 compliant? I look to .gov to show some examples of accessibility
This is a step in the right direction, a thank you to the people who helped make this possible.
They already have some Swift projects [0], which I think is kind of cool. Not sure I would want to use code named "Bro" or "Conman" though. lol.
[0] https://code.gov/#/explore-code/agencies/DOL [1] https://code.gov/#/explore-code/agencies/DOE
[0] https://code.gov/#/explore-code/agencies/DOL [1] https://code.gov/#/explore-code/agencies/DOE
Bro is actually already a widely used and incredibly popular piece of software. However, I don't think the main branch is developed by the US government, so this may be the DOE trying to dodge the law.
I suggest having a search feature across all repositories.
How about open PACER?
That's a Judicial Branch screwup, not Executive Branch.
Check out the RECAP project. So far, that's the best we've got.
Check out the RECAP project. So far, that's the best we've got.
Just an honest question. Can I obtain some estimation of the capabilities of CS people working for the government by just reading all these code. Will I find here a glimpse of code beautifully designed and tailored to solve some really deep problem that require real intelligence?, can I use these code to learn how to code a big project?
Apparently Github supports displaying STL models: https://github.com/nasa/NASA-3D-Resources/blob/master/3D%20M...
Interesting that someone approved the communist reference (or that people didn't perceive that as a communist reference). It's not the 1980s anymore I guess.
Do you mean 'people's'? That would make 'the people's house' as a way to describe US executive mansions and legislature buildings communist references as well.
Where is the IRS code?
Under the Department of Commerce - unsurprisingly - there is nothing from IRS.
A bit more surprising and entirely missing from the list (no source code):
* Department of Education * Department of Health and Human Services * Department of Housing * Department of Interior * State Department * Department of Transportation
And not really expecting anything from: * Department of Defense * Department of Homeland Security
A bit more surprising and entirely missing from the list (no source code):
* Department of Education * Department of Health and Human Services * Department of Housing * Department of Interior * State Department * Department of Transportation
And not really expecting anything from: * Department of Defense * Department of Homeland Security
DHHS:
https://github.com/HHS
There are more if you know where to look. Some of the GitHub organizations are in their infancy, like NIAID's:
https://github.com/niaid
Others are only semi-public; these generally consist forks of software they use or private repos used for external collaboration.
I'm working on getting some of our stuff open-sourced. When it's written by contractors, the intellectual property rules can get complicated. Convincing a contractor to publish code their consultants wrote for a government project, no matter how minor, can be a tricky bit of advocacy. Sometimes they have really skewed ideas surrounding IP that must be overcome before this can happen. (Best case, our code will likely end up hosted by Internet2 or a similar foundation.)
https://github.com/HHS
There are more if you know where to look. Some of the GitHub organizations are in their infancy, like NIAID's:
https://github.com/niaid
Others are only semi-public; these generally consist forks of software they use or private repos used for external collaboration.
I'm working on getting some of our stuff open-sourced. When it's written by contractors, the intellectual property rules can get complicated. Convincing a contractor to publish code their consultants wrote for a government project, no matter how minor, can be a tricky bit of advocacy. Sometimes they have really skewed ideas surrounding IP that must be overcome before this can happen. (Best case, our code will likely end up hosted by Internet2 or a similar foundation.)
I was kinda hoping to see some projects from the NSA. Noooope.
This is awesome initiate by US Gov.
Great so we all have access to useless software code.
Looks like You Don’t Have Javascript Enabled
Code.gov is built on the latest web technology using Javascript. To use Code.gov, please enable Javascript so we can share all of the wonderful open source projects the Federal Government has to offer.
Code.gov is built on the latest web technology using Javascript. To use Code.gov, please enable Javascript so we can share all of the wonderful open source projects the Federal Government has to offer.
I had to unblock third-party scripts to get anything other than an animated pixel-art gif of an American flag.
Horrible name - the Federal Government of the United States is not "the People".
> the Federal Government of the United States is not "the People"
I disagree. It is indeed "government of the people, by the people, for the people". You may not like some particular people, and neither do I: Some are dumb, corrupt, or simply disagree with me.
But if that doesn't represent the people of the U.S., I don't know what else could. If they were non-corrupt, all smart and highly capable, and all agreed with me - that would be some other country (in a fairy tale).
It's not government of, by and for me. I want everyone else to have a vote too, even if I disagree with them.
I disagree. It is indeed "government of the people, by the people, for the people". You may not like some particular people, and neither do I: Some are dumb, corrupt, or simply disagree with me.
But if that doesn't represent the people of the U.S., I don't know what else could. If they were non-corrupt, all smart and highly capable, and all agreed with me - that would be some other country (in a fairy tale).
It's not government of, by and for me. I want everyone else to have a vote too, even if I disagree with them.
> It is indeed "government of the people, by the people, for the people".
In theory at least, and it certainly should be. Often it looks more like a government by the millionaires, for the corporations. But there's increasing attention for turning that around, and this project looks like a small part of that.
In theory at least, and it certainly should be. Often it looks more like a government by the millionaires, for the corporations. But there's increasing attention for turning that around, and this project looks like a small part of that.
[deleted]
I think the idea is that, by making it OSS, it's software that now belongs to "the people".
Only 20% of it by the looks of things ;)
[deleted]
And horrible domain name: How come `code.gov` is US-only? How come it's motto is "Help propel America’s next breakthrough in innovation"? If this in not imperialism, I don;t know what is.
The gov TLD is US only. Other countries have their own, e.g. In the UK most government sites are under gov.uk.
> The gov TLD is US only.
And you think it is normal? Why not gov.us? The US are always giving lessons to the world, but do not even understand that we --the other parts of the world-- would like to have some minimal level of fairness.
And you think it is normal? Why not gov.us? The US are always giving lessons to the world, but do not even understand that we --the other parts of the world-- would like to have some minimal level of fairness.
.us is barely used. The three letter TLDs started out as US only because the whole internet started out as US only. The first country that got its own national TLD was Netherland (.nl). .us was introduced later, but by that time, the ship had sailed.
> Why not gov.us?
Because creator privilege and because life is not fair.
Because creator privilege and because life is not fair.
Civilisation, development of human ethics, laws, all this stuff is our way to adjust for the unfairness of life. Also, back people did not invent our create the buses, was it right to let them seat in the back, when they where even allow to get in?
The US gets the .gov TLD because they made the internet.
Why would we need fairness ? They financed and created the internet, they can have at least a TLD.
Other parts of the world should have made the internet then.
Then you don't know what imperialism is. Simple as that.
Bearing in mind that I'm ecstatic to see this collection of projects and someone clearly put a lot of time and effort into this already, I do have couple points of hopefully constructive feedback here:
- Please list all of your repositories and feature ones that you think deserve special recognition. Seeing all your repos is a 4-step process otherwise: click on the project, click on the link, click on the acknowledgement that I clicked a link, click on the organization name on GitHub. [1]
- Some information on contribution policies, pulling in your README.md, etc would be helpful on this site. [2]
- When I click on the link to go to your repo, I know I'm clicking a link that leaves your site. You even say so. "But you probably knew that already." Don't hijack it, please. [3]
- The activity feed doesn't provide much value in its current form. Most of these GitHub events are completely without context. [4]
EDIT:
I've since opened the following issues on https://github.com/presidential-innovation-fellows/code-gov-.... Anyone who has feedback, definitely let them know (kindly!).
[1] https://github.com/presidential-innovation-fellows/code-gov-...
[2] https://github.com/presidential-innovation-fellows/code-gov-...
[3] https://github.com/presidential-innovation-fellows/code-gov-...
[4] https://github.com/presidential-innovation-fellows/code-gov-...