Mysterious startup Shadow under scrutiny after Iowa caucus meltdown(venturebeat.com)
venturebeat.com
Mysterious startup Shadow under scrutiny after Iowa caucus meltdown
https://venturebeat.com/2020/02/04/mysterious-startup-shadow-under-scrutiny-after-iowa-caucus-meltdown/
144 comments
All software involved in the voting process should (1) not exist except for tallying and communication purposes (no electronic voter records) and (2) be open sourced and verified by known third parties.
Four days ago when the register dug into this they were refused access to test the app[1], they were not given mockups or any information about the UI[1], they weren't told who developed the app[1], they weren't told how the contract was given out to develop the app[1], they weren't told why the app was thought to be necessary[1], and they were told the app was verified by a third party testing firm but that who that was couldn't be disclosed[1].
Iowa moving forward with this app was extremely ill-advised and I'm actually rather sad that the register didn't raise more hell on receiving all these red flags.
1. A red flag...
Four days ago when the register dug into this they were refused access to test the app[1], they were not given mockups or any information about the UI[1], they weren't told who developed the app[1], they weren't told how the contract was given out to develop the app[1], they weren't told why the app was thought to be necessary[1], and they were told the app was verified by a third party testing firm but that who that was couldn't be disclosed[1].
Iowa moving forward with this app was extremely ill-advised and I'm actually rather sad that the register didn't raise more hell on receiving all these red flags.
1. A red flag...
security by obscurity is not security.
It's entirely possible that my ability to follow directions and my status as Not-an-Octegenarian-Caucus-Volunteer is all that kept me out of trouble.
This is most likely the case. This is very much an issue in my day to day with applications like SAP CRM, some people get it, some people learn it, some never get it.
This is most likely the case. This is very much an issue in my day to day with applications like SAP CRM, some people get it, some people learn it, some never get it.
Can you share screenshots or at least tell us what the hard parts might have been?
Dude... That's a frickin' truism of my career and industry! :D
"As a [computer/sw/hw] guy, all I can say for sure is that I'm overwhelmingly grateful that we kept paper backups."
"As a [computer/sw/hw] guy, all I can say for sure is that I'm overwhelmingly grateful that we kept paper backups."
Enough with the conspiracy, mystery, scheming, evil explanations of what's going on. Sufficiently advanced incompetence is indistinguishable from malice, and that's all that's going on here.
This is a bunch of amateur app coders, managed by amateur product designers and sales people, who got paid by clueless amateur voting registrars, to make an app that was rushed, not tested at scale, deployed without training, and allowed to get Iowa into this situation by someone not treating it with the seriousness it deserved.
No more, no less. Just normal human incompetence.
This is a bunch of amateur app coders, managed by amateur product designers and sales people, who got paid by clueless amateur voting registrars, to make an app that was rushed, not tested at scale, deployed without training, and allowed to get Iowa into this situation by someone not treating it with the seriousness it deserved.
No more, no less. Just normal human incompetence.
So the argument is that former members of the Clinton campaign -- highly competent individuals familiar with elections -- put together a company comprised of 'amateur app coders' paid by 'clueless amateur voting registrars'?
The founder, Gerard Niemira and Krista Davis, have a great pedigree. Niemira was formerly at kiva.org. Davis was on the Clinton campaign tech team. These are not incompetent individuals.
Now, you may be right that this is all incompetence. However, your contention that these are 'amateur coders', managed by 'amateur product designers', getting paid by clueless 'amateur voting registrars', is completely ridiculous. The leadership team is clearly professional and tech-savvy..
The founder, Gerard Niemira and Krista Davis, have a great pedigree. Niemira was formerly at kiva.org. Davis was on the Clinton campaign tech team. These are not incompetent individuals.
Now, you may be right that this is all incompetence. However, your contention that these are 'amateur coders', managed by 'amateur product designers', getting paid by clueless 'amateur voting registrars', is completely ridiculous. The leadership team is clearly professional and tech-savvy..
"The leadership team is clearly professional and tech-savvy.."
And yet they failed miserably.
I'm so sick of this attitude. Leadership never takes the fall, it is always the underlings who get the blame.
Their important high visibility project failed. The leadership failed. When you fail, that means you did not perform in a competent way.
And yet they failed miserably.
I'm so sick of this attitude. Leadership never takes the fall, it is always the underlings who get the blame.
Their important high visibility project failed. The leadership failed. When you fail, that means you did not perform in a competent way.
I think you mistook me. I am clearly blaming the leadership team, but not for incompetence. For actual malice. They clearly knew what they should do, and decided not to. That should be something they are punished for, not their employees.
If you're claiming malice, what's the theory of malice? Who benefits from this?
I don't have a theory... I'm just noting that professional neglect at this level is malice, whether due to laziness or actual ill intent.
Like, if I sold someone a piece of software that I knew hadn't been tested and told them it worked great, that is malicious behavior, whether my motive was to make a quick and easy buck, to eat ice cream instead of doing actual work, or to ensure that my customer failed. The behavior is malicious independent of motivation.
Like, if I sold someone a piece of software that I knew hadn't been tested and told them it worked great, that is malicious behavior, whether my motive was to make a quick and easy buck, to eat ice cream instead of doing actual work, or to ensure that my customer failed. The behavior is malicious independent of motivation.
Malice refers to intent, the idea of "malice... without ill intent" doesn't make sense. That's why Hanlon's Razor is worded as it is. By your definition, any shoddy work could be considered malicious.
I get your point though, I think we're just arguing semantics.
I get your point though, I think we're just arguing semantics.
"negligence" is probably the word that's being sought here.
>Who benefits from this?
Probably one of the candidates who had a decent chance of winning the Iowa primary, but was not the most likely one to be picked. I don't like throwing accusations without concrete support for them, so I will just say that the whole involvement of Buttigieg's with the app, as well as his announcement of victory last night (way before votes got actually counted), seems at least suspicious.
Though it is just as likely as simple incompetence, probably even less, so i am leaning more towards incompetence, but I cannot fully dismiss the alternative in the light of all the recent DNC events.
Probably one of the candidates who had a decent chance of winning the Iowa primary, but was not the most likely one to be picked. I don't like throwing accusations without concrete support for them, so I will just say that the whole involvement of Buttigieg's with the app, as well as his announcement of victory last night (way before votes got actually counted), seems at least suspicious.
Though it is just as likely as simple incompetence, probably even less, so i am leaning more towards incompetence, but I cannot fully dismiss the alternative in the light of all the recent DNC events.
Remember, however, this app was commissioned by the Iowa Democratic Party (IDP), not the DNC.
I mean they are clearly not as professional and tech-savvy as their "pedigree" would have us believe or they wouldn't have made so many amateur mistakes. But they clearly are good at convincing non-tech folks in a position to hire them that they are a lot more competent than they really are, so kudos on that?
Those of us who work in software understand that bugs happen even when you follow best practices. But if you don't even try to follow best practices for mission critical software, then roll out in production for the first time at a marquee event with millions watching, that's on you.
Those of us who work in software understand that bugs happen even when you follow best practices. But if you don't even try to follow best practices for mission critical software, then roll out in production for the first time at a marquee event with millions watching, that's on you.
> Those of us who work in software understand that bugs happen even when you follow best practices. But if you don't even try to follow best practices for mission critical software, then roll out in production for the first time at a marquee event with millions watching, that's on you.
Exactly. And this kind of incompetence may be expected from someone who is actually an amateur good at selling themselves. However, the leadership is clearly not comprised of amateurs. Thus, this level of incompetence is professional, almost criminal, neglect.
Exactly. And this kind of incompetence may be expected from someone who is actually an amateur good at selling themselves. However, the leadership is clearly not comprised of amateurs. Thus, this level of incompetence is professional, almost criminal, neglect.
Maybe. But I think it's more likely just another lesson in how far someone who is charismatic, well-connected, and self-promoting can advance, and how much credibility they can amass, while still essentially being a complete fraud who has none of the skills or experience they claim to have.
Some folks take "fake it 'til you make it" to an art form, and they keep right on faking it even after they've made it.
Some folks take "fake it 'til you make it" to an art form, and they keep right on faking it even after they've made it.
> So the argument is that former members of the Clinton campaign -- highly competent individuals familiar with elections -- put together a company comprised of 'amateur app coders' paid by 'clueless amateur voting registrars'?
Yes, that is exactly the argument. Just because you happened to be a member of some noteworthy organizations does not mean you are competent. People drift from golden parachute to golden parachute all the time, especially at an executive level where you get to benefit from people assuming "well he's an executive so obviously he knows what he's doing" and blame your underlings for your failures.
Yes, that is exactly the argument. Just because you happened to be a member of some noteworthy organizations does not mean you are competent. People drift from golden parachute to golden parachute all the time, especially at an executive level where you get to benefit from people assuming "well he's an executive so obviously he knows what he's doing" and blame your underlings for your failures.
Highly competent is not a word heard that often about the failed 2016 Clinton campaign, especially not from people who realize that she lost under the exact same rules that was in place when Obama won.
What the involved people are is highly connected which is rarely the same as competent as demonstrated by the antics of the current crop of in office politicians, and it's a huge part of the reason why crazy outsiders like Trump have a base at all.
The stupid idea that people who's only achievement is climbing the rungs of an failed organization is super cometent and whose ultimate failure could only have been due to nefarious actions of supernaturally competent foreign enemies is silly and seen as such by an huge sway of the actual public as the kind of nonsense you would expect from North Korea or Russia.
What the involved people are is highly connected which is rarely the same as competent as demonstrated by the antics of the current crop of in office politicians, and it's a huge part of the reason why crazy outsiders like Trump have a base at all.
The stupid idea that people who's only achievement is climbing the rungs of an failed organization is super cometent and whose ultimate failure could only have been due to nefarious actions of supernaturally competent foreign enemies is silly and seen as such by an huge sway of the actual public as the kind of nonsense you would expect from North Korea or Russia.
> Highly competent is not a word heard that often about the failed 2016 Clinton campaign, especially not from people who realize that she lost under the exact same rules that was in place when Obama won.
One can be a competent presidential candidate and still lose. That is the point of democracy. Clinton's campaign could be competent while still more people in the right places want Trump to be president.
One can be a competent presidential candidate and still lose. That is the point of democracy. Clinton's campaign could be competent while still more people in the right places want Trump to be president.
Yep but she lost which means that simply being an high ranking member of the 2016 Clinton's campaign cannot be proof of competency on it's own without the rational analysis of the campaign that's being successfully blocked by the emergence of a bunch of conspiracy theories recycled from the 1960ies as mainstream politics by the democratic party of America.
Competent people fail all the time for reasons outside of their control but it's also true that incompetent people can rise to the top of an organization based on luck, connections or by being associated with someone else. So you need something more detailed then having been a part of an failed bureaucracy to justify claims of extreme competency in running software projects.
Competent people fail all the time for reasons outside of their control but it's also true that incompetent people can rise to the top of an organization based on luck, connections or by being associated with someone else. So you need something more detailed then having been a part of an failed bureaucracy to justify claims of extreme competency in running software projects.
Just because they were competent at something somewhat related, doesn't mean they were competent at this particular thing, or that they gave it the attention it needed.
A real-time vote transmission system that requires data integrity and validation is not quite the same as a consumer website, or an email marketing system.
A real-time vote transmission system that requires data integrity and validation is not quite the same as a consumer website, or an email marketing system.
> A real-time vote transmission system that requires data integrity and validation is not quite the same as a consumer website, or an email marketing system.
That's not at all what this was or sounds like. This was a data collection app, from everything I've read. It is meant to replace a phone call.
That's not at all what this was or sounds like. This was a data collection app, from everything I've read. It is meant to replace a phone call.
Ok. So, this is just a data collection app? Trying to have casual inexperienced users accept/trust a developer certificate? Both PINs and 2FA? This is beyond embarrassing.
"...Two people who work for Acronym, speaking on the condition of anonymity because they did not want to risk their jobs, acknowledged that the app had problems. It was so rushed, they said, that there was no time to get it approved by the Apple store. Had it been, it might have proved far easier for users to install.
Instead, the app had to be downloaded by bypassing a phone’s security settings, a complicated process for anyone unfamiliar with the intricacies of mobile operating systems, and especially hard for many of the older, less tech-savvy caucus chairs in Iowa.
The app also had to be installed using two-factor authentication and PIN passcodes. The information was included on worksheets given to volunteers at the Iowa precincts tallying the votes, but it added another layer of complication that appeared to hinder people..."
https://www.nytimes.com/2020/02/04/us/politics/iowa-caucus-s...
"...Two people who work for Acronym, speaking on the condition of anonymity because they did not want to risk their jobs, acknowledged that the app had problems. It was so rushed, they said, that there was no time to get it approved by the Apple store. Had it been, it might have proved far easier for users to install.
Instead, the app had to be downloaded by bypassing a phone’s security settings, a complicated process for anyone unfamiliar with the intricacies of mobile operating systems, and especially hard for many of the older, less tech-savvy caucus chairs in Iowa.
The app also had to be installed using two-factor authentication and PIN passcodes. The information was included on worksheets given to volunteers at the Iowa precincts tallying the votes, but it added another layer of complication that appeared to hinder people..."
https://www.nytimes.com/2020/02/04/us/politics/iowa-caucus-s...
My assumption when seeing that is that those individuals you mentioned signed on to a company for profit and to use their connections thusly. Does not mean they were involved at all in the day to day.
> highly competent individuals familiar with elections
Is this a joke?
There's nothing suggesting competence here and everything suggesting casual nepotism.
Is this a joke?
There's nothing suggesting competence here and everything suggesting casual nepotism.
[deleted]
You may be right, but this will always be the case with electronic voting. A normal child could understand and verify the chain of a non-electronic vote. A very technical adult could not do the same for an electronic vote.
This isn't electronic voting. This isn't even paper voting (though they also use paper backups). This vote is primarily done by people standing in a particular corner.
This is related to the TRANSMISSION of the vote count to the Iowa Dem Party, not the count itself. There was a separate issue from app issues where the different rounds of voting weren't matching up in some precincts. Paper backups of the vote still exist.
This is related to the TRANSMISSION of the vote count to the Iowa Dem Party, not the count itself. There was a separate issue from app issues where the different rounds of voting weren't matching up in some precincts. Paper backups of the vote still exist.
They're hiring!
https://shadowinc.io/jobs
https://shadowinc.io/jobs
"Client Success Representative" sounds like a fun job.
A lack of transparency combined with a reputation (earned or not) for guiding primaries to their desired outcome, it's not hard to see why some people might think this is an example of the DNC putting their finger on the scale in some way. There's no evidence either way, which means confidently claiming incompetence is fairly baseless, as well.
If we want to talk what things are generally true, sure, I'd put money on incompetence without any context, but there IS context to this.
If we want to talk what things are generally true, sure, I'd put money on incompetence without any context, but there IS context to this.
There is no proof of either yet. What makes everyone think this is suspicious is there is proof of election meddling from overseas in the past. So there is proof elections can be modified and I doubt the IT skills groups like these to realize or adequately stop tampering. Without evidence everyone is assuming everything.
Did Russia actually meddle with (read: modify) the election? Or did they just buy ads? I thought it was only the latter.
I read there was unsuccessful attempts and not only by Russia, but if foreigners are willing to do it I absolutely would not put it past interested Americans to try either. The ads were meddling but in a different sense, just like the Russians meddled in Brexit with ads.
There is no evidence of changing votes by any other nation. Ad buying and fake advocacy accounts have been shown to exist for multiple nations. Also, multiple countries have given money to organizations in the US for non-election work that seems to end up on elections. The DC advocacy culture is a wonder to behold from money both domestic and imported.
Most people consider the illegal hacking and distribution of emails as "meddling" with the election and believe it had a greater impact on how people voted than the ad buys did.
If you're just arguing that it's irrelevant because vote totals weren't hacked, fine. That's a dispute over word choice of what it means to "meddle." Russia also hacked into and downloaded data from voting registration systems, which should cause us to re-evaluate our voting systems, but again, that didn't impact vote computations or anything like that.
If you're just arguing that it's irrelevant because vote totals weren't hacked, fine. That's a dispute over word choice of what it means to "meddle." Russia also hacked into and downloaded data from voting registration systems, which should cause us to re-evaluate our voting systems, but again, that didn't impact vote computations or anything like that.
I’m gonna break a rule, but, honestly, HN confuses me. I asked a genuine question and I’m being downvoted for it because (I assume) some people mistake it as trolling.
Anyways, I never said they didn’t do anything. I said, “here’s what I thought: x. Was I wrong?” My use of “(read: modify)” was to clarify that when I said meddle in that instance, I meant actual modification; I was referencing what the parent comment had said.
Anyways, I never said they didn’t do anything. I said, “here’s what I thought: x. Was I wrong?” My use of “(read: modify)” was to clarify that when I said meddle in that instance, I meant actual modification; I was referencing what the parent comment had said.
I didn't downvote you (I promise!). And I hope I answered your question.
For those who did, I think what happens is that it's hard to tell which people are and are not operating in good faith and sometimes people mistake one for the other. But it sounds like you genuinely weren't sure whether Russia had actually hacked into the voting machines. Only into the Podesta, the DNC, and into voter registration systems. There's no evidence to suggest they changed tabulations (and by that I mean they almost certainly didn't), so that's why people haven't made that claim.
For those who did, I think what happens is that it's hard to tell which people are and are not operating in good faith and sometimes people mistake one for the other. But it sounds like you genuinely weren't sure whether Russia had actually hacked into the voting machines. Only into the Podesta, the DNC, and into voter registration systems. There's no evidence to suggest they changed tabulations (and by that I mean they almost certainly didn't), so that's why people haven't made that claim.
Thank you. I like HN because the community is a lot nicer than others, but it’s frustrating being drive-by downvoted when you just asked a genuine question. It reminds me of other online communities where people just downvote because you said something they disagree with.
The only thing I want to add to this discussion is that even though there's no evidence any nation has ever hacked into our voting machines, we still need a paper trail. It could absolutely happen.
We have over 3000 different voting systems - one for each county. Some counties are staffed better than others. They use different types of machines. In some ways, that's a strength. But in others, it's a weakness. There should really be much more discussion of the issue nationally and what needs to be done.
We have over 3000 different voting systems - one for each county. Some counties are staffed better than others. They use different types of machines. In some ways, that's a strength. But in others, it's a weakness. There should really be much more discussion of the issue nationally and what needs to be done.
Oh for sure. The issues with all digital elections are well known (in the tech community), but the governments ignore us. I think Russia was a wake up call for a lot of people that hacking our results is a real possibility, but it sadly wasn’t enough.
Thankfully(?), Georgia (the state) is in the middle of some lawsuits (somewhat) regarding their lack of a paper trail in their elections.
Thankfully(?), Georgia (the state) is in the middle of some lawsuits (somewhat) regarding their lack of a paper trail in their elections.
Not only that, the party itself has a history of meddling against this election's likely winner.
Agree that this is certainly the greatest example of Hanlon's Razor in recent memory. This includes their naming choice.
As anyone in this industry knows, this is very common even with larger, more established companies.
[deleted]
> * Sufficiently advanced incompetence is indistinguishable from malice*
I like what you did there.
I like what you did there.
I agree 99% with your observations.
As you pointed out, this is not just "normal human incompetence", these are layers and layers of incompetence.
As you pointed out, this is not just "normal human incompetence", these are layers and layers of incompetence.
I am a big Bernie Sanders supporter and am always eager to put on my tinfoil hat.
But I have to agree with you. The owner of Shadow's parent company has close ties to the Hillary campaign, a lot of the Shadow employees worked for the Obama campaign, Hillary campaign, and for the DNC.
The scandal here is a bunch of noobs got an extremely important contract through cronyism, and they bungled it horribly.
But I have to agree with you. The owner of Shadow's parent company has close ties to the Hillary campaign, a lot of the Shadow employees worked for the Obama campaign, Hillary campaign, and for the DNC.
The scandal here is a bunch of noobs got an extremely important contract through cronyism, and they bungled it horribly.
I'd surprised you're not more critical of Clinton and her cronies as a Bernie supporter given they abused superdelegates to award Hillary the nomination over Bernie in 2016
Oh believe me, I have plenty of venom for the DNC.
As far as I am concerned they stole the nomination. The DNC was biased against Bernie Sanders and they ratf*ed him every chance they had. Bernie had staff at most of the caucus locations and they kept their own independent tally to ensure there was no funny business.
The bias was so obvious. It damaged Hillary's campaign. I was so disgusted I was tempted to vote for Trump ( I didn't! ), and I am pretty sure a LOT of people tuned out after the circus was over, and didn't bother to vote in the election. It was so obvious the head of the DNC had to step down in shame.
And not to say the shadow bungle was not damaging. It robbed the candidate who won from being able to deliver a victory speech. Instead... everyone delivered a victory speech just in case xD
As far as I am concerned they stole the nomination. The DNC was biased against Bernie Sanders and they ratf*ed him every chance they had. Bernie had staff at most of the caucus locations and they kept their own independent tally to ensure there was no funny business.
The bias was so obvious. It damaged Hillary's campaign. I was so disgusted I was tempted to vote for Trump ( I didn't! ), and I am pretty sure a LOT of people tuned out after the circus was over, and didn't bother to vote in the election. It was so obvious the head of the DNC had to step down in shame.
And not to say the shadow bungle was not damaging. It robbed the candidate who won from being able to deliver a victory speech. Instead... everyone delivered a victory speech just in case xD
Yeah I have a similar story. I was a Bernie supporter but when all that went down I started paying attention to Trump and found out I actually agreed with much of what he was running on
The reports I've seen have the app budget as $60-70k, which seems super low budget for how important it was. If this is cronyism (and not just low bidding), they didn't do a very good job of it.
https://www.thestreet.com/politics/acronym-shadow-iowa-caucu...
https://www.thestreet.com/politics/acronym-shadow-iowa-caucu...
I saw the same thing and I had the same thought: Wow that is low.
My suspicion: The company is all designers and product managers, and they shipped it overseas to the lowest bidder. That is unconfirmed, though.
My suspicion: The company is all designers and product managers, and they shipped it overseas to the lowest bidder. That is unconfirmed, though.
It's funny because I'd bet most of the people that know how to program around here could write a similar app (and likely a betterone at that) in a matter of hours if they were so inclined.
The difference being that you didn't win the contract.
That's pretty immaterial. I was just pointing towards the fact that the simplistic nature of the application in the first place only serves to further highlight the incompetence required for such a catastrophic failure.
Got it. My comment was saying, wryly, the amount of thought you put in just now was probably the maximal level of project design and thinking they did.
Or maybe the situation was treated with seriousness, hence the outcome...
How would you test this type of thing at scale?
Disclaimer: not an app developer!
Disclaimer: not an app developer!
This is a small scale app by any sensible standards and probably not that complex compared to what's being routinely deployed by 2-3 man development teams in the world of business. i routinely handle spreadsheets with larger datasets and more complexity then whats involved here.
There is books written on how to test an new system/feature but in general you have 3 stages:
The first is an automated internal integrity test(sometimes called unit test) where you throw mock data at the systems independent component, in order to test for known classes of edge cases and race conditions.
The second step is implementation where the ops team tries to yank network and hardware resources away form underneath the app until something breaks in order to see if it fails in a sensible way that don't lead to irrecoverable data corruption.
The third stage is acceptance testing which on proprietary solutions is often the only one the end user is 110% in control of where you both test the app with actual user performing standard procedures and an team of specialist trying and break it using whatever knowledge they have on the systems design.
All tree stages have it's own skill set and is often performed by different teams and it's not uncommon to see specialist companies brought in doing part of the QA process for really important new systems but for the most part the entire things just fade into the the daily routine as an newer ending feedback loop.
There is books written on how to test an new system/feature but in general you have 3 stages:
The first is an automated internal integrity test(sometimes called unit test) where you throw mock data at the systems independent component, in order to test for known classes of edge cases and race conditions.
The second step is implementation where the ops team tries to yank network and hardware resources away form underneath the app until something breaks in order to see if it fails in a sensible way that don't lead to irrecoverable data corruption.
The third stage is acceptance testing which on proprietary solutions is often the only one the end user is 110% in control of where you both test the app with actual user performing standard procedures and an team of specialist trying and break it using whatever knowledge they have on the systems design.
All tree stages have it's own skill set and is often performed by different teams and it's not uncommon to see specialist companies brought in doing part of the QA process for really important new systems but for the most part the entire things just fade into the the daily routine as an newer ending feedback loop.
There are apps and libraries that can simulate load.
On production, apparently.
It takes a very dedicated plan to sufficiently test apps at scale when we're talking the scale of 300 million or billions... In this case the scale was 1700 (maybe off by an order of magnitude) when your scale is on the thousands or tens of thousands you can't do outright stupid stuff... but you can do reasonably dumb stuff and be fine - running a test with a hundred or so users should've clearly demonstrated these issues and there are companies out there that excel at resourcing a few minutes of a few hundred folks' time for just this kind of load testing.
That said, this was a pretty pathetic failure to scale, and I don't use harsh language like that lightly. I assume we're going to find out someone's brother or nephew runs the company because, for whatever reason, politics loves nepotism.
That said, this was a pretty pathetic failure to scale, and I don't use harsh language like that lightly. I assume we're going to find out someone's brother or nephew runs the company because, for whatever reason, politics loves nepotism.
Iowa has 1,681 precincts. This is not a scalability problem.
Even the simplest CRUDs will have problems when you have 1681 users who must use it and you can't just bounce out the ones who don't get it and chalk it up to bad user fit.
If your CRUD can't handle 1681 users, then you're almost certainly doing something horribly, horribly wrong.
Like, that's "LAMP stack in 2001 hosting someone's blog about cacti wearing sweaters in Duluth" levels of scale.
Like, that's "LAMP stack in 2001 hosting someone's blog about cacti wearing sweaters in Duluth" levels of scale.
"Scale" in this case is 1,681 precincts, each with a handful of users (the election officials). I'm not sure that was the issue in this case.
This isn't really "scale."
It needs to record something like a dozen numbers from each of the ~1,700 caucuses in the state. You could have used Excel as the database here with no problem.
It needs to record something like a dozen numbers from each of the ~1,700 caucuses in the state. You could have used Excel as the database here with no problem.
I was in the shower this morning thinking about how I would solve this, and something like this is exactly what I would have done. Use simple, familiar, battle tested technology to do the heavy lifting.
Something, something, monorail.
fuqmachine(2)
They named the company Shadow. I am reminded of the movie quote "They literally call themselves Decepticons. That doesn’t set off any red flags?".
I can almost see the name used in countries that have legitimate shadow governments, but in the US this is a really bad name akin to having cavalry in the name when focusing on the reservations (yes, someone did, they though it was cute and they did prove unhelpful). Names are a first impression and this is a really bad one.
I can almost see the name used in countries that have legitimate shadow governments, but in the US this is a really bad name akin to having cavalry in the name when focusing on the reservations (yes, someone did, they though it was cute and they did prove unhelpful). Names are a first impression and this is a really bad one.
This and 'Fraud Guarantee' will go down as the stupidest company names of the decade, whether they were involved in malfeasance or not.
"Fraud Guarantee", supposedly, was meant to mitigate the effect of having "fraud" pop up as the first suggestion when you google for Parnas's name. (See https://lawandcrime.com/high-profile/now-we-know-why-indicte... .)
Hah, in that case it's not as stupid of a name. Reminds me a bit of how Blackwater keeps changing names every few years to avoid the PR nightmare associated with... well, being a bunch of mercenaries killing people for money.
Mysterious startup? Seems identical to almost every small dev shop I've ever seen, complete with bizdev people who over-promise and clients who might not know better. My understanding is that they started development just 2 months ago and never bothered to train users. That's all I need to know.
MSNBC interviewed a precinct captain who stated she had no problems with the app. Last week they were given test PINs for training purposes so training was provided.
Sure, but many did, and many because they were in rooms with large numbers of people and they were depending on wifi and cell service, which often fails in those situations. Then throw in different phones, OSs, versions, etc. Word is that this app was rolled out with no user training.
How loud do all the software developers need to yell "use paper ballots" before people figure out that electronic counting (for quicker results) isn't a terrible idea, but electronic records and simply adding more technology to the system makes voting confirmation and integrity weaker.
This app had no reason to exist - there isn't a reason why they couldn't use almost any other approach to collect the results for publishing - there are very few numbers to keep track of and people are only looking for provisional numbers on election night anyways.
This app had no reason to exist - there isn't a reason why they couldn't use almost any other approach to collect the results for publishing - there are very few numbers to keep track of and people are only looking for provisional numbers on election night anyways.
Paper ballots are already being used. There is an exquisite paper trail, with signed and personally identifiable records being kept until the convention.
[deleted]
Shadow, Inc.? Really? Sounds like a Bond villan's organization, what a strange name, especially with 'mysterious' put before it.
It is an odd choice of a name for a company that wants to build public-facing applications for the Government.
I have no idea what all the features of this app are but nothing I’ve heard makes it sound like this couldn’t have been accomplished using a google form and google sheet
Considering the "butterfly ballot" and the "hanging chad," what on earth made someone think an app was a good idea where people can't even figure out paper?
The process needs to be dumbed-down--kindergaten style. Give everone a jellybean to put in their candidate's jar, and if some dummy eats his jellybean instead of depositing it...
The process needs to be dumbed-down--kindergaten style. Give everone a jellybean to put in their candidate's jar, and if some dummy eats his jellybean instead of depositing it...
The bigger problem wasn't that the app used by precinct campaigns to report back to the Dem Party, it was the convoluted rounds of voting that didn't add up in some precincts and caused the party to have to review all of the paper backups.
So yeah, getting rid of the caucus (which is undemocratic anyway) and letting people just vote for who they want is the solution. Fixing the app doesn't fix that people were too confused in certain precincts to count it correctly.
So yeah, getting rid of the caucus (which is undemocratic anyway) and letting people just vote for who they want is the solution. Fixing the app doesn't fix that people were too confused in certain precincts to count it correctly.
Shadow, we move fast and break Iowa.
Probably being wildly naive here, but wouldn't it be better if there were an open-source solution for this sort of thing that could be used by both parties?
Google forms, not open-source but way more competent than the crony hires. I mean wouldn't Whatsapp's E2E work here?
Every state has a very different process. Even within the same state, Iowa Dems and Republicans have a different process. The work this company did in Nevada for their caucus (which now reportedly will not be used, for obvious reasons) had to be a different solution than the one for Iowa. But yeah, somebody could probably build an open source solution that has the calculator and other functions.
I think they'll likely go back to just transmitting via phone, though.
I think they'll likely go back to just transmitting via phone, though.
I don't get the mystery angle, it's the usual loyalty over competency game where the insiders reward you from having done work for an previous campaign or being an grassroot college activist by making sure you keep getting work.
It's the good old pork barrel and an proud American tradition that overstayed it's welcome with the general public but remains a big part of America's political economy.
It's the good old pork barrel and an proud American tradition that overstayed it's welcome with the general public but remains a big part of America's political economy.
Many comments here and in other threads regarding this app seem to put blame on the DNC. Please note, however, the app was commissioned by the Iowa Democratic Party (IDP), not the DNC.
Why exactly was this app needed at all? Couldn't a $13 node.js app or telegram bot have done the job of receiving a number from under 2000 people just as well?
Apps are cool man!!!
It's kind of part of the general back to the 90ies mentality of the DNC, yes there absolutely no need for an custom app to do this especially given it's a caucus and not a closed box election, and as there is no reason for keeping the raw data secret doing the counting process, an signed message on any platform(it could and should be public) would be sufficient security.
Heck telefaxes with an off the shelf OCR business grade solution at the other end would have been fast effective and likely cheaper but would have meant less opportunity for the Iowa party bosses to mingle with the well connected Washington apparatus.
This just smell of vanity project where a bunch of low power politicians felt their status would increase with an successful app launch only to have the whole thing blow up in their faces.
It's kind of part of the general back to the 90ies mentality of the DNC, yes there absolutely no need for an custom app to do this especially given it's a caucus and not a closed box election, and as there is no reason for keeping the raw data secret doing the counting process, an signed message on any platform(it could and should be public) would be sufficient security.
Heck telefaxes with an off the shelf OCR business grade solution at the other end would have been fast effective and likely cheaper but would have meant less opportunity for the Iowa party bosses to mingle with the well connected Washington apparatus.
This just smell of vanity project where a bunch of low power politicians felt their status would increase with an successful app launch only to have the whole thing blow up in their faces.
AIUI, part of this is the result of 2016. There was an ask for more of the intermediate numbers rather than just straight up count.
gremlinsinc(4)
Wow I disagree with the tone of the article. They're comparing the operations of the Shadow company with the Trump company. That's a false equivalence - Trump as purchasing social media services. Shadow is literally inserting itself into the tally and facilitation of the voting process, and it's got links to Clinton.
That's not social media advertising, that really smacks of straight up voter fraud and abuse to me.
That's not social media advertising, that really smacks of straight up voter fraud and abuse to me.
Obviously Dem vendors will have worked with Dem candidates.
I'm working with a vendor right now that works with Bloomberg, even though my org hasn't endorsed yet. (We wall that Bloomberg consultant off from our work. If we couldn't work with firms that worked with any of the candidates, then we couldn't hire ANYBODY.)
I'm working with a vendor right now that works with Bloomberg, even though my org hasn't endorsed yet. (We wall that Bloomberg consultant off from our work. If we couldn't work with firms that worked with any of the candidates, then we couldn't hire ANYBODY.)
That's not the concern - the concern is that a company which doesn't seem to list it's roster or have any public information about it is now attaching itself directly to the voting systems. The Community has been in an uproar before about how opaque code is in voting machines, why shouldn't we be concerned about not only the code but the employees of a company who are involved in voting?
These aren't voting machines. "Involved in voting" is quite misleading. The app doesn't actually help with the voting part. They help in the calculation of delegates and the transmission.
The votes are recorded on paper. This had a calculator tool (that actually functioned correctly) to count the delegates. The part the failed was the transmission of results to the Iowa Dem Party. The back-up (in addition to paper) was the phone hotline they normally use, but that got overloaded. Additionally, the overall caucus process for reporting -- unrelated to the tech -- was more convoluted this time around and the numbers didn't match for the different rounds of voting. This bigger problem caused the party to have to go back to the paper backups in order to verify results. That's what took so long.
I'd be shocked if anybody actually disputed any of the results. Every campaign keeps their own records of as many precincts as possible and they're announced to everybody attending. Plus, there's the paper. And people record video of these events on their phones these days, too.
In short, this wasn't a voting system, as you claim. It was a reporting system. What's the vendor they used for the phone reporting system? Are you upset that you don't know the names of the employees who run that phone hotline system? I'd be fine with a requirement for that for reporting systems, but nobody has ever asked for it previously for past elections so we shouldn't be shocked we don't have it now.
Also, when you said it had "links to Clinton," I took it as you had a concern about that. Glad we're both in agreement that it's actually quite normal for vendors that Dem Party would and should use. It's normal - encouraged, even! - for Democratic vendors to work with Democratic candidates.
The votes are recorded on paper. This had a calculator tool (that actually functioned correctly) to count the delegates. The part the failed was the transmission of results to the Iowa Dem Party. The back-up (in addition to paper) was the phone hotline they normally use, but that got overloaded. Additionally, the overall caucus process for reporting -- unrelated to the tech -- was more convoluted this time around and the numbers didn't match for the different rounds of voting. This bigger problem caused the party to have to go back to the paper backups in order to verify results. That's what took so long.
I'd be shocked if anybody actually disputed any of the results. Every campaign keeps their own records of as many precincts as possible and they're announced to everybody attending. Plus, there's the paper. And people record video of these events on their phones these days, too.
In short, this wasn't a voting system, as you claim. It was a reporting system. What's the vendor they used for the phone reporting system? Are you upset that you don't know the names of the employees who run that phone hotline system? I'd be fine with a requirement for that for reporting systems, but nobody has ever asked for it previously for past elections so we shouldn't be shocked we don't have it now.
Also, when you said it had "links to Clinton," I took it as you had a concern about that. Glad we're both in agreement that it's actually quite normal for vendors that Dem Party would and should use. It's normal - encouraged, even! - for Democratic vendors to work with Democratic candidates.
> Organizers of the local meetings were supposed to use the app created by startup Shadow to report results from Iowa’s unusual voting system. In the caucuses, people wishing to support a candidate must be physically present at a meeting, where they stand in groups and wait to be counted.
> To make that process more efficient, the Iowa Democratic Party gave local managers the Shadow app to input results. But results were delayed and backup measures — such as calling a hotline — also failed, according to the New York Times.
From the article, I definitely read that as the app is the way that results were reported. I also read it as the phone and paper ballots were the backup systems.
The app absolutely is involved in voting, even if it's not the most authoritative resource, nor should it be. This was discussed last year (for example) in: https://www.desmoinesregister.com/story/news/politics/2019/0...
> What's the vendor they used for the phone reporting system?
Dunno, we should find out, and then find out why that wasn't correctly implemented either. Who's brilliant idea was it to accept election results over the same line which served the helpdesk?
> Are you upset that you don't know the names of the employees who run that phone hotline system?
Yes, you should be too.
> To make that process more efficient, the Iowa Democratic Party gave local managers the Shadow app to input results. But results were delayed and backup measures — such as calling a hotline — also failed, according to the New York Times.
From the article, I definitely read that as the app is the way that results were reported. I also read it as the phone and paper ballots were the backup systems.
The app absolutely is involved in voting, even if it's not the most authoritative resource, nor should it be. This was discussed last year (for example) in: https://www.desmoinesregister.com/story/news/politics/2019/0...
> What's the vendor they used for the phone reporting system?
Dunno, we should find out, and then find out why that wasn't correctly implemented either. Who's brilliant idea was it to accept election results over the same line which served the helpdesk?
> Are you upset that you don't know the names of the employees who run that phone hotline system?
Yes, you should be too.
> I definitely read that as the app is the way that results were reported
That's my point. It's about how caucus managers tell the Iowa Democratic Party how many votes there were, for who, in each round. It has nothing to do with how people actually vote, which is done by walking into certain corners of the room and filling out pieces of paper.
I want a comprehensive, independent audit of all of the things that went wrong, done by an entity that every single campaign would have veto power over selecting. I don't care to know the names of the people who built the phone system, personally, since it'd be part of my recommended audit, but have no problem with a rule that makes them public moving forward because why not. I don't like the idea of making the lives of developers miserable when this seems like a problem at EVERY LEVEL -- at the Iowa Dem Party, DNC, individual campaigns that didn't object, owners of this company, and sure, the developers, too.
That's my point. It's about how caucus managers tell the Iowa Democratic Party how many votes there were, for who, in each round. It has nothing to do with how people actually vote, which is done by walking into certain corners of the room and filling out pieces of paper.
I want a comprehensive, independent audit of all of the things that went wrong, done by an entity that every single campaign would have veto power over selecting. I don't care to know the names of the people who built the phone system, personally, since it'd be part of my recommended audit, but have no problem with a rule that makes them public moving forward because why not. I don't like the idea of making the lives of developers miserable when this seems like a problem at EVERY LEVEL -- at the Iowa Dem Party, DNC, individual campaigns that didn't object, owners of this company, and sure, the developers, too.
I worked as a Precinct Captain for one of the Caucuses in Iowa last night, and I didn't personally have any issues. But I heard from colleagues who definitely did. Also, yesterday we got an email with a 7-page google doc instructing us on how to reset the browser caches on our phones to avoid issues with data carry-over from the Mock Caucuses that we had been holding.
It's entirely possible that my ability to follow directions and my status as Not-an-Octegenarian-Caucus-Volunteer is all that kept me out of trouble.
As a software guy, all I can say for sure is that I'm overwhelmingly grateful that we kept paper backups so we don't have to rely on this software.