Using FOIA Data and Unix to halve major source of parking tickets(mchap.io)
mchap.io
Using FOIA Data and Unix to halve major source of parking tickets
http://mchap.io/using-foia-data-and-unix-to-halve-major-source-of-parking-tickets.html
236 comments
He also spends a lot of time corresponding the old fashioned way to get the data & cleaning it for mechanical evaluation.
His hobbies make me feel bad about my own...
His hobbies make me feel bad about my own...
"I got burned by the CIA and now I'm stuck in the records office fulfilling FOIA requests for conspiracy hobbyists in Ohio!"
a cautionary tale from a public servant
a cautionary tale from a public servant
You could have these hobbies!
I realise you're being positive and helpful, but I don't think many people could actually have these hobbies. The technical side is within reach of most HN readers, but you also need to see the problem in the first place and understand that it's actually solvable. Most people would assume that any effort would be wasted as the authorities would prefer to keep their revenue generator, so they'd never go through with the project. It takes a pretty special and creative person to have the idea, do the work, and actually get results at the end.
You could practice and build the skills necessary to have these types of hobbies!
Nothing this person is doing is magic, and to the extent that genetics or extensive background prep play a role you could still overcome that deficit enough to at least participate in the space, even if you weren't the most amazing, talented person in the world. It's not like making the world better by addressing weird problems is a particularly competitive field.
I'll make a claim that literally every single skill being shown here is something that you personally could learn, and it probably wouldn't take you more than a couple of months to a year of regularly exercising your creative muscles to get to the point where you were doing something useful.
Nothing this person is doing is magic, and to the extent that genetics or extensive background prep play a role you could still overcome that deficit enough to at least participate in the space, even if you weren't the most amazing, talented person in the world. It's not like making the world better by addressing weird problems is a particularly competitive field.
I'll make a claim that literally every single skill being shown here is something that you personally could learn, and it probably wouldn't take you more than a couple of months to a year of regularly exercising your creative muscles to get to the point where you were doing something useful.
Ok, here's the thing. I like playing video games. I get a sense of enjoyment and fulfillment from an evening spent crushing my enemies on the field of battle via a surprise contingent of minotaurs. That's fine. It's good!
I would not get that same enjoyment from doing what this person does, although I fully agree i could probably replicate his method. My hobbies reflect what I enjoy doing, as do those of most people. I can feel nominally "bad" that mine are so incredibly worthless comparatively but still know that I wouldn't get as much utility from doing the "better" thing?
I would not get that same enjoyment from doing what this person does, although I fully agree i could probably replicate his method. My hobbies reflect what I enjoy doing, as do those of most people. I can feel nominally "bad" that mine are so incredibly worthless comparatively but still know that I wouldn't get as much utility from doing the "better" thing?
I realize that actual tangible human emotions are much more complicated than I'm about to make them out to be. There is no "feel better" switch you can flip to make negative feelings go away. But, it is still sometimes helpful to try and recognize when an emotion is illogical and call it out, or at least it's helpful for me when I'm dealing with stuff like this.
If you know that overall the good parts of an activity wouldn't outweigh the bad parts, but you honestly believe that you have a duty to do it anyway, then you should follow your conscience and ignore trying to have a positive life. I don't think that's healthy, but some people disagree with me.
If on the other hand, you know that overall the good parts wouldn't outweigh the bad parts, and you don't think you have a duty to make yourself miserable in your free time, then what the heck do you feel bad about? If your guilt is real, you should acknowledge and take steps to address the cause. If your guilt is not real, then get a rubber band or something and snap it against your wrist every time you feel guilty about comparing yourself to other people.
You can replicate this if you think it would be worthwhile to do so. If you don't think it would be worthwhile to do so, then you shouldn't feel bad about it and it's just your stupid tribal lizard brain that's making you feel that way. And if you do think it would be worthwhile, then do it. I mean, even video games have bad parts. Having bad parts doesn't make a hobby unsatisfying or unenjoyable by default.
The third category in all of this is someone who thinks that overall the good parts of this kind of work would be incredibly satisfying, but they're so scared of the bad parts that they can't start engaging, or they're so scared of being inadequate that they never try to learn or develop their skills. My comment was addressed to that person. If you're not in that category then, I dunno. Flip your "feel better" switch.
If you know that overall the good parts of an activity wouldn't outweigh the bad parts, but you honestly believe that you have a duty to do it anyway, then you should follow your conscience and ignore trying to have a positive life. I don't think that's healthy, but some people disagree with me.
If on the other hand, you know that overall the good parts wouldn't outweigh the bad parts, and you don't think you have a duty to make yourself miserable in your free time, then what the heck do you feel bad about? If your guilt is real, you should acknowledge and take steps to address the cause. If your guilt is not real, then get a rubber band or something and snap it against your wrist every time you feel guilty about comparing yourself to other people.
You can replicate this if you think it would be worthwhile to do so. If you don't think it would be worthwhile to do so, then you shouldn't feel bad about it and it's just your stupid tribal lizard brain that's making you feel that way. And if you do think it would be worthwhile, then do it. I mean, even video games have bad parts. Having bad parts doesn't make a hobby unsatisfying or unenjoyable by default.
The third category in all of this is someone who thinks that overall the good parts of this kind of work would be incredibly satisfying, but they're so scared of the bad parts that they can't start engaging, or they're so scared of being inadequate that they never try to learn or develop their skills. My comment was addressed to that person. If you're not in that category then, I dunno. Flip your "feel better" switch.
This looks sort of like a disagreement over what people mean by 'hobby'.
I think I could do this, from my starting point it wouldn't take me very long to do something useful. But it looks like the meat of this project involved corresponding with a city bureaucracy and cleaning up ugly data to get basic info from it. It sounds awful.
So... if a hobby is just something optional you choose to do and don't make a living at, then yes, I could do this as a hobby. But in the sense where hobbies are things you do for recreation - reading, gaming, woodworking, gardening, etc? This would be far less pleasant than any other hobby I have, and the bureaucracy parts would be actively negative.
I'm glad this happened, and I'm glad if people enjoy doing this sort of thing! But I think when we talk about having productive hobbies, it's worth differentiating "tasks you can achieve" from "tasks you can seek out sustainably without hurting your quality of life". I have productive things in both categories, but I'd only describe the second category as hobbies.
I think I could do this, from my starting point it wouldn't take me very long to do something useful. But it looks like the meat of this project involved corresponding with a city bureaucracy and cleaning up ugly data to get basic info from it. It sounds awful.
So... if a hobby is just something optional you choose to do and don't make a living at, then yes, I could do this as a hobby. But in the sense where hobbies are things you do for recreation - reading, gaming, woodworking, gardening, etc? This would be far less pleasant than any other hobby I have, and the bureaucracy parts would be actively negative.
I'm glad this happened, and I'm glad if people enjoy doing this sort of thing! But I think when we talk about having productive hobbies, it's worth differentiating "tasks you can achieve" from "tasks you can seek out sustainably without hurting your quality of life". I have productive things in both categories, but I'd only describe the second category as hobbies.
If I rephrased to say "you could learn enough to break into a hobby that would be both enjoyable and meaningful", would you agree with that?
Yes, I think so. I expect the ease of finding those hobbies varies a lot by person, but I'd imagine most people have some good options.
No amount of training or practice is going to turn a cat into a dog.
No amount of practice is going to teach me how to see that these problems exist in the first place.
No amount of practice is going to teach me that government types might actually be willing to get rid of their revenue generators.
It takes a special type of person who sees what the actual problem is to begin with, then figures out how to get the data into a format where it all actually makes sense, and then takes the next step of contacting the appropriate authorities to get the problem fixed.
Don't get me wrong, I do understand and appreciate that some people might be able to repeat the same type of success, if only they had more training and practice. But you have to be careful about how you "encourage" that kind of thing.
No amount of practice is going to teach me how to see that these problems exist in the first place.
No amount of practice is going to teach me that government types might actually be willing to get rid of their revenue generators.
It takes a special type of person who sees what the actual problem is to begin with, then figures out how to get the data into a format where it all actually makes sense, and then takes the next step of contacting the appropriate authorities to get the problem fixed.
Don't get me wrong, I do understand and appreciate that some people might be able to repeat the same type of success, if only they had more training and practice. But you have to be careful about how you "encourage" that kind of thing.
[deleted]
> but you also need to ... understand that [the problem is] actually solvable. Most people would assume that any effort would be wasted as the authorities would prefer to keep their revenue generator, so they'd never go through with the project.
Any reasonable person would make such an assumption without a second thought.
I'm still baffled that the city took any action whatsoever in response to this guy's request. How could he have known that they would just roll over and cough up $60k in lost revenue? It's seriously out-of-character for them.
Any reasonable person would make such an assumption without a second thought.
I'm still baffled that the city took any action whatsoever in response to this guy's request. How could he have known that they would just roll over and cough up $60k in lost revenue? It's seriously out-of-character for them.
You’re making the common (and fatal) mistake of evaluating a government on the terms of a private business entity.
Governments are of the people so long as the people maintain them as such.
When you treat a government like a business, it falls behind on it’s maintenance schedule, and begins to resemble one. There are a lot of people who wish the government was their business, and will encourage you to play along. These people work night and day to lower your expectations of what you’re capable of.
Stop falling for it!
The USA had some ingenious (and flawed) founders who set in place some rights and traditions that reserve at least a small finger hold which resembles democracy, by which the people can mobilize effectively.
But, in any nation, that fingerhold can exist when enough people come together en masse.
Governments are of the people so long as the people maintain them as such.
When you treat a government like a business, it falls behind on it’s maintenance schedule, and begins to resemble one. There are a lot of people who wish the government was their business, and will encourage you to play along. These people work night and day to lower your expectations of what you’re capable of.
Stop falling for it!
The USA had some ingenious (and flawed) founders who set in place some rights and traditions that reserve at least a small finger hold which resembles democracy, by which the people can mobilize effectively.
But, in any nation, that fingerhold can exist when enough people come together en masse.
> You’re making the common (and fatal) mistake of evaluating a government on the terms of a private business entity.
> Governments are of the people so long as the people maintain them as such.
"We the people" decided that government has some set of tasks to perform. Those tasks require resources, so "we the people" decided that we should steal some portion of each others' earnings, in order to provide a means that these tasks be accomplished.
The continued legitimacy of this ... arrangement ... is due to Social Contract Theory.
If, for discussion's sake, we temporarily step into the average-person's shoes and accept the Social Contract Theory at face-value, then it is most simple to conclude that what we are really doing is paying the government a lump sum in exchange for some set of services, and by garrulously quarrelling and advertising to each other we can decide who pays how much, and what is the set of services, and to a limited extent how the services ought be provided; under the limitation that if all the lump sums can't pay for all the services, then the difference will have to be made-up by printing more money and thereby reducing everyone's purchasing power (in short: further theft-from-all).
From that we can conclude that the "best" government would be the one which satisfied the majority of the desires of the majority of the people while appearing to take, in return, as little as possible. Does that not sound like the goal of a business?
> The USA had some ingenious (and flawed) founders who set in place some rights and traditions that reserve at least a small finger hold which resembles democracy, by which the people can mobilize effectively.
Correct.
Problem is, that kind of mobilization will cause enough chaos that it's not worth doing it over just a parking-ticket racket.
No, it will likely require some single, unambiguous, flagrant, overt, unapologetic and high-stakes treason against the letter of our principles and procedures ... before the 2nd Amendment's most fundamental purpose is put-to-action.
And I have no faith that the outcome of such an event, will be nontrivially better than the Articles of Confederation; it probably won't even outshine the Constitution of 1788.
But this whole discussion is going far afield enough that my betting-money says we'll soon hear from Fearless Leader Ang...
> Governments are of the people so long as the people maintain them as such.
"We the people" decided that government has some set of tasks to perform. Those tasks require resources, so "we the people" decided that we should steal some portion of each others' earnings, in order to provide a means that these tasks be accomplished.
The continued legitimacy of this ... arrangement ... is due to Social Contract Theory.
If, for discussion's sake, we temporarily step into the average-person's shoes and accept the Social Contract Theory at face-value, then it is most simple to conclude that what we are really doing is paying the government a lump sum in exchange for some set of services, and by garrulously quarrelling and advertising to each other we can decide who pays how much, and what is the set of services, and to a limited extent how the services ought be provided; under the limitation that if all the lump sums can't pay for all the services, then the difference will have to be made-up by printing more money and thereby reducing everyone's purchasing power (in short: further theft-from-all).
From that we can conclude that the "best" government would be the one which satisfied the majority of the desires of the majority of the people while appearing to take, in return, as little as possible. Does that not sound like the goal of a business?
> The USA had some ingenious (and flawed) founders who set in place some rights and traditions that reserve at least a small finger hold which resembles democracy, by which the people can mobilize effectively.
Correct.
Problem is, that kind of mobilization will cause enough chaos that it's not worth doing it over just a parking-ticket racket.
No, it will likely require some single, unambiguous, flagrant, overt, unapologetic and high-stakes treason against the letter of our principles and procedures ... before the 2nd Amendment's most fundamental purpose is put-to-action.
And I have no faith that the outcome of such an event, will be nontrivially better than the Articles of Confederation; it probably won't even outshine the Constitution of 1788.
But this whole discussion is going far afield enough that my betting-money says we'll soon hear from Fearless Leader Ang...
I'm kind of on a negative streak these days, so forgive me, reader, but...why in god's green earth aren't these the government's hobbies?!
When is...dear educated technologists...the last time you even slightly considered working for city government? Alas, our best and brightest aren't serving the people. They toil at advertising.
This is something the public can control. Make government positions competitive with corporate ones and you'll get a bunch more talented (and patriotic) "educated technologists" working in government. Likewise with teachers.
It's not difficult, just costly.
It's not difficult, just costly.
I imagine there are a whole
bunch of “gov is inefficient” types who like having non-competitive rates so reinforce the narrative. This makes it easier for the public to support dismantling gov programs, etc.
I tried to do so but never made it past their interviewing - which was glacially slow and involved a 1/10th chance all things equal even when there were multiple identical positions open. From what I could gather in many cases the agencies were often at the mercy of poor organization from above.
Hey, same here. I live in a very low cost state, so the pay was fantastic (especially compared to what I get now), the job security was there, the benefits were amazing... Unlike some folks, I'm perfectly willing to work for the government (mostly for the benefits).
They really wanted me in that job, too. But then the state froze hiring and several months later the posting expired. It hasn't been re-listed since then and it's been nearly a year now.
They really wanted me in that job, too. But then the state froze hiring and several months later the posting expired. It hasn't been re-listed since then and it's been nearly a year now.
I'm curious what job I would apply for to improve this situation.
I've made an effort to work for local governments. I've even made an effort to volunteer for local governments, and do this sort of technology work directly for them instead of as a FOIA-enabled personal project.
I've never seen a flicker of interest, or found a job posting that would leave any room for this sort of work. I've only ever seen indifferent hostility to the volunteering offers, and while I understand why that could be worse - bureaucratically - than paying staff, it's still not exactly systems making an effort to serve the people.
The US Digital Service was a brilliant and wonderful project to get technologists doing exactly this. It hired a lineup of top-notch staff and got a lot of great stuff done. And, yes, it paid government salaries and appealed to civic duty to recruit. It sounds like a wonderful place to work. Outside of that one national-level pet project from Obama? I mean, I got involved with a local technology/privacy group. They're currently considering suing the town for not following its own surveillance-restricting ordinances about street-facing cameras, because the town found implementing them too hard - and isn't hiring anyone who can, and wouldn't accept volunteer labor to do the work.
It's not just that advertising pays better, it's that advertising doesn't actively avoid working with people for this sort of task.
I've made an effort to work for local governments. I've even made an effort to volunteer for local governments, and do this sort of technology work directly for them instead of as a FOIA-enabled personal project.
I've never seen a flicker of interest, or found a job posting that would leave any room for this sort of work. I've only ever seen indifferent hostility to the volunteering offers, and while I understand why that could be worse - bureaucratically - than paying staff, it's still not exactly systems making an effort to serve the people.
The US Digital Service was a brilliant and wonderful project to get technologists doing exactly this. It hired a lineup of top-notch staff and got a lot of great stuff done. And, yes, it paid government salaries and appealed to civic duty to recruit. It sounds like a wonderful place to work. Outside of that one national-level pet project from Obama? I mean, I got involved with a local technology/privacy group. They're currently considering suing the town for not following its own surveillance-restricting ordinances about street-facing cameras, because the town found implementing them too hard - and isn't hiring anyone who can, and wouldn't accept volunteer labor to do the work.
It's not just that advertising pays better, it's that advertising doesn't actively avoid working with people for this sort of task.
Around two years ago. I applied for a data scientist role with the city of SF and never got a response.
the government, serving the people? Since when? Do you think they install cash grabbing radars on the roads to better serve us?
There's one neat trick to stop 'them' getting your money. It's so easy you'll be amazed!
It's "not speeding" /savedyouaclick
It's "not speeding" /savedyouaclick
> It's "not speeding" /savedyouaclick
No, because they purposely fine you when you are a few kilometers about an arbitrary limit (let's say 55 km/h instead of 50 km/h) which is not speeding in any way and there is no data supporting any kind of increase of accidents at such levels of speeds.
On top of they they use all the dirty tricks in the books (mobile radars, radars right at the exit of a tunnel) which act like traps for anyone that is not constantly vigilant at their current speed. Let's face it, nobody is spending 100% of their attention on the speedometer while driving.
And when "normal" people around you get fined while you know for a fact they are not driving like crazy folks on the road, something is really, really wrong.
No, because they purposely fine you when you are a few kilometers about an arbitrary limit (let's say 55 km/h instead of 50 km/h) which is not speeding in any way and there is no data supporting any kind of increase of accidents at such levels of speeds.
On top of they they use all the dirty tricks in the books (mobile radars, radars right at the exit of a tunnel) which act like traps for anyone that is not constantly vigilant at their current speed. Let's face it, nobody is spending 100% of their attention on the speedometer while driving.
And when "normal" people around you get fined while you know for a fact they are not driving like crazy folks on the road, something is really, really wrong.
Sure that's a bad example, but he does kind of have a point.
In the blog post he estimated it's saved $60,000 in fines. That's a drop in the bucket at the scale we're talking about (a quick Google says the proposed 2018 budget for Chicago was $10.1 billion), but still a decrease. Not only would it cost the government money to employ someone to go through this data and find hotspots like this, someone to go out and evaluate the signs, presumably several someones during the approval process to change signage, and finally people to make and install that signage, but the end result would be to purely cost them more money by decreasing revenue from fines.
I don't personally think governments are inherently evil (though some do seem to try harder than others), but even from a purely capitalistic viewpoint that's a hard sell for anyone who cares about their budget. At the very very best I could see it becoming a token effort that's mostly marketing ("look, we're using big data to make your life better!").
In the blog post he estimated it's saved $60,000 in fines. That's a drop in the bucket at the scale we're talking about (a quick Google says the proposed 2018 budget for Chicago was $10.1 billion), but still a decrease. Not only would it cost the government money to employ someone to go through this data and find hotspots like this, someone to go out and evaluate the signs, presumably several someones during the approval process to change signage, and finally people to make and install that signage, but the end result would be to purely cost them more money by decreasing revenue from fines.
I don't personally think governments are inherently evil (though some do seem to try harder than others), but even from a purely capitalistic viewpoint that's a hard sell for anyone who cares about their budget. At the very very best I could see it becoming a token effort that's mostly marketing ("look, we're using big data to make your life better!").
It's a fair point that it's very difficult to avoid conflict of interest when doing your job better might mean your department has less money to use.
My suggestion: Remove the incentive by divorcing all fine (and similar things like seized goods) revenue from the government budget. Perhaps stipulate that it gets distributed to charities, or is split equally among taxpayers as a tax offset.
My suggestion: Remove the incentive by divorcing all fine (and similar things like seized goods) revenue from the government budget. Perhaps stipulate that it gets distributed to charities, or is split equally among taxpayers as a tax offset.
The government is the only entity that does not have to (and never should) follow capitalistic rules. The government is for the people, and capitalism, by definition, is not.
> The government is for the people
No, the government should be "THE people", not "for" or "by". We introduce many issues with representation of larger groups by very few individuals, that is precisely why democracy/government works best at a local scale rather than State or Nation-wide. It's not very hard to grasp why.
No, the government should be "THE people", not "for" or "by". We introduce many issues with representation of larger groups by very few individuals, that is precisely why democracy/government works best at a local scale rather than State or Nation-wide. It's not very hard to grasp why.
You will note that I did not say “earn a profit”. There is still a cost center here, and even governments (at least those at the local and usually state level) have to abide by a budget. Spending money one year that results in less money the next doesn’t fit into that model in most budgets.
I’m sorry for the confusion, I thought that was blatantly self-obvious.
I’m sorry for the confusion, I thought that was blatantly self-obvious.
It‘s not even about earning a profit. It should not be the goal of the government to collect (part of) its budget via fines, i.e. citizens breaking rules. Instead, it should always be the goal to have no need to fine.
And we can actually estimate how much the government actually saves if it does not need to fine (that much). Let‘s first look at the costs of the status quo:
- it costs X to check rules are followed (here: no cars park where they must not); this is mostly personnel costs\* but note that we may be talking about „manhours spent that could have been spent doing more sensitive/productive things“
- it costs Y to maintain the infrastructure to process and follow up on the fines (here: the $190 million IBM contract\* )
- it costs Z to collect the fines, process the payments (personnel costs), follow up on those that do not pay, court fees (process & personnel costs), jail costs (cause it‘s ’murica), and whatnot.
Now, if there are no\* fines to prosecute, here‘s a few ways the state can gain money:
- Citizens spend less time and money with unproductive work (here: paying fines), leaving more time for work (or relaxation which again increases productivity) and money to spend (raising economic output + sales tax).
- Officers, beuracrats, and judges can spend their time dealing with more important work.
- E.g. in the case of parking fines, businesses affected by cars parking in their way can instead do their business unhampered, i.e. be productive and this increases, again, economic output.
All of that increases tax returns (or reduces tax money spent for dealing with the fines).
\* Obviously, there‘s no way fines will drop to zero, because humans. But minimizing fines allows to minimize capital and infrastructure costs and increases economical output, thus there‘s a net gain that can offset the costs to reach that goal, if not immediately then within a few years.
And we can actually estimate how much the government actually saves if it does not need to fine (that much). Let‘s first look at the costs of the status quo:
- it costs X to check rules are followed (here: no cars park where they must not); this is mostly personnel costs\* but note that we may be talking about „manhours spent that could have been spent doing more sensitive/productive things“
- it costs Y to maintain the infrastructure to process and follow up on the fines (here: the $190 million IBM contract\* )
- it costs Z to collect the fines, process the payments (personnel costs), follow up on those that do not pay, court fees (process & personnel costs), jail costs (cause it‘s ’murica), and whatnot.
Now, if there are no\* fines to prosecute, here‘s a few ways the state can gain money:
- Citizens spend less time and money with unproductive work (here: paying fines), leaving more time for work (or relaxation which again increases productivity) and money to spend (raising economic output + sales tax).
- Officers, beuracrats, and judges can spend their time dealing with more important work.
- E.g. in the case of parking fines, businesses affected by cars parking in their way can instead do their business unhampered, i.e. be productive and this increases, again, economic output.
All of that increases tax returns (or reduces tax money spent for dealing with the fines).
\* Obviously, there‘s no way fines will drop to zero, because humans. But minimizing fines allows to minimize capital and infrastructure costs and increases economical output, thus there‘s a net gain that can offset the costs to reach that goal, if not immediately then within a few years.
What are "capitalistic rules"?
Governments absolutely should be looking to make profits, it's just that they don't necessarily have to make them in dollars.
So say you were able to objectively measure the value a program produced (costs are usually already known). If a program costs 10 units for every unit of value it produces, maybe it isn't a good program. If it produces modestly more value than it costs, it's making society a profit.
Governments absolutely should be looking to make profits, it's just that they don't necessarily have to make them in dollars.
So say you were able to objectively measure the value a program produced (costs are usually already known). If a program costs 10 units for every unit of value it produces, maybe it isn't a good program. If it produces modestly more value than it costs, it's making society a profit.
> Governments absolutely should be looking to make profits
I understand what you're saying, but I think you're stretching the common understanding of "profits" and risking confusion because of it.
Absolutely, government should try to measure the impact of its actions, but expressing that in terms like profit can lead to undesirable consequences like the expectation that a successful self-promoter pretending to be a successful businessman can also be a successful president.
I understand what you're saying, but I think you're stretching the common understanding of "profits" and risking confusion because of it.
Absolutely, government should try to measure the impact of its actions, but expressing that in terms like profit can lead to undesirable consequences like the expectation that a successful self-promoter pretending to be a successful businessman can also be a successful president.
The common understanding of profit is confused!
What are "capitalistic rules"?
In this case, the dogmatic assumption that every endeavour should be financially profitable.
In this case, the dogmatic assumption that every endeavour should be financially profitable.
I would actually counter with the assumption that every endeavor should not be financially unprofitable. Like it or not, governments work on a budget. Aside from the federal one they’re also usually required to stick to it.
I was in no way implying the government was trying to make money off increased fines (quite the opposite with the last paragraph), simply that it would very likely end up costing them more than it saved the taxpayers to support such an initiative at a larger scale, and that would have a very nebulous gain.
Can you imagine being the Mayor of your department at work and proposing to your board of directors a multi-million dollar budget for next year that includes a huge carve out for evaluating all the petty fines and late fees you collected from customers because it’ll make them happier?
Goodwill is one thing, but who is ever going to approve that?
I was in no way implying the government was trying to make money off increased fines (quite the opposite with the last paragraph), simply that it would very likely end up costing them more than it saved the taxpayers to support such an initiative at a larger scale, and that would have a very nebulous gain.
Can you imagine being the Mayor of your department at work and proposing to your board of directors a multi-million dollar budget for next year that includes a huge carve out for evaluating all the petty fines and late fees you collected from customers because it’ll make them happier?
Goodwill is one thing, but who is ever going to approve that?
Then because time is money, you just lose similar amounts of money over time.
No. Maybe if you maximized your time 100% toward earning money. Maybe (but almost certainly not). But if you did that, you'd be in the back of a Maybach or a 7 Series, working, and someone else would be driving. And they could do the speed limit.
Because the people who should be working hard to change government simply aren’t. In the US, city governments ARE “the people”. If you have a city council, start going to meetings. It’s a great way to see how your local government actually works (or doesn’t work), and you might even meet some interesting people in the process. Otherwise, run for local office or try to get on a board or a commission. It’s not sexy, but this stuff, especially things like transportation and zoning, have the most impact at the municipal level, and it’s remarkably easy to get involved.
Truth.
My city has a thing called a target area. The idea being to spiff a region up, and rotate them to keep the city vital and livable overall.
They send a facillitator, who gathers interested people. Some projects get identified and everyone does their part. The people in need of city resources or people get an introduction and help navigating things.
Everyone else plays a role. Labor, outside (not city budget) fundraising, planning, feed the people, organizing, whatever.
My group used it's few years well. Traffic flow changes, a small park made from abandoned property, refurbish the school play areas and equipment.
It took a bit of time and some sweat, but not too much.
And I can drive through today and see that net good.
Some of the people I worked with did exactly what you just said, and for basic, make it better, reasons.
My city has a thing called a target area. The idea being to spiff a region up, and rotate them to keep the city vital and livable overall.
They send a facillitator, who gathers interested people. Some projects get identified and everyone does their part. The people in need of city resources or people get an introduction and help navigating things.
Everyone else plays a role. Labor, outside (not city budget) fundraising, planning, feed the people, organizing, whatever.
My group used it's few years well. Traffic flow changes, a small park made from abandoned property, refurbish the school play areas and equipment.
It took a bit of time and some sweat, but not too much.
And I can drive through today and see that net good.
Some of the people I worked with did exactly what you just said, and for basic, make it better, reasons.
Because it goes completely against the interests of local governments. You will be amazed to see how large a portion of your city and police department's budgets are from fines.
That's true of some fines in some places, but to be clear, it's not really true of parking fines in Chicago. The graph in the article indicates there's about $1 million dollars in tickets issued each year. That's around $ .30 per resident and less than one percent of the billions in revenue the city takes in each year (https://www.cityofchicago.org/content/dam/city/depts/obm/sup..., see page 23 in particular) I did notice that fines in general amount to $300 million a year, so other fines might well be inflated.
I believe the y-axis on that graph (assuming you refer to the last graph) is the number of parking tickets, not the dollar amount of fines. So more like $50 to $100 million in parking fines per year.
You're absolutely right. The graph could have been clearer, but I should have double-checked my assumptions and that $.30 per resident figure should have been a red flag. I apologize for that basic error.
Looking into this more, it sounds like, while there probably is a perverse incentive because more tickets equals a significant increase in revenue, the biggest problem is the punitive fines for minor non-traffic offenses that tend to compound for poorer residents: https://www.motherjones.com/crime-justice/2018/02/how-does-c.... Note the headline is a little misleading, they seem to mean "non-moving violations".
Looking into this more, it sounds like, while there probably is a perverse incentive because more tickets equals a significant increase in revenue, the biggest problem is the punitive fines for minor non-traffic offenses that tend to compound for poorer residents: https://www.motherjones.com/crime-justice/2018/02/how-does-c.... Note the headline is a little misleading, they seem to mean "non-moving violations".
This one change cost the government 60K. If changes like this are implemented all over the city, how will they pay for the IBM contract?
That's nothing at the scale of Chicago. Their seven-cent tax on plastic bags is expected to bring in $9 million dollars this year (http://www.chicagotribune.com/business/ct-chicago-bag-tax-us...).
Perhaps they wouldn't need such an expensive contract if they didn't have to process so many tickets caused by bad signage?
Heaven forbid that bidding be competitive and that contractors be held accountable.
Civil forfeiture?
You’re not wrong, not in the slightest. The benefit is the system allows concerned citizens to step in to make improvements at very little cost.
Democracy requires constant vigilance in many forms. This is one such form.
Matt, we salute you and your efforts! I hope this encourages others to get involved in improving their local government (and perhaps even creates reusable tooling for use at scale [“citizen oversight as code”]).
Democracy requires constant vigilance in many forms. This is one such form.
Matt, we salute you and your efforts! I hope this encourages others to get involved in improving their local government (and perhaps even creates reusable tooling for use at scale [“citizen oversight as code”]).
Right?
They could be. And having a beer or two with govt people will reveal these kinds of desires and ideas.
I have had those chats in the past. Got involved in a legislative effort and was given a sort of insiders view, tour.
What gets in the way, the number one thing, is money. Not lack of it so much as priorities and ripple effects.
Fixing the signs is a net public good. The ripple effect might be revenue targets going down, and the priority being that revenue being made from inane parking tickets all make for a bit of a mess.
The number two is people forgetting or ignoring who works for who and why. There are lots of little fiefdoms, all closely guarded. Barriers where there really should be collaboration.
And on that note, collaboration can be expensive. Sure, we can step out of your way on this, but about that school levy...
They could be. And having a beer or two with govt people will reveal these kinds of desires and ideas.
I have had those chats in the past. Got involved in a legislative effort and was given a sort of insiders view, tour.
What gets in the way, the number one thing, is money. Not lack of it so much as priorities and ripple effects.
Fixing the signs is a net public good. The ripple effect might be revenue targets going down, and the priority being that revenue being made from inane parking tickets all make for a bit of a mess.
The number two is people forgetting or ignoring who works for who and why. There are lots of little fiefdoms, all closely guarded. Barriers where there really should be collaboration.
And on that note, collaboration can be expensive. Sure, we can step out of your way on this, but about that school levy...
Because different people have different priorities.
Remember, you're part of the government.
Remember, you're part of the government.
> Remember, you're part of the government.
This is a really important idea, and I don't think I've seen it expressed so clearly before. Thank you.
This is a really important idea, and I don't think I've seen it expressed so clearly before. Thank you.
Because it's nigh impossible to get someone to understand something (much less actively do large amounts of work to fix it) when their income (or department budget) depends on them not understanding it.
The government is a collection of people with the own motivations. It is more likely that people working in the government would use their time to further their own personal beliefs than to fix a problem that does not affect them
What, work to reduce their revenue?
Nice humor :)
That's what I'm saying!
So many PDFs......
Hey! Thanks for posting this, and the kind words.
Interesting note about getting data like this - Illinois FOIA allows a requester to submit a SQL as part of their request.. so long as they know the tables and columns within the database ;)
Interesting note about getting data like this - Illinois FOIA allows a requester to submit a SQL as part of their request.. so long as they know the tables and columns within the database ;)
If that’s possible then the first two requests should be:
SELECT *
FROM information_schema.tables
SELECT *
FROM information_schema.columnsNot all SQL products keep their schema searchable, but good try.
Your information request has been declined due to unreasonable effort required to answer.
I’m rusty, can you remind me of the SQL statement to run against the master tables to give me the schema? :-)
Not only are you a certified urban hero, you don't even need a costume!
Not only did you save drivers $$$thousands, but you freed up at least one of 'Chicago's Finest' to fight more serious crime.
Not only did you save drivers $$$thousands, but you freed up at least one of 'Chicago's Finest' to fight more serious crime.
So first you submit a FOIA request for the database schema...
Funny enough, I submitted one of those to all of the biggest cities in each state for a list of their databases and the columns/tables within each. Pretty much universal rejection.
The ones that didn't send me a rejection letter all told me that the cost would be somewhere between $400 and $2,000 to complete the work. While it's nice, and I've spent similar on requests before, it's not something I'm exactly interested in doing repeatedly, since this is all coming from my personal savings.
That said... I'm still fighting that fight in Chicago. Chicago used to post a "data dictionary" of 111 of their databases, which had a list of the tables and columns. Problem is... it isn't running anymore since it's no longer being funded by a $300k (!) grant. So, I'm working on requesting a database dump of that. Last request for it was rejected saying that Chicago's Department of Innovation and Technology (DoIT) doesn't mave any of those records. I'm thinking it's because I requested a "copy" of the database dump, rather than just explicitly asking for the data within it.
That opens up a pretty nifty door to work with for future requests ;).
The ones that didn't send me a rejection letter all told me that the cost would be somewhere between $400 and $2,000 to complete the work. While it's nice, and I've spent similar on requests before, it's not something I'm exactly interested in doing repeatedly, since this is all coming from my personal savings.
That said... I'm still fighting that fight in Chicago. Chicago used to post a "data dictionary" of 111 of their databases, which had a list of the tables and columns. Problem is... it isn't running anymore since it's no longer being funded by a $300k (!) grant. So, I'm working on requesting a database dump of that. Last request for it was rejected saying that Chicago's Department of Innovation and Technology (DoIT) doesn't mave any of those records. I'm thinking it's because I requested a "copy" of the database dump, rather than just explicitly asking for the data within it.
That opens up a pretty nifty door to work with for future requests ;).
I'm reasonably sure that in UK FOIA requests have to be handled at cost? It's that true in Chicago.
Mind you the company responsible probably said "the database schema will cost $2k to put in a PDF".
Mind you the company responsible probably said "the database schema will cost $2k to put in a PDF".
It varies by state in the US (what doesn't?), but most states do have a clause in their FOIA law that says something along those lines as well. It may be completely ambiguous ("at cost", fees must be "reasonable", etc.) or it can be more specific ("may not exceed the hourly salary of the lowest paid employee who can do the work", etc.).
Of course the former is pretty worthless, especially if the state doesn't outline any formal appeals process - you'd have to actually sue them for the records, which is rarely going to end up being cheaper. It also doesn't mean that the actual cost couldn't end up being insane. If no one has access to perform that particular query or knows how to get that data it could involve requesting a custom report from whomever developed and/or maintains the system. No one is reasonably going to pay IBM's hourly rate to write a report for them...
Of course the former is pretty worthless, especially if the state doesn't outline any formal appeals process - you'd have to actually sue them for the records, which is rarely going to end up being cheaper. It also doesn't mean that the actual cost couldn't end up being insane. If no one has access to perform that particular query or knows how to get that data it could involve requesting a custom report from whomever developed and/or maintains the system. No one is reasonably going to pay IBM's hourly rate to write a report for them...
And don't forget if their DBA team is all consultants, "at cost" for the state just included that DBA's time, putting together the scope of work for the client, writing the proposal, management approval, etc. Generating a PDF of all tables and all columns in a presentation-level format could definitely cost the state $2k.
I work for a company that has a lot of state consultants (>50% of the workforce) and our salaries are typically 2-2.5x our state counterparts, and most items of work that would take more than an hour or two will require at least one level of management approval on both sides.
I work for a company that has a lot of state consultants (>50% of the workforce) and our salaries are typically 2-2.5x our state counterparts, and most items of work that would take more than an hour or two will require at least one level of management approval on both sides.
Can we do crowdfunding to get that started again?
Yep - I have two ways open now:
A donation to my non-profit, Free Our Info, NFP - which is the company I'm submitting requests for DBs' info through (and many other requests): https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_b...
The other's through Muckrock, for specific requests: https://www.muckrock.com/project/tables-and-columns-from-gov... (I'll add more requests to this list tomorrow)
The other's through Muckrock, for specific requests: https://www.muckrock.com/project/tables-and-columns-from-gov... (I'll add more requests to this list tomorrow)
Do you have any other chicago-related datasets available?
Now wouldn't it be funny if you got them to drop a table this way?
Wouldn't you assign readonly permissions to the account used to fulfill these requests?
Ideally, but until someone asks for it the first time, they probably don't even have an account for the purpose. So, an irritated DBA who has myriad other backlog tasks probably runs it on a copy if smart, and production if lazy or in a hurry. Would not surprise me in the slightest if an important table was dropped in this way.
Yes, but if they're not sanitizing inputs, they might not have set permissions wisely either.
[deleted]
This is one of my all time favorite HN posts, a really good deed done with Unix.
So inspiring!
So inspiring!
Do you have any specific information about this or the relevant regulations?
Perhaps we need a meta-site with best practices and how best to attack open records requests in general.
Perhaps we need a meta-site with best practices and how best to attack open records requests in general.
You should check out muckrock.com!
It's rare to see someone go above and beyond a benevolent hobby to build a complete solution to a people's problem.
Actually Chicago has an extensive open data access platform. Although it's not SQL but it is pretty interesting.
https://data.cityofchicago.org
https://data.cityofchicago.org
Thanks for this! It inspired me to see if Austin has one too. It does, and it appears to be using the same application to serve up the data: https://data.austintexas.gov
This is awesome and inspiring. Any thoughts on how to approach something similar for red light camera violations and accident/police data in Illinois?
I had a buddy in college whose main hobby was sending foias. The amount of information he was able to collect on me and all of our friends since it was a public university was astounding. He wrote a little google map applet that matched the student base's names to their parent's homes. He only showed about 6 of us what he did and we were all horrified and amazed at the same time. I miss CpE college.
It is neat.
(Not all of my post came through)
Great example of value being in data. One that ordinary people can connect the dots on and encourage.
(Not all of my post came through)
Great example of value being in data. One that ordinary people can connect the dots on and encourage.
This is great! I often request records but have never come up with a great use case!
Under Florida public record law, source code produced by state employees is, in very narrow circumstances, a non-exempt public record (the code can't process sensitive data, etc.). I'm considering a future endeavor where I periodically request the code to such projects until the I.T. department decides it's worth the effort to open source it.
I like to think this is a step towards consolidating publicly funded code and reducing duplicate effort. Ahh, imagine making a pull request to your city's website! But I'm getting ahead of myself...
Under Florida public record law, source code produced by state employees is, in very narrow circumstances, a non-exempt public record (the code can't process sensitive data, etc.). I'm considering a future endeavor where I periodically request the code to such projects until the I.T. department decides it's worth the effort to open source it.
I like to think this is a step towards consolidating publicly funded code and reducing duplicate effort. Ahh, imagine making a pull request to your city's website! But I'm getting ahead of myself...
Interesting - any idea what the other circumstance are? Is there a statute for this? Have you considered requesting some specific source code and publishing it yourself? Might make sense to start small here.
I have a lot of experience in making public records requests and would be happy to help.
I have a lot of experience in making public records requests and would be happy to help.
This is awesome. Question though: how is producing license plate data like this not a disallowed privacy invasion? It seems like you could totally track who's parking where and potentially do nasty stuff, if you know (say) someone well-off whom you don't like and who doesn't seem to mind getting tickets on a regular basis.
Thanks!
I thought so to until recently and was honestly kind of surprised they actually gave it to me. They rejected giving license plate info at first, but they've given it out in other, similar, FOIA requests.
Specifically in FOIA's statute, it says:
I thought so to until recently and was honestly kind of surprised they actually gave it to me. They rejected giving license plate info at first, but they've given it out in other, similar, FOIA requests.
Specifically in FOIA's statute, it says:
(c-5) "Private information" means unique identifiers, including a person's social security number, driver's license number, employee identification number, biometric identifiers, personal financial information, passwords or other access codes, medical records, home or personal telephone numbers, and personal email addresses. Private information also includes home address and personal license plates, *except as otherwise provided by law or when compiled without possibility of attribution to any person.*
That is scary, thanks for sharing. And for those here who don't seem to be worried about the safety aspect, note that this means any corporation with a lot of your personal info (insurance, Equifax, etc.) could just submit a FOIA request to get the license plate number of anyone in Illinois (and likely other states) who's parked anywhere illegally, whether intentionally or accidentally... and then strike a deal to gather license plate scanning data to track where you go and what you do. Heck, I wonder if this sort of thing is already happening and we just don't know about it.
I think, because he glossed over it in the post, that you are underestimating how much work Matt does to get the data & how much the governments he works with try to hinder & redact it.
In some cases that is a very good thing. In other cases it’s just them trying to obfuscate and block transparency.
In some cases that is a very good thing. In other cases it’s just them trying to obfuscate and block transparency.
[deleted]
In the US there is an entire industry that scans license plates on public streets and sells the data. Basically no privacy in public.
(random example: https://www.syracuse.com/news/index.ssf/2015/01/private_comp... )
(random example: https://www.syracuse.com/news/index.ssf/2015/01/private_comp... )
I forgot about this, but at the same time that doesn't imply it's legal for a government agency to just give it away. There are obviously industries that gather up all your personal information from address and DoB to SSN but that doesn't mean a government agency can just give it out to a random nobody.
The difference in rational is pretty simple:
A plate identifies a vehicle. A vehicle is not a person. Govt can't give out personally identifiable information.
But a plate identifies a vehicle _and_ its owner—not just a vehicle.
A vehicle’s owner is public information in the US. Credit Karma knows the vehicles I own based on DMV records, and recommends me insurance accordingly.
And they know that because your PII (name, address, social) is mapped to a profile, and that profile is also mapped to license plates and Vins. So you can use one to look up the other info in either direction.
Addresses can be used that way to, but an address as a stand alone data item is used for every public land record in existence.
Think of a license plate as an 'address' for a car.
Think of a license plate as an 'address' for a car.
Criminal proceedings are usually public in most countries. Where do parking tickets fall? Even if not technically criminal, some might, for the same underlying reasons, consider it acceptable for "someone well-off whom you don't like and who doesn't seem to mind getting tickets on a regular basis" to have this illegal behavior on the public record. "Don't want your name tarnished? Don't park illegally."
[Edit: on the other hand, if the ticket is unfair (eg. confusing signage as in this example), then you have a valid point; I just wanted to point out the other side of the coin]
[Edit: on the other hand, if the ticket is unfair (eg. confusing signage as in this example), then you have a valid point; I just wanted to point out the other side of the coin]
I was asking about the legality, not the morality. If it's not technically criminal then that's all that matters.
Also, this is completely missing the point here:
> Don't want your name tarnished? Don't park illegally.
It's not about reputation, it's about privacy -- and safety.
Also, this is completely missing the point here:
> Don't want your name tarnished? Don't park illegally.
It's not about reputation, it's about privacy -- and safety.
Don't know about where you are, but in Ontario Canada (the only place I've gotten in enough trouble to care about the difference ;-) ), there is a criminal code and the highway traffic act. There are illegal things that you can do in your car that may be in either or both areas.
"Criminal" means in the criminal code, but both are illegal. I think that you don't have a right to privacy for either because it obfuscates the application of the law. Indeed, the Japanese government can query the Ontario government to get a list of transgressions that you had while driving a car in Ontario (I know this because they did so when I converted my driver's license to a Japanese one -- and they didn't need my consent).
I think OP's use of the term "criminal" is a bit loose, but I would be surprised if you have any right to privacy for a a fine levied due to a legal infraction. Whether or not you should have a right to privacy is a completely different conversation...
Aside: It was important to me because many years ago I inadvertently drove while suspended. I had an unpaid ticket that I had forgotten about and my license was suspended. The suspension got lost in the mail (first a postal strike and then the delivery person put my mail in the wrong "super box" -- I eventually got it months later). When I was first getting my visa for Japan, I needed to find out if this was a criminal offence or a highway traffic act offence.
"Criminal" means in the criminal code, but both are illegal. I think that you don't have a right to privacy for either because it obfuscates the application of the law. Indeed, the Japanese government can query the Ontario government to get a list of transgressions that you had while driving a car in Ontario (I know this because they did so when I converted my driver's license to a Japanese one -- and they didn't need my consent).
I think OP's use of the term "criminal" is a bit loose, but I would be surprised if you have any right to privacy for a a fine levied due to a legal infraction. Whether or not you should have a right to privacy is a completely different conversation...
Aside: It was important to me because many years ago I inadvertently drove while suspended. I had an unpaid ticket that I had forgotten about and my license was suspended. The suspension got lost in the mail (first a postal strike and then the delivery person put my mail in the wrong "super box" -- I eventually got it months later). When I was first getting my visa for Japan, I needed to find out if this was a criminal offence or a highway traffic act offence.
In WA at least all traffic offenses are public record. You can go search court databases by either name or license plate to get the full details of any tickets and charges in that system.
What you say is true. It's also the case that there's a lot of information that is nominally public--often for very defensible reasons--which historically took significant effort to get at and, even then, only in a very limited and manual way.
Today, a lot of that information is at least a lot more accessible to everyone (though in this case it still took a lot of work) and, furthermore, it can be mashed up with other public or semi-public data.
I'm pretty sure this is something we'll be collectively be coming to terms with for a long time.
Today, a lot of that information is at least a lot more accessible to everyone (though in this case it still took a lot of work) and, furthermore, it can be mashed up with other public or semi-public data.
I'm pretty sure this is something we'll be collectively be coming to terms with for a long time.
Voter registration records being a perfect example - anyone can look up my address because the state makes it incredibly easy to pop my name into a website and there it is.
Of course, historically you could look up the address for most people through the white pages.
But that doesn't change the basic point. If you were to ask if you should be able to look up anyone's physical address by typing their name into a web page, I suspect many people would say no. Yet, here we are.
(I also suspect that many would be really shocked at the amount of info available about them from "deep web" searches much less via a $20 online background check.
But that doesn't change the basic point. If you were to ask if you should be able to look up anyone's physical address by typing their name into a web page, I suspect many people would say no. Yet, here we are.
(I also suspect that many would be really shocked at the amount of info available about them from "deep web" searches much less via a $20 online background check.
Parking tickets aren't criminal offenses.
In a lot of states you used to be able to get the address of a license plate with $5 in admin fees. Not sure if that is still allowed.
[deleted]
The TL;DR is: "any criminal records is public information". Fundamentally, even if it's very minor, it's public information until expunged, cleared, or sealed. I think most people are generally fine with this, it's only really a problem if you consistently are committing offenses.
I wouldn't be that generous. You park 15 mins longer than permitted once outside your workplace, and now your violent ex knows where you now work.
Maybe. But I think the benefit to society of publicly knowing violations outweighs this potential abuse. Also, it would be a problem to allow government to secretly prosecute, even for very minor offenses.
I'm very skeptical about drawing a line from "not sharing license plate numbers tied to address of offense when it comes to traffic fines" to "secretly prosecute".
This is not criminal though?
It is. If you get a parking ticket that means you are being charged with a crime, the crime is just a minor infraction.
Parking tickets are civil, not criminal.
Source? I'm not a lawyer and I don't know Illinois law, but I have generally read these are civil and not criminal offenses, e.g. https://www.wisemantriallaw.com/blog/2012/august/the-differe...
If that were the case, wouldn't the municipality be required to provide a defense attorney? Or is that only for felonies?
Would be nice to use the same skills to help reduce cars illegally parked in the bike lane. Identify areas which cyclists commonly complain about that (e.g., to the city) and encourage them to put up better signs?
When people routinely park in bike lanes, the problem is usually cultural, i.e. people know they're not supposed to do it, but they decide to do it anyway.
Near where I work in Bellevue WA, they recently restriped the road to have a brightly painted bike lane, with double-white lines to make it abundantly clear that you were not supposed to drive in it. Bright red "no stopping" signs were placed on the curb. People still parked right in the bike lane.
It wasn't until they added a concrete barrier that the lane cleared up enough that bikes could use it. And of course, right where the barrier ends, people start parking there instead. The West side seems to have less difficulty understanding this.
Near where I work in Bellevue WA, they recently restriped the road to have a brightly painted bike lane, with double-white lines to make it abundantly clear that you were not supposed to drive in it. Bright red "no stopping" signs were placed on the curb. People still parked right in the bike lane.
It wasn't until they added a concrete barrier that the lane cleared up enough that bikes could use it. And of course, right where the barrier ends, people start parking there instead. The West side seems to have less difficulty understanding this.
I think I know where you're talking about and I bike through there and the biggest problem is that they took away the "3 minute passenger drop-off/pick-up" spot right next to the transit center without giving any other options for passenger loading/unloading.
This is a big issue. I cycle to work everyday (Oakland) and there are at least a dozen people parked or stopped in the bike lanes. The big issue is that there is no where to stop. Parking is cheap/free and not time limited so most cars don't really move and the very few short term spots aren't ever enforced. You can drive around for 15 minutes looking for a spot or you end up in the bike lane. This isn't good but its at least somewhat understandable. This is the case around transit stations and busy restaurant/commercial streets.
> The West side seems to have less difficulty understanding this.
We still have the unmarked white vans over here. I think it's Amazon.
We still have the unmarked white vans over here. I think it's Amazon.
If you're interested in bike lane parking tickets, you might want to check out this article, which was done using the same dataset:
https://www.wbez.org/shows/wbez-news/where-chicago-tickets-f...
https://www.wbez.org/shows/wbez-news/where-chicago-tickets-f...
This problem would solve itself if vandalism laws were changed slightly.
On that note, watching videos from the Russian "Stop a Douchebag" movement can be cathartic, though even their mild form of vandalism is illegal.
https://law.stackexchange.com/questions/16564/would-stop-a-d...
https://law.stackexchange.com/questions/16564/would-stop-a-d...
Man, Russian sidewalks are huge, I sort of understand thinking you could drive on them.
Bike Lane Uprising maintains data on this. Maybe it would be interesting to contact them for the dataset.
Could also signal the area is starved for parking, so the lane should be removed and bicyclists should just ride in the road for those stretches. Small inconvenience to the bicyclists who still get to use the road, big win for the drivers.
Good lord, just park and walk. Or use that bike lane. Cities have an overabundance of expensive parking [1], which does nothing but waste space and act as giant thermal batteries. Everyone loves complaining about traffic, but we don’t seem to have the political will to do something about it (namely: reduce our dependence on cars). Cities should work to reduce their dependence on cars, not increase it at the expense of cyclists and pedestrians, who are actually the life blood of cities (think about how many pedestrians stop into small business compared to drivers speeding by at 35 mi/h).
[1] https://www.strongtowns.org/journal/more-evidence-that-we-ha...
[1] https://www.strongtowns.org/journal/more-evidence-that-we-ha...
And how do you travel more than a couple miles and arrive on-time, during the summer, wearing businesslike attire, without smelling like a cyclist upon arrival?
Don't wear businesslike attire when bicycling in hot weather. You pack it, instead, and ride at a comfortable pace instead of doing speed-racer stuff. Pack a thermos of ice water to drink from occasionally and you won't even break a sweat.
Or use an e-bike so you don't have to pedal as hard.
I live in the Boston area and I see a ton of people in business attire riding bikes.
Or use an e-bike so you don't have to pedal as hard.
I live in the Boston area and I see a ton of people in business attire riding bikes.
> Boston area
That explains. Contrast the climates of Georgia or Florida.
During the summers here, just standing outdooors in the sunlight wearing anything heavier than beachwear, is inadvisable at best.
That explains. Contrast the climates of Georgia or Florida.
During the summers here, just standing outdooors in the sunlight wearing anything heavier than beachwear, is inadvisable at best.
I commute by bike daily in Austin, TX. In the summer I bring a change of clothes and wear a damp athletic shirt and damp headwrap. My commute is relatively short as well. I don't shower when I get to work and most of the time I smell perfectly fine. (Confirmed by people who have no problem contradicting me.) Yes, you'll sweat, but nowhere near as much as most people think you will if you plan it well.
Humidity would make this more difficult. Austin's not as humid as Houston, but it's not a desert either.
Humidity would make this more difficult. Austin's not as humid as Houston, but it's not a desert either.
Boston gets pretty humid in the summer. Drinking ice water towards the end of the ride works really well.
I ride 6 miles to work and pack my work clothes. It only takes me 25-30 minutes. If I arrive to work sweaty, I shower. I know not everyone has the luxury of having a shower at work, but every company I've worked at so far has had one.
On the ride home I change back into my riding clothes and spray sunscreen. Just keep hydrated and you will be fine.
On the ride home I change back into my riding clothes and spray sunscreen. Just keep hydrated and you will be fine.
Perhaps. Still requires more discomfort and ritual than driving a car.
My impression is that most people think cycling regularly is far more difficult than it actually is.
You'll adapt in more than one way. You'll get in better shape, so you won't sweat as much or be as tired by the cycling (or tired at all). More importantly, your attitude towards the discomfort will change. Yes, it takes effort and can be uncomfortable, but it's all perfectly acceptable after a while. Seems like a good example of the hedonic treadmill.
You'll adapt in more than one way. You'll get in better shape, so you won't sweat as much or be as tired by the cycling (or tired at all). More importantly, your attitude towards the discomfort will change. Yes, it takes effort and can be uncomfortable, but it's all perfectly acceptable after a while. Seems like a good example of the hedonic treadmill.
> better shape
I already run about 25 miles per week, but thanks for your concern.
The thing is, I get up at the crack of day so as to do it in coldest weather, I wear appropriate athletic attire, I'm unladen by luggage, and if I feel too tired or something goes wrong then I lose a workout instead of my job.
I already run about 25 miles per week, but thanks for your concern.
The thing is, I get up at the crack of day so as to do it in coldest weather, I wear appropriate athletic attire, I'm unladen by luggage, and if I feel too tired or something goes wrong then I lose a workout instead of my job.
None of the things you mention can't be done while cycling. Many of us get up early to beat the heat, wear reasonable clothes, and drive or take the bus or whatnot if cycling is not possible. I have reasonable cargo capacity on my bike so a normal day's bag is not a problem. Cycling in this case could replace a workout for you which could overall save you time given that it doubles as a commute. I used to be a fairly dedicated runner, but I decided cycling was a better use of my time.
Bus. Subway. Ride-share. Taxi.
> Bus.
Takes 90 minutes to do what my car can do in under 30, and I don't want to wake up yet another hour earlier.
> Subway.
Doesn't run to the area where I live; and would have to run even farther to reach the office. I suspect the NIMBYs want it like that.
> Ride-share.
I'm a miserly bachelor in an office full of respectable familypeople. None of them would dare to live within 2 zipcodes of where I live.
> Taxi.
Is shockingly expensive to use for daily commuting, moreso than financing, fueling and maintaining my own car. Plus, unless rush-hour lasts long enough, each cab will only get one commute ride per rush-hour, so there's no net decrease in traffic for commuting via cab.
Takes 90 minutes to do what my car can do in under 30, and I don't want to wake up yet another hour earlier.
> Subway.
Doesn't run to the area where I live; and would have to run even farther to reach the office. I suspect the NIMBYs want it like that.
> Ride-share.
I'm a miserly bachelor in an office full of respectable familypeople. None of them would dare to live within 2 zipcodes of where I live.
> Taxi.
Is shockingly expensive to use for daily commuting, moreso than financing, fueling and maintaining my own car. Plus, unless rush-hour lasts long enough, each cab will only get one commute ride per rush-hour, so there's no net decrease in traffic for commuting via cab.
It definitely seems like your city is determined to sacrifice everything necessary to uphold their car culture.
Outside of the Pacific-assisted wonder-weather zone of California, summers tend to be hot and humid, and winters aren't particularly mild either. Any amount of travel without the aid of climate control is ... undesirable.
Sure, but things like under- and above-ground rail exist, reasonable bus routes exist, good city planning exists. Punting everything to "well everyone har a car" is effectively giving up.
Perhaps. Then again, judging by the few times I've taken the metro for an inner-city errand, I'm not sure I'd want to ride it to work even if the line did run far enough.
At least, not without ... martial-arts training? Or maybe a "penniless grad-student" disguise?
At least, not without ... martial-arts training? Or maybe a "penniless grad-student" disguise?
In my area, there are places you have to "park and walk" to and places you can just drive to. Second category are the only ones that get my business.
How far a walk is too much for you?
I avoid having to deal with crosswalks if I can. I also regularly make a 1/2 mile drive to a fast food restaurant.
Do you live in a city or a suburb?
It's like a dense suburbish area. There are both plazas/malls/grocery stores with normal parking and areas set up like small imitation city blocks with parallel parking or no immediate parking. I don't like having to deal with street parking either unless it's a nice clear zone.
In my experience (Austin, TX), drivers usually park in the bike lane for convenience, not because parking is unavailable. I talk to many drivers who park in the bike lane about why they didn't park 100 feet away not in the bike lane (which is available and free) and the typical answers are that they didn't know parking in the bike lane was illegal (despite the signs) or that they didn't want to walk (even though it's rarely a long walk to legal parking where I ride).
I can recall a discussion with someone on here about how parking in the bike lane in San Francisco is usually caused by a lack of options. That might be true, but I think most US cities are a lot closer to Austin than SF.
I can recall a discussion with someone on here about how parking in the bike lane in San Francisco is usually caused by a lack of options. That might be true, but I think most US cities are a lot closer to Austin than SF.
It's not a small inconvenience for cyclists, it's a large decrease in safety
A road with street parking probably isn't faster than 30mph.
The road behind my house has street parking and it’s a 45 mph zone.
Yes, small inconvenience. Just slightly less safe, right?
http://www.abc.net.au/news/2018-08-12/new-york-truck-driver-...
http://www.abc.net.au/news/2018-08-12/new-york-truck-driver-...
Reminds me of working with free-form, manual entry order detail information, in a former life.
Hundreds of thousands of records a month. I ended up importing them into Excel(1) and then using... what was that called? An MS/Windows library that came with IE 5 and/or a few other things, that provided regex support (with a few quirks) that was accessible via VBA.
The point was, I could programmatically mine it -- including regex pattern matching and replacement of and within cell contents -- while also having a flexible UI within which to find and handle one-off cases. When the one-off's demonstrated a repeating pattern, I could quickly iterate to add that to the programmatic mining logic.
This included adding color cueing for items of particular interest, manual follow-up. Excel's sorting capabilities to bring potentially related instances into visually displayed groups. And the like.
It ended up working quite well. I might have preferred something else to VBA, and I did use Perl and other stuff, elsewhere (something that also gave me both power and the flexibility to rapidly iterate).
But the point is, with such data, I found it very useful to combine regex and rapid programmatic manipulation, together with a good visual interface (including visual cues, the ability to comment upon instances -- Excel cell-level comments -- etc.) and manual manipulation.
As a final aside, the extensive set of Excel keyboard shortcuts greatly aided in rapidly and effectively navigating and massaging the imported data.
--
1. This was back when Excel had... I think it was a 64K (or a bit less) limit on the number of rows in a sheet.
P.S. I tended to retain the originally imported data in its columns, and to produce my mining of it in other columns. That way, I could always and immediately see what I started with, for any particular record. (And, if things visually started to be "too many columns", well, Excel lets you hide a range of columns from the view. As one example of how its features really helped, on the visual front while doing this work.)
I still had to learn and allow for some quirks Excel exhibited with respect to importing text data. That included making sure the cells/columns being imported into carried the correct/needed formatting designation before importing into them (usually, "Text").
Hundreds of thousands of records a month. I ended up importing them into Excel(1) and then using... what was that called? An MS/Windows library that came with IE 5 and/or a few other things, that provided regex support (with a few quirks) that was accessible via VBA.
The point was, I could programmatically mine it -- including regex pattern matching and replacement of and within cell contents -- while also having a flexible UI within which to find and handle one-off cases. When the one-off's demonstrated a repeating pattern, I could quickly iterate to add that to the programmatic mining logic.
This included adding color cueing for items of particular interest, manual follow-up. Excel's sorting capabilities to bring potentially related instances into visually displayed groups. And the like.
It ended up working quite well. I might have preferred something else to VBA, and I did use Perl and other stuff, elsewhere (something that also gave me both power and the flexibility to rapidly iterate).
But the point is, with such data, I found it very useful to combine regex and rapid programmatic manipulation, together with a good visual interface (including visual cues, the ability to comment upon instances -- Excel cell-level comments -- etc.) and manual manipulation.
As a final aside, the extensive set of Excel keyboard shortcuts greatly aided in rapidly and effectively navigating and massaging the imported data.
--
1. This was back when Excel had... I think it was a 64K (or a bit less) limit on the number of rows in a sheet.
P.S. I tended to retain the originally imported data in its columns, and to produce my mining of it in other columns. That way, I could always and immediately see what I started with, for any particular record. (And, if things visually started to be "too many columns", well, Excel lets you hide a range of columns from the view. As one example of how its features really helped, on the visual front while doing this work.)
I still had to learn and allow for some quirks Excel exhibited with respect to importing text data. That included making sure the cells/columns being imported into carried the correct/needed formatting designation before importing into them (usually, "Text").
I'm pretty sure what I was thinking of is Windows Scripting Host (WSH). At the time, I picked it up as a part of IE 5.
https://en.wikipedia.org/wiki/Windows_Script_Host
https://en.wikipedia.org/wiki/Windows_Script_Host
I did something similar for universities, to help students select their courses:
https://easy-a.net/
I wrote a blog post about it, because it requires a ton of work to get FOIA requested data - this I'm assuming was done in the same painstaking way:
https://austingwalters.com/foia-requesting-100-universities/
I give this props. I'm sure it required a ton of work
https://easy-a.net/
I wrote a blog post about it, because it requires a ton of work to get FOIA requested data - this I'm assuming was done in the same painstaking way:
https://austingwalters.com/foia-requesting-100-universities/
I give this props. I'm sure it required a ton of work
Are there any resources explaining the FOIA process? I'm not sure what types of information is available, what it can be used for, etc and am always amazed with the type of information people are able to get the government to hand over.
I had this question as well and asked someone who did a FOIA request. There is no listing. It's just that, if you notice or can logically conclude that a certain kind of data exists, you can request it. In this example, it is fairly logical that the city has a record of parking tickets that were written out, and so the author requested them.
I'm surprised he asked after license plates, though. I don't know if that is different in the USA, but in Europe that certainly wouldn't fly because of privacy. I wouldn't even have asked because I shouldn't want to have such data. Perhaps one could get an anonymized version to be able to correlate how often a certain plate got a ticket, but not which plate that was. Anyway, the general concept of a FOIA request is the same. (Edit: Oh, someone else remarked this as well: https://news.ycombinator.com/item?id=17754396)
I'm surprised he asked after license plates, though. I don't know if that is different in the USA, but in Europe that certainly wouldn't fly because of privacy. I wouldn't even have asked because I shouldn't want to have such data. Perhaps one could get an anonymized version to be able to correlate how often a certain plate got a ticket, but not which plate that was. Anyway, the general concept of a FOIA request is the same. (Edit: Oh, someone else remarked this as well: https://news.ycombinator.com/item?id=17754396)
I too feel I couldn't pass the $190m cost in the first place. Granted, I can see where the cost ramps up as explained by @morei. Could someone explain whether this is for the 10-year contract or a license of some sort for each year?
If it is annually, they got 17m tickets over 7 years so for 10 years, assuming they issue just over 19m tickets, that means each parking ticket needs to be at least $10 to cover the cost, even at $100 per ticket, IBM is banking on 10% share? That seems excessive to me but I never worked in government so could someone enlighten me on this?
By any chance there's a conflict of interest for government to be willing to make improvement and cut down parking tickets or any other similar source of income? Or maybe that's what public audit is for?
If it is annually, they got 17m tickets over 7 years so for 10 years, assuming they issue just over 19m tickets, that means each parking ticket needs to be at least $10 to cover the cost, even at $100 per ticket, IBM is banking on 10% share? That seems excessive to me but I never worked in government so could someone enlighten me on this?
By any chance there's a conflict of interest for government to be willing to make improvement and cut down parking tickets or any other similar source of income? Or maybe that's what public audit is for?
Wow, great stuff!
Did you give more thought into the address cleaning bit? Or does anyone have an idea how to go about transforming mangled addresses into coordinates?
I have a problem that's been bothering me for months, similar to what you have here: people from an emergency service call-center are inputting the addresses of the emergencies. For emergencies that happen on the public domain, there is often not a specific address, but rather names of landmarks. Something like "Street StreetName / Opposite Train Station Y", which can be written like "st stName / opp tr st y" or some other infinite variations.
I don't have any after-data to corroborate, but I do have previous instances where the operator inputted the same address better. If I can extract the correct landmarks, I think I can do a Google Places search for them, with a cleaned query, like "Store Amazon, Best Street, Ohio" to get coordinates that can fall into an acceptable area.
PS: in the example you gave with Lake Shore Drive, I think you could easily correct the names with an algorithm based on the Levenshtein distance
Did you give more thought into the address cleaning bit? Or does anyone have an idea how to go about transforming mangled addresses into coordinates?
I have a problem that's been bothering me for months, similar to what you have here: people from an emergency service call-center are inputting the addresses of the emergencies. For emergencies that happen on the public domain, there is often not a specific address, but rather names of landmarks. Something like "Street StreetName / Opposite Train Station Y", which can be written like "st stName / opp tr st y" or some other infinite variations.
I don't have any after-data to corroborate, but I do have previous instances where the operator inputted the same address better. If I can extract the correct landmarks, I think I can do a Google Places search for them, with a cleaned query, like "Store Amazon, Best Street, Ohio" to get coordinates that can fall into an acceptable area.
PS: in the example you gave with Lake Shore Drive, I think you could easily correct the names with an algorithm based on the Levenshtein distance
I've put a LOT of thought into address cleaning! And yep - levenstein distance seems to be the way to go.
My current stack is:
1. Send addresses to https://smartystreets.com/ - They gave me a year's worth of unlimited geocoding for free. They also tokenize the addresses, but I had about a 50% success rate with them.
2. Tokenization raw addresses with https://github.com/datamade/usaddress.
3. Use a normalized levenstein distance algo to get ratio of difference.
4. Compare all of the addresses' levenstein distances with each other.
5. Apply logistical regression/gradient ascent algo to tickets by chaining heavilytypo'd addresses to less-typo'd and eventually to a static list of verified-correct addresses.
It works surprisingly well, but there are still a lot of problems that can't easily be solved:
1. Street types (st/ave/blvd/etc) are missing. So, when two addresses have the same street name, it's difficult to pair the two. It's still possible with some probability stuffs and matching the ticketers' paths to the nearest street.
2. Addresses have a LOT of one-off situations. For example, there's a street name called "Avenue A". The street name here is "Avenue", and the street type (usually st/ave/etc) is "A".
3. Lots of four letter streets make levenstein distance very difficult.
Glad you enjoyed it!
My current stack is:
1. Send addresses to https://smartystreets.com/ - They gave me a year's worth of unlimited geocoding for free. They also tokenize the addresses, but I had about a 50% success rate with them.
2. Tokenization raw addresses with https://github.com/datamade/usaddress.
3. Use a normalized levenstein distance algo to get ratio of difference.
4. Compare all of the addresses' levenstein distances with each other.
5. Apply logistical regression/gradient ascent algo to tickets by chaining heavilytypo'd addresses to less-typo'd and eventually to a static list of verified-correct addresses.
It works surprisingly well, but there are still a lot of problems that can't easily be solved:
1. Street types (st/ave/blvd/etc) are missing. So, when two addresses have the same street name, it's difficult to pair the two. It's still possible with some probability stuffs and matching the ticketers' paths to the nearest street.
2. Addresses have a LOT of one-off situations. For example, there's a street name called "Avenue A". The street name here is "Avenue", and the street type (usually st/ave/etc) is "A".
3. Lots of four letter streets make levenstein distance very difficult.
Glad you enjoyed it!
I did enjoy it, yes, and I'm following your idea for my town also (it's open data here). Lucky for me, it's a little bit prettier (I think they have autocomplete on their devices for the addresses).
I already have some preliminary data - in a city with 350k inhabitants, they gave 150k fines last year, totaling 2.5 mil EUR. I can't wait to search for the hotspots
I already have some preliminary data - in a city with 350k inhabitants, they gave 150k fines last year, totaling 2.5 mil EUR. I can't wait to search for the hotspots
Let me know how it goes!
Maybe try https://github.com/openvenues/libpostal
It's a C library with gigabytes of data, so it isn't light weight, but it attacks the problem aggressively.
It's a C library with gigabytes of data, so it isn't light weight, but it attacks the problem aggressively.
What I'm most impressed about is that the author was able to include the FOIA data.
I guess I just learned I half expected each person who wanted FOIA data to have to request it themselves, for their own personal use.
In this case I can see reusing this for interesting reasons (the plates in the .txt.gz have not been removed), so...
I guess I just learned I half expected each person who wanted FOIA data to have to request it themselves, for their own personal use.
In this case I can see reusing this for interesting reasons (the plates in the .txt.gz have not been removed), so...
404 error? (page not found)
Should be better now. Let me know if it persists!
Nope, still a Google 404 page.
Consider putting the site behind Cloudflare or similar content cache provider maybe?
My email is in my profile, feel free to reach out if you want (free) reliable hosting for your blog.
I'm getting intermittent 404s as well. It'll probably be fine once this drops off the front page.
I’m still getting a 404 but going to the top level index worked even though the URLs are the same.
For the record I have no problem seeing it.
404 here. It loaded, initially, but some images were missing, and reloading the page gave me the error.
I really enjoyed this article – not only because of the content but the distraction-free layout makes it a pleasure to read. It’s rare that I come across a site using such minimal and effective graphic design. As a bonus, the site loads quickly and doesn’t rely on a stream of third-party JavaScript files or other web resources. For a first blog post, I’m impressed. If I ever get around to publishing my own blog, I know what to aim for. Keep up the good work. The web needs more of this!
The footer indicates that the web page was generated using bashblog [1] – looks like it might be worth checking out.
[1] https://github.com/cfenollosa/bashblog
The footer indicates that the web page was generated using bashblog [1] – looks like it might be worth checking out.
[1] https://github.com/cfenollosa/bashblog
$190 million?! What does the architecture consist of? A database, some forms and some integrations/api? I'm 90% sure they could have done that with free software and a good support contract with a UNIX provider for far less :/
This is a very common reaction that shows a lack of experience in dealing with scale IT systems.
You're correct that a simple DB with some forms would be cheap.
But integration tends to be crazy expense. For this sort of system, other things that also need to be covered: 1. Billing integration. Including changes to billing codes, bill (fine) printing, testing. 2. Audit integration. Because whenever money is handled, audit follows. 3. Customer support integration. Including UI for customer service, training, testing. This is often a very complex item because customer service already have a zillion systems they have to use and their training requirements are ongoing and expensive, so they want you to integrate with their existing systems instead of giving them a brand new thing, and integrate with their existing training processes, etc etc. 4. Integrate with all those hand-held readers. inc vendor compliance, testing etc. 5. Contract management. You have a contract with the government and they'd like to know that you did what you claim you did. So there's teams of people to deal with on an ongoing basis. 6. Project management. There's more than one person working on this, and a lot of complex integration requiring changes in other systems => extensive project management. 7. Ongoing changes to requirements, often conflicting. All the integration points above are moving targets, so expect that they'll have to be re-done a few times both before and after launch. 8. Arse covering. You now have a large contract with the US Government. You will sued and they will get sued (typically by whomever didn't win the contract). Vast amounts of documentation covering _everything_, including documenting the process by which documents are written => tech writers galore, plus lawyers plus lawyers.
Honestly, this is barely scratching the surface. I haven't even touched the (expensive) work before the contract is even signed.
$190M doesn't go very far!
You're correct that a simple DB with some forms would be cheap.
But integration tends to be crazy expense. For this sort of system, other things that also need to be covered: 1. Billing integration. Including changes to billing codes, bill (fine) printing, testing. 2. Audit integration. Because whenever money is handled, audit follows. 3. Customer support integration. Including UI for customer service, training, testing. This is often a very complex item because customer service already have a zillion systems they have to use and their training requirements are ongoing and expensive, so they want you to integrate with their existing systems instead of giving them a brand new thing, and integrate with their existing training processes, etc etc. 4. Integrate with all those hand-held readers. inc vendor compliance, testing etc. 5. Contract management. You have a contract with the government and they'd like to know that you did what you claim you did. So there's teams of people to deal with on an ongoing basis. 6. Project management. There's more than one person working on this, and a lot of complex integration requiring changes in other systems => extensive project management. 7. Ongoing changes to requirements, often conflicting. All the integration points above are moving targets, so expect that they'll have to be re-done a few times both before and after launch. 8. Arse covering. You now have a large contract with the US Government. You will sued and they will get sued (typically by whomever didn't win the contract). Vast amounts of documentation covering _everything_, including documenting the process by which documents are written => tech writers galore, plus lawyers plus lawyers.
Honestly, this is barely scratching the surface. I haven't even touched the (expensive) work before the contract is even signed.
$190M doesn't go very far!
FWIW in case you wanted to format the list HN does not really support markup, so you need a blank line between list items so they get displayed as separate paragraphs rather than folded into a single one.
Oops. Understood. It's just I've seen other countries with similar projects do a lot better with less, but I guess everything is custom for a particular situation.
Oh wow.
This is so impossible to optimize for :(
The really sad thing is that it's not humanly possible for one, or two or even just five people to do all of this, unless they were all like 19 or something, and then not for very long.
You literally do need all those departments that have the applied knowledge to do what they do well. Voila, $200M.
This is so impossible to optimize for :(
The really sad thing is that it's not humanly possible for one, or two or even just five people to do all of this, unless they were all like 19 or something, and then not for very long.
You literally do need all those departments that have the applied knowledge to do what they do well. Voila, $200M.
20+ years ago I was working for a company that provided systems for secure printing of checks (including payroll checks) and direct deposit notifications. One of the things suggested as a possible enhancement was the ability to email people's direct deposit notifications to them, and I got the assignment to research it.
On a technical basis, it's trivial - you already have the data stream that's going to be sent to the printer, generating a PDF wasn't going to be an enormous roadblock (though it wouldn't have been completely trivial as the source data was PCL not PS - did you know that there was handling for that in Ghostscript, at least on the commercially-licensed side?). Encryption of PDFs also possible, either with separately-licensed open source tools or with some closed-source commercial alternatives. Even ignoring the possibility of email being intercepted in transit, encryption would have been a requirement due to the risk of someone walking up to an unattended desk and simply checking that attached PDF for someone's pay info.
The killer? The infrastructure required to assign and allow people to change their passwords including management, training, etc.. By the time you've built that, you're a chunk of the way to simply providing the payroll information within an online HR system instead.
Like the old trope about the first man on Mars being a technician for an unreliable rover, the bulk of the work and cost isn't always where you'd think it would be.
On a technical basis, it's trivial - you already have the data stream that's going to be sent to the printer, generating a PDF wasn't going to be an enormous roadblock (though it wouldn't have been completely trivial as the source data was PCL not PS - did you know that there was handling for that in Ghostscript, at least on the commercially-licensed side?). Encryption of PDFs also possible, either with separately-licensed open source tools or with some closed-source commercial alternatives. Even ignoring the possibility of email being intercepted in transit, encryption would have been a requirement due to the risk of someone walking up to an unattended desk and simply checking that attached PDF for someone's pay info.
The killer? The infrastructure required to assign and allow people to change their passwords including management, training, etc.. By the time you've built that, you're a chunk of the way to simply providing the payroll information within an online HR system instead.
Like the old trope about the first man on Mars being a technician for an unreliable rover, the bulk of the work and cost isn't always where you'd think it would be.
Which is also why facebook/google/etc login for websites is attractive. Offload the worst part of the work.
Someday you'll work for a government and have to provide a support contract, and that first one will be woefully, absurdly underpriced. Your second one will answer the question you just asked here - why does interacting with the government cost so much money?
10 year support contract is certainly a good chunk of that
The initial product could be free, and that would still be a decent price for 10 years of any real support.
Yep. Have a look at their hourly rates for systems support [1].
All of the entry level positions are above 80/hr.
[1] https://imgur.com/a/ieJj5Po
All of the entry level positions are above 80/hr.
[1] https://imgur.com/a/ieJj5Po
This is the price that the vendor is charging, NOT what the person is getting paid.
The vendor is covering insurance, holiday/sick leave et al, payroll taxes et al, rent, management, carrying the risk of the work going away, etc etc.
It looks high, but it's not nearly as profitable as it seems for either the vendor or the employee.
The vendor is covering insurance, holiday/sick leave et al, payroll taxes et al, rent, management, carrying the risk of the work going away, etc etc.
It looks high, but it's not nearly as profitable as it seems for either the vendor or the employee.
Yep. you're right.One of the many reasons why I'm not in government contracting :)
I know this intersection. It's at the corner of State and Division, in the heart of the Rush Street neighborhood.
The likely reason there are many tickets there is that there are many bars there, and great crowds of people who have had a bit too much to drink. There are also great crowds of cops there every weekend.
Without looking at the data, I'd expect that many of the tickets are getting written in the middle of the night, when people are too inebriated, or too distracted, to read signs carefully.
The likely reason there are many tickets there is that there are many bars there, and great crowds of people who have had a bit too much to drink. There are also great crowds of cops there every weekend.
Without looking at the data, I'd expect that many of the tickets are getting written in the middle of the night, when people are too inebriated, or too distracted, to read signs carefully.
Did the bars close when the new signs went up? Coincidentally or otherwise?
The new signs caused the rate of tickets at the spot to slow down, not stop. According to the charts, there are still 300 tickets per year being written at that one spot. That is huge.
It's not a coincidence that that huge number of tickets is being written in a busy bar area.
It's not a coincidence that that huge number of tickets is being written in a busy bar area.
Who said they stopped? You must have meant to reply to someone else.
Alternatively, if you're a parking meter in Chicago, you now now how to meet quota and can full in the rest of your day with meaningful things.
very interesting example of data analysis and it's practical implications. awsome blog post, and you won't hear me say that quickly.
"This'll be my first blog post on the internet, ever. Hopefully it's interesting and accurate. Please point out any mistakes if you see any!"
KEEP IT UP MATT! and data munging, not sure if it's a word, but it sounds nice :D
"This'll be my first blog post on the internet, ever. Hopefully it's interesting and accurate. Please point out any mistakes if you see any!"
KEEP IT UP MATT! and data munging, not sure if it's a word, but it sounds nice :D
I’m thinking about a similar post. After a long run between different departments in Norway I got out all historic train delays and all form/e-mail contact with the rail company, including the number of people getting money back because of the delays.
What interested me most in this article was mawk/AWK.
Matt/bpchaps, have you shared the results with the aldermen? I'd love to hear about their reactions.
Kind of, but nothing super formal. I've contacted a few of the aldermen's offices, but never the aldermen themselves.
Though, during last mayoral election, some of the mayoral candidates wanted to use parking tickets as part of their campaign, and through some connections I found my way into Bob Fioretti's campaign manager's office to discuss parking tickets, alongside an ex-candidate, Amara Enyia's campaign manager. They were super, super interested - Fioretti's CM calling the work "fucking golden". But.. they both went silent after that, despite Fioretti started using parking tickets as a major part of his campaign. Go figure.
There's a lot more to that story - I'll end up write about sometime later :).
Though, during last mayoral election, some of the mayoral candidates wanted to use parking tickets as part of their campaign, and through some connections I found my way into Bob Fioretti's campaign manager's office to discuss parking tickets, alongside an ex-candidate, Amara Enyia's campaign manager. They were super, super interested - Fioretti's CM calling the work "fucking golden". But.. they both went silent after that, despite Fioretti started using parking tickets as a major part of his campaign. Go figure.
There's a lot more to that story - I'll end up write about sometime later :).
This is great. I wonder if using this data patterns could be found showing when and where tickets are written. Would be interesting to know when and how often certain areas are checked for illegal parking, if such a pattern exists.
This is great, but I can only imagine there being hidden pushback by government agencies if these requests to fix things became more common.
Revenue from parking tickets is easy money for a violation that is generally harmless.
Revenue from parking tickets is easy money for a violation that is generally harmless.
Honestly I dont expect any pushback for nefarious reasons. Its just as easy to ignore a problem thats making the city money, rather than defend for it.
On the one hand, this is a really neat hack. On the other hand, given the current situation with cars, pollution in cities (air quality, noise-level etc), global warming, I would personally think twice before doing something that would make car driving easier and more pleasant, as that simply incentivizes more driving. (getting a ticket might be a small inconvenience, but it does contribute to creating a slight aversion to driving).
Sure, this may seem a bit "evil", and the better solution is reducing negative externalities through taxing, which is a more transparent and ethical solution, but most of us don't have that level of power and influence within local councils and governments.
Sure, this may seem a bit "evil", and the better solution is reducing negative externalities through taxing, which is a more transparent and ethical solution, but most of us don't have that level of power and influence within local councils and governments.
American cities are built for cars - there's no way around that issue. To navigate cities you need some sort of powered transportation vehicle, outside of a few just-about-pedestrian-navigable areas downtown.
Don't punish cars, promote electric cars, scooters, bikes, and other ways of getting around. Dedicate more parking to electric-only spots (with chargers), bike racks and bike share docks, and scooter parks.
Don't punish cars, promote electric cars, scooters, bikes, and other ways of getting around. Dedicate more parking to electric-only spots (with chargers), bike racks and bike share docks, and scooter parks.
How much did the FOIA request cost?
It was free. I've never had to pay for any information coming from an Illinois FOIA request.
I didn't even know about FOIA request until this post thanks for write up
Have you thought about distributing the data, for accessibility and reproducibility? As well as save the taxpayers a little bit of dough in case someone else wants to request the same data.
edit: my bad, didn't see it at the end
edit: my bad, didn't see it at the end
The dataset is linked to at the end of the article (http://freeourinfo.com/tickets/)
[deleted]
That's interesting. I've often had to pay small to medium-sized amounts - always reasonable, I might add - for the price of producing said information.
You can often request that the FOIA officer waive the fee so long as the public's interest outweighs the burden to collect and redact the info. That's almost certainly what happened with my IL requests.
You might want to try postal address lookup/verification software to fix the addresses.
This is incredible, thanks for sharing.
This is awesome!
[deleted]
Is anyone else bothered that they paid 190 million for a database and interface to it?
See that part of the discussion:
https://news.ycombinator.com/item?id=17754670
There's a few good explanations there.
https://news.ycombinator.com/item?id=17754670
There's a few good explanations there.
I see you are neglecting to mention the support contract that came with it, including its very likely absurd SLAs on response times.
This, to me, is so neat.