My Half Workday as a Turker(cs.cmu.edu)
cs.cmu.edu
My Half Workday as a Turker
http://www.cs.cmu.edu/~jbigham/posts/2014/half-workday-as-turker.html
95 comments
I think the way out of this is a basic income guarantee. When a basic income is provided without requiring labor and this basic income meets the needs of an individual the cost of human labor will increase (the cost of everything will increase probably). This system assumes there is enough wealth around for the government to distribute enough to everyone.
This is Paul Krugman's take on it:
http://www.nytimes.com/2013/06/14/opinion/krugman-sympathy-f...
This is Paul Krugman's take on it:
http://www.nytimes.com/2013/06/14/opinion/krugman-sympathy-f...
> the cost of human labor will increase (the cost of everything will increase probably)
How does that solve anything? Sounds like a pretty bad side-effect or is that really an objective with basic income?
How does that solve anything? Sounds like a pretty bad side-effect or is that really an objective with basic income?
Increasing the cost of human labour is an objective; it should encourage more automation and shorter working hours (which are probably more productive anyway, but people are irrational). And hopefully it will reduce the gap between good and bad jobs, giving people more choice of how they want to work.
Increasing the cost of everything is an unfortunate side-effect that will hopefully not be very big.
Increasing the cost of everything is an unfortunate side-effect that will hopefully not be very big.
Makes sense. Thanks!
Andrew McAfee has a number of interesting articles and talks on this. E.g. http://andrewmcafee.org/2014/06/mcafee-autor-edsall-jobs-ski...
The way I see it, automated jobs will still add real value to the economy. Perhaps everyone can receive a share of that surplus through some form of universal income. At that point, assuming most of society's economic value creation is automated, monetary compensation could be completely decoupled from labor, yet everyone can be better off since everyone would be resource rich.
The way I see it, automated jobs will still add real value to the economy. Perhaps everyone can receive a share of that surplus through some form of universal income. At that point, assuming most of society's economic value creation is automated, monetary compensation could be completely decoupled from labor, yet everyone can be better off since everyone would be resource rich.
This question has a 170+ year long history, yet somehow it has come to the forefront against in recent years with people often entirely ignoring its history.
The foundation of the birth of modern socialism in the 1840's was based on the basic premise that technology would drastically increased efficiency and grow the economy to a point where redistribution could eradicate poverty and reduce the amount of labour necessary.
To your specific suggestion, variations over this was the viewpoint of some of the earliest socialist ideologists, who believed it possible to transform society through example and appeals to decency and charity.
Marx was one of the earliest to criticise that view strongly.
As early as the mid 1840's he made the argument that one of the main sources of eventual downfall for capitalism would not be some sudden enlightenment of the elites, but that capitalism would be too successful (the Communist Manifesto starts with a number of paragraphs gushing over the advances that capitalism and the bourgeoise have brought - Marx saw the development of capitalism as absolutely essential for progress) and eventually lead to over-production and under-employment at the same time, causing massive social upheaval and eventually leading to revolution if (though Marx also argument strongly that it was a when, not if) the ruling classes refuse to redistribute voluntarily.
The foundation of the birth of modern socialism in the 1840's was based on the basic premise that technology would drastically increased efficiency and grow the economy to a point where redistribution could eradicate poverty and reduce the amount of labour necessary.
To your specific suggestion, variations over this was the viewpoint of some of the earliest socialist ideologists, who believed it possible to transform society through example and appeals to decency and charity.
Marx was one of the earliest to criticise that view strongly.
As early as the mid 1840's he made the argument that one of the main sources of eventual downfall for capitalism would not be some sudden enlightenment of the elites, but that capitalism would be too successful (the Communist Manifesto starts with a number of paragraphs gushing over the advances that capitalism and the bourgeoise have brought - Marx saw the development of capitalism as absolutely essential for progress) and eventually lead to over-production and under-employment at the same time, causing massive social upheaval and eventually leading to revolution if (though Marx also argument strongly that it was a when, not if) the ruling classes refuse to redistribute voluntarily.
> massive social upheaval and eventually leading to revolution if ... the ruling classes refuse to redistribute voluntarily.
The sad part is that his conclusion is only incorrect because the ruling class figured out (more plausibly, stumbled into) a strategy for doing precisely the minimum amount of redistribution required to avoid revolution.
http://www.guernicamag.com/daily/anya-groner-the-heart-you-s...
The sad part is that his conclusion is only incorrect because the ruling class figured out (more plausibly, stumbled into) a strategy for doing precisely the minimum amount of redistribution required to avoid revolution.
http://www.guernicamag.com/daily/anya-groner-the-heart-you-s...
Maybe, maybe not -- see eg: http://www.politico.com/magazine/story/2014/06/the-pitchfork...
Another part of this is globalization -- you could argue that most people in the US (for example) are quite well off (as contrasted with complete destitution, starvation and lack of clean water) -- and one way to stave off revolution is to simply live on the other side of an ocean, and fill that ocean with friendly nuclear submarines and hangar ships.
But there is no guarantee such a scheme will last forever either -- as you'll likely still have people working in factories, and those factories will be producing (among other things) the weapons used for maintaining the status quo. At the very least it's hard to see how any industrial production could be made immune to strikes and sabotage, if the workers were desperate enough -- and that would likely result in a transfer of wealth (by no means is a "fair" outcome guaranteed, but change seems very likely).
I think it is too early to conclude that the revolution will never arrive. Be that just a small one (redistribution of wealth in the US for example) or a larger one (redistribution including the entire world).
Another part of this is globalization -- you could argue that most people in the US (for example) are quite well off (as contrasted with complete destitution, starvation and lack of clean water) -- and one way to stave off revolution is to simply live on the other side of an ocean, and fill that ocean with friendly nuclear submarines and hangar ships.
But there is no guarantee such a scheme will last forever either -- as you'll likely still have people working in factories, and those factories will be producing (among other things) the weapons used for maintaining the status quo. At the very least it's hard to see how any industrial production could be made immune to strikes and sabotage, if the workers were desperate enough -- and that would likely result in a transfer of wealth (by no means is a "fair" outcome guaranteed, but change seems very likely).
I think it is too early to conclude that the revolution will never arrive. Be that just a small one (redistribution of wealth in the US for example) or a larger one (redistribution including the entire world).
This was true to the point that the IWW had slogans about a four hour, four day workweek: http://www.salon.com/2014/06/01/help_us_thomas_piketty_the_1...
I have said this before. But it may be worth repeating:
"The problem is that the worker who is going to be out of a job doesn't own the company that buys the robots to replace him/her. Ever heard of a lights out factory that has all the former assembly line workers at home with full salary? Didn't think so."
The way capitalist societies are structured doesn't really allow for income distribution that way, the way I understand it.
"The problem is that the worker who is going to be out of a job doesn't own the company that buys the robots to replace him/her. Ever heard of a lights out factory that has all the former assembly line workers at home with full salary? Didn't think so."
The way capitalist societies are structured doesn't really allow for income distribution that way, the way I understand it.
> The way capitalist societies are structured doesn't really allow for income distribution that way, the way I understand it.
That's pretty much true-by-definition when it comes to "capitalist societies", but most advanced modern societies aren't really capitalist, and while the course hasn't been strictly monotonic, they've been overall getting farther from capitalism since the late 19th Century heyday of capitalism.
That's pretty much true-by-definition when it comes to "capitalist societies", but most advanced modern societies aren't really capitalist, and while the course hasn't been strictly monotonic, they've been overall getting farther from capitalism since the late 19th Century heyday of capitalism.
> "The problem is that the worker who is going to be out of a job doesn't own the company that buys the robots to replace him/her. Ever heard of a lights out factory that has all the former assembly line workers at home with full salary? Didn't think so."
Port containerization went a bit like that. Of course, that was back when America had labour unions.
Port containerization went a bit like that. Of course, that was back when America had labour unions.
> The way capitalist societies are structured doesn't really allow for income distribution that way, the way I understand it.
In the fully-automated future, these societies will violently implode if they aren't reconfigured to provide people with something to do and something to eat.
In the fully-automated future, these societies will violently implode if they aren't reconfigured to provide people with something to do and something to eat.
The internet helps globalize the digital labor market, competition from people that live in countries with low costs of living will drive the wages down for jobs that can be done from a computer and bureaucrats will be replaced by efficient databases, humans and paper are not very efficient.
This is natural evolution, IT wages bubble will burst because anyone with the proper education in any place in the world with lower expenses will be able to do it for less.
I hope the same happens with education, internet will change humanity in ways we can't imagine right now.
This is natural evolution, IT wages bubble will burst because anyone with the proper education in any place in the world with lower expenses will be able to do it for less.
I hope the same happens with education, internet will change humanity in ways we can't imagine right now.
In the aftermath of the Bubonic plague in Europe, the shortage of labor led to higher wages for serfs, eventually ending any semblance of what we'd call 'feudalism.'
I'm not advocating a mass die-off of human beings, but any other means of limiting the supply of low-quality labor would have the same effect. If some sort of back-to-the-land subsistence thing became popular among the youth, those who didn't partake might find that a McJob paid more by virtue of there being fewer people willing to do it. Sort of a "shrug" by the lower-class Atlas instead of the elite.
I'm not advocating a mass die-off of human beings, but any other means of limiting the supply of low-quality labor would have the same effect. If some sort of back-to-the-land subsistence thing became popular among the youth, those who didn't partake might find that a McJob paid more by virtue of there being fewer people willing to do it. Sort of a "shrug" by the lower-class Atlas instead of the elite.
>If some sort of back-to-the-land subsistence thing became popular among the youth
This won't become popular because it means accepting a drastic decline in living standards: you lose Internet access, public transportation, access to a variety of goods, the rural area you move to might have a corrupt government or an underfunded police department -- why do you think gun ownership is so popular in small towns? -- you experience a lot of social isolation, etc.
This idea that people should move "back to the land" can only originate from the deepest misanthropy and ignorance of the plight of the lower classes -- or perhaps the most starry-eyed technocratic utopianism, if you think that society can somehow provide poor people in disparate areas with a modern lifestyle -- and I'm not holding my breath. Of course, we could just kick poor people to the curb like this; it seems to be our policy already, city councils won't approve new housing for them, they won't approve the construction of a Wal-Mart, expansion of transit services is always opposed by a chorus of veiled racists, etc.
This won't become popular because it means accepting a drastic decline in living standards: you lose Internet access, public transportation, access to a variety of goods, the rural area you move to might have a corrupt government or an underfunded police department -- why do you think gun ownership is so popular in small towns? -- you experience a lot of social isolation, etc.
This idea that people should move "back to the land" can only originate from the deepest misanthropy and ignorance of the plight of the lower classes -- or perhaps the most starry-eyed technocratic utopianism, if you think that society can somehow provide poor people in disparate areas with a modern lifestyle -- and I'm not holding my breath. Of course, we could just kick poor people to the curb like this; it seems to be our policy already, city councils won't approve new housing for them, they won't approve the construction of a Wal-Mart, expansion of transit services is always opposed by a chorus of veiled racists, etc.
I am a gun owner from a small town. What do I know?
> I am a gun owner from a small town.
That comment plus your handle ('presidentender) might get you a visit from the Secret Service ....
That comment plus your handle ('presidentender) might get you a visit from the Secret Service ....
Gun ownership is popular in my particular small town because of hunting. What other reason would there be? Roaming bandits? Marauding biker gangs?
I would imagine enforcement of minimum wage laws on mturk might take care of that.
The turk is so rarely updated there is probably some kind of disruptive startup opening for someone to set up a middleman service paying an hourly wage to carefully tracked humans.
This might put a stop to the horrible turk habit of claiming a 20 minute survey only takes 2 minutes or writing some kids term paper only takes 30 minutes, which is technically true if you spend no time thinking about it and can type at 250 WPM, but for almost everyone else its a bit unrealistic.
Some communities have sprung up on reddit to highlite great HITs but they're very realtime and seems like as soon as a great HIT is posted to the reddit group, in seconds its gone, used up. A better marketplace dynamic could be set up. Clearly HITs and turkers are not really commodites after all in practice, so commodity market tools do fail. Tools that better match the market would be better for all participants, and the market maker skimming off the top, so its mystifying why they put no effort into it.
The turk is so rarely updated there is probably some kind of disruptive startup opening for someone to set up a middleman service paying an hourly wage to carefully tracked humans.
This might put a stop to the horrible turk habit of claiming a 20 minute survey only takes 2 minutes or writing some kids term paper only takes 30 minutes, which is technically true if you spend no time thinking about it and can type at 250 WPM, but for almost everyone else its a bit unrealistic.
Some communities have sprung up on reddit to highlite great HITs but they're very realtime and seems like as soon as a great HIT is posted to the reddit group, in seconds its gone, used up. A better marketplace dynamic could be set up. Clearly HITs and turkers are not really commodites after all in practice, so commodity market tools do fail. Tools that better match the market would be better for all participants, and the market maker skimming off the top, so its mystifying why they put no effort into it.
If mturk becomes too successful, it'll draw political attention and suddenly minimum wage laws will apply and the business model will implode. Leaving it in a state of neglect until Amazon figures out how to fix that problem would be a strategic solution to that problem.
Subsistence farming is something that almost the entire world tries to get away from as soon as possible, because apart from the food it's near-total poverty.
Also, what land? There isn't a vast reserve of untapped productive land lying around. It's all owned (capital) by someone who won't want you farming on it.
The only humane approach to too many people is a low birthrate movement, which requires universal free contraception. Something which a lot of Americans don't want to see happen.
Also, what land? There isn't a vast reserve of untapped productive land lying around. It's all owned (capital) by someone who won't want you farming on it.
The only humane approach to too many people is a low birthrate movement, which requires universal free contraception. Something which a lot of Americans don't want to see happen.
"The only humane approach to too many people is a low birthrate movement"
That's not a solution, that's another problem. Japan, Europe, most of the USA, and even parts of Africa already have birth rates so low that they're below the replacement level. This causes workforce shortages, requires mass immigration, and causes several new problems in both host and guest countries. Japan itself is in a nose dive. Adult diapers outsell infant diapers. Less people = less customers = less money = less jobs.
"what land? There isn't a vast reserve of untapped productive land lying around."
That's what I thought. But there is. It's in Africa. A large swath of land the size of France got grabbed up during the 2007 Great African Land Grab which no one seemed to notice or care about other than the Oakland Institute. Communal lands were bribed and bought up for as little as 18 cents a hectare with the promise of infrastructure improvements and jobs. Few of which came into fruition.
That's not a solution, that's another problem. Japan, Europe, most of the USA, and even parts of Africa already have birth rates so low that they're below the replacement level. This causes workforce shortages, requires mass immigration, and causes several new problems in both host and guest countries. Japan itself is in a nose dive. Adult diapers outsell infant diapers. Less people = less customers = less money = less jobs.
"what land? There isn't a vast reserve of untapped productive land lying around."
That's what I thought. But there is. It's in Africa. A large swath of land the size of France got grabbed up during the 2007 Great African Land Grab which no one seemed to notice or care about other than the Oakland Institute. Communal lands were bribed and bought up for as little as 18 cents a hectare with the promise of infrastructure improvements and jobs. Few of which came into fruition.
> The only humane approach to too many people is a low birthrate movement, which requires universal free contraception.
No, it doesn't require that: without universal free contraception, economic development and, particularly, strong social safety nets reduced birthrates in much of the developed world to below replacement levels. Note that this is less true of the USA than much of the modern developed world because while the USA has good aggregate economic output, it has fairly weak and unreliable social support systems for a developed country.
People have more children when children are your insurance against poverty due to age, disability, etc. -- as they have been for most of human history. When that becomes less true, they have fewer children.
No, it doesn't require that: without universal free contraception, economic development and, particularly, strong social safety nets reduced birthrates in much of the developed world to below replacement levels. Note that this is less true of the USA than much of the modern developed world because while the USA has good aggregate economic output, it has fairly weak and unreliable social support systems for a developed country.
People have more children when children are your insurance against poverty due to age, disability, etc. -- as they have been for most of human history. When that becomes less true, they have fewer children.
This is called idolizing the pastoral. The Pastoral was/is a popular literary genre/device.
Essentially, when we get disconnected from 'the simple life', we glorify and idolize it.
Having come from back-woods, subsistence farming to a white-collar, higher education job, I can tell you that (a) very few people understand the amount of stress and work required to survive in subsistence farming, (b) seriously, stress; you can starve, or (c) how much it sucks to do without modern conveniences like computers, internet and electricity.
Ever wonder why current day subsistence farmers aren't well known for their poetry or art? Because they spend all of their time finding and growing food.
In the future, I think we'll see a combination of McJob and responsibility for one's own future, much as we've seen for every generation before. The more things change, the more they stay the same.
Essentially, when we get disconnected from 'the simple life', we glorify and idolize it.
Having come from back-woods, subsistence farming to a white-collar, higher education job, I can tell you that (a) very few people understand the amount of stress and work required to survive in subsistence farming, (b) seriously, stress; you can starve, or (c) how much it sucks to do without modern conveniences like computers, internet and electricity.
Ever wonder why current day subsistence farmers aren't well known for their poetry or art? Because they spend all of their time finding and growing food.
In the future, I think we'll see a combination of McJob and responsibility for one's own future, much as we've seen for every generation before. The more things change, the more they stay the same.
I didn't say that opting out of the economy would benefit the people doing it. I said it would increase demand for labor by the workers who remained.
Resuming:
1-Globalization: technology and governments based in short term economics allow work to be made where is cheaper -> wages will be similar everywhere in the world in the near future.
2-Supply and demand: technology decreases the demand of human labor (automated work) and increases the supply (mturk, fiverr, freelancer, etc) -> wages decrease.
1-Globalization: technology and governments based in short term economics allow work to be made where is cheaper -> wages will be similar everywhere in the world in the near future.
2-Supply and demand: technology decreases the demand of human labor (automated work) and increases the supply (mturk, fiverr, freelancer, etc) -> wages decrease.
One solution is a [Basic Income](http://binews.org/) — "an income unconditionally granted to all on an individual basis, without means test or work requirement."
A Basic Income would help solve the problem of poverty and climate change, and should be a human right.
A Basic Income would help solve the problem of poverty and climate change, and should be a human right.
How does it solve climate change?
One theory is that if people aren't forced to expand the economy via polluting methods of producing just to make enough money to live, some of the climate change alleged to be caused in part by that industrial activity will be ameliorated.
Long comment but please bear with me:
To me, it doesn’t make sense to say that money is non-linear or non-constant. Money is not a function of time, therefore it does not make sense to describe it with time-dependent terms such as “linear” or "constant”. Inflation, the change in the purchasing power of money over time, yes. Money, no.
Money is simply a token that we use to represent the intangible concept of value. I think your comment seeks to address a change in what individuals, and by extension and on aggregate, societies, fundamentally value over time.
I’ll return to that in a bit. First let me address the problem with your example: It ignores the compounding returns to performing similar actions over time. Economically: Human capital accumulation. Colloquially: Experience.
When you get your first entry-level salaried job, you are actually a liability to the company. You come in with a low level of human capital - the company is basically paying to train you, all the while putting up with any potential mistakes due to your lack of experience. Why would a company do this?
By hiring you, the company is making an investment in your stock of human capital. It takes on some risk up front in return for some amount of reward in the future when you are wiser. As you gain experience over time, (ideally) you perform increasingly valuable tasks with a higher efficiency. You have accumulated the one thing youth (or old age for that matter) cannot buy for itself: time.
Getting paid for 3 hours for "little to no work" is a drop in the bucket compared to 10 years of solid, salaried employment.
By comparison, one-off jobs which might net you $20 in a couple of hours are a completely different situation where a low level of human capital is not purchased by means of a salaried contract. Rather, it is rented with a much shorter time horizon in mind, generally for menial tasks that require little to no experience.
To connect this idea back to the larger debate of “lowering the value of work”: I think you need to specify what kind of work you are talking about. I agree that the value of human menial labor is constantly decreasing. Why?
The supply of physical, menial tasks (washing dishes or mopping floors) is determined by the inflows of the human birthrate (more humans=more eating=more dishes to wash) and the outflows of automation (dishwashing machines), among other factors. Automation adds value by adding hours to our lives that would otherwise be spent washing dishes. For menial tasks it is generally more cost efficient. Thus the outflow outweighs the inflow (decreasing birthrate) leading to a lower supply of menial labor.
I don’t think we will revert to a hunter-gatherer bartering society in the future. Markets and specialization are much more efficient forms of raising standards of living, which I define as the ultimate goal of “human progress”. I think we will eventually need to reexamine our definition of “standards of living” as we move away from the realm of the physical to that of the virtual. Perhaps one (sad) day food will be unnecessary, as we redesign our bodies to subsist on a different form of energy, or do away with physical bodies altogether. This standard of living does not take into account food, shelter, transportation, etc. – the fundamentals that we strive to improve today through progress.
Anyways, enough with the science fiction and philosophizing. The fundamental lesson here is that the market gets what the market wants. There are a lot of human beings, so menial labor, both physical and mental, is in high supply driving down its relative value/price. Technology is eating jobs from the bottom up. Increase your human capital so that your brain is more valuable than a charged hunk of steel. Be the person who knows how to operate the machine and you will never have to beg for a $20 bus ticket.
To me, it doesn’t make sense to say that money is non-linear or non-constant. Money is not a function of time, therefore it does not make sense to describe it with time-dependent terms such as “linear” or "constant”. Inflation, the change in the purchasing power of money over time, yes. Money, no.
Money is simply a token that we use to represent the intangible concept of value. I think your comment seeks to address a change in what individuals, and by extension and on aggregate, societies, fundamentally value over time.
I’ll return to that in a bit. First let me address the problem with your example: It ignores the compounding returns to performing similar actions over time. Economically: Human capital accumulation. Colloquially: Experience.
When you get your first entry-level salaried job, you are actually a liability to the company. You come in with a low level of human capital - the company is basically paying to train you, all the while putting up with any potential mistakes due to your lack of experience. Why would a company do this?
By hiring you, the company is making an investment in your stock of human capital. It takes on some risk up front in return for some amount of reward in the future when you are wiser. As you gain experience over time, (ideally) you perform increasingly valuable tasks with a higher efficiency. You have accumulated the one thing youth (or old age for that matter) cannot buy for itself: time.
Getting paid for 3 hours for "little to no work" is a drop in the bucket compared to 10 years of solid, salaried employment.
By comparison, one-off jobs which might net you $20 in a couple of hours are a completely different situation where a low level of human capital is not purchased by means of a salaried contract. Rather, it is rented with a much shorter time horizon in mind, generally for menial tasks that require little to no experience.
To connect this idea back to the larger debate of “lowering the value of work”: I think you need to specify what kind of work you are talking about. I agree that the value of human menial labor is constantly decreasing. Why?
The supply of physical, menial tasks (washing dishes or mopping floors) is determined by the inflows of the human birthrate (more humans=more eating=more dishes to wash) and the outflows of automation (dishwashing machines), among other factors. Automation adds value by adding hours to our lives that would otherwise be spent washing dishes. For menial tasks it is generally more cost efficient. Thus the outflow outweighs the inflow (decreasing birthrate) leading to a lower supply of menial labor.
I don’t think we will revert to a hunter-gatherer bartering society in the future. Markets and specialization are much more efficient forms of raising standards of living, which I define as the ultimate goal of “human progress”. I think we will eventually need to reexamine our definition of “standards of living” as we move away from the realm of the physical to that of the virtual. Perhaps one (sad) day food will be unnecessary, as we redesign our bodies to subsist on a different form of energy, or do away with physical bodies altogether. This standard of living does not take into account food, shelter, transportation, etc. – the fundamentals that we strive to improve today through progress.
Anyways, enough with the science fiction and philosophizing. The fundamental lesson here is that the market gets what the market wants. There are a lot of human beings, so menial labor, both physical and mental, is in high supply driving down its relative value/price. Technology is eating jobs from the bottom up. Increase your human capital so that your brain is more valuable than a charged hunk of steel. Be the person who knows how to operate the machine and you will never have to beg for a $20 bus ticket.
I used MTurk off and on through my teens[0] (15-18) and found that the majority of my earnings came in through the 50 word or less summaries. You get to a point where you've read enough PR department press releases that you can write the entire summary (at least to the required standards, which in my experience, is "barely comprehensible English") by looking at the headline. I was able to finish 50 word summaries in under 30 seconds. They values range from .25-.50 and they seem to come from a lot of submitters.
I've had a theory that out in the world somewhere is someone selling a guide to using MTurk to create article summaries and the guide suggests the price range of $.25-.50 because for the entire time that I've been on MTurk (many years now), every time I come back the price range is the same with small fluctuation.
Past the relatively low amount of summaries that are posted daily, money can be made by watching the right forums where people post high paying HITs (the name of the most popular forum eludes me at the moment) that are typically 5-15 minute surveys that pay .5-3.00 each.
By far, the biggest barrier to making a decent wage for me was the number of available HITs. The ones that pay over pennies each are few while the ones that are horrible time invests are everywhere. If I had an unlimited number of summary HITs, I could have made well over minimum wage.
[0] Made about $450 over the time, but this was sporadic, random working periods.
I've had a theory that out in the world somewhere is someone selling a guide to using MTurk to create article summaries and the guide suggests the price range of $.25-.50 because for the entire time that I've been on MTurk (many years now), every time I come back the price range is the same with small fluctuation.
Past the relatively low amount of summaries that are posted daily, money can be made by watching the right forums where people post high paying HITs (the name of the most popular forum eludes me at the moment) that are typically 5-15 minute surveys that pay .5-3.00 each.
By far, the biggest barrier to making a decent wage for me was the number of available HITs. The ones that pay over pennies each are few while the ones that are horrible time invests are everywhere. If I had an unlimited number of summary HITs, I could have made well over minimum wage.
[0] Made about $450 over the time, but this was sporadic, random working periods.
I'm curious, have you found the ability to write 50 word summaries in under 30 seconds to be a useful skill later in life?
It sounds like good practice in concise communication.
It sounds like good practice in concise communication.
Without question it is a valuable skill.
In academic research it is especially important, and made more difficult by the complexity of the subject matter. The most successful researchers can succinctly explain their work (problem, significance, methodology and findings) to a layman. That's how you get funding. This also applies to startup founders seeking investors.
In academic research it is especially important, and made more difficult by the complexity of the subject matter. The most successful researchers can succinctly explain their work (problem, significance, methodology and findings) to a layman. That's how you get funding. This also applies to startup founders seeking investors.
I'd say the above comment is about average, to average+ for hn... so probably not? (Only being partially sarcastic)
[edit: I now realize that that's not enough data, we'd also need how long it takes to write the average hn comment, and how long op took writing his :-) ]
[edit: I now realize that that's not enough data, we'd also need how long it takes to write the average hn comment, and how long op took writing his :-) ]
I'm surprised someone hasn't automated jobs like that, honestly.
Isn't that what Summly was supposed to be? The startup founded by the 17 year old that got acquired by Yahoo for a chunk of change?
I dabble on MTurk every now and then.
It might sound strange, but I find certain kinds of survey type simple tasks incredibly relaxing and some of the HITs clearly connect to what seems to be interesting research.
Pretty much everything the author says here is spot-on and has been for a long time.
MTurk is positively flooded with HITs for generating fake reviews for products and fake content for websites. The best paying HITs by far are translation tasks - transcribing Arabic and Farsi seems to pay the best ($0.30 or more per minute of transcription).
I aim for academic projects with preference for things which require qualification (simple math / reading comprehension tests usually) as they generally pay quite a bit more.
I've earned an average of $0.27 per HIT and have never had one rejected.
Some of the more interesting / unusual tasks I've seen:
* Pinpoint various joints (shoulder, elbow, knee) in each frame of a clip showing a baseball player swinging a bat.
* Drop a pin on an estimate map location based on scenery shown in a short video clip (This was long before MapCrunch/GeoGuessr).
* Manipulate the camera in a scene of flat-shaded objects to bring them into 'correct' perspective.
* Choose 'preferred' structure designs (little houses) which appear to have been created by some sort of genetic algorithm.
It might sound strange, but I find certain kinds of survey type simple tasks incredibly relaxing and some of the HITs clearly connect to what seems to be interesting research.
Pretty much everything the author says here is spot-on and has been for a long time.
MTurk is positively flooded with HITs for generating fake reviews for products and fake content for websites. The best paying HITs by far are translation tasks - transcribing Arabic and Farsi seems to pay the best ($0.30 or more per minute of transcription).
I aim for academic projects with preference for things which require qualification (simple math / reading comprehension tests usually) as they generally pay quite a bit more.
I've earned an average of $0.27 per HIT and have never had one rejected.
Some of the more interesting / unusual tasks I've seen:
* Pinpoint various joints (shoulder, elbow, knee) in each frame of a clip showing a baseball player swinging a bat.
* Drop a pin on an estimate map location based on scenery shown in a short video clip (This was long before MapCrunch/GeoGuessr).
* Manipulate the camera in a scene of flat-shaded objects to bring them into 'correct' perspective.
* Choose 'preferred' structure designs (little houses) which appear to have been created by some sort of genetic algorithm.
"Some of the more interesting tasks I've seen"
I get distracted by categorization jobs, like look at this blueprint and if there is a blueprint number in the title box, enter it. Well, I end up admiring the print. Oh look a giant gear for some kind of mining thing. Why, what an interesting looking architectural drawing.
Similar thing happens with categorize document numbers. Apparently some municipality decided to turn millions of paper scanned pages of contract bids into piles sorted by contract, so I'd end up sitting there reading all about bids to put in some drainage culvert. Who would ever have guessed theres so many steps and processes and backfill and compaction are critical and theres so many inspectors... I mean you can half A a drainage culvert in like one paragraph but if you want it done right its 50 pages of PDF.
These jobs are probably impossible without an excellent short term memory or dual (or more) monitors.
The ones that annoyed me the most were "leave blank if no number" so I'd scour the document for what seemed like forever just to make sure. Inevitably leading me to becoming fascinated by drainage culvert design.
Its very D+D like. Whats behind this dungeon door? Oh, I see, a blueprint of a giant gear from some mining equipment. Fascinating. Like archeology but I'm getting paid for it. Well, not much pay.
I get distracted by categorization jobs, like look at this blueprint and if there is a blueprint number in the title box, enter it. Well, I end up admiring the print. Oh look a giant gear for some kind of mining thing. Why, what an interesting looking architectural drawing.
Similar thing happens with categorize document numbers. Apparently some municipality decided to turn millions of paper scanned pages of contract bids into piles sorted by contract, so I'd end up sitting there reading all about bids to put in some drainage culvert. Who would ever have guessed theres so many steps and processes and backfill and compaction are critical and theres so many inspectors... I mean you can half A a drainage culvert in like one paragraph but if you want it done right its 50 pages of PDF.
These jobs are probably impossible without an excellent short term memory or dual (or more) monitors.
The ones that annoyed me the most were "leave blank if no number" so I'd scour the document for what seemed like forever just to make sure. Inevitably leading me to becoming fascinated by drainage culvert design.
Its very D+D like. Whats behind this dungeon door? Oh, I see, a blueprint of a giant gear from some mining equipment. Fascinating. Like archeology but I'm getting paid for it. Well, not much pay.
"At the time, I was hoping Dmitry would reject my HIT so I could go on a tirade about how unfair it was, and try to get him banned from MTurk or whatever."
I found this quite amusing. You know why this HIT was approved? Because requesters are also punished for rejecting too many HITs! He also would have been banned had Amazon discovered the scheme he was pulling.
I found this quite amusing. You know why this HIT was approved? Because requesters are also punished for rejecting too many HITs! He also would have been banned had Amazon discovered the scheme he was pulling.
This is a serious problem that requesters face. Often the workers who spam everyone with fake work will report anyone who rejects their work as retribution.
This was a great read. I found it particularly interesting that the author came into each encounter 'cold', which is to say without much in the way of additional tooling. They did install a word counter extension but much of their tools were ad-hoc or non-existent.
That suggested that there may be an interesting 'toolware' market for Turkers. For example, document editing; Lets say we build a simple document editing platform (copy editing not typesetting) which includes components that are dropbox like (shared storage), Google docs like etc. And design a workflow around that toolset. So pick a HIT, get a document id, it shows up in your shared storage, use the included tool to edit it, when you're done, click 'done', and have the whole thing resubmit back for evaluation. That might allow you to focus on editing and not get hung up on a bunch of getting it done issues.
Not sure how practical that is, but if there are common Turk workflows it might be useful to build some common tooling for them.
That suggested that there may be an interesting 'toolware' market for Turkers. For example, document editing; Lets say we build a simple document editing platform (copy editing not typesetting) which includes components that are dropbox like (shared storage), Google docs like etc. And design a workflow around that toolset. So pick a HIT, get a document id, it shows up in your shared storage, use the included tool to edit it, when you're done, click 'done', and have the whole thing resubmit back for evaluation. That might allow you to focus on editing and not get hung up on a bunch of getting it done issues.
Not sure how practical that is, but if there are common Turk workflows it might be useful to build some common tooling for them.
> That suggested that there may be an interesting 'toolware' market for Turkers.
In the early days of Mechanical Turk (2005-06), when almost all the HITs were seeded by Amazon themselves and were of a few common types that kept being replenished, there were browser extensions to greatly improve the UI for the common tasks (the default UI is very bad). One of the more profitable tasks was fixing the location of addresses on the new-defunct A9 BlockView (a product similar to what Google would later introduce as Street View). They'd present you about 10 photos of a streetfront that were supposed to be near an address, and you were supposed to pick the one centered around the address (e.g. the store or house entrance). Or else indicate none of the above, if the alignment was seriously off so that the correct address was out of the frame. If you used the regular website UI, you could make maybe $3-4/hr, but most of this was tedious clicking: accept HIT, click a tiny radio button, scroll down, submit HIT, request next HIT, repeat. The browser extension implemented the obvious UI improvement: use the images instead of tiny radio buttons as click targets, and then auto-submit and request the next one on click. I think in one version you could also just hit a number 0-9 on the numpad instead of clicking. With that extension I was making $20-30/hr for a little bit.
In the early days of Mechanical Turk (2005-06), when almost all the HITs were seeded by Amazon themselves and were of a few common types that kept being replenished, there were browser extensions to greatly improve the UI for the common tasks (the default UI is very bad). One of the more profitable tasks was fixing the location of addresses on the new-defunct A9 BlockView (a product similar to what Google would later introduce as Street View). They'd present you about 10 photos of a streetfront that were supposed to be near an address, and you were supposed to pick the one centered around the address (e.g. the store or house entrance). Or else indicate none of the above, if the alignment was seriously off so that the correct address was out of the frame. If you used the regular website UI, you could make maybe $3-4/hr, but most of this was tedious clicking: accept HIT, click a tiny radio button, scroll down, submit HIT, request next HIT, repeat. The browser extension implemented the obvious UI improvement: use the images instead of tiny radio buttons as click targets, and then auto-submit and request the next one on click. I think in one version you could also just hit a number 0-9 on the numpad instead of clicking. With that extension I was making $20-30/hr for a little bit.
I've used MTurk to have interviews transcribed several times, and almost always had flawless results. (sometimes jargon or names trips them up.) I've always been incredibly impressed at the speed they get done, which is substantially better than I've ever achieved doing them myself. I usually aim to pay $10 for a 10 minute segment, which should amount to more than $10/hour for a good typist.
An journal article I submitted a few years ago:
Amazon Mechanical Turk: Gold Mine or Coal Mine?
http://www.mitpressjournals.org/doi/abs/10.1162/COLI_a_00057
Amazon Mechanical Turk: Gold Mine or Coal Mine?
http://www.mitpressjournals.org/doi/abs/10.1162/COLI_a_00057
Last time I looked at MTurk, it was offering money if I would complete some simple task, and, oh, by the way, download this crapware onto your computer and run it.
To be fair, those types of HITs are forbidden by the terms (often removed quickly) and, in my experience, tend to be rare.
That sounds a lot like someones research project: http://threatpost.com/research-project-pays-people-to-downlo...
I checked Mechanical Turk as an example of how to not to price for Newslines, my crowdsourced content site. It seems that most publishers use crowdsourcing as a way to save money, and while that may work for some tasks, you will not get consistent work done.
Rather than crowdsourcing being a way to find cheap writers, content business should see crowdsourcing as a way to find a diverse pool of writers who have different interests or knowledge, who can dip in and out of work, and then pay them fair money for their time.
I pay $1 per post for writers to write 50-100 word posts that follow a very specific format. We provide feedback to the writer to try to make them make money faster. The way I figure it is, if the writer makes money then I can make money. If they stick with you, you don't have to train new people. People won't leave after half a day.
We have already paid out thousands of dollars and our site is growing fast. Writer satisfaction is high. We currently have a waiting list of hundreds of writers wanting to sign up.
http://newslines.org/newslines-rewards/
Rather than crowdsourcing being a way to find cheap writers, content business should see crowdsourcing as a way to find a diverse pool of writers who have different interests or knowledge, who can dip in and out of work, and then pay them fair money for their time.
I pay $1 per post for writers to write 50-100 word posts that follow a very specific format. We provide feedback to the writer to try to make them make money faster. The way I figure it is, if the writer makes money then I can make money. If they stick with you, you don't have to train new people. People won't leave after half a day.
We have already paid out thousands of dollars and our site is growing fast. Writer satisfaction is high. We currently have a waiting list of hundreds of writers wanting to sign up.
http://newslines.org/newslines-rewards/
Initially I found $1/[edit: item] strangely low as an example of "high pay". Having a look at what newslines is actually about, I still think it is low, but not as low as I intially thought (I'd assumed you wanted contribution from domain experts, who'd I guess you'd price at at least 100/hour).
I'd assume anyone that can write well, should be able to make at the very minimum 30/hour writing -- That leaves on average 2 minutes/newsline. I suppose the lesson is that there are many skilled poor people out there.
(Note, this isn't meant as negative criticism, just some observations)
I'd assume anyone that can write well, should be able to make at the very minimum 30/hour writing -- That leaves on average 2 minutes/newsline. I suppose the lesson is that there are many skilled poor people out there.
(Note, this isn't meant as negative criticism, just some observations)
Thank you for your observations, I appreciate the tone. In our case the writers are working to a formula, which for most people is a lot easier than creative writing. So far most of our writers are actually work-at-home moms who are happy to make around $10 an hour, for a task they see as being quite enjoyable - writing news summaries about their favorite celebrities, while sitting at home. They can finish a task in a few minutes, depending on the topic, and bank $10-$20 a day extra cash (or more) fairly easily. Then they can come back the next day for more. On our side we get a 24hr/day pool of talent that wants to keep adding content, without the need for office space and other HR issues.
You are dead wrong if you think any skilled writer can make $30 per hour. Only copywriters and technical writers can do that, and those are not easy jobs to get. Journalism, as an example, is one of the lowest paying professions. Writing jobs on Craigslist are almost always unpaid internships.
I suppose I'm rather out of touch with the not-cs hourly wages in the US. According to:
http://www.forbes.com/sites/susanadams/2012/04/26/is-journal...
Journalists earn (for ~2000 hours/year) between 15 and 28 (without a graduate degree), while lower level CS started at around 25/hour. So I was clearly wrong, but it would also appear journalism isn't especially low paying either?
http://www.poynter.org/latest-news/mediawire/202644/gov-stat...
Seem to mostly agree.
http://www.forbes.com/sites/susanadams/2012/04/26/is-journal...
Journalists earn (for ~2000 hours/year) between 15 and 28 (without a graduate degree), while lower level CS started at around 25/hour. So I was clearly wrong, but it would also appear journalism isn't especially low paying either?
http://www.poynter.org/latest-news/mediawire/202644/gov-stat...
Seem to mostly agree.
Like Jeff, I'm a professor who works on crowdsourcing. For one of my current projects, I am working on a tool to make the Amazon Mechanical Turk marketplace fairer for workers. I have developed a Chrome plugin that tracks the length of time it takes to complete a task, and a web site that aggregates the information across many workers. Crowd-Workers.com allows workers to discovery higher paying work by sorting tasks by estimated hourly rate.
Read more about the tool: http://www.cis.upenn.edu/~ccb/publications/crowd-workers-pos...
Try out our Chrome plugin and our web site: http://crowd-workers.com/landing
Read more about the tool: http://www.cis.upenn.edu/~ccb/publications/crowd-workers-pos...
Try out our Chrome plugin and our web site: http://crowd-workers.com/landing
It's been over a year and a half since i've spent any serious time on Mechanical Turk trying to supplement low income, so some of my experience might be dated. With a bit of a learning curve it's plenty doable to find decently paying jobs in transcription, writing, etc. on MT, especially if you have a strong grasp of the English language. But even with higher-skilled, higher-paying jobs on that platform, the overall experience of working on MT can often end up being plenty demoralizing and unfair in many many ways, such as qualifying for skilled work and also being compensated for work performed.
The most frustrating jobs i've ever performed on the platform were usually approx 5-10 minute tasks that seemed to be decently paying and straightforward, asking you to categorize items based on some arbitrary but seemingly simple system, but later all 5 or more submissions you might have made could be blanket denied / payment refused. In that case not only do you not get payed, but having HIT submissions rejected really seriously hurts your reputation and what jobs you're allowed to apply for on MT; often many more high paying jobs on the system use high acceptance rate and flawless work history as the lowest bar for entry, and so MT workers can be screwed over incredibly by having work rejected through no fault of their own.
To not get seriously screwed newcomers are somehow supposed to know which types of HIT's to definitely avoid that could kill their rating, but without enough information to actually make that judgement. A task might seem easy and give the illusion that it's okay to send 5 or 10 submissions in a row, only to later have them all rejected and serious reputation damage done.
I think it's interesting but also really sad how this mirrors a lot of the unfairness and power imbalance in the larger working world as well. Reputation is everything, but if you just happen to get unlucky to start out your career working for clueless bosses / clients with unrealistic expectations, there's a good chance future job and career prospects are going to be seriously hindered, if not completely derailed.
Transcription work was usually very well paying i found, but doing one long transcription task for a $30 payout is a huge risk the way things were set up. Never mind that you worked for 4 or 5 hours on it; whoever assigned the job can reject it for whatever random reason they want and you get nothing. As nice as it can be to earn a nice wage for more high-skilled tasks, the possibility of being denied any payment for a days work with little recourse is really frustrating and demoralizing; the way Mechanical Turk is structured if anyone's taking a negative financial hit for work done, it's almost without fail going to be the worker.
Though this CMU professor obviously put some work into this write-up, i don't see how somebody can possibly get an accurate picture of what it's like trying to earn an income on one of these platforms by just devoting 4 hours and extrapolating based on that really limited experience. It is really easy to miss all but the most obvious and glaring problems that way, just from not sticking around long enough to even run across them.
There are plenty of MT workers who have spent much more substantial amounts of time on the platform, that are much better able to communicate benefits and pitfalls for the everyday worker. They might not be professors at prestigious universities like CMU, but having a decent amount of hands-on experience should really be the low bar for discussing pros/cons of the platform seriously; just as one example assumptions about longer-term take-home pay in the write-up were pretty naive: many better paying tasks (especially web browsing tasks and surveys paying more than a pittance) are often in limited supply and cannot be repeated either at all, or only a few times per day / week, etc. Even the best employers providing the most decent fairly-compensated work come and go, making a steady healthy income sometimes very difficult to achieve even in the best of circumstances.
Mechanical Turk has experienced a lot of success from the start largely because they created a service that makes it so easy for virtually anybody with internet access and a basic computer, to work from home and perform simple tasks / other types of work when and wherever they want, and actually get paid for it.
By any decent standards of developed-world countries the pay is shit though, and worker protections and benefits are nonexistent; workers can be denied payment and even have their reputation ruined for no good reason, with little in the way of an appeals process. But even now services like MT are still a pretty new thing. Maybe in 10 or even 20 more years most of the bugs will get worked out, and workers of every skill level can be a on more equal footing when negotiating with employers, settling disputes etc.
The most frustrating jobs i've ever performed on the platform were usually approx 5-10 minute tasks that seemed to be decently paying and straightforward, asking you to categorize items based on some arbitrary but seemingly simple system, but later all 5 or more submissions you might have made could be blanket denied / payment refused. In that case not only do you not get payed, but having HIT submissions rejected really seriously hurts your reputation and what jobs you're allowed to apply for on MT; often many more high paying jobs on the system use high acceptance rate and flawless work history as the lowest bar for entry, and so MT workers can be screwed over incredibly by having work rejected through no fault of their own.
To not get seriously screwed newcomers are somehow supposed to know which types of HIT's to definitely avoid that could kill their rating, but without enough information to actually make that judgement. A task might seem easy and give the illusion that it's okay to send 5 or 10 submissions in a row, only to later have them all rejected and serious reputation damage done.
I think it's interesting but also really sad how this mirrors a lot of the unfairness and power imbalance in the larger working world as well. Reputation is everything, but if you just happen to get unlucky to start out your career working for clueless bosses / clients with unrealistic expectations, there's a good chance future job and career prospects are going to be seriously hindered, if not completely derailed.
Transcription work was usually very well paying i found, but doing one long transcription task for a $30 payout is a huge risk the way things were set up. Never mind that you worked for 4 or 5 hours on it; whoever assigned the job can reject it for whatever random reason they want and you get nothing. As nice as it can be to earn a nice wage for more high-skilled tasks, the possibility of being denied any payment for a days work with little recourse is really frustrating and demoralizing; the way Mechanical Turk is structured if anyone's taking a negative financial hit for work done, it's almost without fail going to be the worker.
Though this CMU professor obviously put some work into this write-up, i don't see how somebody can possibly get an accurate picture of what it's like trying to earn an income on one of these platforms by just devoting 4 hours and extrapolating based on that really limited experience. It is really easy to miss all but the most obvious and glaring problems that way, just from not sticking around long enough to even run across them.
There are plenty of MT workers who have spent much more substantial amounts of time on the platform, that are much better able to communicate benefits and pitfalls for the everyday worker. They might not be professors at prestigious universities like CMU, but having a decent amount of hands-on experience should really be the low bar for discussing pros/cons of the platform seriously; just as one example assumptions about longer-term take-home pay in the write-up were pretty naive: many better paying tasks (especially web browsing tasks and surveys paying more than a pittance) are often in limited supply and cannot be repeated either at all, or only a few times per day / week, etc. Even the best employers providing the most decent fairly-compensated work come and go, making a steady healthy income sometimes very difficult to achieve even in the best of circumstances.
Mechanical Turk has experienced a lot of success from the start largely because they created a service that makes it so easy for virtually anybody with internet access and a basic computer, to work from home and perform simple tasks / other types of work when and wherever they want, and actually get paid for it.
By any decent standards of developed-world countries the pay is shit though, and worker protections and benefits are nonexistent; workers can be denied payment and even have their reputation ruined for no good reason, with little in the way of an appeals process. But even now services like MT are still a pretty new thing. Maybe in 10 or even 20 more years most of the bugs will get worked out, and workers of every skill level can be a on more equal footing when negotiating with employers, settling disputes etc.
If there's no real appeal process, what reason is there for any MT employer to make any payouts at all? Honesty and not realising this about the system?
Maybe i read it wrong, but from the article it sounds like the Amazon MT team is at least trying to do more in the way of preventing unnecessary rejections and bad faith task-giving. Not sure if any sort of appeals process was ever put in place, but i really doubt it, since that would create a new non-trivial administrative burden on Amazon that is likely much larger than they are willing to handle.
Seemingly arbitrary rejections always seemed to be a much bigger problem by my estimation, especially for workers already somewhat invested in the system, and needing to maintain approval ratings to even qualify for higher-paying types of work.
If Amazon hasn't implemented anything like a way of expunging old rejection records, it would be a huge benefit to their workers to have some sort of reasonable, semi-automated way of doing so, at least after enough time has passed, or enough more recent positive approvals have occurred.
I'm really out of the loop though, and don't know much about the current state of things. It would just be really nice to have some more in-depth write-ups on the MT worker experience. The article was great but from first-hand experience i know there is a lot more ground to cover.
Senior project / undergrad thesis anyone? Researching / investigating MT from a user interface, user interaction design side of things, giving recommendations for improvements to the worker experience, for HIT providers, etc. would be very interesting and potentially very beneficial for the platform.
Seemingly arbitrary rejections always seemed to be a much bigger problem by my estimation, especially for workers already somewhat invested in the system, and needing to maintain approval ratings to even qualify for higher-paying types of work.
If Amazon hasn't implemented anything like a way of expunging old rejection records, it would be a huge benefit to their workers to have some sort of reasonable, semi-automated way of doing so, at least after enough time has passed, or enough more recent positive approvals have occurred.
I'm really out of the loop though, and don't know much about the current state of things. It would just be really nice to have some more in-depth write-ups on the MT worker experience. The article was great but from first-hand experience i know there is a lot more ground to cover.
Senior project / undergrad thesis anyone? Researching / investigating MT from a user interface, user interaction design side of things, giving recommendations for improvements to the worker experience, for HIT providers, etc. would be very interesting and potentially very beneficial for the platform.
There's an unofficial ratings system for those who post the jobs[1] that most people who use MT with any sort of regularity (including myself back in the day) use. If a requester consistently fails to pay out, they'll stop getting as many workers who are willing to work on their tasks.
[1]turkopticon.ucsd.edu
[1]turkopticon.ucsd.edu
That's a great point! Though still really hoping the default experience has improved in the past year or so as well. Not every new or even semi-experienced user is going to be savvy and aware of external rating systems, or other peripheral tools to help improve the worker experience. I know i wasn't (first time hearing about turkopticon, sadly -_-), but that doesn't reflect at all on how qualified someone is to do the mostly very simple tasks available to work on.
I'd be interested to see how you all did if using either end of Pay4Bugs ( https://www.pay4bugs.com )
I bet you'd learn some great stuff and see some crazy software
I bet you'd learn some great stuff and see some crazy software
Neat! I haven't heard of it before but will definitely check it out. Glad to see more and more options popping up for this sort of flexible, easy payment type work. This kinda thing can do a lot to help people already below, or hovering right above the poverty line, since unforeseen expenses, and not being able to get new sources of income started in time is one of the biggest most daunting problems to have to deal with.
This is an excellent comment.
Looking at your posting history, I can see that you often take the time to write thoughtful comments. Please keep it up.
Looking at your posting history, I can see that you often take the time to write thoughtful comments. Please keep it up.
Thanks. :-)
Kind of reminds me of the saying, "pay peanuts, get monkeys."
What a dehumanizing experience.
Yes, working: how awful.
I hope you're trying to be on the workers side but either way this is a terrible thing to say. The way you look down upon people trying their very best to make a living is not helping them at all.
I hope you're trying to be on the workers side but either way this is a terrible thing to say. The way you look down upon people trying their very best to make a living is not helping them at all.
It's dehumanizing to work and get paid little or nothing at all. You seem to be reading much more in the comment you're replying to than was actually said.
Depends on the sort of tasks you're doing. Someone above mentioned receiving ~$0.27/min of translations. This is a relatively high figure in comparison to a lot of tasks, but if the task you're doing isn't entirely mind numbing then being a Turker isn't as bad as it sounds. $0.27/minute is something like ~$16/hr - if you're working at a gas station with a computer handy, you'd be able to more than double your wage. Are the tasks undervalued? Not sure it's fair for us to say. The HITs keep coming in and they continue to be completed, so something is working.
Depends what sort of gas station you're at, I guess. My sister worked at a gass station one summer. She was mostly alone on the job, kept an eye on everyone who filled gas, made fast food, made sure the shelves were well stocked with products and cleaned. Not much room for going to the toilet much less sitting at the computer, but then her wage was something like 30 USD/h.
First world problems though. It's little or nothing by your standards. In India it would be a very different story.
For those of you like me getting a white screen with no content, the direct link is https://docs.google.com/document/d/1jzipmLrMVIH_FfxK4Hlkadxv...
FYI, Privacy badger is what caused my browser to whitescreen, as it was blocking the pass off to google docs
Interesting article. One hears about MT, but not so much from this perspective.
It's also interesting that the linked page has two scroll bars (tried both Chrome & FF).
It's also interesting that the linked page has two scroll bars (tried both Chrome & FF).
This page is just a container for a google doc in an iframe. Interesting CMS strategy.
I wonder what pays more? Panhandling on the street corner or working on mechanical turk?
Very interesting. Has anyone used Mechanical Turk to get work done?
Know a guy who's iPhone app uses MT to do pseudo-automated video image/motion recognition & processing. He had to train & retain his Turks, who seem quite content doing the work as an ongoing project. (Not sure if he wants further details revealed.) Between this and the article, seems the untold story may be that blind/random "turking" may be frustrating & unprofitable, but finding & developing a relationship on a project can work well for all involved.
I've used MTurk for a very subjective task: "Value the object in the image" for which I paid $0.10 per valuation (a maximum of 30 seconds was spend on the task).
The valuations were fair but the workers of Mechanical Turk may have a different view of how much something "home made" or "retro" may be worth (typically far lower than the object would be worth for someone in the market for said object).
The valuations were fair but the workers of Mechanical Turk may have a different view of how much something "home made" or "retro" may be worth (typically far lower than the object would be worth for someone in the market for said object).
I worked for years on systems built on top of MTurk (building trust systems, scaling tasks, etc)
The hardest thing is always task design, as the author discusses. It's quite challenging to create a task that will work in all browsers, across cultural and language boundaries, and to pay enough to incentivize the worker (while simultaneously little enough to make all the effort worthwhile).
The final effect is a bit dehumanizing on both ends. The workers feel alienated and mistrusted by the people requesting the tasks. The people requesting the tasks feel that the workers are lazy / "scammers" trying to cheat them.
We don't fully even know how to think about this kind of work yet - ideas like "hourly wage" don't seem to fit for someone working completely independently, at their own pace, possibly distracted or impaired.
Still, despite the trouble, it seems like one of the most promising fields for interaction design. There's the possibility to really improve peoples' lives (or make them worse).
The hardest thing is always task design, as the author discusses. It's quite challenging to create a task that will work in all browsers, across cultural and language boundaries, and to pay enough to incentivize the worker (while simultaneously little enough to make all the effort worthwhile).
The final effect is a bit dehumanizing on both ends. The workers feel alienated and mistrusted by the people requesting the tasks. The people requesting the tasks feel that the workers are lazy / "scammers" trying to cheat them.
We don't fully even know how to think about this kind of work yet - ideas like "hourly wage" don't seem to fit for someone working completely independently, at their own pace, possibly distracted or impaired.
Still, despite the trouble, it seems like one of the most promising fields for interaction design. There's the possibility to really improve peoples' lives (or make them worse).
i've used it a couple times in a lame attempt to find leads. IIRC, I paid $.40 to find a name/phone number/website of construction equipment rental companies in 50 different cities. They did exactly what I wanted them to do, although I never did anything with the results. One guy seemed to pick up most of the jobs, so I tipped him an extra $10.
I've collected experimental data using Mechanical Turker. In my experience, you have to do a lot of validation to make sure the data you are getting is reasonable. You have to kick people out who aren't paying any attention at all and just want to click through your survey to get paid. (Similar techniques also have to be employed for undergraduate participants who just want to get done with your survey so they can earn their 10 points of extra credit).
If nothing else, Turkers are good at making sure your data collection instruments are working the way that they are supposed to.
If nothing else, Turkers are good at making sure your data collection instruments are working the way that they are supposed to.
> In my experience, you have to do a lot of validation to make sure the data you are getting is reasonable.
Do you ask any 'trick' questions that are the cognitive equivalent of a captcha?
Do you ask any 'trick' questions that are the cognitive equivalent of a captcha?
I've turked for fun and you'd be shocked how many incredibly long psych surveys have scale of 1 to 10 questions like "enter a 5 as the answer to this question if you expect to get paid". Also if a survey asks "how much do you love the new windows phone" you can fully expect the opposite question perhaps on the next page "how much do you hate the new windows phone". Presumably they correlate the data somehow?
I've never run into a serious college professor category of trick question.
Turk is a grind game if you don't need the money. If you're susceptible its going to be a little moth to flame for you and all that matters is your meaningless number goes up, or if you're easily bored like me, you'll turk for like 20 hours one week and then not think about it again, until this HN question, in fact.
I will say that any researcher type would be horribly disappointed by turking because I did it while watching TV, or drinking a beer, and I have a low tolerance not being much of a drinker. Speaking of beer, I am told there is, or was, something of a fad of trying to turk fast enough to pay for your booze. There is no way humanly possible to turk fast enough to drink at a bar, but cheap freshman class beer in a dorm is totally realistic. Can you pay for that six pack on mturk before you finish drinking it? So... I don't think anyone could get homework assignments done on mturk. At least if you want to pass the class or have graduated from grade school. The linked article example is far beyond anything I ever wrote for mturk.
The most insulting and infuriating turk jobs I ran into were not on the author's list, basically being paid to be spammed to. Here's 20 full page variations of a home equity mortgage spam, now pick your favorite one for a nickle. Or call this number and talk to this scammer for a minimum of 5 minutes and then come here and fill out a survey for like 10 cents, at least on the surface its for "quality control" but you know its really to give you the hard sell.
Life is cheap on the turk, and its full of work hits that pay the equivalent of a buck an hour or less, which is eventually super demotivating / depressing. You hear about cool high paying hits but end up filling out 20 page surveys about B.S. for like ten cents, and getting sick of it.
I've never run into a serious college professor category of trick question.
Turk is a grind game if you don't need the money. If you're susceptible its going to be a little moth to flame for you and all that matters is your meaningless number goes up, or if you're easily bored like me, you'll turk for like 20 hours one week and then not think about it again, until this HN question, in fact.
I will say that any researcher type would be horribly disappointed by turking because I did it while watching TV, or drinking a beer, and I have a low tolerance not being much of a drinker. Speaking of beer, I am told there is, or was, something of a fad of trying to turk fast enough to pay for your booze. There is no way humanly possible to turk fast enough to drink at a bar, but cheap freshman class beer in a dorm is totally realistic. Can you pay for that six pack on mturk before you finish drinking it? So... I don't think anyone could get homework assignments done on mturk. At least if you want to pass the class or have graduated from grade school. The linked article example is far beyond anything I ever wrote for mturk.
The most insulting and infuriating turk jobs I ran into were not on the author's list, basically being paid to be spammed to. Here's 20 full page variations of a home equity mortgage spam, now pick your favorite one for a nickle. Or call this number and talk to this scammer for a minimum of 5 minutes and then come here and fill out a survey for like 10 cents, at least on the surface its for "quality control" but you know its really to give you the hard sell.
Life is cheap on the turk, and its full of work hits that pay the equivalent of a buck an hour or less, which is eventually super demotivating / depressing. You hear about cool high paying hits but end up filling out 20 page surveys about B.S. for like ten cents, and getting sick of it.
I haven't found the need to use "tricks." Basic questions work pretty well. I've done a couple of things. 1) Ask, "Do you have any problems reading English?" Some people with poor English skills, or people being honest, or people not reading carefully enough will click "Yes" thinking that I'm asking them if they speak English. 2) If sound is required, I've played an audio clip and asked people what was playing. They have to be able to identify what was happening from a list of words. This makes sure that they have headphones and can speak English. 3) I sometimes will ask people an obvious question about something that was presented on the previous page to make sure people pay attention. For example, if they have to read an essay on baseball, and the word "baseball" is mentioned 20+ times in a paragraph, I might ask them what sport the essay was about. A surprising number of people miss questions like that. 4) As mentioned by another poster, tell them to select a specific choice (e.g. "Strongly disagree"). I try not to do anything too nitpicky out of respect for people's time and because people aren't perfect.
It may be worth noting that making surveys that actually provide good data, perhaps especially longitudinal data (eg: how does Android popularity change over the period 2014-2024), is both science and art. It's also compounded if you try to collect answers in several languages. See eg:
http://www.europeansocialsurvey.org/methodology/improving_qu...
http://www.europeansocialsurvey.org/methodology/improving_qu...
Can anyone suggest any alternatives to Mechanical Turk that offer a better experience for the worker?
Check out CrowdFlower. The workers are mturk workers, but I believe that there's more attention placed on the worker.
For example, say you are broke and wandering the streets of a big city and really want to buy a bus ticket to another city for $20 or whatever. No matter how much your net worth is, you can't get that $20 easily. You could try asking various businesses to do some work for cash, you could try a song and dance routine, etc, but your choices all amount to various forms of begging.
Meanwhile you can be sitting in an office somewhere and make that money in less than 3 hours even at minimum wage, even if you do little or no work. You can even sell something on craigslist if you’re home. I suppose in desperation you could sell blood plasma, but that’s one of very few lifelines.
Nobody thinks $20 is worth much, but when you don’t have it, it’s very expensive indeed.
To me, the root of the problem is whether you are resource rich or resource poor. So the web, by virtue of being intangible, is almost quintessentially resourceless. The trend seems to be lower and lower wages for increasingly onerous labor. In other words, if you have money, you get a real world return greater than the value of your money. But if you don’t have money, then getting it requires an expenditure of resources and effort larger than the value of the money itself.
At some point in the near future, acquiring money will be so expensive from a labor standpoint that it will be cheaper to simply do things yourself and live outside of mainstream society by bartering goods and services. This really bothers me, because that shouldn’t be the goal of progress. The paradox is that even though every new fiverr and mturk create more jobs, they lower the value of work. Right now this is affecting developing countries by creating a race-to-the-bottom economy, but the futurist in me looks all the way to the end and sees how so many jobs today (especially the non-production ones like administrative/clerical work) will eventually be automated by technology and thereby make the purchase of capital through labor even more expensive.
Does anyone see a way out of this? Did I miss something fundamental?