The odd reality of life under China's all-seeing credit score system(wired.co.uk)
wired.co.uk
The odd reality of life under China's all-seeing credit score system
http://www.wired.co.uk/article/china-social-credit
56 comments
Cheating in online competitive gaming is a form of lying. I play CSGO myself and it is extremely frustrating when people aimbot or wallhack. Based on personal experience playing against aimbot is like banging head against the wall, it definitely lowers my quality of life. Moreover a single person with aimbot can depopulate entire server pretty quickly.
A person chooses to negatively affect quality of life of 10-20 other people. I can't find a word to describe that behavior other than asshole. With that in mind the issue can be reframed to - should assholes have a lower credit score?
I do not know. I certainly do not agree with Chinese methods, because I think positive reinforcement works better. With that said, I am curious what long term effect Chinese population hacking will have on their society.
On using positive reinforcement to shape the society. Imagine a city park where lights pulse green when someone picks up a piece of trash from the ground and throws it away into the trash can. Or a park that remembers people that do not litter and greets them with a positive reinforcement signal when they walk in.
A person chooses to negatively affect quality of life of 10-20 other people. I can't find a word to describe that behavior other than asshole. With that in mind the issue can be reframed to - should assholes have a lower credit score?
I do not know. I certainly do not agree with Chinese methods, because I think positive reinforcement works better. With that said, I am curious what long term effect Chinese population hacking will have on their society.
On using positive reinforcement to shape the society. Imagine a city park where lights pulse green when someone picks up a piece of trash from the ground and throws it away into the trash can. Or a park that remembers people that do not litter and greets them with a positive reinforcement signal when they walk in.
> Or a park that remembers people that do not litter and greets them with a positive reinforcement signal when they walk in.
People who litter will have points deducted from their score and they'll no longer be allowed in parks. Unless they have money of course. "Donating to charity" can increase your score, so the rich will be able to whatever the fuck they want and the poor will lose access to parks.
> playing against aimbot is like banging head against the wall, it definitely lowers my quality of life
Some friendly advice from a fellow human - go take a nice long walk in a park. Enjoy the fresh air and sunshine, while you still can.
People who litter will have points deducted from their score and they'll no longer be allowed in parks. Unless they have money of course. "Donating to charity" can increase your score, so the rich will be able to whatever the fuck they want and the poor will lose access to parks.
> playing against aimbot is like banging head against the wall, it definitely lowers my quality of life
Some friendly advice from a fellow human - go take a nice long walk in a park. Enjoy the fresh air and sunshine, while you still can.
> People who litter will have points deducted from their score and they'll no longer be allowed in parks. Unless they have money of course. "Donating to charity" can increase your score, so the rich will be able to whatever the fuck they want and the poor will lose access to parks.
The whole point of the above comment was, in opposition of Chinese system, to only provide positive interactions. The surveillance genie is out of the bag and there is no going back. For better or for worse cameras are everywhere, it is only a matter of time until there is AI watching most of them. The traditional system is based heavily on punishment, so it does not take a lot of imagination to extend it to the automated enforcement.
Indeed one of the worst possible outcomes is when your karma is directly tied into your banking account. In that world the rich have nearly unlimited indulgences. I was trying to imagine a world where the technology is used to provide positive feedback instead of punishments.
> Some friendly advice from a fellow human - go take a nice long walk in a park. Enjoy the fresh air and sunshine, while you still can.
I do that quite often, but every once in a while I want some variety such as competitive games.
The whole point of the above comment was, in opposition of Chinese system, to only provide positive interactions. The surveillance genie is out of the bag and there is no going back. For better or for worse cameras are everywhere, it is only a matter of time until there is AI watching most of them. The traditional system is based heavily on punishment, so it does not take a lot of imagination to extend it to the automated enforcement.
Indeed one of the worst possible outcomes is when your karma is directly tied into your banking account. In that world the rich have nearly unlimited indulgences. I was trying to imagine a world where the technology is used to provide positive feedback instead of punishments.
> Some friendly advice from a fellow human - go take a nice long walk in a park. Enjoy the fresh air and sunshine, while you still can.
I do that quite often, but every once in a while I want some variety such as competitive games.
> go take a nice long walk in a park. Enjoy the fresh air and sunshine, while you still can.
What if the park is polluted with litter? The point is that people can degrade your quality of life in many places, even if it's as "trivial" as leisure.
What if the park is polluted with litter? The point is that people can degrade your quality of life in many places, even if it's as "trivial" as leisure.
I might be in the minority but I always try and pick up litter when I'm walking in a park, I feel even better about it knowing that I've left it a little cleaner than when I started. I know there are others that think the same way as I've seen them and had a friendly chat in the past.
In that vein, Overwatch announced a new endorsement system yesterday, where people can mark others for "sportmanship", "good teammate", and "shot-caller", and they're going to highlight players who regularly get them.
https://mic.com/articles/189690/online-shooter-overwatch-to-...
They've been getting more and more requests for ways to penalize bad players, so I thought this was an interesting direction to push for features. A little more sophisticated that the general "commend a teammate" feature that was more or less just an upvote.
https://mic.com/articles/189690/online-shooter-overwatch-to-...
They've been getting more and more requests for ways to penalize bad players, so I thought this was an interesting direction to push for features. A little more sophisticated that the general "commend a teammate" feature that was more or less just an upvote.
> Or a park that remembers people that do not litter and greets them with a positive reinforcement signal when they walk in.
Sounds like a scene from a dystopian movie set in the future. Like a park in a city where the populace has lost its soul and everything is sterile and white and hopeless.
I would vandalize that positive reinforcement robot with some choice Newspeak words of my own.
Sounds like a scene from a dystopian movie set in the future. Like a park in a city where the populace has lost its soul and everything is sterile and white and hopeless.
I would vandalize that positive reinforcement robot with some choice Newspeak words of my own.
CITIZEN: OUR AUTOMATIC TEXT ANALYSIS HAS DETECTED INTENTION OF VANDALISM AND YOUR REPUTATION WAS LOWERED 50 POINTS. YOU ARE NO LONGER ALLOWED TO ENTER PUBLIC PARKS.
IF YOUR REPUTATION FALLS BELOW ZERO YOU WILL BE ENCOURAGED TOWARDS THE NEAREST CITIZEN ENHANCEMENT CENTER.
YOUR REMAINING REPUTATION IS 42.
IF YOUR REPUTATION FALLS BELOW ZERO YOU WILL BE ENCOURAGED TOWARDS THE NEAREST CITIZEN ENHANCEMENT CENTER.
YOUR REMAINING REPUTATION IS 42.
The effect of a lower credit score in this case is disproportionately large compared to the impact of the assholeness.
> when someone picks up a piece of trash from the ground and throws it away into the trash can.
Reminds me of the Cobra effect. Cause trouble, fix it - unlimited positive reinforcement.
Reminds me of the Cobra effect. Cause trouble, fix it - unlimited positive reinforcement.
PC bang/internet cafes in China are filled with computers preloaded with aimbots. Every new multiplayer game maker quickly learns to isolate this part of the world with separate server pool, or face sale decline as people leave your game and never come back.
The problem with these sorts of systems, both those backed by technology and those backed by old-fashioned prejudice, is that it exerts an upward force on those who are already on top while simultaneously pushing downwards on those who are on the bottom. Each of the examples listed here have this effect: having a high credit score means you get cheaper healthcare, have more opportunities to find a well-off partner, don't need to pay deposits on car rentals, get to travel, etc. It seems as though if you're on top, you're going to stay there, meanwhile if you're on the bottom, you're going to stay there also.
What I'd like to know, as an American sitting literally a world away from the people under this system, is to what extent wealth and influence purchase a high social credit score? Some factors are obvious: those with more money are less likely to incur debts, for instance. But what about connections to administrators? I imagine there isn't a politician in all of China who doesn't magically have a sterling social credit score.
I wonder whether the net result is a society in which a high standing resulting from material and social wealth leads to perks that in turn lead to more material and social wealth.
What I'd like to know, as an American sitting literally a world away from the people under this system, is to what extent wealth and influence purchase a high social credit score? Some factors are obvious: those with more money are less likely to incur debts, for instance. But what about connections to administrators? I imagine there isn't a politician in all of China who doesn't magically have a sterling social credit score.
I wonder whether the net result is a society in which a high standing resulting from material and social wealth leads to perks that in turn lead to more material and social wealth.
The same way, for instance, Chinese wealth has purchased high social rewards in America; university placement for their children, investments, politics, etc. Wealth and influence seem to ahhh trump all, so to speak.
Let me give you a counter example: https://en.wikipedia.org/wiki/Jia_Yueting
He is on the travel ban list due to credit problems in China, yet apparently rich and free in the U.S.
The first released travel ban list has 169 names on it: https://www.creditchina.gov.cn/toutiaoxinwen/201806/t2018060...
(according to Google Translate) "On June 1, the first list of restricted-riding trains was publicized on the “Credit China” website. The list of public opinions contained a total of 169 persons, of which 21 were provided by the State Administration of Taxation, mainly involving non-payment or underpayment of tax payables, illegal receipts of value-added tax invoices, etc.; the CSRC provided 31 persons involved in overdue performance of securities and futures Responsible parties related to listed companies failed to fulfill their public commitments within the overdue period; the Civil Aviation Administration provided 86 people, mainly related to finding trouble in airplanes, blocking traffic in airport security, occupying, impinging on security inspection channels, and assaulting other people, and checking out carry-on countries in airport security inspections. Laws and regulations stipulate that dangerous goods (stuck lighters) are subject to administrative penalties; the railway company provides 31 people, mainly involving smoking on EMU trains or smoking in non-smoking areas of other trains, without ticket or station ) Travel by car and refusing to make up for tickets etc. The public can click https://hmd.creditchina.gov.cn/ to view."
He is on the travel ban list due to credit problems in China, yet apparently rich and free in the U.S.
The first released travel ban list has 169 names on it: https://www.creditchina.gov.cn/toutiaoxinwen/201806/t2018060...
(according to Google Translate) "On June 1, the first list of restricted-riding trains was publicized on the “Credit China” website. The list of public opinions contained a total of 169 persons, of which 21 were provided by the State Administration of Taxation, mainly involving non-payment or underpayment of tax payables, illegal receipts of value-added tax invoices, etc.; the CSRC provided 31 persons involved in overdue performance of securities and futures Responsible parties related to listed companies failed to fulfill their public commitments within the overdue period; the Civil Aviation Administration provided 86 people, mainly related to finding trouble in airplanes, blocking traffic in airport security, occupying, impinging on security inspection channels, and assaulting other people, and checking out carry-on countries in airport security inspections. Laws and regulations stipulate that dangerous goods (stuck lighters) are subject to administrative penalties; the railway company provides 31 people, mainly involving smoking on EMU trains or smoking in non-smoking areas of other trains, without ticket or station ) Travel by car and refusing to make up for tickets etc. The public can click https://hmd.creditchina.gov.cn/ to view."
> I wonder whether the net result is a society in which a high standing resulting from material and social wealth leads to perks that in turn lead to more material and social wealth.
Why do you have to wonder and why do you need to look to China to learn this? This is how America worked and works, way before China got to it.
I know it's hard to introspect and all, but I'm glad having a China that looks more like America (than before) is making Americans finally see our own country more clearly.
Why do you have to wonder and why do you need to look to China to learn this? This is how America worked and works, way before China got to it.
I know it's hard to introspect and all, but I'm glad having a China that looks more like America (than before) is making Americans finally see our own country more clearly.
Under such a system, what happens to an individual if errors are made?
If you have 1000 million citizens, your social credit computation better have a sub-ppm false positive rate, or else you risk ostracizing thousands of innocent people.
Maybe on the whole it's worth it for the Chinese government.
Also, I wonder if increasing adoption of this system will create a black market of 'score hackers', people that try to fuzz the algo's into giving free or cheap social credit....
If you have 1000 million citizens, your social credit computation better have a sub-ppm false positive rate, or else you risk ostracizing thousands of innocent people.
Maybe on the whole it's worth it for the Chinese government.
Also, I wonder if increasing adoption of this system will create a black market of 'score hackers', people that try to fuzz the algo's into giving free or cheap social credit....
A callous government which oversees the lives of 1,000 million citizens isn't concerned with thousands of false positives.
All governments are callous. Can you give an example of a bureaucracy that isn't callous?
I don't think they were implying that there are uncallous governments, just emphasizing that particular feature of the Chinese government.
Some governments are more callous than others.
not many that disregard human rights or are as callous as the chinese govt.
When you get to 1 billion, your representatives will be as callous. (Already on the way there, actually.) Government is just a mirror of the people, even in China.
I disagree. I argue that the Chinese government and the one I live under (in the United States) could hardly be more different. The government of China was established for the benefit of the rulers (the Party). The American Founding Fathers said "Government is like fire: It makes a good servant, but a poor master." Our Constitution was written primarily to constrain the government.
Only two countries have gotten to 1 billion inhabitants, and I'd say that the government of India is much less callous or calculating than that of China.
Only two countries have gotten to 1 billion inhabitants, and I'd say that the government of India is much less callous or calculating than that of China.
Founding intention does not matter. Modern Russia was founded on the idea that all people are equal and that workers should be respected, that doesn't mean that that happens.
The founders of Communist Russia intended that the state be dissolved by 1980, that it be replaced by councils that work together in coöperation, such that everyone has a say. Did that bear fruit? No.
Did the founding fathers intend for the United States to spy on every single one of their citizens? No. Does it happen? Yes. Did the founding fathers intend for the state to be able to detain foreigners of the US indefinitely without trial? No. The patriot act allows it, however.
You cannot judge a country by founding principals and safeguards alone, they are easy to repeal and change; you can only judge a country on the current legislature and how much the current Constitution constrains the government.
The founders of Communist Russia intended that the state be dissolved by 1980, that it be replaced by councils that work together in coöperation, such that everyone has a say. Did that bear fruit? No.
Did the founding fathers intend for the United States to spy on every single one of their citizens? No. Does it happen? Yes. Did the founding fathers intend for the state to be able to detain foreigners of the US indefinitely without trial? No. The patriot act allows it, however.
You cannot judge a country by founding principals and safeguards alone, they are easy to repeal and change; you can only judge a country on the current legislature and how much the current Constitution constrains the government.
I feel that this discussion has wandered. Are you arguing that the Chinese government is concerned with false positives in their social scoring system?
I'm arguing that the US and China are closer than you might think with regards to the goals of surveillance and tyrrany. The difference in founding intentions are irrelevant.
> I wonder if increasing adoption of this system will create a black market of 'score hackers'
Well, Campbell's law...
> "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
Well, Campbell's law...
> "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
Just for those interested:
"All Systems will be Gamed: Exploitive Behavior in Economic and Social Systems," in Complexity and the Economy, W. B. Arthur (2014)
http://tuvalu.santafe.edu/~wbarthur/Papers/All%20Systems%20G...
I suppose there are two sorts of errors:
System errors: falsely attributing you with certain penalties (or credits). The workaround here is the same as with a bank - provide a statement and a mechanism to dispute entries.
Fine errors: penalising you when you don't deserve it. This is like anything else in the justice system, although it has the advantage that it should be an accumulation of a larger number of smaller inputs (positive and negative), so individual errors should act like noise and remain negligible.
System errors: falsely attributing you with certain penalties (or credits). The workaround here is the same as with a bank - provide a statement and a mechanism to dispute entries.
Fine errors: penalising you when you don't deserve it. This is like anything else in the justice system, although it has the advantage that it should be an accumulation of a larger number of smaller inputs (positive and negative), so individual errors should act like noise and remain negligible.
"...a mechanism to dispute entries"
I seriously doubt that the Chinese government is interested in inviting dispute. Dispute is exactly what such a surveillance system is designed to suppress.
I seriously doubt that the Chinese government is interested in inviting dispute. Dispute is exactly what such a surveillance system is designed to suppress.
[deleted]
There's a chinese girl in my russian class, so I asked her about it. She thought it was a great idea so people can't leave without paying their debts, and can't get further into debt.
Until your inability to leave your current city for better opportunities in another stifles your ability to get out of the first debt, of course.
A valid argument.
"Mortals never sell me their souls. They give them away."
-Satan, "The Broken Sword" by Poul Anderson
That's a defense of a financial credit score system, not a social credit score system. And it's no defense of a top-down authoritarian score system.
China doesn't have free speech or free press, so unfortunately a Chinese resident can only form "low-information" opinions about issues outside their immediate experience; as they spend their entire life under the brainwashing of an authoritarian government.
China doesn't have free speech or free press, so unfortunately a Chinese resident can only form "low-information" opinions about issues outside their immediate experience; as they spend their entire life under the brainwashing of an authoritarian government.
Many Americans say we don't have free speech here in Europe. Does that mean I can only form a "low-information" opinion? I'm curious.
> Many Americans say we don't have free speech here in Europe
I've never heard an American or European say that (other than a case of hyperbole here or there. What could this refer to?
I've never heard an American or European say that (other than a case of hyperbole here or there. What could this refer to?
Since you brought this up: Do you agree that Europeans don't have free speech?
Yepp, and it's not a problem IMHO. Being from Europe, I tend to disagree with the american form of free speech, as I believe meaningful free speech is precious, and we shouldn't allow some right wing extremists to take advantage and ruin it by using it as an excuse for hateful language that contributes precisely nothing to political debate.
Prime example of brainwashing right there I guess.
for some reason i haven't seen many people discuss china as a totalitarian country.
but is there really any doubt, at this point? the government seeks to control an ever-greater range of human behaviors, from the banal -- like borrowing an umbrella -- to the political.
it's unclear to me whether the chinese people want this sort of system for whatever reason or whether it's being forced onto them. what is clear to me is that this system would warrant a prompt and violent revolution if a similar implementation was attempted here.
once these systems take root, become normalized, and start to change the way that people behave, a society's chance at freedom is finished...
but is there really any doubt, at this point? the government seeks to control an ever-greater range of human behaviors, from the banal -- like borrowing an umbrella -- to the political.
it's unclear to me whether the chinese people want this sort of system for whatever reason or whether it's being forced onto them. what is clear to me is that this system would warrant a prompt and violent revolution if a similar implementation was attempted here.
once these systems take root, become normalized, and start to change the way that people behave, a society's chance at freedom is finished...
> it's unclear to me whether the chinese people want this sort of system for whatever reason or whether it's being forced onto them.
These become indistinguishable by "manufacturing consent".
These become indistinguishable by "manufacturing consent".
The groupthink is quite strong in China. If you agree with the policies of the government, you get to live happily in society. If not, you're gonna have a bad time.
It doesn't take a genius to decide that you'd be better off consenting, whether that consent has been manufactured or outright enforced.
It doesn't take a genius to decide that you'd be better off consenting, whether that consent has been manufactured or outright enforced.
The chinese accept their authoritarian regime simply because of the immense economic growth they’ve had over the past few decades. Not long ago they were starving under Mao. No one revolts under good times and risks going back to starvation.
Washington Post has been hammering that nail intensely:
https://www.google.com/search?q=china+totalitarian
Perhaps others have not, considering the Post's dominance of the Google front page.
https://www.quora.com/Is-China-a-totalitarian-state
makes a good point:
> Authoritarian yes. Totalitarian, no.
> The difference between authoritarian and totalitarian is that in an authoritarian state, what is not banned is allowed. In a totalitarian one, what is not banned is required.
and that China is so much better than it was in the 1950s, that is deserves a better evaluation.
Still, from the perspective of a mostly-free society, the distinctions between Authoritarian and Totalitarian are blurred.
Perhaps others have not, considering the Post's dominance of the Google front page.
https://www.quora.com/Is-China-a-totalitarian-state
makes a good point:
> Authoritarian yes. Totalitarian, no.
> The difference between authoritarian and totalitarian is that in an authoritarian state, what is not banned is allowed. In a totalitarian one, what is not banned is required.
and that China is so much better than it was in the 1950s, that is deserves a better evaluation.
Still, from the perspective of a mostly-free society, the distinctions between Authoritarian and Totalitarian are blurred.
Aren't there like four different Chinas?
You have the modern mainland, which is where all these problems exist, the rural mainland which this probably doesn't affect as much, Hong Kong which is autonomous, and Chinese Taipei which might as well give up the ghost by this point.
You have the modern mainland, which is where all these problems exist, the rural mainland which this probably doesn't affect as much, Hong Kong which is autonomous, and Chinese Taipei which might as well give up the ghost by this point.
The unit of account is different. Totally untethered individual freedom is unknown in East Asia and paternalistic societies have been around forever to enforce group norms.
Different values for different folks. Some people are happy to live in conservative, stable, conformist, and low-risk societies (whether because they are brought up in such or not, though usually the former). Some people are not.
Different values for different folks. Some people are happy to live in conservative, stable, conformist, and low-risk societies (whether because they are brought up in such or not, though usually the former). Some people are not.
"Totally untethered individual freedom is unknown in East Asia"
Where in society does 'untethered individual freedom' exist? I live in Canada, a very 'free' country, yet there are many rules and laws to follow. Enough that I probably break the law multiple times per day.
Where in society does 'untethered individual freedom' exist? I live in Canada, a very 'free' country, yet there are many rules and laws to follow. Enough that I probably break the law multiple times per day.
but is there really any doubt, at this point?
There's never been any doubt among people who know China's recent history.[0]
The problem is that for the most part, Americans under 30 or so don't know about it. They are frequently confused when I remind them that China is communist. They grew up thinking of it as just another country, but with a slightly different government system. Like Germany or Japan.
If you tell them that the Chinese government murdered millions of its own people, they're dumbfounded until they, themselves, look it up.
The path that China is taking isn't a surprise to most older people (including diplomats and politicians). And it's why China is working so hard to whitewash its global repuration where it can, or attempt to erase history, where it's able.
[0]https://en.wikipedia.org/wiki/Cultural_Revolution
There's never been any doubt among people who know China's recent history.[0]
The problem is that for the most part, Americans under 30 or so don't know about it. They are frequently confused when I remind them that China is communist. They grew up thinking of it as just another country, but with a slightly different government system. Like Germany or Japan.
If you tell them that the Chinese government murdered millions of its own people, they're dumbfounded until they, themselves, look it up.
The path that China is taking isn't a surprise to most older people (including diplomats and politicians). And it's why China is working so hard to whitewash its global repuration where it can, or attempt to erase history, where it's able.
[0]https://en.wikipedia.org/wiki/Cultural_Revolution
How close China has become to something out of 1984 is truly disturbing
1984 wasn't written as fantasy horror fiction. It was an informed and reasoned extrapolation of the then-present into the future, with some adventure fiction elements.
Sadly, this turn of events was not unexpected.
Sadly, this turn of events was not unexpected.
Here's more scary literature: https://www.engadget.com/2018/02/22/china-xinjiang-surveilla...
Tangentially related, the Daniel Suarez sequel novel, Freedom(TM) presents a similar scenario. People will have a rating system based on their interactions with others (think eBay) and you can see that rating. Four stars out of a base of 81 interactions - from the postman to your boss. I have a hunch that China's generally compliant, anti-antagonistic society, makes the Chinese system easier to implement. In a western country, I think it would turn south really, really quickly and for much worse.
Sesame Credit really creeps me out, but give it a few years and you'll see LifeHacker articles on "10 Ways to Raise Your Sesame Credit Score!"
Valve knowingly allows its cheat countermeasures to impact human lives in this way? I'd like to hear their response to this claim.