This is a great article. For more background on this growing industry, I recommend a recent academic study on the use of surveillance technology at the Los Angeles Police Department. The book documents the LAPD's use of data brokerage firms that collect and aggregate info from public records and private sources (e.g., Palantir), as well as automatic license plate readers (ALPR’s) which record vehicles are they move around the city, and Suspicious Activity Reports from police and civilians, which include reports of mundane activities such as using binoculars, drawing diagrams, or taking pictures or "video footage with no apparent aesthetic value." All this data ultimately gets parked in Fusion Center facilities built after 9/11 where federal, state and local law enforcement agencies collaborate to collect, aggregate, analyze and share information. As the author observes, "The use of data in law enforcement is not new. For almost a century, police have been gathering data, e.g., records of citations, collisions, warrants, incarcerations, sex offender and gang registries, etc. What is new and important about the current age of big data is the role in public policing of private capitalist firms who provide database systems with huge volumes of information about people, not just those in the criminal justice system."
Sarah Brayne (2020) "Predict and Surveil: Data, Discretion, and the Future of Policing," Oxford University Press
For a contrarian argument, it is worth considering the advice of Michael Porter in his seminal study "Competitive Strategy: Techniques for Analyzing Industries and Competitors" (published in 1980):
> Some argue that firms should not choose competitive positions at all but concentrate on staying flexible, incorporating new ideas, or building up critical resources or core competencies that are portrayed as independent of competitive position. I respectfully disagree. Staying flexible in strategic terms renders competitive advantage almost unobtainable. Jumping from strategy to strategy makes it impossible to be good at implementing any of them. Continuous incorporation of new ideas is important to maintaining operational effectiveness, but this surely is not inconsistent with having a firm strategic position (xv-xvi)... A strategic position is a path, not a fixed location (xiv).
In other words, there is no such animal as a "fully diversified" entrepreneurial business (unless your business is simply investing money in a diversified stock portfolio). The very concept of a (non-financial) capitalist enterprise implies some degree of specialization. Even YC, to some extent, specializes and focuses their investments in certain high-tech industries.
Here are some classics in cognitive psychology with direct quotes that illustrate some of the main themes:
Richard Rhodes (1986) Making of the Atomic Bomb
"In scientific work, creative thinking demands seeing things not seen previously, or in ways not previously imagined; and this necessitates jumping off from 'normal' positions, and taking risks by departing from reality. The difference between the thinking of the paranoid patient and the scientist comes from the latter's ability and willingness to test out his fantasies or grandiose conceptualizations through the systems of checks and balances science has established – and to give up those schemes that are shown not to be valid on the basis of these scientific checks. It is specifically because science provides such a framework of rules and regulations to control and set bounds to paranoid thinking that a scientist can feel comfortable about taking the paranoid leaps. Without this structuring, the threat of such unrealistic, illogical, and even bizarre thinking to overall thought and personality organization in general would be too great to permit the scientist the freedom of such fantasying." (p. 151)
Daniel Kahneman (2011) Thinking, Fast and Slow
"In the current view of how associative memory works, a great deal happens at once. An idea that has been activated does not merely evoke one other idea. It activates many ideas, which in turn activate others. Furthermore, only a few of the activated ideas will register in consciousness; most of the work of associative thinking is silent, hidden from our conscious selves. The notion that we have limited access to the workings of our minds is difficult to accept because, naturally, it is alien to our experience, but it is true: you know far less about yourself than you feel you do." (p. 52)
Thomas Kuhn (1962) Structure of Scientific Revolutions
"One suspects that something like a paradigm is prerequisite to perception itself. What a man sees depends both upon what he looks at and also upon what his previous visual-conceptual experience has taught him to see. In the absence of such training there can only be... a 'bloomin’ buzzin’ confusion'" (p. 113)
Sarah Brayne (2020) "Predict and Surveil: Data, Discretion, and the Future of Policing," Oxford University Press
https://www.amazon.com/Predict-Surveil-Discretion-Future-Pol...