I hate to interrupt and go against the flow here, but it seems that some main points have not been mentioned.
First, the OP is talking about his 'business', no doubt a software and Web site based business, but still a business.
Here is the history of database for businesses in four easy steps:
Step 1.
In the 1950s with computers and then and before for a few decades, people did business computing with, say, files and decks of punched cards.
Step 2.
In the 1960s it was noticed that a small change in the business could result in too much programming; so the idea of 'database' was born. There for new purposes could add fields, links, etc. to the database without need to revise old software -- a huge point.
Approaches in the 1960s included hierarchical and network.
Step 3.
In the 1970s the mature conclusion from the experience of the 1960s was that a 'database' should be 'relational' with tables and relations. Each table had one column for each quantity and one row for each instance of those quantities. E.g., maybe in one table there was one row for each customer and one column for each quantity about a customer -- customer number, phone number, billing address, etc. A table usually had a key, and the table was in 'third normal form' so that each row was a function of the key, the whole key, and nothing but the key. Tables with relatively few columns could be combined, as needed for ad hoc purposes, into new, possibly larger, tables with 'join' operations based on 'relations'.
Leaders in this movement included E. Wong at Berkeley and M. Blasgen at IBM's Yorktown Heights lab. The Wong work resulted in INGRES, and the Blasgen work resulted in R* and later DB/2.
As database theory, the story was in good shape as of 1980. See, e.g.,
Jeffrey D. Ullman, 'Principles of Database Systems, Second Edition', ISBN 0-914894-36-6, Computer Science Press, Rockville, MD, 1982.
A relational database is usually programmed in 'structured query language' (SQL), and it is standard. SQL is easy to learn and fun to use.
Relational database systems take data integrity and security, system reliability, and data backup and recovery very seriously. A key to the functionality is the 'log file', and one result is the ability to backup a database while it is executing reads, writes (inserts, updates, deletes). If there is a hardware failure, say, power outage, in the middle of a large operation, then when the power comes back on the database will be put back in a 'consistent' state. There are extensions for parallelism, redundancy, and distributed implementation. The algorithms are advanced, and the software is mature and very thoroughly tested. E.g., a complicated operation is carefully 'complied' into a 'plan', and that with smart use of caching data in main memory commonly results in shockingly good performance. If a complicated operation is needed, then it will be difficult to beat the performance of just stating what is desired in SQL and letting the database software do the rest. There are mature tests of performance; see, the Transaction Processing Performance Council work at
The performance achieved on the tests, from simple to very complicated, is astoundingly high.
For a business, relational database brings high 'flexibility': With well designed tables, relational puts the business in the best position to meet the data manipulation needs a few years in the future where the needs are not understood now. This one point is likely the most important reason for a business to take relational database seriously.
Well known relational database systems now include SQL Server, Oracle, DB/2, MySQL, and some more.
Curiously, conceptually the files in the punched card days were much like relational database now. So, one deck of cards corresponded to one database table. All the decks together corresponded to the whole database. In one deck, each card corresponded to a database row, and each field on the card corresponded to a database column. It was common to sort a deck on a column. Given two decks sorted on a common column, it was common to 'collate' the two decks and create a third, and that operation is essentially a relational database 'join'.
Broadly, the basic operations needed to do well manipulating database data have been known for nearly 100 years.
Step 4.
As the Internet and Web servers grew in importance, there have been needs for special purpose database systems with, compared with relational database, more performance but less functionality and generality. So, we have NoSQL, key/value stores, etc.
Most experienced programmers know some SQL, and even if they do not it should be easy for them to learn and start having fun with SQL quickly. A program in SQL is just simple text and is easy to type, comment, and read. For the fun, J. Bezos at Amazon had fun early on when he saw how easy it was to get some really nice information about Amazon's business from a simple query typed in as SQL.
For using relational database in a startup, the main management issue will be the need for the expertise of a 'database administrator' (DBA). For a startup, first, hopefully it's reasonable just to f'get about a DBA, keep everything simple, and do the work in-house. For background, read the chapter on database in, say,
Jim Buyens, 'Web Database Development, Step by Step: .NET Edition', ISBN 0-7356-1637-X, Microsoft Press, Redmond, Washington, 2002.
and then read the first chapter of Ullman above.
Second, as the work gets more challenging, get a good phone number, call, and ask questions of some person who does DBA work full-time. Paying a high hourly rate to such a person will still be much cheaper than having a good DBA in-house. E.g., for working through some tricky issue in security or performance monitoring will be much easier if call someone who considers such things full-time. If get to be a big company, then, sure, have lots of in-house database expertise.
So, net, for the BUSINESS, use relational database. For some special purposes in, say, high performance Web site architecture, maybe start by just using relational database but when more performance gets to be crucial move to special purpose tools.
Really, the special purpose tools are not directly competitive with relational database. E.g., at my Web site, I'm keeping all the important data in relational database. But in one case, for one special purpose, some data in the relational database is read into some simple files and used read-only with very high performance, maybe off SSD or just in virtual memory. In another case, I'm implementing a simple key/value store, a baby Redis, so simple it is easier for me to program what I want than even to read the Redis documentation. What I am programming can knock the socks off relational database in several respects and do well for my company until it is so big I can let others worry about what else to do. But, for my BUSINESS, including all the important data in my Web server farm, that's in relational database. Period.
> I think you misread and/or misinterpreted OP's point
His "point" was not clear: What was more important in the article, that there was a Google+ community discussing some "problems" or the "problems" he listed of concern to that community?
The leading problem listed was difficulty in getting published, and I suggested a way to get published: Do well on "new, correct, and significant" and, in particular, 'mathematize' the field.
Responding to that "problem" was fully appropriate.
In more detail, one of the best ways to mathematize CS is more in probability, especially as it connects with 'information'. It does connect; trust me on that one.
That was a good and helpful suggestion and should have been welcome.
> Later came these repeated and almost manic allegations of some secret cabal of MODs (using your capitalization) who are after you.
There's a LOT of evidence that some MODs are angry with me and for the past month or so have been attacking me personally. For several reasons I've listed on this thread, much of the downvoting had to be from MODs. Apparently HN is a PG PC sandbox, and MODs will downvote anything they feel violates the PC norms of HN and do so without responding.
PG won't respond to deny this.
I would have been a fool to ignore the clear evidence that I was being attacked and a wuss not to defend myself. It's a scandal for HN, and PG has not responded.
On another user ID I was 'hell banned' here at HN.
Today I finally responded in plain terms to defend myself. If by defending myself I get banned from HN, then so be it. But for now the HN community will see that HN attacks some users personally.
I tried to help CS researchers get published. My claim, easy enough to understand, is that the research paradigm of CS is largely bankrupt, and my recommendation is to further mathematize the field. They are in a "wretched mess" if only from their statement of their "problem".
> puerile delusion of grandeur on display was mildly amusing and continues to amuse me
Insulting nonsense. There is no delusion involved. I was quite apparently attacked and have been several times for about a month now. It's personal, not technical or anything else.
> if I may put it in my own words
My own words for my own statement were much more appropriate: To repeat, CS is close to 'information technology', and there 'information' should be taken seriously mathematically. The R-N theorem is one important approach.
The "lack of background" of the Editors in Chief and chaired professors of computer science was surprising and shocking. I used some group theory and probability based on measure theory and got a new family of statistical hypothesis tests both multidimensional and distribution-free and applied them to ASAP detection of anomalies in server farms and networks, a good CS problem, and bluntly too much of the best of the CS community couldn't handle the math.
So, there is a big, huge gap between some of the best of current CS and what it would take to do at all well applying some 'modern probability' to some CS problems. Due to this gap, my suggestion to learn the math is appropriate and should be seen as helpful.
In simple terms, if a student wants a research career in CS, then as an undergraduate it is much more important for them to major in math than CS. And likely similarly at the Master's level. That is a surprising point but potentially quite helpful and should be welcome.
"On other hand if you can show that your favorite Neveu calculus can be proved sound according to some computable logic, that would indeed be helpful."
Nonsense. Neveu is based just on set theory, axiomatic set theory if you wish, the foundations as in Bourbaki if you wish, or P. Suppes, essentially the same as all of math for the past 100 years or so. For the importance of that material, there is nothing anyone should have to "prove". 'Computability' is not directly relevant.
> For example, to follow up on your own example, ... such a submission will just get dropped
I never had a submission dropped in the sense of 'rejected'. For the paper in question, I sent copies to some journals and just asked if they would like a formal submission. Some journal Editors in Chief said that their journal couldn't review the math. With one such I wrote tutorials before he gave up. So, I didn't make a formal submission and, thus, never got rejected or "dropped".
One journal welcomed a formal submission, and we went forward. They had problems getting reviewers who could read the paper, and reluctantly I suggested a qualified friend who did a good review. Eventually the Editor gave up, and for more reviews apparently the Editor in Chief walked the paper around his campus, had the CS guys say the problem was good and had some math guys say the math was good, and the paper was accepted, in an archival journal. I was invited to present the paper at a conference but declined. I just wanted to publish the thing and be done with it; I had no desire to go to a conference.
It's clear: In being mathematized, the CS community is very short on the needed math. I'm sure it's happened before, e.g., Hamming and the start of coding theory based on finite field theory, likely poorly known in the CS community then. For a researcher with the math, there are important CS problems that can be solved fairly easily just sitting there and, thus, are good research opportunities.
E.g., in Feller II is renewal theory, and it has some obvious applications to a lot that goes on in a server farm or network. For more, in Neveu is martingale theory, and there is one of the strongest inequalities in math (knock off the strong law of large numbers in one line) and more, and it's easy to find martingales in nearly any stochastic system. Should be able to get some nice, new, strong inequalities in many algorithms and processes important in CS. There's plenty that can be done with stochastic optimal control. There are the applications of 'machine learning', that is, statistics done very badly, and can solve the problems much better with statistics done well.
For the paper I wrote, there's much more that could be done; one could run off a dozen or so papers as a 'stream' in roughly the same direction by changing some of the assumptions better to fit various real situations.
So, sure, I'm suggesting some 'field crossing', long known to be a good approach to 'innovation'.
Moreover I'm suggesting exploiting some of the best, rock solid math of the last century. So this is a very sound, conservative suggestion.
That the CS community is very short on this math is a big point and a good research direction for anyone wanting to publish in CS.
Indeed, as is painfully obvious, CS has been taking intuitive and heuristic approaches far too seriously and neglecting solid math approaches. This situation makes CS look dumb, but the flip side is a terrific opportunity.
Again, with the right math from the last century, now should be able to knock off important CS problems by the dozens like shooting fish in a barrel.
HN and you don't like this remark. Fine with me.
I tried to offer some help, but you don't want to hear it and want to criticize me for offering. Fine: I won't offer. I've already deleted the post.
With your remark on 'computability', there's no more reason to respond to you.
I wrote a good post. The CS community lost out.
Can lead a horse to water but can't make him drink.
> The main criteria for research work are "new, correct, and significant". Do well on all three of these and have NO PROBLEMS getting reviewed and published.
So, I said that have to "do well".
So your
> You couldn't be more wrong.
is a bit strong.
I've published maybe a dozen papers, and I've never had a paper rejected or needing significant revision.
I've never been interested in publishing, and maybe the situation has changed a lot in the last few years while I've being doing other things.
Whatever, my original post to help CS profs get published was not wanted, and I've deleted it. The CS profs can struggle on their own.
early in the article was that the 'community' had a "serious problem" which was getting published. So I outlined how to get published, and that should have been a welcome contribution.
It wasn't very clear in the article if the article was mostly about the community or the 'problems' of the community. Whatever, I gave a solution to what seemed in the article to be the most important "problem" of the community.
But my solution wasn't welcome, so I deleted it. CS profs can struggle on their own to get published without my help.
Because there is a pattern that would not apply to the relatively random users and needs a very persistent user, that is, a MOD. Because the downvotes happen QUICKLY, as if from a MOD. Because the downvotes don't come with replies. Because only a small fraction of users can downvote. Because part of the pattern of what gets downvoted is material that is relatively advanced technically and, really, a challenge to CS.
My original post was a nice contribution to CS people having trouble getting published. But, if people don't like such a contribution, then they won't have it. So, I deleted it. It's gone. The CS profs can continue to struggle to publish without my help.
My original post was downvoted within minutes of when I posted it, and there was no reply. This is NOT nearly the first time. There is a pattern, and I'd be a fool not to conclude that one or more HN MODS are personally angry at me.
"schizophrenic"? Total nonsense; look up the meaning of the word, a very specific medical term. Not even paranoid. I'm just defending myself against a pattern over the past few months at HN. There is a pattern of my being attacked, and this one post is just the most recent example.
As far as I can tell, I'm responding to every response that can use a response.
I've more than once asked PG to correct this. We will see what he does. This situation is a scandal at HN.
This is NOT nearly the first time. There is a pattern, and the only reasonable explanation is that one or more HN MODs just do NOT like a lot of my posts and jump to downvote ASAP with anger.
A guess that fits the facts of what they don't like is anything critical of any other media outlet, anything about CS, or anything about venture capital. But it is necessary to guess since the downvoters essentially never respond to the substance of my posts. The situation has clearly become personal for them and, then, me.
"I can’t speak for other disciplines, but within computer science, I’ve always felt that these meta discussions were inadequate — in terms of volume, vigor, and format. They happened mostly at conferences were largely limited to more senior/well-connected researchers, and lagged behind some of the serious problems that have accumulated such as restrictive publisher copyrights and very low acceptance rates at journals and conferences."
So, the claim here I noticed was:
"some of the serious problems that have accumulated such as ... very low acceptance rates at journals and conferences."
So, a big topic in that community was a "serious" problem getting published. So, I outlined how to get published. Simple.
In a word, my outline said to 'mathematize' CS. That suggestion is potentially earth shaking, likely not already in the community, and should be a welcome contribution.
"Far cry" in what sense? The AAAI IAAI conference was about the best could expect from the AAAI.
I published in JIS because it has a wider audience than, say, another Elsevier journal on 'theoretical computer scince' or some such, which I did consider.
For JOTA, I contend that such applied math is closer to the important future of CS than nearly any journal in CS. If I am correct here, and my theme in my post was that CS needs to 'mathematize', then I have been helpful.
There's no "Big Shot", another insult, involved. I just listed where I got published. From the article, one of the leading topics on the Google+ community was the difficulty of getting CS published, and I indicated that I had essentially been successful even though I wasn't much interested.
For the academic 'pecking order' of the journals, I have never cared since I've never had any academic aspirations.
But for someone having trouble getting published, the journals I listed are a big step up from no publications. So, if they can do what I did, then they would be ahead. So, my advice should have been seen as helpful, which was clearly the intention.
That lament was one of the leading topics in the Google+ community that was the the subject of the article.
Yes, the article was mostly about the community and not the issues of the community. But the article did list some of the leading issues of the community, and I responded to one of them. Uh, the issues of that community are more important than just the existence of the community! So it should have been appropriate for me to have commented on one of the issues of the community.
I was attacked, downvoted, for my main comment, and then defended myself. This is NOT nearly the first time for such. Usually the result of the downvote is to send my comment to the end of the thread where it is effectively lost, which is clearly the intention of the MOD.
Why a MOD? Because only a small fraction of the users can downvote, and users instead of the MOD attacking me personally write rebuttals. So far I have not a single rebuttal to any of the content of my original post. I did a great service to some lost CS profs -- outlined for them how to publish and, then, got attacked by a MOD.
Then I defended myself. You found my defense offensive. It was, but not nearly as much as that of the attack of the MOD.
The attacks are from downvoting, not upvoting. Only a small fraction of HN users can downvote. With so much downvoting, it has to be the MODs who are doing it. It's so consistent that it's a personal attack on me.
Yup. But in this case I know all too well just what the heck I'm talking about, unfortunately. Hint: I live in Dutchess County, NY and for years worked in Westchester Country, NY.
Besides, the NYT wrote just a puff piece. Now just why would the highly self-esteemed, formerly highly revered, long pseudo-objective NYT do that?
On close examination, you may find that their Research Division is mostly just a patent shop.
Likely their 'services' work is mostly just 'help desk'.
Likely much of what they are doing is what they did decades ago called 'facilities management' or 'service bureau'.
Likely for new products, they find a small company with a good customer list, buy the company, and have the IBM sales force include that company's products in the list of products available. That is, they don't really try to 'develop' such products on their own.
So, in many companies, the CIO can 'bring in IBM' and go play golf.
Likely a good CIO could give faster results, more innovative and valuable results, at lower cost. But not all CIOs are good.
First, the OP is talking about his 'business', no doubt a software and Web site based business, but still a business.
Here is the history of database for businesses in four easy steps:
Step 1.
In the 1950s with computers and then and before for a few decades, people did business computing with, say, files and decks of punched cards.
Step 2.
In the 1960s it was noticed that a small change in the business could result in too much programming; so the idea of 'database' was born. There for new purposes could add fields, links, etc. to the database without need to revise old software -- a huge point.
Approaches in the 1960s included hierarchical and network.
Step 3.
In the 1970s the mature conclusion from the experience of the 1960s was that a 'database' should be 'relational' with tables and relations. Each table had one column for each quantity and one row for each instance of those quantities. E.g., maybe in one table there was one row for each customer and one column for each quantity about a customer -- customer number, phone number, billing address, etc. A table usually had a key, and the table was in 'third normal form' so that each row was a function of the key, the whole key, and nothing but the key. Tables with relatively few columns could be combined, as needed for ad hoc purposes, into new, possibly larger, tables with 'join' operations based on 'relations'.
Leaders in this movement included E. Wong at Berkeley and M. Blasgen at IBM's Yorktown Heights lab. The Wong work resulted in INGRES, and the Blasgen work resulted in R* and later DB/2.
As database theory, the story was in good shape as of 1980. See, e.g.,
Jeffrey D. Ullman, 'Principles of Database Systems, Second Edition', ISBN 0-914894-36-6, Computer Science Press, Rockville, MD, 1982.
A relational database is usually programmed in 'structured query language' (SQL), and it is standard. SQL is easy to learn and fun to use.
Relational database systems take data integrity and security, system reliability, and data backup and recovery very seriously. A key to the functionality is the 'log file', and one result is the ability to backup a database while it is executing reads, writes (inserts, updates, deletes). If there is a hardware failure, say, power outage, in the middle of a large operation, then when the power comes back on the database will be put back in a 'consistent' state. There are extensions for parallelism, redundancy, and distributed implementation. The algorithms are advanced, and the software is mature and very thoroughly tested. E.g., a complicated operation is carefully 'complied' into a 'plan', and that with smart use of caching data in main memory commonly results in shockingly good performance. If a complicated operation is needed, then it will be difficult to beat the performance of just stating what is desired in SQL and letting the database software do the rest. There are mature tests of performance; see, the Transaction Processing Performance Council work at
http://www.tpc.org/
The performance achieved on the tests, from simple to very complicated, is astoundingly high.
For a business, relational database brings high 'flexibility': With well designed tables, relational puts the business in the best position to meet the data manipulation needs a few years in the future where the needs are not understood now. This one point is likely the most important reason for a business to take relational database seriously.
Well known relational database systems now include SQL Server, Oracle, DB/2, MySQL, and some more.
Curiously, conceptually the files in the punched card days were much like relational database now. So, one deck of cards corresponded to one database table. All the decks together corresponded to the whole database. In one deck, each card corresponded to a database row, and each field on the card corresponded to a database column. It was common to sort a deck on a column. Given two decks sorted on a common column, it was common to 'collate' the two decks and create a third, and that operation is essentially a relational database 'join'.
Broadly, the basic operations needed to do well manipulating database data have been known for nearly 100 years.
Step 4.
As the Internet and Web servers grew in importance, there have been needs for special purpose database systems with, compared with relational database, more performance but less functionality and generality. So, we have NoSQL, key/value stores, etc.
Most experienced programmers know some SQL, and even if they do not it should be easy for them to learn and start having fun with SQL quickly. A program in SQL is just simple text and is easy to type, comment, and read. For the fun, J. Bezos at Amazon had fun early on when he saw how easy it was to get some really nice information about Amazon's business from a simple query typed in as SQL.
For using relational database in a startup, the main management issue will be the need for the expertise of a 'database administrator' (DBA). For a startup, first, hopefully it's reasonable just to f'get about a DBA, keep everything simple, and do the work in-house. For background, read the chapter on database in, say,
Jim Buyens, 'Web Database Development, Step by Step: .NET Edition', ISBN 0-7356-1637-X, Microsoft Press, Redmond, Washington, 2002.
and then read the first chapter of Ullman above.
Second, as the work gets more challenging, get a good phone number, call, and ask questions of some person who does DBA work full-time. Paying a high hourly rate to such a person will still be much cheaper than having a good DBA in-house. E.g., for working through some tricky issue in security or performance monitoring will be much easier if call someone who considers such things full-time. If get to be a big company, then, sure, have lots of in-house database expertise.
So, net, for the BUSINESS, use relational database. For some special purposes in, say, high performance Web site architecture, maybe start by just using relational database but when more performance gets to be crucial move to special purpose tools.
Really, the special purpose tools are not directly competitive with relational database. E.g., at my Web site, I'm keeping all the important data in relational database. But in one case, for one special purpose, some data in the relational database is read into some simple files and used read-only with very high performance, maybe off SSD or just in virtual memory. In another case, I'm implementing a simple key/value store, a baby Redis, so simple it is easier for me to program what I want than even to read the Redis documentation. What I am programming can knock the socks off relational database in several respects and do well for my company until it is so big I can let others worry about what else to do. But, for my BUSINESS, including all the important data in my Web server farm, that's in relational database. Period.