Foundations of Information(faculty.washington.edu)
faculty.washington.edu
Foundations of Information
https://faculty.washington.edu/ajko/books/foundations-of-information/#/
19 comments
Yeah that's really interesting but information is exactly equivalent to energy and matter, with conservation relations and everything.
Information which is not present in spacetime literally does not exist.
That said, I invite you to communicate some of this other kind of information which exists outside of spacetime to me to demonstrate otherwise.
I'd be waiting here, but alas, without time and space...
Information which is not present in spacetime literally does not exist.
That said, I invite you to communicate some of this other kind of information which exists outside of spacetime to me to demonstrate otherwise.
I'd be waiting here, but alas, without time and space...
[deleted]
Another book which talks about this:
Information. https://en.m.wikipedia.org/wiki/The_Information:_A_History,_...
Information. https://en.m.wikipedia.org/wiki/The_Information:_A_History,_...
Highly reccomended. I’ve read it several times, and learn something new every time.
I love the table of contents - initially this sounded vague but it’s a great breakdown of the entire landscape of “information”. It’s also very useful to have estimated reading time next to each topic. I wonder what the “next level deeper” looks like, since the author frames this as introductory material.
Skimmed through some of the chapters. Was disappointed to find no references to Shannon's theory of communication, or to some ideas about the economics of information (e.g. moral hazard, adverse selection).
That's probably just a function of my personal interests, though.
That's probably just a function of my personal interests, though.
I've not gone through all the chapters, but in Chapter 3 - Data, information, knowledge, Shannon's theory is mentioned [0].
>> Entropy is a measurable physical property of a state of disorder, randomness, or uncertainty. For example, the sequence of letters aaaaa has low entropy, conceptually, as it is the same letter appearing multiple times; it could be abbreviated to just 5 a’s. In contrast, the sequence of letters bkawe has high entropy, with no apparent pattern, and no apparent way to abbreviate it without losing its content. Shannon’s view of information was thus as an amount of information, measured by the compressibility of some data.
[0]: https://faculty.washington.edu/ajko/books/foundations-of-inf...
>> Entropy is a measurable physical property of a state of disorder, randomness, or uncertainty. For example, the sequence of letters aaaaa has low entropy, conceptually, as it is the same letter appearing multiple times; it could be abbreviated to just 5 a’s. In contrast, the sequence of letters bkawe has high entropy, with no apparent pattern, and no apparent way to abbreviate it without losing its content. Shannon’s view of information was thus as an amount of information, measured by the compressibility of some data.
[0]: https://faculty.washington.edu/ajko/books/foundations-of-inf...
Yup, noted in my comment below [1]. Thanks for the pointer though!
[1] https://news.ycombinator.com/item?id=26939479
[1] https://news.ycombinator.com/item?id=26939479
No reference to algorithmic information theory either, or Kolmogorov. All the interesting aspects of information are missing.
Imagine the eager young minds' disappointment upon learning that, in fact, information theory has nothing at all to do with the thinly veiled radical feminist agenda the author has apparently substituted for the curriculum of the course.
Instead of skimming genius use the index.
Sorry, these websites-as-books are still foreign to me :)
There are indeed references to Shannon. Still nothing about the other stuff, though.
There are indeed references to Shannon. Still nothing about the other stuff, though.
Thanks for posting this. I’ve been interested in the field of Information Science/Studies since discovering it recently. First via Semantic Web and Linked Data, then Taxonomy and Knowledge Management, then PKM and Zettelkasten. Just finished Cataloging the World about Paul Otlet. It’s a fantastic book.
I can recommend one of the referenced books:
James Gleick (2011). The information: A history, a theory, a flood. Vintage Books.
James Gleick (2011). The information: A history, a theory, a flood. Vintage Books.
I couldn’t get into it and stopped reading about 40 pages in. Does it get better and I should just retry?
Excellent start!
You might want to refer to the mathematical foundations of
information that is in disagreement since empirical
information is always subject to disagreement.
You might want to refer to the mathematical foundations of
information that is in disagreement since empirical
information is always subject to disagreement.
Looks extremely interesting! I originally it mean information theory but this is more about the spread of knowledge(?). I wish there was a ePub of this!
Some distinctions about Data which are not mentioned (or which escaped my skimming) are:
Data is physical (not ephemeral, like information).
Every bit of data can be mapped to a point in 3D space where it is ‘stored’ on some piece of matter (tape, disk, flash memory).
In other words, data has location in space, it can be weighted.
Encoding information as data actually means altering matter (the storage medium - piece of paper, disk tape, CD, RAM) in a way in which we can later infer data by examining the state of the physical storage medium.
And
Data is in time
It has a T0. Every single bit of data that exists ‘appeared’ in space (on storage) at a precise moment in time.
Every bit flip in the stored data has a point in time, so we can derive a 'history' time axis for data.
Data has an implicit refcount - the number of physical storage mediums where it is encoded. Data is lost when the last medium is damaged.
Just planting some seeds in case someone finds these distinctions useful.