Are you me? Wish that I had known the insertion order trick, though it isn't straightforward to implement with the stack I was using at a previous gig. (Tabula + Naive parsing + Pandas Data Munging). I can expand on a few issues challenges I've run into when parsing PDFs:
# Parser drift and maintenance hell
Let's say that you receive 100 invoices a month from a company over the course of 3 months. You look over a handful of examples, pick features that appear to be invariant, and determine your parsing approach. You build your parser. You're associating charges from tables with the sections their declared in, and possibly making some kind of classification to make sure everything is adding up right. It works for the example or two pdfs you were building against. It goes live.
You get the a call or bug report: it's not working. You try the new pdf they send you. It looks similar, but won't parse because it is--in fact--subtly different. It has a slightly different formatting of the phone-number on the cover page, but identical everywhere else. You change things to account for that. You retest your examples, they break. Ok, two different formats same month, same supplier. You fix it. Chekhov's Gun has been planted.
A month passes, it breaks. You inspect the offending pdf. Someone racked up enough charges they no longer fit on a page. You alter the parser to check the next page. Sometimes their name appears again, sometimes not, sometimes their next page is 300 pages away. It works again.
A few more months later, a sense of deja-vu starts to set it. Didn't I fix this already? You start tracking three pdfs across 3 months:
pdf 1 : a -> b -> c (Starts with format a, change to be same as pdf 2, then changes again)
pdf 2 : b -> b -> c (Starts with one format, stays the same, changes the same way as pdf 1)
pdf 3 : b -> a -> b (Starts same as pdf 2, changes to same as pdf 1 first month, same as pdf 3)
What's the common factor between these version changes? The return address is determining the version.
PDFs are slightly different from office to office, with templates drifting slightly each month in diverging directions. You have to start reevaluating parsing choices and splitting up parsers. It's difficult to account for incurring linear maintenance cost for each new supplier and amortize that over a sizeable period of time. My arch nemesis is an intern who got put to work fixing the invoices at one office of one foreign supplier.
# PDFs that aren't standards compliant
In this case, most pdf processing libraries will bail out. Pdf viewers on the other hand will silently ignore some corrupted or malformed data. I remember seeing one that would be consistently off by a single bit. Something like `\setfont !2` needed to have '!' swapped out for another syntactically valid character that would leave byte offsets for the pdf unchanged.
TLDR: If you can push back, push back. Take your data in any format other than PDF if there is any way that is possible.
So not an expert in this area, would love if someone corrects me. My understanding is generally FOL is infeasible. Propositional logic even can be computationaly difficult [1]. My understanding is that most of the semantic web stuff is done using a description logic of some flavor. These will be named based on the properties of the logic. The important thing is that they are generally decidable, and you can use something like MALET or some other solver to infer things from your database or ontology.(You give up some expressivity for decidability) Not sure how much stuff is going on with that these days. Played with a petrology ontology in protegé some back in college, but haven't followed the space. I remember OWL being important, but can't remember why at the moment.
[1] For example if you try to figure out if a formula is satisfiable. You can for sure do this using truth tables. The catch is that you're looking at 2^n complexity where n is the number of propositions in your formula.
As someone who is currently picking up Clojure, that chapter just sold me on the book. I really liked the sign/referent/sense distinction and discussion. I knew it from philosophy/semiotics, but every other place I come across it instantly goes down another rabbit trail. I'm sure the balance you strike will vary language to language, but it really was useful seeing it discussed in terms of naming in software.
On a side note. I haven't been able to find full episodes anywhere besides Delta's in flight entertainment. Confusingly not listed under Adult Swim, but only shown in the All Titles section.
Haven't explored internet options up there. But the UP of Michigan, it's beautiful country and it's been far too long since I've had a pasty. It doesn't have the scale of Washington or the weather of California, but it is so much beautiful unspoiled wilderness. It's really hard to describe. Honestly the best show to capture it is Joe Pera talks with you.
On a shin splints related side note. Something not sure intuitive for non runners. If possible take the 1 minute-ish analysis at the shop to find out how your feet roll while running. If they roll to much inside (pronate) your appropriate shoe has more support in that area. If you roll too much outside (supenate), they add it on the outside. If you dont roll too much, shoes for either or these can cause shin splints or stress fractures in the medium to long term (still preferable to inactivity). Most of the time this is the key thing to consider when buying running shoes.
If you're interested at all. Here's a great interview from the 60s or 70s he did. Wish something of this level was on public access these days. Not sure how accessible it is, but really makes me wish a PhD in philosophy made sense/afford to do.
Taking a wild guess, but I would figure it's about distinguishing statements about truth values and statements about sets. If you think of it as a function, takes in two booleans and returns a booleen. Likewise ⊂ takes a thing and a set and returns a booleen. I've never really been a fan of ⊃, and always personally preferred an arrow.
Not something I would put into production, but had a fun application of this on a side project. I had the `id` of a function (but not the function object itself) and needed to recover the function. In CPython, `id` of a function corresponds to it's memory address (not sure if that can be overridden). By casting the id to a PyObject, I was able to recover the original.
# Parser drift and maintenance hell
Let's say that you receive 100 invoices a month from a company over the course of 3 months. You look over a handful of examples, pick features that appear to be invariant, and determine your parsing approach. You build your parser. You're associating charges from tables with the sections their declared in, and possibly making some kind of classification to make sure everything is adding up right. It works for the example or two pdfs you were building against. It goes live.
You get the a call or bug report: it's not working. You try the new pdf they send you. It looks similar, but won't parse because it is--in fact--subtly different. It has a slightly different formatting of the phone-number on the cover page, but identical everywhere else. You change things to account for that. You retest your examples, they break. Ok, two different formats same month, same supplier. You fix it. Chekhov's Gun has been planted.
A month passes, it breaks. You inspect the offending pdf. Someone racked up enough charges they no longer fit on a page. You alter the parser to check the next page. Sometimes their name appears again, sometimes not, sometimes their next page is 300 pages away. It works again.
A few more months later, a sense of deja-vu starts to set it. Didn't I fix this already? You start tracking three pdfs across 3 months:
pdf 1 : a -> b -> c (Starts with format a, change to be same as pdf 2, then changes again)
pdf 2 : b -> b -> c (Starts with one format, stays the same, changes the same way as pdf 1)
pdf 3 : b -> a -> b (Starts same as pdf 2, changes to same as pdf 1 first month, same as pdf 3)
What's the common factor between these version changes? The return address is determining the version.
PDFs are slightly different from office to office, with templates drifting slightly each month in diverging directions. You have to start reevaluating parsing choices and splitting up parsers. It's difficult to account for incurring linear maintenance cost for each new supplier and amortize that over a sizeable period of time. My arch nemesis is an intern who got put to work fixing the invoices at one office of one foreign supplier.
# PDFs that aren't standards compliant
In this case, most pdf processing libraries will bail out. Pdf viewers on the other hand will silently ignore some corrupted or malformed data. I remember seeing one that would be consistently off by a single bit. Something like `\setfont !2` needed to have '!' swapped out for another syntactically valid character that would leave byte offsets for the pdf unchanged.
TLDR: If you can push back, push back. Take your data in any format other than PDF if there is any way that is possible.