HackerTrans
TopNewTrendsCommentsPastAskShowJobs

pani5ue

no profile record

comments

pani5ue
·vor 3 Jahren·discuss
Use makefiles for as long as you can..

If your DAGs get too complicated for makefiles, it's time to rethink and restructure your DAGs
pani5ue
·vor 4 Jahren·discuss
If you haven't yet, you should look at the full text search capabilities of SQLite and postgres. Could simplify your search part of the workflow a bit
pani5ue
·vor 4 Jahren·discuss
I'm doing this for a customer right now!

Using Textract from Amazon to do the OCR. I dump the raw json in a SQLite database.

Information extraction is trickier. I extract useful things like lines and pages, locations of lines, set up various columns and now can search the SQLite db either with python code or SQL queries.

There's of course cost for running textract OCR on AWS. There are open source solutions like Tesseract