HackerTrans
TopNewTrendsCommentsPastAskShowJobs

linuxrebe1

123 karmajoined 6 năm trước

Submissions

Turning a pile of documents into a searchable useable knowledge base

github.com
192 points·by linuxrebe1·Hôm kia·56 comments

Release of MonVisoer 0.1.0

github.com
1 points·by linuxrebe1·tháng trước·0 comments

comments

linuxrebe1
·Hôm qua·discuss
I just had a wild thought. Combine Hister with my RepoSearch app. Point it at a companies Internal github/gitlab and have a searchable knowledge base of your git repos.
linuxrebe1
·Hôm kia·discuss
hmmmm :)
linuxrebe1
·Hôm kia·discuss
I've run it on a VM with 4G ram and no GPU. It runs, But I really recommend 8G ram at least. If you have a GPU (like I do) with 4G vRAM that is ideal. Will get this in the readme. Thanks for the suggestion. I really tried to build this to minimal spec.
linuxrebe1
·Hôm kia·discuss
I fixed this in version 0.9.1 (just released) thanks for the bug (seriously)
linuxrebe1
·Hôm kia·discuss
v0.9.1 is in the repo and packages have been built. It now does all of the ODT formats.
linuxrebe1
·Hôm kia·discuss
By restricted for personal use I mean it's not networked. It's running on your system only. It's not a networked commercial product able to do SSO etc. It's not an enterprise level product.
linuxrebe1
·Hôm kia·discuss
I went with the python libraries (pydoc and pyxls for example), because it's portable and doesn't require a big download to a users system if they don't already have it installed.
linuxrebe1
·Hôm kia·discuss
Will have version 0.9.1 out later today to support ODF formats.
linuxrebe1
·Hôm kia·discuss
Consider it yes, However having experience in this ... not really. For now there is a file called Decisions.md in the repo that is my "notes to self" if you will about where and what I need to do.
linuxrebe1
·Hôm kia·discuss
I use rclone to "mount" them locally. Then it becomes searchable.
linuxrebe1
·Hôm kia·discuss
I was building for mac on a VM install of OSX. Will take this as a bug, Thanks.
linuxrebe1
·Hôm kia·discuss
Noted the bug.
linuxrebe1
·Hôm kia·discuss
thankyou
linuxrebe1
·Hôm kia·discuss
I'm actually thinking of this for a commercial product feature. However, if you use a tool like Rclone on Windows, Linux or Mac. Mount the s3 bucket and you can then run DocuBrowse as if the s3 bucket were local.
linuxrebe1
·Hôm kia·discuss
Yes, not supporting it is an oversight I will correct.
linuxrebe1
·Hôm kia·discuss
- Filling a need I personally have. - Learning how to leverage AI for real world use not just to fill up a data center. - Personal knowledge -developing skills

Pretty much in that order
linuxrebe1
·Hôm kia·discuss
I creating DocuRepo as well. though not as fleshed out.
linuxrebe1
·Hôm kia·discuss
Let me look into this
linuxrebe1
·Hôm kia·discuss
Interesting observation. However it seems to me it uncovers an age old issue. "Works on my laptop" The real blindness I see in the process is failing to use the tools at hand to properly model production, document production etc. The change was made prior to the person making the change either bothering too, or being able to, understand what the constraints really are.
linuxrebe1
·Hôm kia·discuss
I had an issue. A documents folder with over 12k objects in it. A hodgepodge of folders and sub-folders. That over time had created a mess that no amount of file movement was ever going to make it usable. I wanted: 1) To keep my data local 2) be able to filter out PII and other data 3) Be able to find and delete duplicates 4) Get short synopsis of what a document is 5) Semantic and keyword search 6) All of this kept local to me requiring no internet access and no tokens spent to train someone elses AI.

The result I call DocuBrowser and in it's current form is FOSS (GPL-3) licensed for your personal use. The UI is in your browser. The AI models used are held local and are tiny, Available for Linux(RPM,Deb, and tgz) Windows and Mac. Let me know what you think and thanks for taking the time to try it out.