HackerTrans
TopNewTrendsCommentsPastAskShowJobs

mingtianzhang

no profile record

Submissions

[untitled]

1 points·by mingtianzhang·16 days ago·0 comments

[untitled]

1 points·by mingtianzhang·3 months ago·0 comments

Show HN: Open KB: Open LLM Knowledge Base

6 points·by mingtianzhang·3 months ago·2 comments

ClawdReview – OpenReview for AI Agents

5 points·by mingtianzhang·5 months ago·0 comments

Show HN: ClawdReview – OpenReview for AI Agents

3 points·by mingtianzhang·5 months ago·0 comments

[untitled]

1 points·by mingtianzhang·8 months ago·0 comments

[untitled]

1 points·by mingtianzhang·8 months ago·0 comments

[untitled]

1 points·by mingtianzhang·8 months ago·0 comments

[untitled]

1 points·by mingtianzhang·8 months ago·0 comments

[untitled]

1 points·by mingtianzhang·8 months ago·0 comments

[untitled]

1 points·by mingtianzhang·9 months ago·0 comments

[untitled]

1 points·by mingtianzhang·9 months ago·0 comments

Do we still need OCR? An implementation of a pure vision-based agent

pageindex.ai
7 points·by mingtianzhang·9 months ago·1 comments

Do We Still Need OCR?

pageindex.ai
4 points·by mingtianzhang·9 months ago·2 comments

Reasoning-based RAG for long document question answering

pageindex.ai
1 points·by mingtianzhang·9 months ago·1 comments

PageIndex Chat – Human-Like Long Document AI Analyst

pageindex.ai
6 points·by mingtianzhang·9 months ago·1 comments

PageIndex Chat – Human-Like Long Document AI Analyst

pageindex.ai
4 points·by mingtianzhang·9 months ago·1 comments

Show HN: In-Context Index for In-Context Retrieval

github.com
5 points·by mingtianzhang·9 months ago·0 comments

DeepMind's paper reveals Google's new direction on RAG: In-Context Retreival

arxiv.org
6 points·by mingtianzhang·9 months ago·1 comments

From Claude Code to Agentic RAG

vectifyai.notion.site
3 points·by mingtianzhang·9 months ago·0 comments

comments

mingtianzhang
·3 months ago·discuss
Hi, thanks for the feedback. Our biggest differentiation is that OpenKB can handle long PDFs and images—something that isn’t trivial or easily “vibe-coded.” We will try to find some ways to compare with other projects! Thanks for the valuable feedback!
mingtianzhang
·8 months ago·discuss
VLM can already process both the document images and the query to produce an answer directly. Do we still need the intermediate OCR step?
mingtianzhang
·9 months ago·discuss
We discuss the limitations of the classic OCR pipeline and provide a pure vision-based RAG system for document analysis (https://github.com/VectifyAI/PageIndex/blob/main/cookbook/vi...)

Any feedback is welcome!
mingtianzhang
·9 months ago·discuss
We actually don't need OCR: https://pageindex.ai/blog/do-we-need-ocr
mingtianzhang
·9 months ago·discuss
This blog examines the inherent limitations of the current OCR pipeline in the context of document question-answering systems from an information-theoretic perspective and discusses why a direct, vision-based approach can be more effective. It also provides a practical implementation of a vision-based question-answering system for long documents.
mingtianzhang
·9 months ago·discuss
PageIndex Chat is the world's first human-like long-document AI analyst. You can upload entire books, research papers, or hundred-page reports and chat with them without context limits, all in the browser. Unlike traditional RAG or "chat-with-your-doc" tools that rely on vector similarity search, PageIndex builds a hierarchical tree index of your document (like a table of contents), and then reasons over this index to retrieve and interpret relevant sections. It doesn’t search by keywords or embeddings — it reads, understands, and reasons through the document like a human expert.

What makes it different:

- Reasoning-based retrieval: Understands structure, logic, and meaning, not just semantic similarity.

- Page-level references: Every answer includes precise citations for easy verification.

- Cross-section reasoning: Connects information across sections and appendices to find true answers.

- Human-in-the-loop: You can guide, refine, and verify its reasoning.

- Multi-document comparison: Analyze and contrast multiple reports at once.
mingtianzhang
·9 months ago·discuss
PageIndex Chat is the world's first human-like long-document AI analyst. You can upload entire books, research papers, or hundred-page reports and chat with them without context limits, all in the browser.

Unlike traditional RAG or "chat-with-your-doc" tools that rely on vector similarity search, PageIndex builds a hierarchical tree index of your document (like a table of contents), and then reasons over this index to retrieve and interpret relevant sections. It doesn’t search by keywords or embeddings — it reads, understands, and reasons through the document like a human expert.

What makes it different:

- Reasoning-based retrieval – Understands structure, logic, and meaning, not just semantic similarity. - Page-level references – Every answer includes precise citations for easy verification. - Cross-section reasoning – Connects information across sections and appendices to find true answers. - Human-in-the-loop – You can guide, refine, and verify its reasoning. - Multi-document comparison – Analyze and contrast multiple reports at once.
mingtianzhang
·9 months ago·discuss
PageIndex Chat is the world's first human-like long-document AI analyst. You can pload entire books, research papers, or hundred-page reports and chat with them without context limits, all in the browser.

Unlike traditional RAG or "chat-with-your-doc" tools that rely on vector similarity search, PageIndex builds a hierarchical tree index of your document (like a table of contents), and then reasons over this index to retrieve and interpret relevant sections. It doesn’t search by keywords or embeddings — it reads, understands, and reasons through the document like a human expert.

What makes it different:

- Reasoning-based retrieval – Understands structure, logic, and meaning, not just semantic similarity. - Page-level references – Every answer includes precise citations for easy verification. - Cross-section reasoning – Connects information across sections and appendices to find true answers. - Human-in-the-loop – You can guide, refine, and verify its reasoning. - Multi-document comparison – Analyze and contrast multiple reports at once.
mingtianzhang
·9 months ago·discuss
Instead of relying on vector databases, DeepMind proposes:

1. The LLM itself selects the most relevant documents — no vector database needed.

2. The selected documents are then placed directly into the context for generation.

This kind of in-context retrieval approach greatly improves retrieval accuracy compared to traditional vector-based retrieval methods.
mingtianzhang
·9 months ago·discuss
Hi, thanks for your inspiring questions.

1. What happens when the TOC is too long? -- This is why we choose the tree structure. If the ToC is too long, it will do a hierarchy search, which means search over the father level nodes first and then select one node, and then search its child nodes.

2. How does the index handle near misses, and how do you disambiguate between close titles? For each node, we generate a description or summary to give more information rather than just titles.

3. For documents that are not in a hierarchy, it will just become a list structure, which you can still look through.

We also write down how it can combine with a reasoning process and give some comparisons to Vector DB, see https://vectifyai.notion.site/PageIndex-for-Reasoning-Based-....

We found our MCP service works well in general financial/legal/textbook/research paper cases, see https://pageindex.ai/mcp for some examples.

We do agree in some cases, like recommendation systems, you need semantic similarity and Vector DB, so I wouldn't recommend this approach. Keen to learn more cases that we haven't thought through!
mingtianzhang
·9 months ago·discuss
The current OCR approach typically relies on a Vision-Language Model (VLM) to convert a table into a JSON structure. However, a table inherently has a 2D spatial structure, while Large Language Models (LLMs) are optimized for processing 1D sequential text. This creates a fundamental mismatch between the data representation and the model’s input format.

Most existing pipelines address this by preprocessing the table into a linearized 1D string before passing it to the LLM — a question-agnostic step that may lose structural information.

Instead, one could retain the original table form and, when a question is asked, feed both the question and the original table (as an image) directly into the VLM. This approach allows the model to reason over the data in its native 2D domain, providing a more natural and potentially more accurate solution.
mingtianzhang
·9 months ago·discuss
Thanks, any feedback is welcome!
mingtianzhang
·9 months ago·discuss
Thanks for the reminder, I have edited the comment.
mingtianzhang
·9 months ago·discuss
Edited version:

We try to solve a similar problem to put long documents in context. We built an MCP for Claude to allow you to put long PDFs in your context window that go beyond the context limits: https://pageindex.ai/mcp.
mingtianzhang
·9 months ago·discuss
Thanks for the great question! We actually use a reasoning-based, vectorless approach. In short, it follows this process:

  1. Generate a table of contents (ToC) for the document.

  2. Read the ToC to select a relevant section.

  3. Extract relevant information from the selected section.

  4. If enough information has been gathered, provide the answer; otherwise, return to step 2.
We believe this approach closely mimics how a human would navigate and read long PDFs.
mingtianzhang
·9 months ago·discuss
thanks!
mingtianzhang
·9 months ago·discuss
Great post, I am wondering if this system includes financial report analysis.
mingtianzhang
·9 months ago·discuss
Many RAG systems handle in-document references (like “see appendix for details”) by building graphs or other preprocessing structures. The idea is to make sure cross-references are resolved before retrieval.

But with reasoning-based RAG, you don’t need that extra layer. The LLM itself can read the document, notice the reference, and then “jump” to the appendix (or wherever the reference points) to extract the answer. In other words, instead of pre-building structure, the model reasons its way through the content.

An example of reasoning-based RAG with PageIndex MCP is attached. In this example, the query asks for the total value. The main text only provides the increased value and refers to the appendix table for the total value. The LLM then looks up the appendix to find the total value and explains its reasoning process.

This raises an interesting question: how much preprocessing do we actually need for reasoning-augmented RAG, and when is it better to just let the model figure it out?
mingtianzhang
·10 months ago·discuss
This paper introduces a method that allows you to fine-tune black box embedding models (e.g. those vectors obtained with ChatGPT API). It shows there is around 10% improvement in various domains. Any feedbacks are welcome.
mingtianzhang
·10 months ago·discuss
Check out this MCP: https://pageindex.ai/mcp, which allows you to chat with any long PDFs (hundreds of pages) beyond the context limit of Claude or ChatGPT.