One important aspect of the success of AlphaGo and its successor is the game environment is closed domain, and has a stable reward function. With this we can guide the agent to do MCTS search and planning for the best move in every state.
However, such reward system is not available for LLM in an open domain setting.
The company I work for try to add AI/LLM on everything, instead of trying to improve/fix the underlying problem, they now just add the magic AI and everything is “perfect” now.
As an ML engineer and AI developer, I don’t see the real value at all, not to mention the added cost of using LLM
I looked at the supposed “research” article, there’s nothing to read except few charts to show off the “improvements” over current models. No discussion of the training method or dataset whatsoever.
If I remember correctly, the last decent research paper the company published was probably the InstructGPT paper.
Anyone know how to use GraphRAG to build the knowledge graph on a large collection of private documents, where some might have complex structure (tables, links to other docs), and the content or terms in one document could be related to other documents as well?
In case anyone is interested, we have a very simple notebook which try to trick GPT-4o to reveal the structure of the prompt/response related to function calling, and it's located at:
I'm wondering if the tensor parallel settings have any impact on the performance. My naive guess is yes but not sure.
According to the article:
"""
AMD Configuration: Tensor parallelism set to 1 (tp=1), since we can fit the entire model Mixtral 8x7B in a single MI300X’s 192GB of VRAM.
NVIDIA Configuration: Tensor parallelism set to 2 (tp=2), which is required to fit Mixtral 8x7B in two H100’s 80GB VRAM.
"""
I think the main problem is the way we turn the raw mathematics symbols or equations into tokens, and these suboptimal tokenization may decreases the performance