HackerTrans
TopNewTrendsCommentsPastAskShowJobs

simonbutt

no profile record

Submissions

Evaluating Multimodal LLMs Using the Google IO 2024 Puzzle

hiresynth.ai
3 points·by simonbutt·2 years ago·2 comments

Am I using it wrong or has the ChatGPT Bing plugin degraded?

chat.openai.com
1 points·by simonbutt·3 years ago·1 comments

Google PaLM Pricing matches gpt3.5 for chat

cloud.google.com
1 points·by simonbutt·3 years ago·4 comments

comments

simonbutt
·2 years ago·discuss
A couple weeks ago I rebuilt the https://hiresynth.ai landing page and docs using vitepress and was super impressed with the speed of implementation and how pretty it was by default.

Would recommend for any text heavy static site.
simonbutt
·2 years ago·discuss
While opensource models allow you to explore wider topics, I wonder how well mixtral holds context on longer exchanges?
simonbutt
·2 years ago·discuss
Wrote a post on multimodal LLMs (GPT-4V, Gemini Ultra, Claude3 Opus) attempting the puzzles at https://hiresynth.ai/blog/googleio_puzzle_multimodal_eval.ht.... Seems very easy for humans, far harder for LLMs currently
simonbutt
·2 years ago·discuss
GPT-4V, Claude 3 Opus and Gemini Ultra go head to head in solving GoogleIO Puzzle 2024
simonbutt
·3 years ago·discuss
This feels like the start of a lawsuit. If Tesla is state of the art and requires 5x less funding per charger than competition; what is the reasoning behind picking the competitors?
simonbutt
·3 years ago·discuss
This feels as if they've tacked the bing "chat with gpt4" to chatgpt while the previous version was able to browse multiple places and provide (usually) decent responses.

Have you found any custom instructions to improve the way it browses and responds?
simonbutt
·3 years ago·discuss
Once in a while HN provides absolute gems. Would never have known about this anti procrastination feature otherwise!
simonbutt
·3 years ago·discuss
Logic for azure chatgpt's "infinite context" summarisation is in https://github.com/microsoft/azurechatgpt/blob/main/src/feat...

*Edit Azure chatgpt, would be amazed/disappointed if chatgpt used langchain.
simonbutt
·3 years ago·discuss
Google released PaLM pricing per character at $0.0005/1k (for chat, instruct is 0.001). With OpenAI tokens at approx 4 chars per token, this matches $0.002/1k tokens for 3.5. Let's hope OpenAI retaliate, cut prices to maintain market share and start a happy race to the bottom/affordable genAI for users.
simonbutt
·3 years ago·discuss
That first photo has to be Midjourney. Have we hit the turning point where AI art is suddenly embedded in everything?
simonbutt
·3 years ago·discuss
You won't catch everything but I've found if you specify that it's okay to respond that it doesn't know the answer and essentially accuracy on a smaller subset of answers > responding to all, it often will say it doesn't know rather than making something up.

Another option could be - assuming this is an LLMChain that adds the most similar (n) embedding(s) to the prompt before passing to LLM - basic entity recognition to find the asset name from the query initially and dynamic prompt that if the most similar results don't contain the asset name, don't provide examples and respond saying you don't know.
simonbutt
·3 years ago·discuss
Q: "Tell me about Stonks GPT?"

A: "Stonks GPT is a trading name for Stocks & Breads, a UK-based provider of bread and breakfast products. The company was founded in 1894 and is headquartered in London. It is a leading provider of bread products in the UK, with a portfolio of over 50 brands. Stonks GPT is a subsidiary of Unilever PLC, which owns the brands including Pepperidge Farm, Lipton, and Marmite. The company's core business is the production of bread and breakfast products, with a focus on quality and freshness."

There are a lot of hallucinations here. Recommend really specifying in the prompt that if it doesn't know or is unsure, say that and don't make up facts. Understand it's alpha rn and this is all free UAT but for trading related products, accuracy and trust in the product is vital
simonbutt
·3 years ago·discuss
Product market fit