HackerTrans
TopNewTrendsCommentsPastAskShowJobs

0101111101

no profile record

Submissions

Semantic search engine for ArXiv, biorxiv and medrxiv

arxivxplorer.com
149 points·by 0101111101·السنة الماضية·31 comments

comments

0101111101
·السنة الماضية·discuss
Amazing!
0101111101
·السنة الماضية·discuss
True, but similarly if your embeddings are any good they'll capture interesting associations between authors, topics and your search query. If you find any interesting author overlap results I'd be very interested!
0101111101
·السنة الماضية·discuss
One chunk embedded together
0101111101
·السنة الماضية·discuss
Exactly! A near property of the matryoshka embeddings is that you can compute a low dimension embedding similarity really fast and then refine afterwards.
0101111101
·السنة الماضية·discuss
That's a nice idea! Might take a look this weekend!
0101111101
·السنة الماضية·discuss
Thanks!
0101111101
·السنة الماضية·discuss
Looks cool! You can input either a search query or a paper URL on arxiv xplorer. You can even combine paper URLs to search for combinations of ideas by putting + or - before the URL, like `+ 2501.12948 + 1712.01815`
0101111101
·السنة الماضية·discuss
Sadly I couldn't find a public API for chemrxiv, but would be happy to be proven wrong!
0101111101
·السنة الماضية·discuss
Do they have a public API/dataset?
0101111101
·السنة الماضية·discuss
Sure! I first used openai embeddings on all the paper titles, abstracts and authors. When a user submits a search query, I embed the query, find the closest matching papers and return those results. Nothing too fancy involved!

I'm also maintaining a dataset of all the embeddings on kaggle if you want to use them yourself: https://www.kaggle.com/datasets/tomtum/openai-arxiv-embeddin...