Launch HN: Hello (YC S22) – A search engine for developers

273 pointsby wayyhace 4 años199 comments

Hi HN, we’re Michael and Justin from Hello Cognition (https://beta.sayhello.so). We're building a better search engine for software developers. Hello saves you time by synthesizing clear explanations to technical questions along with code snippets from the web, showing them right on the search page.

We’ve found that most technical searches fall into a few categories: ad-hoc how-tos, understanding an API, recalling forgotten details, research, or troubleshooting. Google is too broad and shallow of a search tool to be good at this. Even after sifting through the deluge of spammy, irrelevant sites pumped full of SEO, you still have to manually find your answer through discussion boards or documentation. Their “featured snippet” approach works for simple factoid queries but quickly falls apart if a question requires reasoning about information across multiple webpages.

Our approach is narrow and deep — to retrieve detailed information for topics relevant to developers. When you submit a query, we pull raw site data from Bing, rerank them, and extract understanding and code snippets with our proprietary large language models. We use seq-to-seq transformer models to generate a final explanation from all of this input.

For our honors theses at UT Austin, we researched prototypes of large generative language models that can answer complex questions by combining information from multiple sources. We found that GPT-3, GPT-Neo/J/X, and similar autoregressive language models that predict text from left to right are prone to “hallucinating” and generating text inconsistent with the “ground truth” document. Training a sequence-to-sequence language model (T5 derivative) on our custom dataset designed for factual generation yielded much better results with less hallucination.

After creating this prototype, we started actively developing Hello with the idea that searching should be just like talking to a smart friend. We want to build an engine that explains complex topics clearly and concisely, and lets users ask follow-up questions using the context of their previous searches.

For example, when asked “what type of semaphore can function as a mutex?”, Hello pulls in the raw text from all five search results linked on the search page to generate: “A binary semaphore can be used as a mutex. Mutexes and semaphores are two different types of synchronization mechanisms. A mutex is a lock that prevents two threads from accessing the same resource at the same time. A semaphore is used to signal that a resource has become available.” We're biased, of course, but we think that the ability to reason abstractly about information from multiple web pages is a cool thing in a search engine!

We use BERT-based models to extract and rank code snippets if relevant to the query. Our search engine currently does well at answering applicable how-to questions such as “Sort a list of tuples by the second element”, “Set a response cookie in FastAPI”, “Get value of input in React”, “How to implement Dijkstra's algorithm.” Exclusively using our own models has also freed us from dependence on OpenAI.

Hello is and will always be free for individual devs. We haven’t rolled out any paid plans yet, but we’re planning to charge teams per user/month to use on internal data scattered around in wikis, documentation, slack, and emails.

We started Hello Cognition to scratch our own itch, but now we hope to improve the state of information retrieval for the greater developer community. If you'd like to be part of our product feedback and iteration process, we'd love to have you—please contact us at [email protected].

We're looking forward to hearing your ideas, feedback, comments, and what would be helpful for you when navigating technical problems!

207 comments

gbro3nhace 4 años

Thought I'd try this on a problem I've been researching today (which I resolved) where my service worker for offline PWA usage was working for everything except audio files.

I searched the following in say hello.so.

"Service worker fails on request for audio file"

I got back a couple of results related to general service worker use but none that get close to discussing the core problem that lead to the solution.

The same query in Google returns several results that together pointed me to the solution (it was around range headers in requests for media data types).

This is just one example though. I think the problem you are trying to fix is worth the effort. I just wonder if this is where humans are still stronger than computers - gathering unstructured data to use in problem solving.

wayyhace 4 años

The description of the steps you took is super helpful feedback - thanks! Hello performs best on "how-to" questions at the moment. We're still working to improve troubleshooting type queries.

gbro3nhace 4 años

No problem. Good luck with the project.

CodeSgthace 4 años

That'll be a difficult adaptation for potential users to make. I think most of us have been conditioned to phrase our queries a certain way to achieve the best results from Google.

Then again maybe that's just me.

wayyhace 4 años

You're right in that moving away from Google is a huge behavioral change. While our model does prefer natural language queries, what I was really referring to are the different categories of developer searches. Ad-hoc how-tos are a type of search where the developer knows what to do but not exactly how e.g "how to set a cookie in fastapi." Troubleshooting searches are things like copy-pasting a compile error, or "why is X behavior not working."

yojohace 4 años

I may not be representative, but the vast majority of my Google searches are troubleshooting.

If I need to know how an API works, I go straight to the docs.

Put another way, troubleshooting is a several times per week task for me, and doing something net new is a once a month task which may not result in a search query.

You’re going to have a hard time staying on my radar focusing on the less frequent case.

Possibly the traffic patterns are different for more junior devs, but my hazy recollection of my early career is spending most of my time troubleshooting.

wayyhace 4 años

Thanks for the perspective - we're working on improving our functionality for troubleshooting.

harrisonjacksonhace 4 años

Is the assumption you are making that most developers would go to search first? rather than when they hit a blocker or error?

wayyhace 4 años

We will support troubleshooting type searches in the future. It's just not available right now.

mlejvahace 4 años

Hey Michael and Justin, congrats to the launch!

My co-founder and I were building the same product as you are some time ago [1]. We managed to scale it to around 5k WAU before we decided to pivot for various reasons.

If you think there might be any useful information and experience we could share with you, please shoot me an email - [email protected]. I'd love to help in any way I can to help you guys succeed.

[1] https://www.producthunt.com/products/devbook

moneywoeshace 4 años

Do you mind sharing why you pivoted?

danenaniahace 4 años

Congrats on the launch! I love this idea. I've thought for a long time that something like it should exist. Google results are often lacking in this realm.

I've played around just a bit and clicked some of the preset examples and like what I'm seeing so far. I bookmarked it and will try it out more as I code over the next few days.

Main initial feedback: I'd really like to see version/last-updated-at info accompanying all results. One of the biggest problems with Google for code stuff is finding outdated examples and docs. Even better would be a dropdown that lets me see results depending on the version of the language/framework/tools I'm using.

wayyhace 4 años

Thanks for trying it out and good point - we'll look into adding version info

ianbutlerhace 4 años

Hey so I built a search engine doing largely the same thing (and also interviewed with YC during the W20 time) and ultimately we pivoted away due to lack of interest from the developer teams we were pitching, often the startups we were pitching didn't have enough accumulated internal knowledge for the paid plan to be useful. For the ones who did (at the like 200+ person 5+ yrs in business mark) we still weren't seeing the problem being painful enough where companies wanted to pay to solve it.

How do you see navigating this space when this can be considered a nice to have versus a strict need?

wayyhace 4 años

Right now we're primarily focused on building a search tool that developers love. Would love to chat more about your experience - shoot us an email at [email protected]

mudshace 4 años

Awesome work guys! A couple of knee jerk reactions while playing around with this:

1. In my work (also at UT actually: Hook 'em), we've found that the hallucination problem is, in part, lessened by over-parametrizing the model. Places that have the budget to do this have noticed that the performance of ml4code transformers increases linearly for every 1e3 increase in the number of parameters (with no drop off in sight). Love to hear your thoughts on this.

2. I'm concerned that finding code snippets from a short form query is underspecifing the problem too much and may not be the best user-interaction model. Let's compare your system to something like Github Copilot. I pass a query:

> how to normalize the rows of a tensor pytorch

With GitHub Copilot, I can demonstrate intent in the development environment itself with an IO example / comment / both and interact more efficiently. If I see errors in the synthesized snippet, I can change the query in >1 second etc. Etc. This is hard with a search engine style interactive environment. For this query, I had to navigate to the website, type in the query, check the results (which were wrong for me btw. Y'all might need to check correctness of the snippets), copy back the result, maybe go to the relevant thread and parse more closely etc. A good question to keep in mind here would be to figure out how to make this process more interactive.

3. Finally, I just want to say that the website is phenomenal, even on mobile. Kudos on the frontend/backend/architecture side of things.

Also, don't let my or anyone else's comments take away from the awesome work y'all have done!!! I pulled out that example from a paper I read recently called TF-coder. They have a dataset of these examples as part of their supplement material. All the best!

izolatehace 4 años

Congrats on the launch! Looks promising, so I'll try it out for a couple of days.

One feature request at first glance: please default to the system font stack for code snippets. I see you're currently using Consolas, a Microsoft typeface, which is not pleasant to see as a mac user.

You can use this to default to the system font on every platform:

    font-family: "SF Mono", "Monaco", "Inconsolata", "Fira Mono", "Droid Sans Mono", "Source Code Pro", monospace;