Show HN: Continue – Open-source coding autopilot(github.com)
github.com
Show HN: Continue – Open-source coding autopilot
https://github.com/continuedev/continue
103 comments
In my experience working with GPT4, if I give enough context on types, other functions definitions and the libraries I use, I get very accurate results. But it is a tedious task to copy paste from multiple places (type definitions, function definitions, packages, etc.).
In addition to the selected lines, does Continue support getting related definitions from the language server and inject them in the prompt? That would be huge.
In addition to the selected lines, does Continue support getting related definitions from the language server and inject them in the prompt? That would be huge.
This is very near on the roadmap, and we agree it will be awesome!
As of now, if there are collections of definitions that you frequently reference, you could save them in the system message, or write custom slash commands that let you prefix your prompt with these definitions.
As of now, if there are collections of definitions that you frequently reference, you could save them in the system message, or write custom slash commands that let you prefix your prompt with these definitions.
> if I give enough context on types, other functions definitions and the libraries I use, I get very accurate results.
It's almost like .. coding it yourself!
It's almost like .. coding it yourself!
After I copy every piece of code that is relevant, ask it to do something, then correct it's errors using human knowledge, then ask senior devs if it makes sense to revise my prompts and repeat, fix bugs from reviews from other humans, it's like this magical thing I get the mostly right answer!
For smaller pieces, you are right. But as a BE engineer, I was prototyping a concept and I didn't know much about Typescript+React+React-Router-Dom+React-Hooks-Form, etc etc. So I listed the libraries, a few lines of boilerplate and API definitions that RTK toolkit generated for me. Then asked GPT to generate the full page. It was much faster than I could "code it myself".
And that's why it is a "pain point". These all can be done automatically.
And that's why it is a "pain point". These all can be done automatically.
If you ever need to extend that full page to do something else or something additional you might find the time not so different after all. Because you still don't know much about Typescript+React+React-Router-Dom+React-Hooks-Form and now you have to read that stuff and try to make sense of it, even if just for passing it back into the bot.
Actually writing code is a minority of my time spent developing software, I don't need to trade that for need to spend time to know what and more importantly why something is where it is.
I guess it does work if you just want it to generate a webpage and you never want to then add any functionality. Or a glorified boilerplate generator for stuff not on the prod path.
Actually writing code is a minority of my time spent developing software, I don't need to trade that for need to spend time to know what and more importantly why something is where it is.
I guess it does work if you just want it to generate a webpage and you never want to then add any functionality. Or a glorified boilerplate generator for stuff not on the prod path.
> I don't need to trade that for need to spend time to know what and more importantly why something is where it is.
I had two options: 1. Learn everything first and then start prototyping. 2. Start prototyping and learn along the way. I chose the latter. But instead of searching stackoverflow and putting the pieces together, I used GPT and learned from the generated outputs in the context of my own problem.
It may not save a lot of time in larger projects because as you said, one should ultimately learn the "what and why", but it definitely provides a more pleasing experience. And I guess the time-saving part becomes more relevant with better tooling, like the suggestion I had.
I had two options: 1. Learn everything first and then start prototyping. 2. Start prototyping and learn along the way. I chose the latter. But instead of searching stackoverflow and putting the pieces together, I used GPT and learned from the generated outputs in the context of my own problem.
It may not save a lot of time in larger projects because as you said, one should ultimately learn the "what and why", but it definitely provides a more pleasing experience. And I guess the time-saving part becomes more relevant with better tooling, like the suggestion I had.
Fair - I guess I'm just used to the old ways and hated things showing up without me specifically telling them to show up there first. But then, I still learn while prototyping, ChatGPT might be easier since it does give you a template to go off of.
But then, I don't think this really go beyond entry level prototyping, any complexity and chatGPT doesn't have enough tokens.
But then, I don't think this really go beyond entry level prototyping, any complexity and chatGPT doesn't have enough tokens.
And driving to the store is basically just like walking there yourself!
I'd say it's more like driving vs assisted driving with shouting inputs (e.g. TURN LEFT! WATCH FOR THE PEDESTRIAN! CRUISE AT 70 MPH! SLOW DOWN THERE'S ANOTHER CAR IN FRONT! USE THE SECOND RAMP TO EXIT!). The only benefit is that you don't have to hold the wheel any more, the negative being that it still only works like 90% of the time, and you end up with laryngitis.
I have been experimenting a lot lately, and I would much rather copy paste high quality output(via providing context) than playing guessing games.
It's not like you have to be coding all the time.
Things will of course change as tools evolve.
It's not like you have to be coding all the time.
Things will of course change as tools evolve.
Couldn't agree more—it's worth the extra effort to know exactly what enters the prompt. But control isn't mutually exclusive with removing the need to copy/paste. Continue lets you highlight code with perfect precision, and this is much lower effort.
I'd say it is a UX concern. It could show you the things it referenced in the prompt. So you'd only hit enter or customize it if needed :shrug:.
I also found this tedious and made a tiny vscode extension to make it less tedious
https://marketplace.visualstudio.com/items?itemName=TomJenni...
https://marketplace.visualstudio.com/items?itemName=TomJenni...
I'm not sure about your plans but you should consider implementing this based on the language server protocol model. Having it run as a local service that editors and other applications can interact with. LSP was a huge success and seems like a good model to follow.
We have something similar in our plans. While the LSP doesn't directly offer all of the APIs we need, we want to be an "extended LSP", and integrate an LSP as part of the Continue server.
An alternative to in LSP is tree sitter https://github.com/tree-sitter?type=source
This looks great.
A few observations:
- As soon as I sent it the first prompt, it tried to connect to windows.net. Why is that? Is this call safe to block?
- When opening a new window, macOS asks me to give access to VS Code for a lot of folders: Desktop, Downloads, Documents... The only new thing is the Continue plugin. Why would it ask that?
- It looks like it needs to redownload "Python packages" every time I open a new window. I wonder if this could be optimised for a quicker startup.
- It tries to connect to meilisearch.com . What information is being sent over? What is this used for?
A few observations:
- As soon as I sent it the first prompt, it tried to connect to windows.net. Why is that? Is this call safe to block?
- When opening a new window, macOS asks me to give access to VS Code for a lot of folders: Desktop, Downloads, Documents... The only new thing is the Continue plugin. Why would it ask that?
- It looks like it needs to redownload "Python packages" every time I open a new window. I wonder if this could be optimised for a quicker startup.
- It tries to connect to meilisearch.com . What information is being sent over? What is this used for?
I'm not aware of any reason we would be connecting to windows.net. I would be surprised if VS Code did not already have access to Desktop / Documents / etc. but if this is the case, then Continue reads the currently open files as context for the LLM. It would be very useful to hear more about the details of these two cases so we can try to better reproduce and solve the problem. Would you be interested in opening a new issue? (https://github.com/continuedev/continue/issues/new)
Continue will cache Python packages in ~/.continue/server/env after the first download, but there might be something else causing slower startup. Will look into this as well!
Meilisearch is an open-source search library. We connect to meilisearch.com only in order to download the software, which then runs completely locally to power search in the dropdown as you type. The line of code where we do this is here: https://github.com/continuedev/continue/blob/ce76e391775034c...
Continue will cache Python packages in ~/.continue/server/env after the first download, but there might be something else causing slower startup. Will look into this as well!
Meilisearch is an open-source search library. We connect to meilisearch.com only in order to download the software, which then runs completely locally to power search in the dropdown as you type. The line of code where we do this is here: https://github.com/continuedev/continue/blob/ce76e391775034c...
subprocess.run(f"curl -L https://install.meilisearch.com | sh", shell=True)
This is fair feedback. We will move to packaging it with our extension to avoid running arbitrary code.
> I'm not aware of any reason we would be connecting to windows.net.
If that connection is denied, the following error occurs:
Why would I give VS Code access to these directories? All my code resides elsewhere, it has no reason to access those, and it hasn't in the last couple years of this particular installation. Not only it hadn't accessed them, but it hadn't even asked me for access until I installed your extension.
If that connection is denied, the following error occurs:
HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x118377110>: Failed to establish a new connection: [Errno 9] Bad file descriptor'))
Why would I give VS Code access to these directories? All my code resides elsewhere, it has no reason to access those, and it hasn't in the last couple years of this particular installation. Not only it hadn't accessed them, but it hadn't even asked me for access until I installed your extension.
"Continue works with any LLM, including local models using ggml"
Which of the local models have you seen the best results with?
UPDATE: Found a note here https://github.com/continuedev/continue/blob/ce76e391775034c... that says "While these models don't yet perform as well, they are free, entirely private, and run offline." and points to the documentation here for how to run one, but doesn't really recommend a specific model: https://github.com/continuedev/ggml-server-example
Which of the local models have you seen the best results with?
UPDATE: Found a note here https://github.com/continuedev/continue/blob/ce76e391775034c... that says "While these models don't yet perform as well, they are free, entirely private, and run offline." and points to the documentation here for how to run one, but doesn't really recommend a specific model: https://github.com/continuedev/ggml-server-example
I've had the best luck with WizardLM-7B. It runs at ~4 tokens/sec on my M1 and has decent outputs for simple situations. While it can't handle complex refactors like gpt4, I believe there's room to improve both via prompting and codebase-specific fine-tuning.
And good point! I've just added this as the recommendation in the README.
And good point! I've just added this as the recommendation in the README.
I tried to change config.py to use ggml setting, but I did not see any request going to local llama-cpp.server. It keeps on asking for openai key. The local llama-cpp.server is up and I was able to make swagger calls for completion.
Assuming that may be I missed something, I deleted ~/.continue and trying to start from scratch (vs code message: Starting Continue Server ...). I do see FastAPI is up on http://localhost:65432/ still vscode dialog shows same message. Where I can see the logs what its trying to do? What am I missing?
Anyone else able to use local ggml?
Assuming that may be I missed something, I deleted ~/.continue and trying to start from scratch (vs code message: Starting Continue Server ...). I do see FastAPI is up on http://localhost:65432/ still vscode dialog shows same message. Where I can see the logs what its trying to do? What am I missing?
Anyone else able to use local ggml?
Hey there. I realized that the documentation is less clear than it should be on this, so I've made an update: https://continue.dev/docs/customization#local-models-with-gg...
The additional thing you need to do is set "default_model" to "ggml" in ~/.continue/config.py.
The additional thing you need to do is set "default_model" to "ggml" in ~/.continue/config.py.
Why is nobody in this space making good gains on UX?
I have tried almost every co-pilot solution, including GitHub Co-Pilot with Chat and Labs, Cody, and a few random extensions.
And for some reason, I still default to using ChatGPT, even with the massive drawback of copying and pasting.
I haven’t seen any breakthroughs with these developer experiences. Every still feels suboptimal.
I have tried almost every co-pilot solution, including GitHub Co-Pilot with Chat and Labs, Cody, and a few random extensions.
And for some reason, I still default to using ChatGPT, even with the massive drawback of copying and pasting.
I haven’t seen any breakthroughs with these developer experiences. Every still feels suboptimal.
Check out the Rubberduck Extension [1] for VS Code. It's not super well publicized but I've actually found it basically hits a perfect middle ground between copilot and copy/pasting into ChatGPT. You can give it a prompt to edit code and it will stream the answer from GPT-4 into VS Code and show you a live diff with your current code. It's actually pretty well done, I use it dozens of times a day for even super minor things (I'm lazy).
Actually, it looks like this project is fairly similar in some ways. A little more full featured.
[1] https://marketplace.visualstudio.com/items?itemName=Rubberdu...
Actually, it looks like this project is fairly similar in some ways. A little more full featured.
[1] https://marketplace.visualstudio.com/items?itemName=Rubberdu...
Author here - thanks for the mention! Re publicizing: I can never get any attention for my projects, no matter where I post them. Sites like this one (where I tried posting the project multiple times) feel very gamed to me at this point.
Sorry! I didn't mean to imply you had done a bad job publicizing it, I know marketing this kind of stuff is a whole separate job. I'm honestly just surprised the project hasn't gotten more attention organically since it's awesome. Your extension is at least as good as the OP here, if not better now that I've had time to try them both out. I tried probably a dozen half baked VS Code GPT integrations and yours is by far the most polished and usable. Way, way too good to only have 17k downloads on the marketplace. I've been recommending it to my coworkers and friends non-stop.
My one suggestion: I hate to say it but the name RubberDuck and the icon you've used kind of undersell the seriousness of the project. I think the whole "hacker" branding thing is kind of lame but it could be worth some tweaks to make it seem more "cool" :/. Also your website seems to be down!
My one suggestion: I hate to say it but the name RubberDuck and the icon you've used kind of undersell the seriousness of the project. I think the whole "hacker" branding thing is kind of lame but it could be worth some tweaks to make it seem more "cool" :/. Also your website seems to be down!
Oh yeah, never thought that - was just frustrated at the lack of success in my marketing efforts.
Thanks for the feedback, good points!
Thanks for the feedback, good points!
I'm trying your extension. I have held the same suspicions for over a decade. Practically given up on trying to get notice for my projects, the situation feels so random and/or pay to play.
Thanks! I’ll give this a try for sure.
I'm one of the people building Cody (https://cody.dev), which you mentioned. Re: suboptimal, yeah, these are all very early and there is so much room for improvement. Do you have a sense at least of what would be less suboptimal/more optimal for you?
I think there are several problems with these co-pilot solutions, but the most obvious ones are the context switching problem and the lack of steer-ability.
Let's say I want to build a complex React component that does a lot of stuff under the hood. Perhaps it needs to handle multiple inputs, it needs to show certain children UI conditionally based on the state and it also needs to push the state to a global redux store.
Perhaps the component also depends on some utils and other root components that pass props to it.
None of these solutions seem to acknowledge that what would improve my productivity is to have a proper way to model the problem for the AI the same way I'm understanding it. Allowing me to build the blast radius of the problem, instead of expecting the AI to infer it.
In the case of the React Component what I want is:
1) bootstrap the component
2) modify it to address requirements that emerge as I explore the needs.
3) look at dependencies such as schema files or root components and suggest modifications that align with the desired functionality or allow me to point out at required changes in dependencies.
When I say that these solutions are suboptimal, what I mean is that there's no straight-forward way for me to engage in these code generation tasks in a way that doesn't feel fragmented.
What I literally want is go to a file, tell the AI to consume all the context of that file and its dependencies and then modify it to add features, fix bugs, improve code or extend functionality. I then want to have the ability to accept those changes as a reviewer or suggest changes. And I want to do without forking away from my current context.
Nobody has gotten this experience right because what most players in this space are doing is building these restrictive form factors or atomized features like text brushes, global chats or in-line prompt to code generation.
I appreciate having these, but ultimately what I want is to have an AI system that accepts a context (a file or set of files) and a prompt and then generates the requested code modifications.
None of these extension allow me to do this effectively and if they do then it seems that ability is being diminished by a poor user-facing abstraction.
In my opinion, this is a design problem.
Let's say I want to build a complex React component that does a lot of stuff under the hood. Perhaps it needs to handle multiple inputs, it needs to show certain children UI conditionally based on the state and it also needs to push the state to a global redux store.
Perhaps the component also depends on some utils and other root components that pass props to it.
None of these solutions seem to acknowledge that what would improve my productivity is to have a proper way to model the problem for the AI the same way I'm understanding it. Allowing me to build the blast radius of the problem, instead of expecting the AI to infer it.
In the case of the React Component what I want is:
1) bootstrap the component
2) modify it to address requirements that emerge as I explore the needs.
3) look at dependencies such as schema files or root components and suggest modifications that align with the desired functionality or allow me to point out at required changes in dependencies.
When I say that these solutions are suboptimal, what I mean is that there's no straight-forward way for me to engage in these code generation tasks in a way that doesn't feel fragmented.
What I literally want is go to a file, tell the AI to consume all the context of that file and its dependencies and then modify it to add features, fix bugs, improve code or extend functionality. I then want to have the ability to accept those changes as a reviewer or suggest changes. And I want to do without forking away from my current context.
Nobody has gotten this experience right because what most players in this space are doing is building these restrictive form factors or atomized features like text brushes, global chats or in-line prompt to code generation.
I appreciate having these, but ultimately what I want is to have an AI system that accepts a context (a file or set of files) and a prompt and then generates the requested code modifications.
None of these extension allow me to do this effectively and if they do then it seems that ability is being diminished by a poor user-facing abstraction.
In my opinion, this is a design problem.
I think CodeAssist ( https://codeassist.tech ) is a tool that would let you do that (I'm the creator). It has a chat, can execute actions (write code), puts context from other files into the prompt and has some long-term memory.
The problem is that at the moment it's slightly buggy and the UI is not so polished. I would recommend to submit your email to the newsletter on the website. Once I feel it's good enough, I'll send an email to everyone. Don't worry that I'll spam your inbox, I haven't sent any email yet to people who subscribed. I'll just send it when I feel the tool is good enough.
The problem is that at the moment it's slightly buggy and the UI is not so polished. I would recommend to submit your email to the newsletter on the website. Once I feel it's good enough, I'll send an email to everyone. Don't worry that I'll spam your inbox, I haven't sent any email yet to people who subscribed. I'll just send it when I feel the tool is good enough.
We very much agree, and are aiming to tackle exactly this problem with Continue. Curious to know more about what you've tried and learned. What are the limitations that cause you to return to ChatGPT?
On another note: we purposefully refer to ourselves as an "autopilot" rather than "co-pilot". While at first glance pedantic, we think there is real value behind the idea that developers should be in control. We want to get out of their way, rather than injecting AI everywhere. An autopilot should feel like less of a separate entity (a pair-programmer) and more of an extension of yourself, a part of your own brain that you can call on to more easily complete particular tasks.
On another note: we purposefully refer to ourselves as an "autopilot" rather than "co-pilot". While at first glance pedantic, we think there is real value behind the idea that developers should be in control. We want to get out of their way, rather than injecting AI everywhere. An autopilot should feel like less of a separate entity (a pair-programmer) and more of an extension of yourself, a part of your own brain that you can call on to more easily complete particular tasks.
> more of an extension of yourself, a part of your own brain that you can call on to more easily complete particular tasks.
I like this idea but I gotta say when I think "autopilot" I think "does it for you" and imagine ceding control rather than retaining it. I definitely assumed after reading the name that it was going to be a tool to minimize developer involvement as much as possible, similar to how AutoGPT is supposed to require less supervision.
I like this idea but I gotta say when I think "autopilot" I think "does it for you" and imagine ceding control rather than retaining it. I definitely assumed after reading the name that it was going to be a tool to minimize developer involvement as much as possible, similar to how AutoGPT is supposed to require less supervision.
Hey. I gave it a try to Continue and I think this is going in the right direction at least for me. I guess opinions on how this should work are subjective.
But I do really like that functionality of attaching context to the query! Love that. I replied to the founder of Sourcegraph in this thread and I think that should answer your question as well.
I'm excited about your approach and even more about the fact that you made it open source! Thanks for that. If this sticks for me I can see myself contributing to it.
But I do really like that functionality of attaching context to the query! Love that. I replied to the founder of Sourcegraph in this thread and I think that should answer your question as well.
I'm excited about your approach and even more about the fact that you made it open source! Thanks for that. If this sticks for me I can see myself contributing to it.
Excited to hear all of this! (and appreciate the extensive thinking) Please don't be shy about asking questions in our Discord if you end up wanting to contribute or have more feedback :)
This looks really interesting. Have you spent much time investigating how to provide code context to the LLM without needing to manually highlight all the relevant code?
That’s been a major focus of my open source AI coding assistant project [0]. It seems super promising to exploit the semantic structure of code to help LLMs understand complex codebases [1].
Also, there’s a small discord [2] where a few of us are sharing learnings about building these coding assistants. Please join if you’d like to compare notes.
[0] https://github.com/paul-gauthier/aider
[1] https://aider.chat/docs/ctags.html
[2] https://discord.gg/FTYDTRKZ
That’s been a major focus of my open source AI coding assistant project [0]. It seems super promising to exploit the semantic structure of code to help LLMs understand complex codebases [1].
Also, there’s a small discord [2] where a few of us are sharing learnings about building these coding assistants. Please join if you’d like to compare notes.
[0] https://github.com/paul-gauthier/aider
[1] https://aider.chat/docs/ctags.html
[2] https://discord.gg/FTYDTRKZ
We've tried embedding search multiple times, and are also looking into the LSP. We actually removed the former for the time being because it reduces control/transparency when files are automatically being selected. Soon however we plan to release a Continue add-on that lets you type '@magic' or '@search' or '@embed' or something to deliberately include the results of a vector search in your context.
Looks like cool work you're doing, would love to learn more—thanks for the invite!
Looks like cool work you're doing, would love to learn more—thanks for the invite!
Looks like this is similar to GitHub Copilot Chat [0], just open source right? I like that you're supporting open models as well rather than just ChatGPT. Is there a way for your extension to read the file you're in as input before you ask any questions, so that it has the context of what you want to do?
[0] https://github.blog/2023-07-20-github-copilot-chat-beta-now-...
[0] https://github.blog/2023-07-20-github-copilot-chat-beta-now-...
Right now we are similar, but the open-source part is really key. We think that the ability to write custom plugins will make for a completely different kind of product.
And yes, by default Continue sees your open file, but you can also highlight multiple code snippets or type '@' to include context from outside your codebase, like GitHub issues.
And yes, by default Continue sees your open file, but you can also highlight multiple code snippets or type '@' to include context from outside your codebase, like GitHub issues.
I've been using Copilot and Copilot chat for a while now, and I really struggle to see why I should use any of these ChatGPT wrappers over it. I know some support local inference with open models, but they're just not that good in July 2023.
Sure, it's M$, sure it's not open source (IIRC), but all you get with these alternatives is a wrapper around the ChatGPT API, or in some cases a lesser model. No one's solved the whole directory context problem yet, no one seems to be doing much the Copilot suite can't.
Copilot is currently reasonably priced, and pretty much guaranteed support and development going forward. There's pull requests and cli in the pipeline [0].
I'd need directory context inference on a quality open model to be convinced to use anything else, and we're just not there yet.
[0] https://github.com/features/preview/copilot-x
Sure, it's M$, sure it's not open source (IIRC), but all you get with these alternatives is a wrapper around the ChatGPT API, or in some cases a lesser model. No one's solved the whole directory context problem yet, no one seems to be doing much the Copilot suite can't.
Copilot is currently reasonably priced, and pretty much guaranteed support and development going forward. There's pull requests and cli in the pipeline [0].
I'd need directory context inference on a quality open model to be convinced to use anything else, and we're just not there yet.
[0] https://github.com/features/preview/copilot-x
I use both Copilot and ChatGPT, they're entirely different things. Copilot works best for one line autocomplete, ChatGPT works for writing entire functions, if needed. This is to be expected as Copilot currently uses a much older LLM, and it is due to be upgraded to GPT 4 soon.
Are you on the Copilot Chat[0] preview? It's ChatGPT in VSCode, and it's nice. Not quite ChatGPT-level quality, but good enough.
[0] https://docs.github.com/en/copilot/github-copilot-chat/using...
[0] https://docs.github.com/en/copilot/github-copilot-chat/using...
Seems interesting will definitely give it a try.
Few observations/ questions from reading the documentation-
> Continue will only be as helpful as the LLM you are using to power the edits and explanations
Are there any others apart from gpt4 suitable for programming copilot tasks ?
> If files get too large, it can be difficult for Continue to fit them into the limited LLM context windows. Try to highlight the section of code that include the relevant context. It's rare that you need the entire file.
Most of the value and real world use case benefits come from usage in a brownfield development where a legacy code isn’t well understood and is large (exceed current LLM context ?)
> telemetry through posthog Can organisations setup their own telemetry and development data collection to further analyse how and where the Copilot is being used ?
> Finops How does one get visibility of token / api usage and track api spends ?
Are there any others apart from gpt4 suitable for programming copilot tasks ?
> If files get too large, it can be difficult for Continue to fit them into the limited LLM context windows. Try to highlight the section of code that include the relevant context. It's rare that you need the entire file.
Most of the value and real world use case benefits come from usage in a brownfield development where a legacy code isn’t well understood and is large (exceed current LLM context ?)
> telemetry through posthog Can organisations setup their own telemetry and development data collection to further analyse how and where the Copilot is being used ?
> Finops How does one get visibility of token / api usage and track api spends ?
Appreciate the deep read into the docs!
> We've found claude-2 very capable, especially for chat functionality, and especially in situations where you're looking for the equivalent of a faster Google search, even smaller models will do. For inline edits, gpt4 well outperforms others, but we've only optimized the prompt for gpt4. There's a LOT of tinkering to be done here, and it seems clear that OSS models will be capable soon.
> Definitely value there. We have an embeddings search plugin heading out the door soon, but we very consciously avoided this for a while - it obstructs understanding of what code enters the context window, and we think transparency is underrated.
> Yes! You could have your own PostHog telemetry by simply switching out the key, but we also deposit higher quality development data on your machine (we never see it). Benefits being both 1) understanding ROI of the tool, and 2) being able to train custom models.
> This is a reasonable request! We'll add a feature for this. Right now, you can use the usage dashboard of whichever provider's key you use.
> We've found claude-2 very capable, especially for chat functionality, and especially in situations where you're looking for the equivalent of a faster Google search, even smaller models will do. For inline edits, gpt4 well outperforms others, but we've only optimized the prompt for gpt4. There's a LOT of tinkering to be done here, and it seems clear that OSS models will be capable soon.
> Definitely value there. We have an embeddings search plugin heading out the door soon, but we very consciously avoided this for a while - it obstructs understanding of what code enters the context window, and we think transparency is underrated.
> Yes! You could have your own PostHog telemetry by simply switching out the key, but we also deposit higher quality development data on your machine (we never see it). Benefits being both 1) understanding ROI of the tool, and 2) being able to train custom models.
> This is a reasonable request! We'll add a feature for this. Right now, you can use the usage dashboard of whichever provider's key you use.
Been using Continue for a few weeks in combination with GH co-pilot. Overall it's been a solid experience. After a few days of adjusting, it's become my go to because I don't feel like I need to leave VSCode to get questions answered. Although there are constraints, the edit functionality works ~80% of the time after figuring out how to prompt it.
It's clear the team is shipping a ton too since almost every day I see VSCode popup about restarting my editor for the new version of Continue.
Excited to see where things go with this!
It's clear the team is shipping a ton too since almost every day I see VSCode popup about restarting my editor for the new version of Continue.
Excited to see where things go with this!
Hey! Thanks for this tool. I was testing and paying copilot, including the new chat integrated tool, but I feel your work flow proposal is more compelling.
That said, I'm not sure what's the difference between providing my own openapi key or not. The "Customization" doc is not entirely clear on what using my own key enables me to do.
For instance, is this required for gpt4? What are the limits of the free trial key?
I don't want to evaluate this not knowing which model is really using, and it's not clear what difference the key makes.
Edit: Also when I asked the Continue chat how to change my key and the model being used, it said that this is not possible since the key is yours, instead of pointing me to the "Extension Settings" inside the extension tab using the cog wheel.
That said, I'm not sure what's the difference between providing my own openapi key or not. The "Customization" doc is not entirely clear on what using my own key enables me to do.
For instance, is this required for gpt4? What are the limits of the free trial key?
I don't want to evaluate this not knowing which model is really using, and it's not clear what difference the key makes.
Edit: Also when I asked the Continue chat how to change my key and the model being used, it said that this is not possible since the key is yours, instead of pointing me to the "Extension Settings" inside the extension tab using the cog wheel.
We wanted to make it as easy as possible for people to try Continue, so we allow 250 free requests. These use gpt4. If you plug in your own API key in VS Code settings, it will also use gpt4 by default.
We'll update the /help messaging so it knows this, and you can read more about choosing a model here: https://continue.dev/docs/customization
We'll update the /help messaging so it knows this, and you can read more about choosing a model here: https://continue.dev/docs/customization
LocalAI seems like a interesting project that allows self-hosted can be easily called over the network. Open AI compatible interface https://github.com/go-skynet/LocalAI
Thanks for sharing! We've been recommending something similar with these llama-cpp-python bindings https://github.com/abetlen/llama-cpp-python#web-server, but I'll have to check that out
A cursory look through the source reveals the presence of three different telemetry suites of which only Posthog looks to be properly documented. I could have overlooked it, but do you have any more information on what Segment and Sentry are doing there?
Just deleted: https://github.com/continuedev/continue/commit/eba2f57a6462f...
Neither were doing anything, we simply forgot to `npm uninstall` after (a while ago) playing around to decide which service to use. Thanks for pointing it out.
Neither were doing anything, we simply forgot to `npm uninstall` after (a while ago) playing around to decide which service to use. Thanks for pointing it out.
That clears it up, cheers and all the best with this project!
The extension is not available on https://open-vsx.org/ ? (The market place for VSCodium)
I don't find this clear in the website what LLM it is really using.
Is it sending my code and queries over the internet? Or can it use a local language model?
By default it uses the gpt4 API, but you can optionally use any language model you'd like, including a local model. Can read more here: https://continue.dev/docs/customization#change-the-default-l...
Self ref: One more that is highly extensible and fully open source (MIT): https://github.com/ppipada/vscode-flexigpt
With multiple ai providers, customizable prompts, select to ask, comment to ask, history saving, SO search, command line exec, etc.
With multiple ai providers, customizable prompts, select to ask, comment to ask, history saving, SO search, command line exec, etc.
Thank you for supporting an open source ecosystem.
A couple of thoughts.
1. It would be nice if we could ingest external resources like web pages.
2. Deep integration with the debugger (Limitation mentioned in your docs). Code pilot seems to be very superficial here.
I know the issues above are limited by the token context. However, there's not a lot of innovation in these two aspects for code generation.
A couple of thoughts.
1. It would be nice if we could ingest external resources like web pages.
2. Deep integration with the debugger (Limitation mentioned in your docs). Code pilot seems to be very superficial here.
I know the issues above are limited by the token context. However, there's not a lot of innovation in these two aspects for code generation.
The first is already possible today! Using "ContextProviders", you can specify external resources to use as context. For example, we have a GitHub Issues ContextProvider that lets you type '@issue' and then search through open issues in your repo. We haven't yet built a ContextProvider for web pages, but this is a great idea. Theoretically, just type '@<URL>', then the LLM will know about the page.
The second sounds quite interesting. I'm curious whether you envision as far as the language model stepping through a program on its own? We had considered working on this at one point in time.
The second sounds quite interesting. I'm curious whether you envision as far as the language model stepping through a program on its own? We had considered working on this at one point in time.
All right.
Honestly I could see it both ways depending on the needs of the developer. The language model could step through autonomously or with the developer.
Thinking of how this could look like abstractly through a UI. Picture sequential timeline which consists each step the developer or the model has made through the code. A timeline that the developer could fast forward or rewind which has some implications for the LLM.
There are three main areas the developer could give LLM inputs.
1. Before debugging
The user could give input (Trace back, a question, ingested context (URL) before initializing debugging, instruction steps, autonomously or guided.
Autonomously given the context of before debugging, it could even optionally set its own breakpoints.
2. At any step during debugging
LLM might influence how the developers steps through the code.
3. Post debugging
Now imagine each step through the debugger being logged to which the user could ask questions within the context of the history of the debugger. LLM could have different scoped contexts relative to the timeline of the steps of interest.
Imagine the developer steps through 15 points in the code. Developer could ask about step 7 or a range of steps post debugging. LLM could be limited scope to only the previous steps including 7 or a range. Alternatively it could have knowledge of the entire step history. The underline assumption there is limiting context in some cases improves output of LLM.
Honestly I could see it both ways depending on the needs of the developer. The language model could step through autonomously or with the developer.
Thinking of how this could look like abstractly through a UI. Picture sequential timeline which consists each step the developer or the model has made through the code. A timeline that the developer could fast forward or rewind which has some implications for the LLM.
There are three main areas the developer could give LLM inputs.
1. Before debugging
The user could give input (Trace back, a question, ingested context (URL) before initializing debugging, instruction steps, autonomously or guided.
Autonomously given the context of before debugging, it could even optionally set its own breakpoints.
2. At any step during debugging
LLM might influence how the developers steps through the code.
3. Post debugging
Now imagine each step through the debugger being logged to which the user could ask questions within the context of the history of the debugger. LLM could have different scoped contexts relative to the timeline of the steps of interest.
Imagine the developer steps through 15 points in the code. Developer could ask about step 7 or a range of steps post debugging. LLM could be limited scope to only the previous steps including 7 or a range. Alternatively it could have knowledge of the entire step history. The underline assumption there is limiting context in some cases improves output of LLM.
This is fantastic thinking. Actually I'm quite excited about the combination of fault-localization techniques and LLMs; taking a traceback, pruning the call-graph, and seeking out bugs. There's a real future in all of this
Profiling code then optimizing autonomously would be interesting to see as it edits the code.
Any plans for supporting Jetbrains IDEs?
Focusing on VS Code for at least the next few weeks, but we've planned from the start! You can read more here (https://continue.dev/docs/how-continue-works), but the Continue server abstracts over the IDE APIs, so it will be easier to support any IDE, even letting you run Continue in "headless mode" from the a Python script, the CLI, or in CI/CD.
I use both pycharm and Webstorm, so an extension would be great! On the plus side, since they are all basically reskins of IntelliJ, one plug-in should work for all of the jetbrains suite.
Looks like it's trying to send my data to "meilisearch" without asking me :/
No longer: https://github.com/continuedev/continue/commit/8db5b39170229...
Definitely not intending this type of behavior. We want to do everything to keep Continue completely private.
Definitely not intending this type of behavior. We want to do everything to keep Continue completely private.
Ah, good to see, easy mistake to make. Thanks.
Why is it there in the first place?
If you mean Meilisearch, we use it to provide a snappy dropdown search experience, like here: https://youtu.be/3Ocrc-WX4iQ?t=86
We chose Meilisearch because it works entirely on your machine (now that analytics are turned off ;) )
We chose Meilisearch because it works entirely on your machine (now that analytics are turned off ;) )
How are you going to monetize this? I would be very nervous building a business around an extension calling someone elses API. You only have to look at the drama that played out at reddit a few weeks ago, to see how risky it is.
Given it can run on other engines as well as self-hosted or local models, I don't think "calling someone elses API" is a major issue in relation to the reddit situation. If anything I'd argue this being both open choice and open source negates the concern of losing access to the software. As for continued development, it's about the same as any self-funded project: It's free, so don't expect the world, and if you end up relying on it as a company, it might make sense to forward some effort back to the project to keep it stable.
Great question! We plan to make money by helping organizations continuously collect, analyze, and train on their development data. This will let them a) understand ROI of using LLMs for coding and b) use models that are constantly improving from internal usage in their codebase.
As for the issue of being cut out from the OpenAI API, we would survive such a situation because we work with any model, including open source.
As for the issue of being cut out from the OpenAI API, we would survive such a situation because we work with any model, including open source.
Is there any LLM based coding assistant than can read from terminal, thus catching error output?
We were actually doing this a couple weeks ago, but due to limitations of VS Code, it was slightly buggy (issues mainly around knowing the terminal window size when setting up a pseudo-terminal). That work still exists here, and we hope to find a solution eventually: https://github.com/continuedev/continue/blob/main/extension/...
My open source tool aider does this. In the chat you can use /run to launch any shell command and return the output to GPT. It’s super helpful for running the code it just changed or related tests and letting GPT figure out how to fix problems.
Here are the docs:
https://github.com/paul-gauthier/aider#in-chat-commands
Here is a chat transcript that shows gpt-4 fixing code after being shown failing test output using the /run command:
https://aider.chat/examples/add-test.html
Here are the docs:
https://github.com/paul-gauthier/aider#in-chat-commands
Here is a chat transcript that shows gpt-4 fixing code after being shown failing test output using the /run command:
https://aider.chat/examples/add-test.html
What do you think about starcoder
We've experimented with it quite a bit, and at one point used it as our default model for inline edits. As they discuss in section 5.1 of the paper (https://arxiv.org/pdf/2305.06161.pdf), they've trained on data formatted as "<commit_before>code<commit_msg>text<commit_after>code<eos>", which was convenient for our use case. Unfortunately, it tended to repeat itself and for more complex edits it doesn't match the raw ability of gpt4. I'm optimistic about open-source models though, and the data Continue lets users collect for themselves will help the open-source community get the data they need to compete.
does this require gpt-4? I can't afford such a luxury sadly.
> "Continue works with any LLM, including local models using ggml or open-source models hosted on your own cloud infrastructure, allowing you to remain 100% private. While OpenAI and Anthropic perform best today, we are excited to support the progress of open-source as it catches up (https://continue.dev/docs/customization#change-the-default-l...)."
Our GitHub is https://github.com/continuedev/continue. You can watch a demo of Continue and download the extension at https://continue.dev
— — —
A growing number of developers are replacing Google + Stack Overflow with Large Language Models (LLMs) as their primary approach to get help, similar to how developers previously replaced reference manuals with Google + Stack Overflow.
However, existing LLM developer tools are cumbersome black boxes. Developers are stuck copy/pasting from ChatGPT and guessing what context Copilot uses to make a suggestion. As we use these products, we expose how we build software and give implicit feedback that is used to improve their LLMs, yet we don’t benefit from this data nor get to keep it.
The solution is to give developers what they need: transparency, hackability, and control. Every one of us should be able to reason about what’s going on, tinker, and have control over our own development data. This is why we created Continue.
— — —
At its most basic, Continue removes the need for copy/pasting from ChatGPT—instead, you collect context by highlighting and then ask questions in the sidebar or have an edit streamed directly to your editor.
But Continue also provides powerful tools for managing context. For example, type ‘@issue’ to quickly reference a GitHub issue as you are prompting the LLM, ‘@README.md’ to reference such a file, or ‘@google’ to include the results of a Google search.
And there’s a ton of room for further customization. Today, you can write your own
- slash commands (e.g. ‘/commit’ to write a summary and commit message for staged changes, ‘/docs’ to grab the contents of a file and update documentation pages that depend on it, ‘/ticket’ to generate a full-featured ticket with relevant files and high-level instructions from a short description)
- context sources (e.g. GitHub issues, Jira, local files, StackOverflow, documentation pages)
- templated system message (e.g. “Always give maximally concise answers. Adhere to the following style guide whenever writing code: {{ /Users/nate/repo/styleguide.md }}”)
- tools (e.g. add a file, run unit tests, build and watch for errors)
- policies (e.g. define a goal-oriented agent that works in a write code, run code, read errors, fix code, repeat loop)
Continue works with any LLM, including local models using ggml or open-source models hosted on your own cloud infrastructure, allowing you to remain 100% private. While OpenAI and Anthropic perform best today, we are excited to support the progress of open-source as it catches up (https://continue.dev/docs/customization#change-the-default-l...).
When you use Continue, you automatically collect data on how you build software. By default, this development data is saved to `.continue/dev_data` on your local machine. When combined with the code that you ultimately commit, it can be used to improve the LLM that you or your team use (if you allow).
You can read more about how development data is generated as a byproduct of LLM-aided development and why we believe that you should start collecting it now: https://medium.com/@continuedev/its-time-to-collect-data-on-...
Continue has an Apache 2.0 license. We plan to make money by offering organizations a paid development data engine—a continuous feedback loop that ensures the LLMs always have fresh information and code in their preferred style.
— — —
We’d love for you to try out Continue and give us feedback! Let us know what you think in the comments : )