Building the next generation IDE
Adventures in building open source tools to re-imagine how we code with AI.
Since I got into open source LLM I became interested in how tools like GPT-4 could re-invent how we write code. I’ve created a few projects this summer that explore this deeper.
Backseat Pilot
My first iteration of this was a VSCode extension designed to work with Llama models, Backseat Pilot. There were hundreds of VSCode extensions for ChatGPT, so this project is simply extending it so it will work with your local LLM.
What is different from Backseat Pilot than Github Copilot (at the time at least) was that its not tabbed or autocompletion based, but instead the LLM output goes directly into the editor, thus the “backseat”. This is particularly useful for refactoring code where you highlight a block and it fixes potential errors.
WizardCoder Sandbox
As new open source language models continued to be released I kept close tabs on which models performed better at coding. The Can AI Code project by Mike Ravkine made it easy to do this, and the modified StarCoder + WizardLM combo has stayed on top of the leaderboards for some time.
The quality is good for basic coding and the cost is much lower with ggml models on a Mac. Since WizardCoder requires slightly different backend I helped create a repo for running your own API.
With WizardCoder running on CPU instances for pennies an hour I started a project to showcase its coding skills. This is WizardCoder Sandbox. This is a first of its kind “What you prompt is what you get” HTML editor (WYPIWYG?).
To get good results you need to do some prompt engineering, but following a simple inline styles only, html only yields good results and demonstrates the potential of chatting with the IDE.
Codetree.ai
I wanted to take the WizardCoder Sandbox project to the next level, specifically with writing basic Javascript React.js apps. I discovered a project on Github called Codetree by Philippe Tedajo that does this.
Codetree uses Web Assembly with esbuild to compile Javascript code in realtime within the browser to an iframe preview tab. I added the chat using GPT-4. I plan to extend this to work with WizardCoder/Open Source LLMs soon!
Future of coding LLMs
There will be many different re-imaginations of software development with the LLM, this is just scratching the surface. Its inevitable these integrations are deeper customized specific languages or IDEs. Github CoPilot and VSCode is the most obvious example of this. Developers will always gravitate towards homegrown solutions, so I’m hopeful that tools like Codetree.ai will make software development accessible to more people!