Developers range from traditional IDE users who incrementally adopt AI to new AI-first users who embrace chat extensions from the start.
GitHub Copilot supports models from major providers, requiring a fast-paced, often ad-hoc integration process with about a two-week lead time for new checkpoints.
Offline evaluations use deterministic assertions, similar to test cases, to reliably measure the performance and quality of non-deterministic AI model outputs.
Different AI models exhibit unique personalities and excel at specific tasks, such as Claude for web development and GPT for backend work.
VS Code supports connecting local models via a "bring your own key" feature, though built-in models benefit from an optimized coding harness.
Before a new model launch, the team spends two weeks running offline evaluations, optimizing system prompts, and conducting internal dogfooding for qualitative "vibe checks".
Users can manually A/B test different models using local sessions, and the community can contribute by suggesting improvements to the open-source system prompts.
The best practice for model selection is to use the latest available version for optimal quality and to experiment with different models for various tasks.
The VS Code team remains committed to its open-source culture, valuing community contributions and accepting that forks are a natural part of the ecosystem.
New contributors can use AI tools to understand large codebases like VS Code, making it easier to start with small, valued contributions.
Julia Kasper is Senior Product Manager for VS Code & GitHub Copilot at Microsoft.
Jobs that call for the skills explored in this talk.