AI Coding Assistants

Everything you need to know about how AI is transforming coding

Author

Saahil Dama
September 03, 2024

“It’s the AI’s job to be fast, but it’s your job to be good.” – Isaac Lyman

Introduction

What AI coding assistants can do

AI coding assistants can help with writing basic scripts and queries, making incremental changes to code, debugging, and optimizing computer resource usage, among other tasks.

These tools can be used for “pair programming,” a style of programming where one person writes the code as the “driver” and the other checks the work while keeping an eye on the big picture as the “navigator.” With AI, the human takes the navigator role to plan, think about design, and review any code produced by the tool.

Copilot and other AI coding assistants have turned programming into prompting, allowing developers to tweak half-formed code and nudge large language models (LLMs) instead of coding from scratch.

Market size

The global AI coding tools market size is expected to increase from $4.91B in 2024 to $27.17B by 2032, growing at a CAGR of 23.8%. Dealroom data shows that AI-based coding assistants raised $753M so far in 2024, bringing the total since January 2023 to ~$1.2B.

Industry impact

Usage levels

57% of software CTOs and engineering leaders surveyed by Bain say they are actively rolling out generative AI-based coding assistants. About 75% of adopters say that generative AI coding assistants have met or exceeded their expectations and that speed to market is the biggest advantage offered by the technology.

76% of respondents on a Stack Overflow survey say they are using or planning to use AI tools in their development process in 2024, up from 70% in 2023. 72% viewed AI tools favorably or very favorably, lower than 77% in 2023 potentially due to disappointing results from usage. 43% feel confident about AI accuracy, while 31% have some degree of distrust.

Gartner predicts that 75% of enterprise software engineers will use AI coding assistants by 2028, up from less than 10% in early 2023. It believes that building an effective value story will require software engineering leaders to move from a narrative of cost reduction to higher value generation for the organization.

McKinsey found that generative AI increased developer speed for code documentation by 45% to 50% and for code generation by 35% to 45%. For high-complexity tasks, however, time savings dropped to under 10%. The study also found that code quality was marginally better in AI-assisted code when developers iterated with tools to achieve that quality, suggesting that the tech is “best used to augment developers rather than replace them.”

Overview of key players

In its first-ever Magic Quadrant for AI Code Assistants, Gartner identified 12 companies as leaders, visionaries, challengers, and niche players.

Concerns and controversy

Legal issues

Some developers filed a class-action lawsuit against GitHub, Microsoft, and OpenAI in November 2022, arguing that the companies were unlawfully copying their code. In a July 2024 order, a California judge dismissed three claims and ruled that Copilot’s code was not identical enough to the developers’ copyrighted code. One claim relating to breach of open-source licenses was allowed to continue.

Code quality

Code churn grew from 3-4% between 2020 and 2022 to an average of 5.5% in 2023, the year in which code written by AI programming assistants increased dramatically. “Churn” refers to code that was reverted, removed, or updated within two weeks of being pushed to the repo. While code does get written faster, more time and resources are spent in understanding existing code and fixing bad code.

A Stanford study found that participants using an AI assistant based on OpenAI’s codex-davinci-002 model produced significantly less secure code than those who didn’t have access. Furthermore, those who used the AI assistant were more likely to believe that their code was secure than those who didn’t.

Coding skills assessment platform CodeSignal found that out of over 1,000 developers surveyed about AI-generated code, 55% had concerns about the quality, 48% were worried about security and/or privacy, and 46% believed it could result in lower demand for developers.

A report by cybersecurity startup Snyk found that 96% of 537 people working in software engineering and security use AI coding tools, with over half using these tools most or all of the time. These tools’ polish and ease-of-use, however, has led to a misplaced sense of confidence being placed in them. 91.6% of the respondents said that the tools generated insecure code suggestions at least some of the time.

Job implications

Experts believe that software developers will continue to have jobs, especially when novel situations arise and something that nobody has seen before needs to be represented in code. John DeNero, Computer Science Teaching Professor at UC Berkeley, draws parallels to language translation: human linguistic experts remain critical despite Google Translate and other AI-based tools.

Major players

Big tech companies

Google unveiled Gemini Code Assist at its Cloud Next conference in April 2024. Code Assist offers developers code generation and completion capabilities, including based on best practices, organizational governance, and a natural language description of the code. It can handle over 20 programming languages.

CEO Andy Jassy wrote Amazon’s AI assistant for software development, Q Developer, helped reduce the average time to update an app to Java 17 from 50 developer-days to a few hours, saving 4,500 developer-years of work. This improvement is worth $260M in annual cost savings.

According to sources, Apple is building an AI-based tool to help app developers create code more quickly. The company plans to release this tool as early as this year as part of the next major version of its developer platform, Xcode.

Replit launched Replit Teams in April 2024, allowing real-time collaboration for developers with an AI agent automatically fixing coding errors. Replit’s ambition is to provide “a fully autonomous pair programmer — one that feels like working with another teammate.” The startup has raised over $220M and laid off 30 employees, or 20% of staff, in May 2024.

Unicorn (or almost-unicorn) startups

Cognition AI claims it software development assistant, Devin, can finish entire projects on its own, including deploying apps/websites and fixing bugs. A source said that the startup raised $175M at a $2B post-money valuation in April 2024. Microsoft also announced a partnership with Cognition AI to bring Devin to its customers. Devin’s demo, however, led to criticism in a popular YouTube video titled ‘Debunking Devin: “First AI Software Engineer” Upwork lie exposed.’ The video split the coding community with its claims that Devin didn’t do the task that was actually requested on Upwork, fixed bugs that were created by itself, and more.

Codeium provides features for code suggestions, chat for purposes like explaining code, and more within existing code editors. Founded by two MIT grads, the startup grew from zero to over 300,000 users in less than fifteen months. It raised a $150M Series C led by General Catalyst at a $1.25B post-money valuation in August 2024, bringing its total funding to ~$243M. Codeium released a coding engine called Cortex, which it claims can process up to 100M lines of code at once.

Magic is building AI models to generate code and automate various software development tasks. It recently raised $320M from Sequoia, former Google CEO Eric Schmidt, Jane Street, and others, bringing its total funding to $465M. The startup has a team of about two dozen people and no revenue to speak of.

Augment emerged from stealth in April 2024 with a $227M Series B at a $977M post-money valuation. Investors included Sutter Hill, Lightspeed, and former Google CEO Eric Schmidt. CEO Scott Dietzen argues that Augment’s opportunity is that tools like GitHub Copilot and ChatGPT help individual programmers but don’t “address the complexity of programming at scale” by failing to take advantage of knowledge in an organization’s existing codebase.

Poolside AI is developing a specialized LLM for developers. The Paris-based startup was founded by former GitHub CTO Jason Warner and is in talks to raise $450M at a $2B pre-money valuation, according to sources.

Other hot startups

Cursor is building a code editor that is built on top of GPT-4, Claude, and other LLMs. The startup, founded by four friends who met at MIT, raised a $60M Series A led by a16z and has over $10M in ARR. Some AI features that integrate directly into developer workflows include next action prediction, natural language edits, and chatting with the codebase. Use cases include building a video editor, chatbot, and Chrome extensions.

YC-based Cosine announced Genie, a coding assistant that can solve bugs, build features, refactor code and more either fully autonomously or with the user. The startup claims Genie is the “best AI software engineer in the world,” scoring 30% on the benchmark test SWE-Bench, beating the 19% scored by Amazon’s Q and Factory Code’s Droid’s and 13.8% for Cognition’s Devin. It raised $2.5M in August 2024.