Best AI Coding Assistant Tools in 2023

Written By: Ashish Tripathi, Niharika Kargudri & Jenny Li

The recent surge in the popularity of AI Coding Assistant Tools is quite noticeable. This is especially true among developers due to their ability to accelerate coding processes and minimize errors. Popular tools in this category include GitHub Copilot, Tabnine, AWS CodeWhisperer, etc.

Amidst the AI buzz of 2023, this engaging topic took the forefront in our internal API tech radar discussions. Participants displayed enthusiasm, seeking a comprehensive rundown of AI assistant tools and their potential impact on daily engineering workflows—considerations extended to whether these tools heighten developer efficiency or introduce feedback and troubleshooting complexities with AI-generated code. The intricacies of the learning curve and the appropriateness of integration into ongoing client projects were also explored.

To keep ourselves updated on emerging technological trends, Mantel Group decided to dive deep into this landscape with an in-depth assessment of these tools. Establishing internal competency to offer tailored recommendations is imperative in this evolving landscape.

The purpose of this assessment is to recommend a set of best practices to be adopted when using any AI coding assistant tools; as well as to identify the pros and cons of various popular AI coding tools to inform and assist the team with selecting and using them for future client (or personal) projects.

The basics: What exactly is an AI Coding Assistant tool?

An AI-based coding assistant is a sophisticated software tool that utilizes Artificial Intelligence (AI) technology, such as Large-Language-Models (LLMs), to assist developers in the process of writing, editing, debugging, testing, and documenting code. Besides many other functions, AI tools do this mainly either by generating code based on prompts or suggesting code for auto-completion as it is written in real-time.

It basically acts as a virtual companion, providing intelligent suggestions, recommendations, and automated assistance to optimize the software development process and boost productivity.

The Assessment Approach

The assessment was done in three distinct phases:

Phase 1: Data Collection

This step involved the assessment participants conducting initial high-level research to find a list of all of the available and relevant AI coding tools on the market. Around 32 tools were identified in this phase. Some initial data regarding them was also collected based on the following rubric.

The initial list of tools found in phase 1 of the assessment included a ton of different tools, each at different levels of maturity and used for different use cases. 

The tools we listed down are

Phase 2: Tool Selection

In this phase, the assessment participants came together and went through the initial list of tools from phase one. A final shortlist was created of the tools that looked the most promising. The tools that seemed to have good maturity, a wide range of features, a large clientele, good market rapport/reviews, good language support, and extensive use cases were selected to go through to the next assessment round.

A total of 8 tools were selected for the deep dive and hands-on that are covered in more detail later.

A couple of the tools that we did think were worth looking into but couldn’t due to their unavailability to the general public at the time were Jetbrains and Mutable.

Some other tools that were not a part of this assessment but are worth mentioning for their unique use case are: 

HyperTest: Helps generate regression test suites for many use cases and real-world scenarios.

SapientAI: Generative AI for writing comprehensive Unit Tests

AI2SQL: Write efficient and accurate SQL queries quickly

Top 8 AI Coding Tools

The tools that underwent extensive assessment and were deemed worthy of being top-tier by the assessment team are listed in detail below. It is important however to note that things may have changed since the time of writing this blog given the pace with which the AI industry is evolving.

GitHub Copilot

Launched in 2021 and powered by Codex, GitHub Copilot uses generative AI models developed by GitHub, OpenAI, and Microsoft. It has been widely recognized in the developer community to help accelerate coding. This tool has also been hailed as the “AI Pair Programmer”. 

The languages supported by Copilot are effectively all languages that were available within GitHub publicly at the time of training e.g. Java, Python, JavaScript, TypeScript, Ruby, Go, C#, C++, SQL, and many more. It is available for use in all major IDEs like VScode, JetBrains and even Neovim. It boasts a huge amount of client adoption with over 400 business customers. 

Some of the notable consumers are Dell, SAS, Shopify, and Duolingo.

Pros

• Easy Setup: GitHub Copilot integrates with most major IDEs and has little to no learning curve. Has both individual and business tiers to support a wide array of clients
• Code Autocompletion: Helps finish code by suggesting and adding code while it’s being written
• Code Generation: Good at generating code when it is given instructions in natural language
• Contextual Suggestions: Ability to suggest meaningful names for functions and variables based on analyzing what the code is doing
• Constant Improvement: It learns from how the developer codes and gets better at giving suggestions the more it is used
• Evolves with time: Copilot learns as you code and keeps getting better with time. It can then use this context and learnings to generate better code. It adapts to your style of coding
• Wide array of features: Copilot supports a lot of major languages and IDEs. Apart from autocompletion, it can also generate code from comments, write algorithms in code from scratch, provide coding shortcuts, and write tests. This makes it a very versatile tool
• Copilot Chat: There is also a chat interface available that can be used to ask Copilot questions directly in the IDE. It can be used for code suggestions and descriptions and also for writing unit tests
• Copilot Labs: This is an experimental feature of Copilot that uses machine learning to improve the developer experience. It tries to improve developer experience with real-time suggestions, collaboration, and best practices
• Copilot for CLI: A copilot for your terminal

Cons

• Limited Context: Does not always understand the full context of a project while suggesting code. It does not do well with external config files. It also takes a while to get to full efficiency with the developer’s style of coding
• Tries to do a lot: It generates a lot of suggestions and this can be distracting. It may also make errors due to not having proper context

Differentiator

With its wide array of language and IDE support and industry adoption, GitHub Copilot is one of the top AI coding tool choices out there and it seems to be getting better at what it does with time.

AWS CodeWhisperer

From one of the key players in the cloud industry themselves, AWS CodeWhisperer is an AI coding tool that accelerates the writing of code by providing real-time code suggestions and autocomplete functionalities.

It integrates with major IDEs and supports a wide number of languages such as Python, Java, JavaScript, TypeScript, C#, Go, Rust and more. CodeWhisperer is still fairly new. The preview version of CodeWhisperer was launched in June 2022 and general availability was opened up in April 2023.

Its clientele includes some major players like Accenture, Amazon Ads, and TCS.

Pros

• Easy setup: Getting started with AWS CodeWhisperer is quick and easy and one can get started within minutes. It is intuitive to use and doesn’t have much of a learning curve
• Autocompletion: Provides real-time code suggestions when coding and autocompletes lines of code
• Test Generation: Can generate unit tests for functions pen in your IDE window
• Code References: Ability to reference code (Usually when used to write large blocks of commonly used code or frequently used algorithms)
• Code quality and security checks: Can perform quality and security checks on code
• Good with AWS Services: CodeWhisperer performs exceptionally well when writing code related to AWS services. It also works directly with AWS. Eg- You can use CodeWhisperer directly in the AWS lambda console
• Referencing Ability: Codewhisperer has an interesting referencing ability where it can reference any code that it might generate. This is a unique one that we haven’t yet seen in other coding tools
• Command Line Integration: Can be integrated with major command line tools like Iterm and Terminal. It provides IDE style completions within the CLI

Cons

• Not mature yet: Codewhisperer is still fairly new and is still finding its niche. Out of the languages it does support, some of them don’t have all the features available. It is also a tad slow and makes silly mistakes sometimes
• No API Support: The only way to use CodeWhisperer is to add it as an IDE extension
• Privacy Concerns: Privacy concerns on individual plan (Code might be used to improve their model, but there is an option to turn this off)

Differentiator

Shines and outperforms when working with code that talks to AWS services. The referencing ability is also very unique and insightful. CodeWhisperers command line integration is also very useful and unique.

Replit AI

Formerly known as Ghostwriter, Replit is a browser-based IDE that provides an interactive space for developers to write, test, and collaborate with other developers in various programming languages. It supports more than 50 languages and its user base is estimated to be 22.5 million from across 200 different countries as of April 2023 according to this blog.

Pros

• Good Documentation and Support: Replit provides great documentation about how to use Replit AI, including videos, diagrams, and well-thought-out examples. It also has a large community base
• Code Completion: Prompt suggestions based on the current context in real-time during the coding process
• Explain code: Gives line-by-line explanations for code
• Transform code: It understands code and can help refactor it to improve code readability and efficiency
• Generate code: can generate code based on a given natural language prompt
• Chat: Can answer any coding-related question right in the Workspace
• Easy to get started: Almost no setup is required, and it is intuitive to start using
• Large language support: Replit AI is one of the AI coding tools with the largest number of supported languages on the market

Cons

• Privacy Concerns: The free tier of Replit is not private. This raises some privacy concerns especially when working with sensitive code
• Expensive: The cost of getting a paid Replit plan is around $30 Australian dollars, per month. This seems expensive considering that other tools such as Github Copilot are around half the price
• Convenience or Constraints?: Replit comes with the advantage of having almost no setup requirements. However, this can be annoying for someone who likes to code using a customized setup

Differentiator

With its large language support and free tier, it is ideal for newbies or students as Replit works out of the box. Also, no need to configure the environment locally since its a browser-based tool.

Tabnine

Tabnine is an AI-powered code completion assistant that empowers developers and helps accelerate software development. For this assessment, only the free tier of Tabnine was tested due to some constraints. This means that only its autocomplete features were tested.

However, it was clear that Tabnine is a very highly regarded tool in the developer community.

Some of the major clients of Tabnine are Samsung, LG, and Reasonlabs.

Pros

• Easy to get started: The learning curve is not too big and it is easy to start working with the code suggestions. It has huge language and IDE support making it suitable for a wide range of use cases
• Autocomplete: Provides code suggestions in real-time when coding
• Tabnine Chat: The chat feature runs in the IDE and provides a way for developers to interact with the tool using natural language
• Flexible installation: Tabnine for organisations can be hosted on-cloud or on kubernetes clusters in-house
• Secure: Secure SaaS & Self hosting available for extra security (private AI models trained and hosted on local server or private cloud). User code remains private and is not used to train models other than private code models
• Smart suggestions: Adapts to user’s codebase and coding style when generating suggestions

Cons

• “Chat” function is in Beta, and only available to enterprise users: but looks like it has the most useful features such as Code generation, Unit test generation, documentation generation, Code improvement and code explanation features
• Resource intensive: Can be quite resource intensive, particularly on RAM. It needs proper hosting either on-premises or on-cloud to make the best use of it
• Limited Local Machine Power: Advanced completions can’t be run locally due to processing power required

Differentiator

Tabnine is not only one of the most matured tools in this space but the way it handles code privacy and security is quite refreshing. Private code remains private and this will entice a lot of bigger organisations.

Cody (by Sourcegraph)

Cody is a free and open-source AI coding assistant that was launched by Sourcegraph in March 2023. It is designed to explain, refactor, and write code by leveraging the context of the entire codebase. Some of the major clients using Cody are Nine, Hashicorp, Indeed and Yelp.

Pros

• Ability to utilize 3rd party Libraries: One of the flagship features of Cody is its auto code completion. It is not limited to features that are provided by the standard library of the programming language
• Autocomplete: It generates code suggestions based on a single line, or a whole function, configuration file, or requirement documentation
• Chat: Can answer code-related questions based on the context of your codebase 
• Commands: Provides easy-to-use commands for common actions, such as adding code documentation, generating unit tests, and more
• Code reusability: Ability to find and efficiently reuse code across codebases
• Flexible usage options: Currently, Cody can be integrated with IntelliJ, VS Code, and Neovim. It also has its own desktop app making it very flexible in the way developers can choose to use it
• Open-source tool: Cody is open-sourced and can be found on GitHub.This makes it easy for the community to contribute, resulting in faster development cycles, bug fixing, and more
• Codesearch: Its codesearch ability allows finding and fixing code in any repository, host and language

Cons

• Rate limit: As of today, Cody is free to use for individual developers with a rate limit, which means that if the rate limit is hit, the developer would have to wait for the next day for the rate to be replenished before sending requests again. Currently, there is no option to have a paid plan unless you are at an enterprise level. It also only supports up to 10 repositories for free
• Cody desktop application: At the moment, you are unable to view code using the desktop app and it needs to be used in conjunction with an IDE to be able to use it properly
• Security: The repositories linked to Cody are set to OpenAI to create embeddings to help with performance. This is a potential security concern

Differentiator

One of the only open-source AI coding tools on the list. Its Codesearch ability is also very unique since it enables code reusability, pinpointing root causes of issues, and finding vulnerable/buggy code. This ensures faster turnaround times when completing tasks.

Codeium

Codeium is an AI-powered code assistant that aims to help developers write more efficient code. It streamlines coding by reducing the need to write boilerplate code, offering natural language searching capabilities when working with unfamiliar codebases.

Developers can install the Codeium extension in the IDE of their choice and seamlessly integrate Codeium into their workflow.

A couple of notable clients of Codeium are Dell and Anduril.

Pros

• Huge Language Support: Codeium has trained its model with more than 70 programming languages including popular languages for front-end, back-end, and mobile development
• AI-Powered Autocomplete: It generates code suggestions as you type up the function name, comments, or even just the first few words of the line
• Intelligent Search: Developers can use plain English to search for relevant files and code by asking questions like “Where do we do X?” or “How does Y use Z?”, allowing them to quickly make changes in unfamiliar codebases
• AI-Powered Chat: You can prompt Codeium with commands such as “generating documentation for me” and let it handle the rest of the process
• Huge IDE Support: Codeium provides integration with more than 40 popular IDEs such as Visual Studio Code, JetBrains and Neovim. Mobile developers can also integrate Codeium with Android Studio or Xcode. It is even possible to install Codeium as a Google extension and use it with your Machine Learning projects on Google Colab
• Free forever for individuals: Codeium is committed to offering this tool to individual developers for free forever. There are different pricing options for enterprises and teams
• Encryption in Transit: The encryption feature makes users of Codeium less vulnerable to security attacks such as the man in the middle attack

Cons

• Lack of documentation: The Codeium website doesn’t have a lot of documentation. While there is an option to join their Discord Server, it might be less accessible for developers who would prefer not to do that
• Data Security and Privacy: Codeium collects telemetry data such as latency, feature suggestion, and suggestion acceptance information. It also collects code snippet data to be used to directly improve functionality, usability, and quality. Developers do have the option to opt out anytime

Differentiator

Codeium is SOC 2 (Service Organization Control 2) compliant. That means companies with a strong emphasis on security, privacy, and availability of the platform may find Codeium as a preferred option.

Refraction

Launched in 2023, Refraction is powered by OpenAI. Despite its recent launch, this innovative tool redefines the coding experience by offering 19 features including but not limited to function generation, documenting code, and unit test generation.

Pros

• Versatility: It supports functionality ranging from function generation to regex generation, creating CI/CD pipelines to creating SQL queries, converting CSS styles from one framework to another to even converting code from one language to another
• Generate Regex: Refraction can generate a Regex for a pattern you like
• Generate Diagrams: Uses Mermaid to generate diagrams based on requirements
• Generate CSP (Content Security Policy): Helps generate a content security policies depending on requirements
• Supported languages: Huge language support with the possibility to use up to 56 languages
• Easy to use: With the option to use either the Web UI or the IDE extension, developers can get started quickly and easily

Cons

• Some key features are limited to the Web UI: Currently, the IDE extension of Refraction does not provide as many features as the Web UI. Therefore, the developers would often find themselves manually copying and pasting code between the Web UI and their IDE, which is a less-than-optimal workflow
• Pricing: Refraction offers a 7-day free trial. To keep using it longer, it is around $12 Australian dollars, per month for a Pro plan or around $21 Australian dollars, per user per month for a Team plan. One of the most important aspects of the trial period is that you can only use the Web UI, not the IDE extension, and send up to 10 code generations, which is not a lot

Differentiator

Lots of great/unique features such as diagram generation, CSP (Content Security Policy) generation, and CICD pipeline generation that we haven’t seen with the other tools on this list.

CodeStory (Aide)

Aide was launched by CodeStory and is marketed as an AI-powered IDE that can do everything you do with you, i.e., it understands the codebases, explains code, helps debug, suggests commit messages and much more.

The vision for Aide was for it to be used as an AI pair programmer.

Pros

• AI-powered IDE: Aide is a one-stop shop that focuses on having AI assistance seamlessly integrated into the IDE, leveraging rich developer tools and features, and working collaboratively with the developers
• Debug and Explore: Outline the issue you are having with the code and Aide will guide you through finding, diagnosing, and resolving the issue 
• Natural language search: Ability to provide commands and ask questions in plain English
• Grouped git diffs: It summarises related changes for you since your last commit. This gives a visual representation of your changes and helps you decide whether to break these changes down into several commits instead 
• Commit message helper: Help write better commit messages based on the tasks completed 
• Pricing: Free. Aide does not cost anything as of the time of writing this blog.
• Easy to get started: It is easy to transfer settings and extensions from other browsers into Aide

Cons

• Limited language support: at the time of research, only 4 languages were supported by Aide, namely TypeScript, JavaScript, Python, and Golang
• Limited Platform Support: Aide currently only offers support to macOS machines
• Missing features: Aide is relatively new to the market. Hence some features are yet to be included, such as auto-complete for example

Differentiator

Offers Local Model Support which is unique compared to the other tools that are hosted on the cloud or require on-prem hosting. This also means complete security with no telemetry collection.