GitHub Copilot vs. Amazon CodeWhisperer
Tech giants compete in everything. Now the topic of interest seems to be AI-powered coding assistants. One of the main technological innovations of the last year was GitHub Copilot, an AI pair programmer developed by Microsoft and OpenAI. It created quite a stir in the tech world and received a lot of appreciation (and some criticism too).
Amazon woke up from its year-long slumber and launched direct competition with Copilot. He was named CodeWhisperer.
This launch took place at re: conference on mars held recently.
Strike at the right time
Interestingly, this comes days after Microsoft announced the general availability of GitHub Copilot to all developers for $10/month or $100/year. It will be free for verified students and maintainers of popular open source projects. Until this announcement was made by GitHub, it was also available as a technical preview. Currently, there are no details on the cost of CodeWhisperer.
Github CEO Thomas Dohmke said, “With over 1.2 million developers in our technical preview over the past 12 months, people who started using GitHub Copilot were quick to tell us that it’s become an indispensable part of their daily workflows.”
He added that in files where Copilot is enabled, almost 40% of the code is written by GitHub Copilot in popular coding languages, like Python, and the company expects this to only increase.
GitHub Copilot pulls context from comments and code to instantly suggest individual lines and entire functions. It is powered by Codex (OpenAI’s pre-trained generative language model) and is available as an extension for Visual Studio Code, Visual Studio, Neovim, and the JetBrains IDE suite.
Not just a copy of Copilot, says top Amazon executive
While interacting with Tech Crunch, Vasi Philomin, Amazon’s vice president for AI services, mentioned that Amazon did not create CodeWhisperer to give an alternative to Copilot. He said that with CodeGuru and DevOps Guru, the company laid the groundwork for the launch a few years ago.
- CodeGuru – In 2020, Amazon announced the general availability of Amazon CodeGuru. It is an ML-based development tool that provides intelligent recommendations to improve code quality and identify the most expensive lines of code in an application.
- DevOpsGuru – Last year, Amazon announced the general availability of Amazon DevOps Guru, an ML-powered service that gives users an easier way to measure and improve an application’s operational performance and availability and reduce costly downtime.
Deliver what it claims?
Since the launch of Copilot, it has generated a lot of curiosity and studies have been carried out to see if it delivers what it claims. According to the article entitled “Is GitHub Copilot a substitute for human pair programming? An Empirical Study,” the authors conducted an experiment with 21 participants where they focused on productivity and code quality.
For the experimental design, a participant was given a project to code under three conditions presented in random order.
- Pair programming with Copilot
- Human pair programming as pilot
- As a browser
The codes generated from the three trials were analyzed to determine how many lines of code on average were added in each condition and how many lines of code on average were removed in the next step. “The results suggest that although Copilot increases productivity as measured by lines of code added, the quality of code produced is lower by removing more lines of code on the next trial,” the authors added.
Another study titled “Asleep at the keyboard? GitHub Copilot code contribution security assessment” showed that code designed by Copilot could include bugs or design flaws that an attacker could potentially exploit. The authors created 89 scenarios for which Copilot developed code, resulting in 1,692 programs. The authors found that about 40% of this code had bugs that could pose security risks.
CodeWhisperer takes into account your coding style
Developers need to keep up to date with different programming languages, frameworks, software libraries, and cloud services. Now, Amazon says developers can speed up the development process with CodeWhisperer by simply writing a comment in their IDE’s code editor.
“CodeWhisperer automatically analyzes the comment, determines the cloud services and public libraries best suited for the specified task, and recommends a code snippet directly in the source code editor,” Amazon explains.
Jeff Barr, Chief Evangelist for AWS, added in a blog post that CodeWhisperer will continuously review user’s code and feedback and present them with syntactically correct recommendations. “Recommendations are synthesized based on your coding style and variable names and are not just snippets,” he added.
Security is also a priority area for CodeWhisperer
CodeWhisperer also makes security a priority, Amazon claims. It provides security scans for Java and Python to help developers find vulnerabilities in their projects and build apps responsibly. It also includes a benchmark tracker that detects whether a code recommendation may be similar to particular training data. Developers can then easily find and review the sample code and decide whether or not to use the code in their project.