Programmers spend a lot of time writing code. Tools like code editors can help us along with syntax suggestions, snippets, debugging suggestions, and so on. But what if we had a tool that used artificial intelligence (AI) to help us write much more substantial portions of code? That’s what GitHub Copilot is all about.

I was recently scrolling through Twitter when I saw this tweet from the official GitHub account:

I was amazed by the idea of AI helping me to write code (or even do all the heavy work), so I went ahead and visited the GitHub Copilot page.

Skipping all the content, I went to the bottom of the page searching for a way to test this out. I encountered a banner that was a call to action to sign up for GitHub Copilot technical preview.

After some days (or weeks) waiting, I got granted access to the technical preview, and now I can let AI code for me … or can I?

Read more to learn what GitHub copilot is, my experience with it, and how it’ll impact you … or why maybe not.

What is Copilot?

Simply put, GitHub Copilot is an AI tool that provides you code suggestions based on comments and the context of the file you’re editing.

Copilot is the result of a collaboration between GitHub and OpenAI, which is heavily backed by Microsoft. It’s powered by a brand new AI system named Codex, which is based on the GPT-3 model.

GPT-3 stands for the third generation of the Generative Pre-trained Transformer — a language model capable of generating sequences of text from simple prompts. Codex is derived from this model, which is capable not only of text, but also code generation in some of the most popular languages.

Copilot has been trained with billions of lines of code from publicly available repositories on GitHub, so your code has probably improved this AI tool in some way (we’ll get into details later).

Although it supports most programming languages, it currently works the best with Python, JavaScript, TypeScript, Ruby, and Go.

Let’s see how GitHub Copilot works, and what it’s currently capable of.

GitHub Copilot in Action

Copilot is incredibly easy to install. In case you have access to the technical preview, just download the VS Code extension by searching for it on the Extension tab and activating it.

It then requires you to log in to your GitHub account, so it can confirm you have access to the technical preview.

For now, the only way to use Copilot is on VS Code, and it may remain the same for some time according to Copilot’s page.

Most of the following examples will be using Python, since it’s one of the languages this AI tool is really good with.

How Code Suggestions Work

GitHub Copilot generates multiple suggestions for you based on the context of the file you’re editing. Mainly, it gives you suggestions based on the comments you’ve made in the file, and the code you’ve written before.

Once Copilot has a code suggestion, it’ll ask you to use it. Let’s test out Copilot by creating a function that computes the average of a dataset. The only thing I’ll provide to Copilot is a comment and the name of the function.

As you can see, the text in gray is suggested by Copilot, and I can accept it by pressing Tab. But if I don’t like the first suggestion, I can walk through more suggestions with Ctrl + ], or see a bunch of solutions from a side panel with Ctrl + Return.

Impressive, it isn’t? But let’s set a different challenge. Now, Copilot has to create a main function that lets the user enter some space-delimited numbers. It should split these numbers and pass the resulting list to the compute_average function, before printing the result.

Lastly, I’m going to ask Copilot to call the main function using the execution entry point __name__ == '__main__'.

And that’s how GitHub Copilot wrote a functional Python script based only on the commands I gave to it. Of course, the code isn’t perfect. For instance, the compute_average function could be reduced to sum(dataset) / len(dataset), but the overall result is pretty good.

Testing Copilot with Simple Problems

Let’s start with the function every developer must know: FizzBuzz. I’ll write the problem statement, name the function and let Copilot do the work

What about a leap year function? In this case, I’ll only provide a simple docstring.

Now, a simple palindrome checker.

Another neat thing about Copilot is that it can also provide suggestions in comments and docstrings. In the above example, it completed the definition of a palindrome!

Lastly, a simple password generator. I provided a long description, and the modules I wanted to use. Surprisingly, I got exactly what I wanted.

To conclude this section, Copilot is extremely good at suggesting simple, bite-sized solutions from our comments.

Now let’s test how this AI pair programmer performs in more complex environments.

Complex Copilot usage

First, let’s use Copilot to solve common algorithms problems. For example, an iterative binary search implementation.

Don’t worry if you don’t understand the code; at first, I didn’t either. Here comes one of the downsides of using this kind of tool. You may implement code provided by Copilot without actually understanding its meaning.

We’ll see more downsides later, but you should take this into account in case you have access to the technical preview.

Aside from this, the solution above is excellent (probably extracted from a DSA GitHub repository). It’s readable code that, with few moments of analysis, I managed to understand.

But you can’t always rely on Copilot suggestions. Most of the time you’ll need to review the suggestions a couple of times before integrating them into your codebase.

We’ve tested Copilot with simple problems and algorithms. It would be a good idea to use it in real-world solutions.

A Django project

Django is one of the most-used Python frameworks. Let’s see how Copilot interacts with Django in a blog app.

I created a Post model class and specified the fields I wanted it to have. (A model in Django is the Python code representation of a database table.) This is what Copilot suggested to me.

Pretty good, but immediately after I accepted the suggestion, I got another one. Copilot wanted to create a Comment model! This was what I get.

Copilot generated a working and well-documented model, without me even asking it to do so.

Once again, Copilot suggested another possible model — a Tag model — and the only thing I had to type was class:

class Tag(models.Model):
    """Tag model
    name: CharField
    """

    name = models.CharField(max_length=100)
    post = models.ManyToManyField(Post)

    def __str__(self):
        return self.name

I was stunned, so my next challenge was to let Copilot test the models it created. I fired up the test.py file inside my blog app and give Copilot some little hints (like importing the models, creating the Test class name).

Finally, I found a weakness! Copilot wasn’t able to write a whole test suite for the code it created. But nonetheless, Django is a third-party package, and I was amazed that Copilot had managed to create three models by itself.

Closing this Django adventure with Copilot, I have to say that the code wasn’t perfect. There were things to improve like the help_text definitions, and the usage of some crucial model arguments like verbose_name, but in general Copilot surpassed my expectations.

Chatting with Copilot

Copilot is somewhat GPT-3 based, which means it may be able to understand natural language in a plain text file and establish a conversation based on this.

Wow, it seems Copilot has a pretty good liking for Python, and so do I. Looking forward to the conversation.

Aside from its code suggestion capabilities, it seems Copilot also has good negotiation skills.

Being serious about the possible usages of this, Copilot could help you to write proper documentation, or copywriting in case you’re working in text file formats, like Markdown, RST, or LaTeX.

For instance, it could help you to write articles (like this one), or even a book.

My Thoughts on Copilot

Copilot is an extremely fun-to-use tool. At first glance, it’s really enjoyable to code with it, and I spent hours testing it out.

If you spend a couple of days coding with it, you’ll get used to it, and it’s really useful in some cases, like writing documentation or getting started with new technology.

However, it’s not perfect, and below is a list of things that may be a problem when building a programming project.

Slow Code Completition

Currently, (and remember that Copilot is still in technical preview), code completion could be a little bit clunky.

Especially with line-by-line suggestions, I was able to type my solutions before Copilot could even show off (taking into account my average typing speed is 48 words per minute, which isn’t that impressive compared to other developers).

This is understandable because it retrieves suggestions from the Internet, which can cause some kind of delay. Of course, this depends on your internet speed.

Also, Copilot often offers more extensive completion, as shown in the above tests, which is much more resource-demanding (on the server Copilot is in) than a simple function method suggestion.

Flow Interuption

In these early stages, it’s not a tool you’d use while building a serious project.

Certainly, you don’t know when Copilot suggestions will arrive, and when they do, they may interrupt your workflow because you get distracted with the autocompletion.

I experienced this a lot, since it can be annoying to review every piece of code it suggests to me. Fortunately, Nat Friedman (GitHub’s CEO) has addressed this issue on Twitter, and Copilot may have a UI to indicate whether it’s producing a code suggestion or not.

Broken Code Suggested

Sometimes, Copilot suggestions don’t work. As they confirm in GitHub Copilot page, it does its best to give you optimal code completion, but that doesn’t mean every code snippet provided by Copilot will work perfectly.

Whether you have access to the technical preview, or you’re a reader from the future, to get the most of Copilot you should try to provide the best docstrings and function names you can (similar to the above code snippets).

Also, as Copilot is trained with public GitHub repositories, it might suggest code snippets using old libraries or modules, so it’s important to review each large bunch of code it offers up.

With Time it May Generate Dependency

Copilot is a really powerful tool, but with time you might become dependent on it.

It’s like having a Stack Overflow client directly in your editor — one that tries to understand your code and give you the ten best possible solutions.

Don’t get me wrong: it’s fantastic, and with time it may become one of the most-used tools in the industry, but with time it could become an issue to depend heavily on it. This little detail combined with the fact that some results provided by Copilot may not work as expected, leads to the following issue …

Copilot suggests code you may not understand

As showed earlier with some code samples, Copilot suggests a large bunch of code, and it’s not always easy to understand what it’s doing.

It can be tempting to just let Copilot do the hard work, which can affect beginners (and advanced developers) in their learning process. Most of the time, the most important thing about coding is error troubleshooting, and the code completion provided by Copilot may affect the development of this skill.

To avoid this, it’s important to review the code suggested by Copilot and to understand what it’s doing.

Copilot is a brand new tool, and it presents a lot of bugs. I created some posts on the GitHub Copilot’s discussion page (available for technical preview users), and I’ve already received feedback from the community.

I’m sure this tool will solve most of these issues before it gets released.

Controversy around Copilot Training

It’s known that Copilot is trained on the basis of public GitHub repositories, and this has caused certain reactions by the community. Some people are mad because free and open-source code is being used to train Copilot.

As you may know, most open-source licenses (GNU, BSD, APACHE, etc.) allow you to use, modify and distribute software, with the only condition of using the same license. However, Copilot is meant to be a commercial product.

According to Creative Commons, the use of publicly available data to train AI models doesn’t infringe copyright by default — at least for CC licenses. Nat Friedman has also addressed this topic on a Twitter thread:

Copilot (as they proclaim in its page) is just a code synthesizer, therefore it’s really weird to get a literal suggestion extracted from the training set.

This tool still is in technical preview, which means it could output personal data from GitHub repositories, but we can expect this problem to be solved before it’s officially released.

Don’t forget that TabNine — another AI code completion — has also been trained with open-source code from GitHub, so this has already happened a couple of times.

In conclusion, it’s generally accepted that AI models will be trained with public data. However, ethical concerns about this practice will be a strong discussion topic in the next few years.

Are we training GitHub Copilot?

GitHub Copilot is constantly learning from our code style, and preparing itself to meet our needs. However, it won’t share private code as suggestions for other users. I consider this extremely important, because sometimes I edit environmental files that store sensitive information, directly on VS Code, and it would be terrible if these files were shared with others.

On the other hand, just like any other AI model, Copilot learns from its users, and it’s known that there’ll be telemetry for some actions like accepted or rejected suggestions. If you’re curious, make sure to check out the Copilot telemetry page.

Is Copilot Going to Replace Developers?

For now, Copilot is just what it claims to be: an AI helper for developers. I don’t think it will replace developers, at least in the short or midterm.

It can’t understand a real-world problem, plan a solution, build it and show it off to the world — tasks that developers (and humans in general) are good at.

Maybe, in the future, a tool like GitHub Copilot will be a game changer in the programming industry — not by stealing jobs, but by making developers more productive. We’ve been improving developers’ experience (code editors, debugging tools, etc.) since the last century, and now with the rise of AI technology, we can expect the creation of much more tools using it.

Remember that before humans landed on the moon, “Computers” were what we called people extremely skilled in solving mathematical operations by hand. Computers as we know them today have destroyed countless jobs, but also generated new ones, and ended up creating one of the biggest industries worldwide.

Copilot Alternatives

For now, Copilot is a unique technology. But if the only thing you need is a good AI completion system, there are some other options to choose from.

Copilot isn’t publicly available yet, so you may need a more established product to code in your preferred language.

On the other hand, if you’re new to coding, I strongly suggest you make use of these alternatives, since they help you without implementing code logic for you. Once you’re more experienced, you could go with Copilot as your AI pair programmer.

Intellisense

If you’re a VS Code user, you’ve probably been using Intellisense since day one. This is the default code-completion system on VS Code, and you can install support for every programming language you’re using. The only setup you need to do is to fire up the Extension tab and search for the language you want to install support for.

Kite

Kite is an AI-powered code completion service that you can install in almost every popular code editor. It offers a free plan, which, in most cases, is more than enough, and a full-featured copilot (not like GitHub Copilot) tab which lets you view Python documentation without firing up a browser.

Unfortunately, Kite’s main target is Python, so it’s worth looking at TabNine before making a decision.

TabNine

TabNine, previously named Codota, is another AI completion tool used by millions of developers. Currently, it works with over 30 languages and can be installed in 15 IDEs.

Interestingly, TabNine has also been trained with public code available on GitHub, and it uses GPT-2 (the previous version of GPT-3) to generate code completion. If you don’t have access to Copilot’s technical preview, TabNine may be a good alternative.

One thing to note is that these autocompletion systems must run on your machine (at least their free versions), which means they’re much more resource-consuming in your system than GitHub Copilot. Remember that Copilot brings all the code suggestions from a remote server.

Conclusion

It’s quite impressive to see an AI-powered tool generate code. GitHub’s intention isn’t to replace programmers, but to help them elevate their productivity while coding, especially with repetitive coding tasks, like writing good docstrings in functions or classes.

After using Copilot for a while, I noticed some problems, but overall it provides good (yet not perfect) code suggestions. I tested solutions for some popular code problems, and I’m quite happy with the results. Taking this into account, I think Copilot shouldn’t be used by complete beginners, for the reasons I explained above.

The project is quite fresh, so it’s not the ideal partner when building a serious project, but in the future, it may well emerge as one of the most-used coding tools.

Maybe the “AI pair programming” statement isn’t a reality yet, but I’m sure it will be in the future.

Finally, I want you to know that Copilot is helping me to write this conclusion.