We enlisted the help of Copilot AI and asked about user experiences and the results were promising. Nine out of ten Crasmani's Creative Nerds want to continue using it. The increase in work productivity and improved quality also serve our clients.

7 March 2023 was the day when GitHub Copilot for Business AI became part of all our developers' toolkit at Crasman. Copilot is an AI-based virtual “programming buddy” that provides suggestions and assists developers in writing code faster.

The Copilot, developed by OpenAI and integrated into the developer's code editor, is a generative pretrained OpenAI Codex language model. It should be noted right from the start that GitHub does not store data processed by the Copilot for Business version, nor is it used for AI training.

About a month after trialing Copilot, we conducted an anonymous survey with our developers to investigate experiences with AI and its impact on developers' daily lives.

Overall, the AI survey results reinforce the understanding that AI is a fundamental change in developers' daily lives, and there is no return to the past. 68% of the developers who responded to the survey use Copilot as an aid every day.

The amount of interesting and quality work has increased

• 75% of respondents feel more productive

• 69% of respondents can focus more on doing interesting work

• 41% of respondents feel the quality of their programming work has improved

• Generally, Copilot has reduced frustration and increased the meaningfulness of work

And perhaps the most crucial question - do you want to continue using Copilot?

What problems or concerns arose during the Copilot trial?

The AI based on a language model is not infallible, and sometimes Copilot's code suggestions do not meet the need.

We also noticed that Copilot works better with certain programming languages than others. For instance, Typescript is handled much more smoothly compared to PHP, where code suggestions often contained errors or did not function.

The developer must always critically evaluate the code suggestions made by Copilot. There is no change in that regard, as the developer remains responsible for the program code they produce, even if it is generated with Copilot's assistance.

Copilot is currently available only for a limited set of code editors, such as Visual Studio Code. If you are accustomed to using a particular code editor, it is possible that Copilot is not available for it.

Copyright issues related to program code have raised concerns within the international developer community, as Copilot has been trained with source code available from public sources.

Developers may find it difficult to be certain whether a code snippet suggested by Copilot is license-free or carries the risk of including licensable parts.

• GitHub has responded to these concerns by increasing Copilot's transparency. In future, Copilot will indicate how many times a suggested code snippet appears in GitHub's code repositories, assisting developers in evaluating the eligibility of the code snippet.

• Copilot's code suggestions can be limited so that it does not present options available in the same form across multiple GitHub public code repositories, which are most likely to fall under an open source license.

• Short code snippets are not automatically copyright-protected, but they must be sufficiently autonomous and original to receive copyright protection (the so-called threshold of originality).

Copilot accelerated work by up to half

Based on these results, it's clear that the majority of developers have warmly welcomed Copilot AI, and it has increased the joy in their work. One developer mentioned in the open feedback of the survey that they would go to the barricades if the use of Copilot did not continue after the pilot phase.

Copyright issues require precision, but the same issues arise even when picking code examples from programming-related discussion boards to support one's work.

Furthermore, GitHub has investigated the impact of Copilot on work productivity and developer satisfaction. In the study, the increase in productivity was measured by giving the same programming task to developers divided into two different groups.

The developers in the Copilot-using group completed the task on average 55% faster than the developers in the control group. GitHub's research results were very much in line with the feedback collected from Crasman's developer community. 

Increased work productivity and improved quality are factors that also serve our clients. And not to mention the added value that comes from a satisfied, smiling creative nerd.

Crasman Insight follows the arrival and impact of AI on digital commerce. Explore previously published insights.

Mikko Tikkanen, Crasman's Chief Technology Officer, is a challenger of technologies, tools, and processes and a continuous developer of business operations.