Generative AI: What does it mean for software development?

Computing's research finds IT leaders positive about the new tools, which are already starting to have a profound effect

The arrival of generative AI in the public consciousness has been met by a mix of excitement, cynicism and trepidation. What do IT leaders make of it?

Aside from students who no longer have to choose between partying all night and turning in a 2,000-word essay, among the most enthusiastic about these new tools have been software developers.

Writing boilerplate code, documentation, discovery and even suggesting whole code blocks are all tasks that AI can tackle competently when provided with well-crafted prompts, making coding one of the more clear-cut use cases for the technology.

What's more, coders adopt new tools frequently and as a matter of course, and unlike in some other domains, automation is very much part of innovation.

In addition, for developers AI has not arrived as a bolt from the blue with ChatGPT, Bard et al. IDE vendors have been adding automation features forever, from simple autocomplete and syntax highlighting to sophisticated code completion and interactive suggestions, so in some ways the latest developments are just another option. Moreover, AI-enhanced tools like GitHub Copilot had already been available for a year before ChatGPT burst onto the scene.

Having said that, many developers we speak to really do see this as a step change. Being able to post questions in natural language and refine prompts based on the answers is certainly something new, as is the wide range of topics the latest AI models can tackle thanks to their vast training datasets.

More pros than cons

As part of our latest research into DevOps, we asked 150 UK IT leaders working in software-producing organisations what they thought about the new developments.

19% of the cohort was already using AI-enhanced development tools, the most popular being GitHub Copilot and ChatGPT. Amazon CodeWhisperer and Intellicode were among others mentioned.

The general sentiment among users and non-users alike was enthusiasm, as indicated by responses to a range of common statements about AI-enhanced tools, both positive and negative.

The five most-picked statements were all positive. They concerned the usefulness of AI-assisted developer tools for discovery; speeding up coding; implementing boilerplate code; documentation and comments; and testing.

After this, however, a few worries started to seep into the responses, namely concerns about plagiarism and copyright; encouragement of bad coding habits; lack of integration with other tools; difficulty debugging and checking the correctness of the AIs' suggestions; and cost.

Those currently using the tools were more likely to see them as positive. They were also more likely to mention more niche uses, such as translating code into other languages and onboarding new developers. But for most it was all about automating repetitive tasks.

"A major advantage is code completion and boilerplate code because it almost always predicts that correctly," said a user of Intellicode in an IT company.

However, useful as these tools are, they are still rough around the edges and need to be used with care. Some respondents were suspicious about the hype surrounding the new models, and worried about the hallucinations of LLMs, which are known to "invent" plausible-sounding answers and make mistakes such as importing libraries with non-existing version numbers.

Others were worried about security. How much the tools learn from the work they are processing and where might confidential code end up?

"They're not secure enough as our code is of interest to competitors," commented a technical manager at an engineering firm.

Change is coming

Indeed, security, confidentiality and accuracy all need to be tightened up before these tools will gain the trust of some respondents. However, there's no doubt as to the direction of travel and the rapid rise of the capabilities of these tools.

GPT-4 — which went public after the survey was sent — can reportedly already write more than 500 lines of code and is far more precise and accurate than ChatGPT. It also outperformed all human coders on Amazon and Google online challenges. GPT-4 is making its way into GitHub Copilot.

Google's own Bard does not yet support code, but the company says that goal is very much in its sights.

There is little doubt that AI will change the business of coding, probably in a profound way. The pressures to cut costs and improve productivity, mean that for the more routine parts of the profession, including those aspects that tend to be outsourced will see cuts, possibly extensive ones. It will also make the more skilled engineering-oriented roles more productive — hugely so potentially — and if history is any guide will create new specialisms and careers in its wake. "Prompt engineering" is one such role that's already being mooted.

Because there's a lot more to producing software than generating lumps of code.

People who understand the nuances of design, the shifting demands of customers and markets, and the importance of maintaining the code will be essential for the foreseeable future — if only to hone code that's been written by machines.