GitHub Copilot
Introduction
With the hype of AI, I’ve felt the need to have a gander & give it a try myself, granted, some of the tools out there are pretty awesome. But I thought I’d try out GitHub Copilot since this is an AI tool that’s meant to help improve the experience when writing software.
My honest opinion is that it simply didn’t come close to what I expected. Honestly, I was quite disappointed. But it also reinforces my theory that software engineers aren’t going to be replaced anytime soon. One thing that it seriously struggled with was context, it would make suggestions that kinda made sense. However, it didn’t make sense in the context of the code that I was writing.
Architecture
To clarify, I decided to throw it into the deep end, the reality of the work that many software developers have to deal with on the daily basis. Sadly, not many of us get the luxury of writing very simplistic scripts or small & fun websites. Most of us do have to deal with some large scale enterprise systems. Due to the nature of large scale systems, there’s a lot of business logic that takes place behind the scenes. And there’s a lot of architectural decisions that need to be made due to many, many factors, from cost through to security.
An example might be the use of the outbox pattern, where you have two systems working in parallel. One is essentially the ‘main’ application, which you want to be lightning fast, you want it to provide an excellent user experience. The other, you don’t care so much about, provided it’s stable, reliable & gets the job done, it’s happy to slowly but surely plod along, completing long running tasks behind the scenes. It might be that you’re an SME & you can’t quite justify the likes of event driven architecture or the use of microservices, so you might need a simple PHP web application that uses some cron jobs behind the scenes.
That’s a relatively simplistic, low cost & low risk solution for a large number of small companies that don’t have a big budget. But it’s subject matters like that where I’ve found AI tools to quickly fall over. In my experience, they’ve just spout the ‘hype’ that one might read online, using references to the big 3 cloud providers & all this fancy microservice oriented architecture.
Strictly Coding
But even if we give the likes of GitHub Copilot the benefit of the reality that the subject matter above somewhat goes beyond the realm of just writing code. Even if you throw Copilot into a very large codebase, where you’re working with complex result objects, using concepts such as railway programming, etc. It might make some valid suggestions here & there, but in reality it’s nothing you’ve not already thought about. At best, it might help save you some time with typing, but even then, I’ve found it to make some suggestions whereby I’ve had to completely refactor or even remove parts of the suggested code.
It would not consider business rules that are deeply nested in other classes, even using patterns that a software developer would have no trouble in understanding such as CQRS. I simply found Copilot didn’t consider a great deal about the code that was potentially nested within the stated classes.
Conclusion
While it’s fun to tinker with new toys, such as GitHub Copilot, it just left me feeling pretty disappointed. Considering all the hype you hear online, you’d expect it to be a pretty capable tool, but sadly, I found it less helpful than a very inexperienced junior software developer. This has just reinforced my theory that software engineers are safe for the time being, it’s going to be a while before we have to worry about our jobs being replaced by bots & AI driven tooling.
Now one thing that I will give AI tools credit for is things like helping developers find vulnerabilities within the code base, in addition to help improve performance of the applications we build. I’ve recently heard how some AI tool out in the wild helped improve a sorting algorithm quite substantially, which is awesome. But it’s things like that, where there’s a very limited amount of context, AI is fantastic. But as far as I’m aware, at this point in time, context is a killer for AI, granted, this may not always be the case.