In October 2023, I held a workshop at WordCamp Germany about AI tools one could use to help with coding. While we talked about and tested various tools, it was clear that ChatGPT and GitHub CoPilot were our favorites.

Just three months later, I gave a small presentation on the same topic and had to review my opinion based on new tools that had emerged since then.

I am finding myself in a situation where I no longer have a clear favorite. Surely, I can start reviewing this list in another three months to see the tools evolve.

ChatGPT, Claude, and Bard

I will only quickly cover the AI chat tool section. Three months ago, ChatGPT was a clear winner regarding quality and consistency when it came to code-related questions.

I feel that Anthropic’s Claude and Google’s Bard got closer. There are still differences in how they approach solutions, but these are often nuances, and deciding which one is better becomes more subjective.

I recently found myself in a situation where Bard was the least hallucinating tool for non-coding tasks.

While ChatGPT remains my default choice as an AI chat and brainstorming tool, I can also recommend the others.

One thing I should spend more time on is custom GPTs. So far, I have only created one to help me review WordPress code.

CoPilot

GitHub’s CoPilot is my default choice regarding code completion within my IDE (PHPStorm by JetBrains).

I have been a happy customer of CoPilot from the beginning. It is surprisingly good for understanding my code style and naming conventions for its suggestions and saves me a lot of time with mundane tasks.

I am missing CoPilot Chat, a feature only available in Visual Studio Code, which, as GitHub and CoPilot, belongs to Microsoft.

I am on the waitlist for this integration into PHPStorm. The frustration about this is why I look for AI coding tools that offer more within my favorite IDE.

JetBrains AI

Just last week, I was given access to JetBrains AI. Since they are the developer of PHPStorm, I expected great things.

One can access many options by right-clicking anywhere in the code.

Some of these features open a chat, some write code where you are or in a new window.

So far, I haven’t gotten code completion to work, as I am used to with CoPilot. The code generation I have seen so far didn’t blow my mind. There were some really weird results.

Generating tests didn’t work at all.

I also had to activate AI every time I started PHPStorm. I only tested writing commit messages once since I usually do that in the Terminal. It seemed to work, though a bit wordy.

I will keep an eye on JetBrains AI since I expect them to figure out the best integration of AI into their tools.

Cody by Sourcegraph

An AI coding tool I have already presented in October was Cody by Sourcegraph.

It integrates code completion and chat into PHPStorm.

Both worked decent enough. The setup is clunky, though.

Cody needs to index my repository, which only worked with a public one on GitHub. Doing that was far from straightforward, and I don’t remember how I eventually did it.

Three months later, I can no longer find an option to connect the repo, and there is no trace of that in their docs.

Their support eventually told me I needed to request indexing via their Discord (!).

After doing that, Cody’s chat still occasionally tells me that it doesn’t know my code. Even when selecting the code I want to interact with, then right-clicking on it and selecting an option from the Cody menu fails to complete my task.

The frustrating thing is that it previously worked, and I had enough hopes for this tool to recommend it.

Now, I will wait for them to figure out the quicks and hopefully improve their docs. With chat working with JetBrains AI and code completion from CoPilot, Cody might be the first AI coding tool to ditch, even though it is entirely free for now.

CodiumAI

The last tool I tested was CodiumAI.

It is not a code completion tool or chat. Instead, it focuses on testing and reviewing code.

Within PHPStorm, I can ask it to explain, test, and make suggestions for JavaScript or Python code—unfortunately, not PHP.

I like that it does that in a structured interface instead of a loose chat format.

There should also be a PR review integration, but so far, I only got an error in PHPStorm that there is no git repository – which isn’t true. Unfortunately, I didn’t receive support for this issue yet.

I am very eager to get PR reviews to work within PHPStorm. One can ask CodiumAI to do a PR review on a public GitHub repository, which I tried out here.

Suggestions like these would be great while writing code, not only after committing them.

Conclusions

The choice of the proper coding tools became tougher. Ideally, one of them will directly combine all the features in a pleasant UI in PHPStorm. Given the current development speed in this area, it should only be a matter of months.

Regarding code completion, it will be hard to match CoPilot since they have a lot of material to take from through GitHub.

JetBrains has the chance to build the best integration into their software.

CodiumAI could have a chance if they focused on the use case of tests and reviews instead of being an overall coding tool and extended support for PHP and PHPStorm.