ChatGPT writing code, some experiments.
Not going to replace programmers anytime soon.
I have been investigating the code generation process by ChatGPT by running experiments.
Summary
In the current implementation it is useful for simple problems. ChatGPT can also find published libraries (GEMs for example) or utility programs easier than using other mechanisms. The major limitations are that the code is not production ready without guard and at limits. It frankly fails at more complex code for a published interface.
It is a useful tool but does not replace professional programmers.
Experiments
The first experiment was to handle the logic to implement a published interface. This seemed like a good test since the language for the interface is restrictive.
The interface I choose was packet extraction from the CCSDS Packet Telemetry definition (standard for deep space communication). Packet extraction is not trivial if you implement all of the required capability. This is code I know well in that I have created the logic in multiple languages in my work supporting deep space spacecraft.
The code generated was the minimum subset of the problem and ignored all of the more complex aspects. It would…