Proceed with caution when using ChatGPT for production code

Written by

Walter G.

Published on

May 30 2023

Modified on

May 31 2023

Filed under

artificial-intelligence

There is alot of noise these days when it comes to the forthcoming A.I. apocalypse. Developer's are worried that a chatbot is going to take their jobs soon and everybody else is probably unaware of what a chatbot is. If you're reading this, then you're probably in the former camp.

I personally don't share any of those fears, at least not just yet. I love what ChatGPT is capable of doing right now, and I use regularly in my day to day development work. But having been a developer for the past 20 years, it's pretty obvious to me where it falls short. Mainly in being left to its own devices to do as it wishes. It's just not quite there yet.

And I'm not knocking it in any way, because it just isn't designed to be a developer (at least not yet). But before I get to the list of challenges that it faces, let's take a look at what ChatGPT can get right. Because there's a fair amount that the little A.I. engine that could can do.

Simple utility functions

I tend to write alot of utility functions in my day to day life. They can range from data validation to data formatting to creating React components that achieve some visual effect. Think custom time select as an example or some kind of calendar widget.

Just recently I needed to generate a function that formatted a date in its 'time ago' equivalent, like you would see on many blog sites. And while I could write one from scratch, or scour through my large codebase looking for a previous version that I wrote, or even look up this article that I wrote about it, I decided to ask ChatGPT to do it for me. And the result was great as it did exactly what I asked it too generating the following:

    function getTimeAgo(datePosted) {
  const MINUTE = 60 * 1000; // milliseconds in a minute
  const HOUR = 60 * MINUTE; // milliseconds in an hour
  const DAY = 24 * HOUR; // milliseconds in a day

  const currentTime = new Date().getTime(); // current timestamp
  const postedTime = new Date(datePosted).getTime(); // posted timestamp

  const timeDifference = currentTime - postedTime; // difference in milliseconds

  if (timeDifference < MINUTE) {
    const seconds = Math.floor(timeDifference / 1000);
    return `${seconds} seconds ago`;
  } else if (timeDifference < HOUR) {
    const minutes = Math.floor(timeDifference / MINUTE);
    return `${minutes} minutes ago`;
  } else if (timeDifference < DAY) {
    const hours = Math.floor(timeDifference / HOUR);
    return `${hours} hours ago`;
  } else {
    const days = Math.floor(timeDifference / DAY);
    return `${days} days ago`;
  }
}

A quick glance at the code and you can tell that it's pretty much on point with the query. It's not the shortest function, or the most optimal, and it doesn't account for plural versus singular values, but it kind of sort of works.

This is where ChatGPT shines pretty well as far as development goes. Because these functions aren't at all difficult to analyze and parse and there are probably thousands of versions of this exact code scattered across the web.

But more importantly, they aren't vital in any technical sense. They could provide a poor UI/UX experience to the user if it's wrong, but it won't cause any business related damage to a website.

But here's another query that I recently asked on a topic that's a bit nuanced and somewhat difficult to get an answer to.

I, and probably yourself as well, dislike working with timezones in any programming language. And unless you're using a library, getting a clean list of all valid timezone codes that are currently in use around the globe is somewhat challenging.

Not only are you dealing with different global regions, but you might also have to account for DST versus non-DST offsets.

ChatGPT also had trouble with that request, and here's how it responded:

function getAllTimezones() {
  const timezones = [];

  // Get all available timezones
  const dtf = Intl.DateTimeFormat(undefined, { timeZone: 'UTC' });
  const timeZoneOptions = dtf.resolvedOptions().timeZone;
  
  // Split timezones by '/' and push the key name to the array
  timeZoneOptions.split('/').forEach((zone) => {
    const parts = zone.split('_');
    if (parts.length > 1) {
      timezones.push(parts[1]);
    } else {
      timezones.push(parts[0]);
    }
  });

  return timezones;
}

Just eyeballing the code, you can probably tell that something is a bit off. Not just a bit off. The function is wrong essentially.

It only generates a single timezone entry of 'UTC', and then attempts to split a string with a character that appears nowhere in the string, only to return that original single value.

I obviously didn't use the code. But it did get me thinking. If a new junior developer, up and coming in the world, were to stumble on this same situation, I don't know that they would catch the issue. They might just copy and paste and continue forward on their way to the next function.

And this might be more detrimental to a website than the above 'time ago' feature. You might be working with some kind of scheduling feature and having this level of error could potentially cause the entire application to not work.

Of course a big company with proper code reviews in place would catch these issues without a problem. At least in small amounts. But I would imagine that if the use of coding bots became the norm with hit or miss functionality, it will undoubtedly affect the quality of software being released for everyone in the future.

I will say this though. After some questioning and further training, ChatGPT was able to provide a much better answer. It essentially just told me to use the moment.js library to retrieve all timezones. Fair enough.

And this isn't anything new. Anytime there's automation in any industry, you're allowing for some percentage of known automation errors. And error correcting code for A.I. isn't quite there just yet.

Until we have that in place, I would say that you should absolutely continue to use ChatGPT (or whichever bot you use) for help with your daily coding needs. But proceed with caution if you're going to be using this code in a production environment.

Always double check your code, run the appropriate tests and use your best logical judgement.

Comments

No messages posted yet

Add comment