Can ChatGPT Help me code?
Everywhere you turn around at the moment there is some sort of AI disrupting industry after industry. Today's flavour 'ChatGPT' is making the rounds.
So what problem can we throw at it?
If you have been following me on twTwitterou may have seen that during Hacktoberfest last year, I started up a new open source project that aims to be a free open API that provides mocked data to be used when building out applications.
We have been implementing endpoint after endpoint based on tickets raised on our GitHub repository. One of which below is asking for us to make an endpoint that would return a random success or error code based on weights provided.
ageddesiSo let's see what ChatGPT knows about maths and node express apps with the Aim of generating code we will actually pull into the codebase.
First things first, I Ask the major question.
I started off by trying to digest the ticket raised into one sentence that I thought would be a good basis to build from and wanted to try to ask the questions as if I was giving this to a Junior developer for them to work on or even as an interview question.
Here is the question which ended up being the first prompt to the library.
How do I make an express node endpoint that provides a random success of fail code based on a weighted randomness supplied in the request body
ChatGPT Response:
To create an endpoint in an Express app that returns a random success or fail code based on a weighted randomness value supplied in the request body, you can follow these steps:
First, install the chance npm package, which provides utility functions for generating random data. You can do this by running the following command:
npm install chanceIn your Express app, require the chance package and create a new router for your endpoint:
const express = require('express');
const chance = require('chance').Chance();
const router = express.Router();
Define the endpoint route and handler function. In the handler function, use the chance.weighted method to generate a random value based on the weight supplied in the request body. For example:
router.post('/random', (req, res) => {
const { weight } = req.body;
const result = chance.weighted([true, false], [weight, 100 - weight]);
if (result) {
res.send({ success: true });
} else {
res.send({ success: false });
}
});
Make sure to parse the request body in your Express app before the router is mounted. You can do this using the body-parser middleware:
const bodyParser = require('body-parser');
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use('/api', router);
Now, when you make a POST request to the /random endpoint with a request body that includes a weight property, you will receive a random success or fail code based on the weight value provided. For example:
POST /api/random
{
"weight": 80
}
RESPONSE
{
"success": true
}Let's get more complicated
Well, colour me impressed that is a decent start. I have seen even some juniors struggle to give that good of a go in the first run at it. It has recommended a third-party plugin to help with the maths and clearly shows returning a result based on the weight provided in the request.
So let's see if we can up the ante a bit. So we get a weighted response based on the weight we requested from the body. However, in the ticket, it was raised based on a specific model type being provided. So let's add another question providing that example as an input to build against.
How do I update the weight to return based on this modal
{ "successes": { 200: 50 }, "failures": { 400: 25, 403: 10, 404: 15 } }
ChatGPT Response:
To update the endpoint to return a success or failure code based on the weights specified in the model you provided, you can follow these steps:
In the handler function, first extract the successes and failures objects from the request body:
router.post('/random', (req, res) => {
const { successes, failures } = req.body;
Use the Object.keys method to get an array of the keys in the successes and failures objects. These will be the codes that the endpoint can return.
const successCodes = Object.keys(successes);
const failureCodes = Object.keys(failures);
Use the Object.values method to get an array of the values in the successes and failures objects. These will be the weights for each code.
const successWeights = Object.values(successes);
const failureWeights = Object.values(failures);
Use the chance.weighted method to generate a random value based on the weights for the success codes, and another for the failure codes:
const successResult = chance.weighted(successCodes, successWeights);
const failureResult = chance.weighted(failureCodes, failureWeights);
Use an if statement to determine whether to return a success or failure code based on the random results generated. You can also use the Number constructor to convert the code strings to numbers before sending the response:
if (successResult) {
res.send({ code: Number(successResult) });
} else {
res.send({ code: Number(failureResult) });
}
});
Now, when you make a POST request to the /random endpoint with a request body in the format you provided, you will receive a random success or failure code based on the weights specified in the model.
Let's remove the need for external packages
Now we're starting to take shape. We are taking into consideration of the status's that have been imported and their weights attached to them. This is utilising the external package we pulled in before.
I would like however to see if ChatGPT can do this without importing an external package as this will add potential bloat to the application.
Can I do the same code without needing to import chance
ChatGPT Response:
Yes, you can generate random values based on weights without using the chance npm package. One way to do this is to use the JavaScript Math.random function to generate a random number between 0 and 1, and then use a series of if statements to determine the result based on the weights specified in the model.
Here's an example of how you could update the endpoint to do this:
router.post('/random', (req, res) => {
const { successes, failures } = req.body;
const successCodes = Object.keys(successes);
const failureCodes = Object.keys(failures);
const successWeights = Object.values(successes);
const failureWeights = Object.values(failures);
let successWeightSum = 0;
for (const weight of successWeights) {
successWeightSum += weight;
}
let failureWeightSum = 0;
for (const weight of failureWeights) {
failureWeightSum += weight;
}
const randomValue = Math.random();
let result;
if (randomValue < successWeightSum / (successWeightSum + failureWeightSum)) {
let successWeightTotal = 0;
for (let i = 0; i < successCodes.length; i++) {
successWeightTotal += successWeights[i];
if (randomValue < successWeightTotal / successWeightSum) {
result = Number(successCodes[i]);
break;
}
}
} else {
let failureWeightTotal = 0;
for (let i = 0; i < failureCodes.length; i++) {
failureWeightTotal += failureWeights[i];
if (randomValue < (successWeightSum + failureWeightTotal) / (successWeightSum + failureWeightSum)) {
result = Number(failureCodes[i]);
break;
}
}
}
res.send({ code: result });
});
This code generates a random value between 0 and 1 using Math.random, and then uses a series of if statements to determine the success or failure code based on the weights specified in the model. It first checks whether the random value is less than the total weight of the success codes divided by the total weight of all codes. If it is, it iterates through the success codes and weights, adding up the weights and checking whether the random value is less than the current weight total divided by the total weight of the success codes. If it is, it sets the result to the current success code and breaks out of the loop. If the random value is not less than the total weight of the success codes divided by the total weight of all codes, it does the same thing for the failure codes.
You know what I think we need some tests
Damn AI, you got some skills. Obviously, there is some repeated code in here that I would indeed refactor and frankly, I forgot to ask the AI if it could do that lol. So maybe next time I will check if it can identify the duplicated code.
Next up though as with all code that makes it to this project. I need some tests. I am no good at Math either so how can I validate that what ChatGPT has produced will work? Let's give it a simple request with the project's test framework of choice and see how it fairs.
Can you write me Jest unit tests for this endpoint, please?
ChatGPT Response:
Sure! Here are some Jest unit tests that you can use to test the endpoint that returns a random success or failure code based on weights:
const request = require('supertest');
const app = require('../app');
describe('POST /random', () => {
it('should return a 200 code with weight 50', async () => {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 50
},
failures: {
400: 25,
403: 10,
404: 15
}
});
expect(res.statusCode).toEqual(200);
expect(res.body.code).toEqual(200);
});
it('should return a 400 code with weight 25', async () => {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 50
},
failures: {
400: 25,
403: 10,
404: 15
}
});
expect(res.statusCode).toEqual(200);
expect(res.body.code).toEqual(400);
});
it('should return a 403 code with weight 10', async () => {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 50
},
failures: {
400: 25,
403: 10,
404: 15
}
});
expect(res.statusCode).toEqual(200);
expect(res.body.code).toEqual(403);
});
it('should return a 404 code with weight 15', async () => {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 50
},
failures: {
400: 25,
403: 10,
404: 15
}
});
expect(res.statusCode).toEqual(200);
expect(res.body.code).toEqual(404);
});
});
These tests send a POST request to the /random endpoint with the model you provided, and then use the expect function to assert that the response status code is 200 and the code property in the response body is one of the codes specified in the model.
A problem with the tests!
OK any experience coder is going to see the major issue with the tests here. But for the sake of the experiment I am going to play dumb and inform it that the tests randomly fail based on the result being returned. What say you AI?
The expect sometimes returns the wrong code why is this and how do I fix it ?
ChatGPT Response:
The reason the tests are sometimes returning the wrong code is because they are all testing the same endpoint with the same request body, but the endpoint is generating a random code based on the weights specified in the model. This means that the tests are not deterministic, and the results of the tests will vary depending on the random values generated.
To fix this, you can update the tests to send different request bodies to the endpoint, each with different weights for the success and failure codes. This will allow you to test the different code paths in the endpoint more reliably.
For example, you could update the tests to the following:
const request = require('supertest');
const app = require('../app');
describe('POST /random', () => {
it('should return a 200 code with weight 50', async () => {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 50
},
failures: {
400: 50
}
});
expect(res.statusCode).toEqual(200);
expect(res.body.code).toEqual(200);
});
it('should return a 400 code with weight 50', async () => {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 50
},
failures: {
400: 50
}
});
expect(res.statusCode).toEqual(200);
expect(res.body.code).toEqual(400);
});
it('should return a 200 code with weight 75', async () => {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 75
},
failures: {
400: 25
}
});
expect(res.statusCode).toEqual(200);
expect(res.body.code).toEqual(200);
});
it('should return a 400 code with weight 25', async () => {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 75
},
failures: {
400: 25
}
});
expect(res.statusCode).toEqual(200);
expect(res.body.code).toEqual(400);
});
});
With these updated tests, you can test the different code paths in the endpoint more reliably, as the request bodies are different and the weights for the success and failure codes are different.
Can we test the weights?
Hmmm, so it has given the reason why these tests would not work correctly. Did you pick up on that word 'deterministic'. Well let's try rephrasing the question and see what it comes back with this time.
Can I make the tests check the weighted results more accurately
ChatGPT Response:
Yes, you can make the tests more accurate by adding more tests and increasing the number of requests made to the endpoint. This will allow you to test the different code paths in the endpoint more thoroughly and ensure that the weighted randomness is working correctly.
One way to do this is to use a loop to make multiple requests to the endpoint with the same request body, and then use the expect function to assert that the correct number of each code is returned.
For example, you could update the tests to the following:
const request = require('supertest');
const app = require('../app');
describe('POST /random', () => {
it('should return a 200 code with weight 50', async () => {
const numRequests = 100;
let numSuccesses = 0;
let numFailures = 0;
for (let i = 0; i < numRequests; i++) {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 50
},
failures: {
400: 50
}
});
if (res.body.code === 200) {
numSuccesses++;
} else {
numFailures++;
}
}
expect(numSuccesses).toBeGreaterThanOrEqual(numRequests / 2 - numRequests / 10);
expect(numSuccesses).toBeLessThanOrEqual(numRequests / 2 + numRequests / 10);
expect(numFailures).toBeGreaterThanOrEqual(numRequests / 2 - numRequests / 10);
expect(numFailures).toBeLessThanOrEqual(numRequests / 2 + numRequests / 10);
});
it('should return a 400 code with weight 50', async () => {
const numRequests = 100;
let numSuccesses = 0;
let numFailures = 0;
for (let i = 0; i < numRequests; i++) {
const res = await request(app)
.post('/api/random')
.send({
successes: {
200: 50
},
failures: {
400: 50
}
});
if (res.body.code === 200) {
numSuccesses++;
} else {
numFailures++;
}
}
expect(numSuccesses).toBeGreaterThanOrEqual(numRequests / 2 - numRequests / 10);
expect(numSuccesses).toBeLessThanOrEqual(numRequests / 2 + numRequests / 10);
expect(numFailures).toBeGreaterThanOrEqual(numRequests / 2 - numRequests / 10);
expect(numFailures).toBeLessThanOrEqual(numRequests / 2 + numRequests / 10);
});
it('should return a 200 code with weight 75', async () => {
const numRequests = 100;
let numSuccesses = 0;
let numFailures = 0;
for (let i = 0; i < numRequests; i++) {
const res = await request(app)
.post
So where does that leave us?
Do I think that Chat GPT is ready to take over a Juniors role and allow Senior Developers to just pump it questions and get fully-fledged code answer?
The short answer is NO. You only have to take a few moments looking through the code it produces to notice code that the code can be cleaned up a lot and there are more efficient ways of doing stuff than it recommends a lot of the time. This means I will not be using the code provided during this experiment and will not be putting it into our codebase. However, as I am rubbish at Math. It has given me a look into how the endpoint could be coded and will be working on it over the next week or so when I have time.
However! I do think this tool is going to be useful as an amalgamation tool for google search results when you are trying to understand some problem you are having. I have also found it very useful to debug complicated or minified code.
So personally I can not wait to see where this goes and how I can continue to use it as an extension to my current daily toolset.
What are your current thoughts on ChatGPT in dev? let me know in the comments below.