Transcription
Writing tests isn't usually a developer's favorite part of the job. But what if I could write my test by just describing what I want in plain English or a natural language? Well, in today's video, I'm going to show you how to use prompt driven development to build out and hopefully create a complete working test suite for a web application which I have created using Golang. Let's jump into it and see how far we can get.
So before we jump into the prompt driven development and start to use GitHub Copilot to create some really cool tests for this application, I thought it'd be good to actually run through and show exactly what this application is doing. So as with many weather applications, you're going to have a user request that comes in. We're using Go. So we're going to jump into the main.go file, which is the main sort of parallel file for a lot of this application. That's going to call the config file. So load the API key, general configuration, everything we need. So just over here we call this file grab everything we need. From there we're going to go and create a weather client. And that weather client is then going to determine whether we are using a UI or whether we're using a CLI based weather app at this point. Then it's going to go and grab the data with an API call from the open weather map API and return that in a JSON format. So that's going to give us things like the humidity and the temperature and the location and all those kind of things. So that's on the face of it what is happening.
Now, for you to see what this actually looks like, I've got it open in simple browser in VS Code. If I want to know what the weather in London is like right now, I will type in London and click search. 16° C. Pretty simple. But now, what we need to do is use some prompts to go and generate some tests and give us a real good foundation to work on as a developer.
So, prompt driven development is all about guiding an LLM or an AI assistant to help create purposeful code based on descriptions. So rather than me writing every line of code, I'm writing natural language and having the AI assistant write that for me, but I'm guiding it. And there's a lot of two and throw. So let me show you how to do this or what I'm going to do. So first of all, I'm in VS Code and I'm going to open up a chat window with GitHub Copilot. So my Asia mode over here. Going to make sure that is in Asia mode and I'm going to choose I'm going to leave it on Claude Sonic 4.5. Let's let's have a bit of fun. So the prompt I'm actually going to use is create a blank set of test files for all of the files I need to write unit test for in this application. Now prior to writing this I've also got an agent.mmd file which is outlining everything I needed to know about the test framework that I want to create. So I've outlined the testing pyramid. So the end to end test integration unit tests etc. I've even outlined exactly how I want these tests to sort of be written. So the standard test format, I'm going to have the function name. I'm then going to arrange it. So I'm going to have the action, the execution, and then the assertion and verifying the results. Very typical to what you would do in a lot of testing. Uh so I'm really giving it the understanding and the foundation of what I expect it to do. So this is another means of guidance.
So let's go ahead and run this command. So I'm going to say create a blank set of test files for all the files I need to write unit test for in this application. And hopefully it's going to take in as reference my agents.mmd file and start to write up exactly what it thinks I need to do. So I'm not asking it to write test at the moment. I'm just guiding it and trying to get it to write or create these files for me. The ones which I know are in those I need to write tests for.
Awesome. So now we can see that it has finished writing or creating these blank test files for me. It's given a summary of what it's done. So you can see it's created config test, server test, cli test, etc. And it's all within my directory. But if I was to go and look into one of those. So let's take the server test.go. You can see there's a lot of to-dos. It's asking or told me that this is a test I need to write. Test for a new server. We've also got test handle whether API success. And I'm sure there will be probably a invalid one as well somewhere in here. But you can see it's gone ahead and created these test files for me. It's given me a great foundation to now start building off of. And I'm going to cherry pick a couple of these tests that I want to write with Copilot in just a moment. But let's go and dig a little bit deeper and see if there's any other things in here which is done. It all looks pretty standard and there's a few skips in there as well. So uh we can see that they're not implemented yet. So once they're implemented, we can then unskip that test. This is very normal uh behavior from any developer. We'll always see people skipping tests. Copilot is absolutely uh no different here. I'm going to go ahead and keep these cuz these look good to me.
So, our next challenge is to actually go and write some tests now because if I was to go and run this and sort of compile this test suite, everything's of course going to pass. Nothing's going to fail. They're all to-dos. They're all skipped. There's nothing really there. They're just existing in a file. So actually what we need to do is go and create them. Now I think if we go and look at the weather test.go this is actually where we're making the API request the open weather map. So this is where we're going to fetch the data and get it back. So I think it's a really good example to see if we can write some tests that maybe include some mocking. So in my agents.md file I actually have a mocking strategy which I've written out. and a mock client is going to allow us to not have the server running at the time of the test suite being run. So the test so so the server doesn't need to be running for the test to be passing and for the test to get mock data. So this is essentially what we're doing here. So I've written out a mocking strategy and this is probably going to be a really good way to test this and see how far we can get with copilot.
Now I'm actually going to write the prompt make me some uh unit test for the weather API. Very vague but pretty straightforward because I've got so much context around it. I don't need to write really elaborate prompts. use the mocking strategy I have outlined to mock any data needed for a rest request. So, this is just going to really double down on the fact that I want it to mock the data as opposed to try and spin up a test which starts a client and a server and all sorts of stuff that goes along with it just for the purpose of a test. So, let's go ahead and see if we can get this to write some tests for us. Hopefully, it should pick up the fact that I'm in the weather test.go rather than trying to create a big elaborate suite of tests. Uh, but we'll see. I was pretty vague with this prompt and and let's just see how far we can get with it.
This is great. This has picked up everything I needed it to. And I was super vague with my prompt, but I'm trying to guide it using lots of different contexts from both the chat window and from my agent. file. And what it's managed to do is it's managed to pick up the fact that I only want to test in the weather test.go file. No other file has been touched. It has actually gone off and created a mock server for us. Here we can see that it has created this mock server http test new server. It's kind of contained within itself. Uh, and it's giving us back a lot of mock data for us, which is brilliant. This is exactly what I wanted. It looks like it has got no compilation errors in terms of the syntax. So, we're going to find out and see what's going on in terms of the test. They kind of look like they're all somewhat written. They're all pulling in the mock server, which is great. Uh, they're all pulling that in. They are somewhat returning the data. This one here is the API error. So, we would expect a 401. Yes. test any of the fours or fives. So 401, 429 and 500. It has also used table testing. So it has picked up from my agent.md file that in my strategy I have asked it to use table test formatting. So we can see here we've got multiple inputs per test allowing it to test multiple times and not have like lots of different tests in one go.
So let's go ahead and see if we can run this test and see if it passes. So we're going to do we're going to allow Copilot to write a terminal command. So go test. We're going to have verbose. So, we're going to have D minus D. Super. It looks like it is passing. Fabulous. Okay, brilliant. We've got a passing test suite. And of course, as a developer, I'm now going to go in and check out and see what's going on. Uh, whether there needs to be any more finessing, anything needs to be sort of tidied up, all these kind of things. As a developer, I can now go in and modify. We're just going to let it finish here and do whatever it's doing.
So, it looks like what it previously failed to do was understand the JSON formatting. So, since we want to get JSON back from each request, we want to make sure that the formatting is is good for the CLI especially. Uh, and of course the structure and everything we're going to be placing it into or mapping it to in the program. So, it's picked that up. It's managed to iterate on it. And now, of course, let's go and run the test again and see if anything has changed. Doesn't look like anything has, which is which is brilliant. So, we can actually go ahead and just stop this. Uh, we can go ahead and get test coverage, all these kind of things, but we're not going to. I'm just going to skip over those and then stop Copilot from running. It's done everything I need it to do for this one, uh, test file, which is brilliant.
So, with all this being done, I guess there's only really one thing left for us to do, and that's to create the rest of the test suite. So, I'm going to write in again a pretty vague prompt, and I'm going to say write a full test suite for all of the remaining uh test files. Include code coverage so we know how much code is being tested. And this should be a pretty blanket statement just to say, hey, copilot, I've now given you the agent.md file. I have given you a current test file which is weather test. I've showed you or prompted you to create mock data and mock tests with that server and go ahead and go and create all of the rest now. Hopefully it should be able to do this. Pretty straightforward.
Okay, lovely. So it looks like it has gone off and read in the codebase understood what it needed to do and it went off and created a config test or unit test in the config file test for the server file and test for the CLI file as well. So we can go ahead and just click on these. We can go and check out what it's been doing inside there. And you can see it's removed the skip just like it did before. And it's added a number of different tests for us again using table test format just like I outline out outlined in agents.d.
So let's go ahead and allow this. Let's go and uh run some test cover and see how much coverage we have now got. Lovely. Looks like we've got some failed tests. Now this is a really good option and a really good chance for us to show how Git Copilot is going to iterate over these failed tests and see what it needs to do to fix them. And there we have it. It's recognized the fact that I've got an API key in my M file. There's going to be obviously a little bit of uh issues with running tests with M files because obviously I've got API keys and whatnot in there. So it needs to go and figure out how to amend that. So where it loads it properly, whether it loads that in or not, etc.
Okay, so it looks like it's kind of fixed something in the end to end testing and the config test. So let's go and run cover again to see how much coverage we have and see if it's passing. Okay, so it looks like we've got 90% coverage of certain statements. So I'm guessing that is actually it looks like 90% statement 90% coverage over most of it actually. But there is Oh, so that's just the weather the weather app, but it is still failing somewhere. So, it's obviously going to figure out Yeah, we've got still got a problem with the M file. So, let's allow this, but it's just going to uh pipe out exactly what's going on with the M file. Going to read that output into itself and try and help figure out what's going on. So, it's still saying the M file exists. The test need to be temporarily renamed to ignore it. Uh, let's just update the name and handle it properly. So, we're going to see how it handles this. Obviously, as a developer, I'm going to come back and have a good eye over exactly what it's doing and how it's doing it.
So, we're going to run test coverage again. Super. That looks much better. We have now got passing test suite, which is great. And now what it's going to ask to do is it's just trying to go off and do all these extra things that you can do with Go. You can pipe out and do like a cover profile or a test coverage MD file, all these kind of cool things. So, we're not going to bother with that one. We've got everything we need. We've got a full suite of working tests at the moment. I'm now going to go over and check them out and see what needs to be done from a developer perspective. So, we're going to skip that and then end copilot.
So, there we have it. That is what we are using as prompt driven development to help write a test suite with GitHub Copilot inside VS Code just using a whole lot of context and a lot of massaging. So, we've got some great tests from end to end tests. We've got a whole load of unit tests. And by the looks of it, we also have some integration tests as well. If you like this video, if you learned something, give us a like, give us a subscribe, and check out some of the other videos we have. See you in the next one.