Transcription
Most creators here on YouTube show you how to create applications with Cloud Code extremely quickly. And that is all well and good. Only an important topic is almost always left out in all videos. But that's exactly where the problem lies. This step in software development is extremely important, and professional developers pay very close attention to ensuring this step is not forgotten. And that is the topic of testing. By this, I don't mean that you yourself go and test the various functions of your application. That would be far too time-consuming in the long run. Imagine you've been developing an application for months, and after every update, you want to make sure that the new functions have no impact on the old ones, and then you would have to go and fundamentally test all the functions of the application every time, but rather I'm talking about automated tests. And honestly, if you're not using this yet, then there will certainly be one bug or another in your application, and you won't even know it. That's why I want to show you in today's video what test levels actually exist for your project and when you need which, how Cloud Code writes and executes tests for you, and how you can have entire user flows automatically tested end-to-end. With this, you won't just hope that your application works flawlessly, but you'll know that it works flawlessly. There are three different types of tests, and I want to show you a brief overview of them now. First of all, we have unit tests. This means they test individual functions in isolation. Typical use cases are input or field validations, calculations, or formatting. For example, you could check if an entered email address is valid or not. Then we have integration tests, which test whether multiple parts work together flawlessly. Typical use cases are user registration, payment, or various data operations. An example would be: does the registration work, including database entry, and does the user then receive a confirmation email? And the last level is end-to-end tests. Here, complete user flows are tested, including in the browser. For example, the user opens the page, fills out the form, clicks submit, and then sees a confirmation at the end. This is the most complex form of testing, but also the one that comes closest to a real user experience. We move on this pyramid from many small tests at the bottom to few large tests at the top. With Cloud Code, the nice thing is that you don't have to worry about what kind of test you need for your functions, for your feature, or for your application. If you specifically ask Cloud to create tests for a particular process, it will automatically select the right test framework for your stack. We will now look at all three levels in practice, and for that, I will set up a dummy project. For this, I will use my AI Coding Starter Kit. This is essentially a project template for modern web applications. The whole thing is already pre-configured with the most important skills for a professional development workflow. In this template, we simulate a real development team. You can, of course, clone this for free. I'll put the link in the video description. In addition, there is also a complete setup and user manual, where everything is explained step by step, so that even as a beginner in Cloud Code, you are fully covered. There you will also find the Git repository, which you can simply clone. Here you can also see the workflow from the requirements engineer, who gathers requirements, through the solutions architect, who then does the tech design, and then the frontend and backend developers, who do the implementation. Then we have the QA engineer, who will also be important in today's video, because they will thoroughly check your function. And finally, there is the DevOps engineer, who ensures that your app is published correctly. I have now simply started a fresh project, and we will now create a small project management tool for our example. I will now guide you directly through the process in a quick overview. I have now started a new session. This means I have cloned the project or the data kit and am now starting in. So, please help me initialize a new project. Specifically, I want to develop a small test case. It should be a simple project management tool with a Kanban board, on which I can create tasks and drag and drop them back and forth. Before that, there should be user registration and user login. For now, everything should be stored locally in the browser cache. There is no real backend yet. So, I'll submit it now. Exactly. What we can see now is that the Requirement Skill is called first. This means the project is now being initialized. It directly checks if a Product Requirements Document already exists, if there are already features, and so on. And if not, it will now start asking me questions to better understand the project. Exactly. So now it starts this "Ask User Question" mode and starts by asking me questions. So, who are the primary users of the tool? I'd say individuals. Then, what Kanban columns do we need? Quite classic. Then, what should a task include? We have a multi-select here, and the last topic is email and password for user registration. Exactly. Moving on. Now it gives me a summary of what it has done. So, it's now writing the feature specification, and it has Feature 1: User Authentication, Feature 2: Kanban Board and Task Management Drag and Drop in Feature 3. So, that's it. I can now approve it, and then it will create the feature specification for me. There is a dedicated folder here under Features. That's where it will land shortly. It has now created its to-do list and will work through it piece by piece. And then we'll take a quick look at the specification. So, Phase 1 is completed. The Requirements Engineer is done. This means we now have the feature specification, and it's also suggesting how to proceed. Specifically, it recommends a build order in which the features are built or implemented one by one. And it also specifies the next step. Namely, I should now execute the Architecture Skill to have the tech design done before the implementation begins. But let's take a quick look at a feature. We can find them in the Features folder. We have the index.md. This is, so to speak, the project management for Cloud Code itself. It monitors the status and so on. There we see our three features that have been created. And if we look into a feature, that's also quite exciting to see. We have the User Stories it has created, then the Acceptance Criteria, which will also be important later for end-to-end testing, and things like Edge Cases, to cover special cases right from the start, technical prerequisites, and so on. Exactly. And the remaining topics will be filled in shortly. Therefore, I'll just say: "Yes, proceed, so that it does the tech design for Feature 1." The tech design is now complete. Let's take a quick look at what it has documented here. So, for the tech design, it has listed the component structure here. Specifically, it has precisely outlined which components it needs for the User Authentication feature. Then, how data storage works here with Local Storage being used, and the various tech decisions it has made, documented along with the packages needed for implementation. So, I will now do the tech design for all other features afterwards, and then we will see each other in the next step. The tech design is now done. This means I can proceed with the next step, namely the Front-End Skill, and carry out the implementation of the features. I will do that directly now. Yes, please continue in the same way. With the Frontend Skill, there's a small specialty, as it also asks for the design again. We haven't provided that yet. We could have enriched it ourselves in our initial request, or we can provide it now. By the way, this is the result we have just produced, our project management tool. Super simple. I can log in here, of course, but I have to register first. I'll do that quickly. So, I'll log in now and put this in here. And there we are in our, well, very simple Kanban board. I can also create new tasks directly with a description and priority. And the whole thing will be created accordingly. Moving it around doesn't work yet. That was part of ticket number 3 or feature number 3. We haven't implemented that yet, but exactly. Fundamentally, authentication and task management on the project board work here. So, the implementation is now complete for now. We can proceed with the QA Skill, i.e., Quality Assurance. However, before we execute this, I want to take a quick look at the skill with you. For that, we'll open the Cloud folder. Then we go to Skills and then open the QA folder and look at the Skill MD. Here we see how this skill is structured. We have a small header at the top, where we describe what the skill actually does and when Cloud should call it. Then there's the role description. Then we have some prerequisites that Cloud should check before it even starts to carry out this workflow. And then it performs various work steps, including writing unit tests. So, and this should ideally happen before we write end-to-end tests. We'll look at that in more detail shortly. For this, it will create various test files with a testing framework called Jest during the execution. The location where everything is stored is also mentioned here, meaning colocated. That is, the test files are placed directly next to the code files. There is also a description here of what exactly should be tested with unit tests and what should not, so that Cloud has very precise instructions. I'd say let's proceed here. Please start QA for feature number 1. So, the QA process is now complete, and we can look at the results directly in the feature file, because everything is wonderfully documented here. So, if you scroll down here, there's the QA results section, and we can see that all acceptance criteria have been successfully checked. There are two findings here for the feature: one cosmetic and a generic error message that could be replaced with a more specific one, but that's all fine for now. And now we want to look at the unit tests that the process or the skill has just set up. For that, we'll go into the source code. Specifically, we find it under lib. For the authentication service, it has created various unit tests. And we can see that quite well here. There are indeed quite a few that it has set up, and it has grouped them. So, we see, for example, unit tests for password hashing. There are 1, 2, 3, 4 of them. Then comes the next group for session management. Then we have some for the registration workflow. And I'd say let's take a quick look here. So, this group for the registration workflow, it tests the following: it tests whether registration can be successfully completed. Then we have the check: store the password as a hash, never plain text. This checks whether the password is indeed stored encrypted and not in plain text, and so on and so forth. So, there are various unit tests that have been created, just as for login and logout. What is not tested are pure UI components that only display something. For that, there are end-to-end tests later. The core idea behind unit tests is that they run extremely fast, they precisely test a single building block, and they give you immediate feedback if something is broken. And Cloud executes these test functions one after another during testing, or rather, Cloud Code actually starts a test runner via the terminal using the command `npm test`, and this test runner, in turn, checks the various functions, and if something goes wrong, an error is reported back to Cloud Code, and Cloud can then fix the bug. Of course, you don't necessarily have to use the template and the skill for this. You can, of course, also have Cloud Code write a unit test just like that. This means that if you have implemented a function, you simply ask Cloud if it makes sense to write a unit test for this function. You can specifically ask it to write test cases for normal behavior, for special cases, i.e., edge cases, or for error cases. If you haven't done so yet, it's best to specify directly that it should be colocated, and then it will also write the test files directly next to the code files. If you like the video so far, I would be extremely happy about a subscription from you. Currently, almost 90% of viewers have not subscribed to the channel, and with this one click, you would really help me a lot. So, thank you very much, and let's continue. Let's move on to the second level, integration tests. Here, it's no longer about testing individual functions, but about how multiple functions work together. A typical example would be user registration. An integration test could then test the following process. Namely, a user can register, their account is created in the database, and they receive a confirmation email. In the Starter Kit, the Backend Skill covers the creation of integration tests, as most application logic is created here. Therefore, let's take a quick look at how exactly this integration test is created. For my small project here, I have run the Backend Skill again, which is responsible for writing the integration tests. Since the use case itself is quite thin, as we don't have a real backend, it has created one here, but it's more of a nice-to-have than sensible. So, the file is located here in the same directory, named accordingly with "integration," and here we see again exactly what is being done. Since we don't have a real backend in this sense, but everything is only stored locally, as I said, it's more of a nice-to-have. Of course, one wants to check system transitions with such integration tests. So, here again, we see a clear description of what is actually being tested. For example, the entire registration flow from login, logout, lifecycle, etc., also wonderfully documented here. Exactly. And this is how it looks for all other integration tests, so that you can get a very good overview. I would like to briefly show you what instructions Cloud Code receives for creating integration tests. For that, we'll go into the Backend Skill, and here you see again, there's the corresponding header with the documentation or description. Then we have the role again. Then there are again various prerequisites that must be met before the Backend Skill starts, or rather, before the work instructions are processed. And here there is a corresponding point, namely here in my skill under point 6: Write Integration Tests. Here, primarily for each API route, an integration test is created, and then both the happy path and validation errors, and so on. All of this should be covered by integration tests. The focus here is primarily on the API routes, because the template works with Supabase as a backend by default. You can, of course, adapt this to any other backend. If you want to have Cloud manually create an integration test, you can simply do so with the following prompt, for example: "Write me an integration test for a registration workflow. The test should check API, database entry, and response." However, you can also let Cloud itself determine which processes are sensible to test. By the way, in our case here, integration tests do not use a real database, but simulate one instead. There are two approaches, and both are actually quite legitimate. Approach 1 is to work with mock data. This is also done in the template. We use a fake Supabase client and not actual database access. The advantage is that testing is extremely fast and no external setup is required. Of course, this can also mean that the setup behaves differently than in a real scenario with a real database. This means that the tests may pass here, but could still fail in the production environment. Then there is a second approach with a real test database. For this, you would have to set up a separate Supabase project solely for the tests, which would then be reset before each test run. The disadvantage of this is, of course, that testing is a bit slower and takes longer, and requires more setup in advance. However, you can then also test the real interaction completely and ensure that everything really works. Let's move on to the last level, end-to-end tests. As mentioned, end-to-end tests test entire user flows. This means a real browser is launched, pages are accessed, forms are filled out, and buttons are clicked. This is definitely the most realistic form of testing, but also the most complex. Therefore, it is often only used for the core flows of an application, i.e., things like login and registration, checkout or payment, and then precisely the features for which your users ultimately pay you. For end-to-end tests, we need a small helper tool, and we use Playwright for that. Playwright is essentially the standard tool for end-to-end tests for web applications. It was developed by Microsoft and is open-source, therefore also free. You can install Playwright very easily via the terminal, simply with the command `npx playwright install`. Alternatively, you can also ask Cloud Code to install it in the project for you. So, and now you can ask Cloud to write the end-to-end tests. You can either do this manually by simply telling Cloud: "Write me an end-to-end test with Playwright for the login flow. This means, open the login page, enter email and password, click login, and check that the dashboard is accessed at the end." Cloud will then create a test folder and within it, for the respective tests, a `*.spec.ts` file. When you want to run the test, you simply enter the command `npx playwright test` in your terminal, and then all files, all test files, will be executed automatically. Alternatively, you can, of course, also ask Cloud to do it for you. In the AI Coding Starter Kit, end-to-end tests are automatically written by the QA Skill. We work with feature specifications, and these specifications contain acceptance criteria. These are testable conditions that must be met for a feature to be accepted in the end. And the skill writes its own end-to-end test for each of these criteria. This means you don't have to worry about it again, and the framework takes care of it for you. Our QA Skill has already set up the appropriate end-to-end tests. However, you can always manually instruct Cloud to provide end-to-end tests. This is found in a separate folder. It is stored by default in the Tests folder, and here we find a file with the corresponding name of the ticket. Therefore, it can be very well assigned. So, if we look at this now, it's also a rather long file. However, we can see the workflow very precisely that Cloud or the test runner goes through here. In this case, it's the entire registration flow. This is also nicely described in plain text here. This means we know exactly what each block does. For example, here the registration form is checked for the corresponding fields. So, it checks if the email field exists, if the password field exists. This is matched using the IDs. And only then does it proceed to the next test. It then checks if the email validation, if the email validation works, then the password validation, and so on. Then we scroll further down. It then checks if there is a duplicate email during registration, so that it is rejected accordingly. And it continues in the same way here. Here it continues with the login. Then we have session persistence and so on, logout. So, everything from the registration flow is tested here. So, we want to play this through now. Since we are using Playwright to execute this test, we must ensure that it is installed. I will do that now by opening a new terminal and then using the command `npx playwright install`. So, the package will be downloaded and installed, and then we can have the test executed shortly. So, Playwright is installed. So, I'll just tell it: "Please test the end-to-end test with Playwright for Feature 1." So, the end-to-end tests are now complete, and this ran in the background. Several things were identified, but they were automatically fixed in the end. In the end, you see 26 out of 26 tests are green, and here's a summary of the fixes. Playwright runs in headless mode by default. This means the browser opens in the background, and the process runs. So, you usually don't notice anything. You can also run it visually. To do this, simply enter `npx playwright test -ui` in your terminal, and then the same test will be executed, and additionally, your browser session will be started. I'll pull this up. So, you should be able to see it. Exactly. Here you can see the tests on the left side and the preview on the right side. And if we start this now, it should run through completely. So, all tests completed, as you can see here. If we click on it again, we can see exactly what happened here, which page was accessed, which fields were checked, etc. And yes, and you can look at this again to understand it. Now there is a second approach that can be followed. So far, the processes in an end-to-end test are clearly defined. Now, however, you can also let Cloud Code operate the Playwright CLI so that it navigates interactively through the app and decides for itself what is tested. While the tests we have just written are completely free, this approach costs a certain amount of tokens with each test run. This is simply because Cloud then opens your application, takes a screenshot, analyzes it, then finds out where to click next, then navigates to the corresponding page, then takes another screenshot, and so on. This is not a better or worse approach in itself. It simply serves a different purpose. My recommendation would definitely be to integrate the first approach into your development workflow as standard. This means you always write test files for your core workflows or have Cloud Code write them. And if you then want to add exploratory testing, you can follow this approach in certain cases and let Cloud loose on your application. So, to be complete again, it's not about achieving 100% test coverage of your application. That would definitely be overkill. However, the most common mistake is that no testing is done at all. And this can lead to changes breaking existing functionalities, and you not even noticing it. And with such a test strategy, we later ensure that your core functionalities work at all times. This means that what you should always test is your business logic, i.e., calculations, validations, rules contained within them, then your API endpoints, i.e., checking that the correct responses are returned or that error handling occurs when incorrect inputs are sent. Then, of course, authentication and authorization, and also your core feature, for which your users ultimately pay. So, even if you don't use my personal framework, where everything is already implemented, just have Cloud create a test strategy for you. For example, simply give it the following prompt: "Analyze my project and create a test strategy. Which areas need unit tests? Which integration tests? Which end-to-end tests? Please prioritize by impact. What would hurt the most if it broke, for example?" So, I hope I was able to give you a good overview of the topic of testing. As I said, I believe this is one of the drier topics, but an essential topic if you want to develop software professionally with AI. If you don't want to worry about the whole topic during development, then simply download my free AI Coding Starter Kit. The whole topic is already pre-configured there. This means the entire topic of testing is already considered there. You can find the link to it in the video description, and as I said, there is also a complete user manual, so that even as a complete beginner in this area, you can find your way around directly. If you liked the video, please give it a like and subscribe. I would be very happy about that. And otherwise, I hope to see you in the next video. Until then, take care. M.