Transcription
[Music]
So, N8N is a service for automating your data work. It can be a full-fledged backend or just a link between services that lack pre-configured integration. How to use this service is entirely up to you.
Today, we will talk about cloud hosting and its pricing. We will go through the journey from renting a virtual server to a fully configured working environment. We will examine the interface in detail and the workspace.
And we will summarize. This is the Assistant project. Here, we discuss services. I am your host and founder, Nikita Ryansky. Let's get started.
[Music]
From my experience, I can say that I use N8N Production for both small projects and for working with large data sets, including their segmentation, matching, slicing, and sending the results of this processing to various data sources.
Working with data in N8N, as well as configuring workflows, is very intuitive. There is a canvas where nodes are placed, through which your data passes and is processed. Data can come into the incoming node, go through certain steps that you configure, and be sent to external sources, applications, and services that you specify.
A large number of nodes are already pre-configured to process your data. But if the standard functionality is not enough for you, there are always code nodes, which allow you to write all the necessary data processing in JavaScript that you require.
[Music]
If you don't have enough time for installation and configuration, or if you already have running workflows that need to be understood and optimized, contact us using the details in the description of this video for consultations or complete turnkey data work.
We rent a dedicated server for you, fully configured, protect it from external sources, set up data processing exactly according to your parameters and requests, and deliver it in a ready-to-use format in the shortest possible time. Contact links will be in the description.
[Music]
You can use the N8N service via a subscription. On the official website, you will find a range of three tariffs, but there is also a fourth, the Community plan. This is the one we will examine in more detail today, performing the installation and configuration ourselves.
The main pricing in N8N is based on workflow executions, which are the data processing tasks that each workflow will perform.
On the Starter plan, which starts at 20 euros, you get 5,000 executions per month, and only 5 active workflows can be run simultaneously to listen for external calls and be triggered from external services and applications.
The Pro plan is divided into two types: for 50 euros, you get 10,000 executions and 10 active workflows, and for 120 euros, you get 60,000 executions and 25 active workflows per month.
We will look at the Enterprise plan today, but all the nuances and details are discussed directly with the N8N company.
Our option today is the Community plan, which is free to use and can be installed on your own server. In any of the paid plans, the company N8N takes care of hosting, guaranteeing the continuous operation of your specific installation and the availability of your workflows for more than 99.5 percent of the time.
Despite the price differences in the plans and the complete absence of cost in the Community plan, all the main functions of the web service are available on any plan. All possible nodes and triggers through which these workflows can be called are available everywhere.
There is a slight difference in global variables, which are only available on the Pro plan, but from experience, I can say that global variables can be passed from one workflow to another via HTTP calls.
A key limitation is the workflow timeout. On the Starter plan, the optimal execution time for a single workflow is 5 minutes, which is very little for processing large data sets, but more than enough for transferring data from point A to point B.
On the Pro plan, this limitation is 40 minutes, which is sufficient for processing a small set of analytical data. In the Community version, there is no such limitation; in this case, the ceiling for capacity and timeout is your server and its technical specifications.
It is also worth noting the aspect of collaborative work with the service. If you need to invite more than one developer, share access rights, share configured workflows with external participants, or share access keys to accounts used in executing workflows, the Pro plan is the only option. The Starter and Community plans do not provide such capabilities.
Self-hosting options for installing the N8N service on your server are numerous, but our installation option today will look as follows: we will rent a VPS, that is, a virtual server on a clean Ubuntu, install Docker for container management, set up the web application Partainer, and then install the N8N service through it.
We will also connect our own domain through a proxy manager to conveniently manage webhooks and share them with external participants.
So, for the installation, we will need to perform several steps. Some of them will include commands for the terminal, while others will involve commands for the Docker Compose file. Everything we are about to do will be in text form, with a link in the description.
The first step in the setup will be the domain name. What it will be is completely unimportant; the main thing is that you have access to DNS settings so that we can configure the direction for this domain to our rented server.
You can purchase domain names and register a virtual server with a provider like Bget. The link for registration will be in the description of this video.
When we move to rent a virtual server on Bget, we have a very convenient configurator that allows us to flexibly set the technical specifications of our equipment. This is very convenient at the beginning for setting the minimum technical requirements for our service, as well as later when the load on our workflows theoretically increases.
From my experience, I can show that the minimum technical requirements for installing N8N will be sufficient with 2 CPU cores, regardless of their power, and 2 GB of RAM. This is necessary so that you are not surprised when most workflows end with an error.
You may notice that we specify quite a small amount of RAM, but during the setup of N8N and the configuration of the Docker Compose file, we will specify several parameters that allow us to optimize the amount of RAM on the server and not use it unnecessarily.
A storage volume of 10 GB is quite enough to start with. In the setup, we will also specify storage optimization, but gradually it will need to be increased on the server.
For the operating system, we specify Ubuntu 22.04. We will not add SSH keys for the server today; we will simply set a static password and also specify a name for our server. After that, we can proceed to its creation.
When the server is created, we will have access to its statistics. We can monitor the load on the CPU cores, RAM consumption, and storage usage after installation and during the operation of your workflows.
From time to time, I recommend keeping an eye on the free disk space on your server and not letting it reach zero. Most errors in workflows can be related to the fact that the services simply ran out of disk space for data processing.
And the parameter we will need for further configuration is the server address. We will copy it.
If you like this detailed format of the review, give this video a thumbs up, turn on the notification bell so you don't miss new releases, and subscribe to our channel.
Now, let's continue.
To connect to the server and work with it, we will need a terminal. I use the Termius terminal, which is available in the Microsoft Store. If you want to install it and forget about updates, it is also available on the official website with all its installers.
After you install the terminal, you need to add a new host. Here, in the settings, we specify the server address we copied. The username will be root, so we get super administrator rights in the Ubuntu operating system, and the password we set earlier.
After saving the settings, double-click on the server to connect to it.
So, we are connected to the server. A very convenient feature of Termius is the ability to save snippets. Snippets are code fragments that we often use in our work. For this video, I saved snippets and even organized them by number so that we can execute all commands sequentially and install all the necessary services and utilities on our server.
So, let's get started.
As you will be copying snippets from an external source and pasting them into the terminal, you will need the keyboard shortcut Command + V or Control + V, and Command + Shift + V or Control + Shift + V. This is the combination for pasting text into this terminal.
The first command updates all packages on the server. After pasting the command, press Enter.
You can tell that the server has completed all operations and the terminal is waiting for the next command by the text at the bottom of the terminal: root@internal identifier of our server and a blinking cursor. This means that the terminal has finished the previous operation and is waiting for our next command.
The next operation is to upgrade all packages and again update the packages for the correct installation of the kernel. We will need additional utilities. If they are already installed, they will simply be updated, or nothing will happen if they are not. This command will install these utilities in our operating system.
Next, we will import the key for accessing the Docker repository, add the repository itself, confirm the command, and wait for its execution. After that, we will update the packages again.
Here, we will check that we have correctly connected to the Docker repository. We should see a response like this: a list of candidates for installing the Docker kernel that are available to us.
We will install the Docker kernel, and along with the Docker kernel, the Docker Compose utility will also be installed. But besides it, we need the Docker Compose plugin, which differs only by a hyphen between the words.
In the next command, we will install the Docker Compose plugin, confirm the operation, and check the installation.
We will check the Docker version; everything is correct. We will check the version of the Docker Compose utility; everything is also correct. The version has been identified, and we will check the version of the Docker Compose plugin. The installation has gone smoothly.
If any of your responses from the terminal at this point are completely different or return an error, go back to the very beginning and go through all the steps sequentially up to this point.
Next, we will install the web service Partainer in a couple of simple steps. To install it, we will need to create a directory for it and run Docker, specifying the ports on which the Partainer service will be hosted and linking the directories that will be connected between the physical storage and the storage inside the Docker container.
We will also specify a parameter that forces the container to restart automatically every time it stops. We will perform the installation and launch.
After loading the installation and starting the container, we need to wait a little while for it to fully activate and become available on the ports. To confirm that Partainer is running, we will execute the command to check running containers in Docker.
Here, we see one running container, which is our service that we just installed. By default, Partainer is available on port 9443. We enter the address of our server followed by :9443, the port address of our container, and press Enter.
Before the server address, we also add the HTTPS protocol, which we will use to connect. Upon the first connection, the browser will inform us that this connection is not secure. This is normal because no domain is connected to our server yet, and we are trying to access it via a secure protocol directly.
We ignore this warning and proceed to the service. Upon the first connection, Partainer will ask us to set a super administrator password. Here, we enter it and remember it or write it down to create a superuser.
So, we are inside and starting work. We go to our local server, and this screen will begin the installation of the N8N service.
[Music]
The installation process inside Partainer will be divided into two parts: first, we will install the N8N service itself, and then we will install the proxy manager to redirect from our domain directly to the service.
We will install the containers in packages, that is, stacks. For this, we go to the corresponding menu item on the left and click "Create Stack." We name it, and here we must specify the configuration with which our containers will be loaded, installed, and launched in this stack.
In this configuration, the most interesting part will be setting up a dedicated database connection, making your N8N instance more resilient to failures and capable of processing large data sets.
I copied and pasted a template that will be in the description of this video. Let's break it down in detail.
In this configuration, we explicitly declare the directories where the persistent data of our database will be stored from the application itself. Otherwise, after restarting the server or container, all data, workflows, and processing will be lost.
We declare the installation of the database; in this case, it is PostgreSQL version 11. This version of the database has proven to be the most robust and stable for processing any data.
We also specify the installation parameters, the directories where the data will be stored, and check the database's availability every 5 seconds.
In the N8N installation parameters, we specify the image we will download for installation, the restart parameter every time the container stops, the connection parameters to the database, and their variables.
We will also specify the authentication methods in the service, indicating the webhook URL that will be used in the webhook nodes so that we can trigger workflows from outside.
Next, we have parameters that optimize our installation. The Max Old Space parameter indicates how much data the N8N service keeps behind itself after processing. If this data is no longer needed, it will be automatically deleted from its memory.
In this case, we set a limit of 8 GB. If we specify less, errors may arise. By default, this number is quite small, but if we specify less, errors may start to appear.
During data processing, for example, files in JSON format weighing half a megabyte expand, and this file starts to weigh not half a megabyte but significantly more.
8 GB is quite enough for processing large data sets. The next parameter specifies where N8N should store these permanent data during processing.
Not all data needs to be processed in a matter of seconds in RAM, which is also more expensive to maintain on a virtual server. Here, we explicitly indicate that the service should use the hard disk of our server for this, which is much more economical to expand and maintain than RAM.
Not all data formats can be processed this way, but this eliminates most errors during processing of large data sets. By the way, a large data set is not only one that contains many objects for processing but also the content within each cell.
If you pass only one object for processing, but one of its parameters contains a lengthy text description, such as a white paper in one cell, and you want to use regex functions to process the entire text, this array will also be quite large.
To expand all this within the server, significant computational power will be required. The next parameter specifies the maximum initial file size that we can upload.
Not all data in N8N needs to be processed, but they need to be transferred from one endpoint to another. In this case, it could be an image file or a PDF document. Here, we specify this size with a margin of 4 GB.
The time zone is also a very important parameter. This will allow us to correctly orient ourselves in the timing of workflow execution, properly configure scheduled launches, and accurately track logs of what happened with our data.
Sometimes your server or container will restart, so it would be useful for the service to clean up unnecessary data it used during processing. The last parameter does just that, removing data it has already processed and no longer needs during the service startup.
The last parameters are the port through which our service will be available. By default, this is port 5678. Here, we link two identical ports because there is nothing else on our server for now.
But if your server already has this port occupied, you can change this parameter to another. The first parameter of these two will direct all incoming traffic through the proxy manager.
The last parameters specify the link to the database and the directory where N8N will store its data. If you have already encountered the installation of N8N or other applications using Docker Compose, you know that variables are passed to the installation using a .env file.
Here, we will also specify these settings but in a slightly different way. We scroll down the page, switch to advanced mode for editing variables, copy the values from the .env file, and paste them here.
After that, we switch back to simplified mode. All parameters that are usually specified in the .env file are already here. The key parameters on the left column should not be changed; they are already defined in the Docker Compose file and will be used for mapping.
On the right side, you can change them as you wish. Here, we specify the username and password for the database for the root user in N8N, as well as the name of the database that will be created for our instance.
With this, the installation of the N8N service itself is complete. We can start the stack. After we start the stack, Partainer will load the necessary images specified in the configuration, install them, and run them together.
Great! We have installed and verified that we have linked the PostgreSQL version 11 database and the N8N service instance, which is now available on port 5678.
Now, let's move on to installing the proxy manager. We also go to the stacks tab in the left menu and create a new stack. We name it and paste the template from the description of this video.
This configuration is much simpler than the previous one. We define the service itself, how it will restart, and that this service will be available on ports 80, 81, and 443. This means that the proxy manager will listen for all requests via HTTP and HTTPS protocols.
In addition, we specify the network that this proxy manager will use, but we indicate it as external so that when the container or the entire server restarts, we do not duplicate the network inside the Partainer settings.
Before starting this stack, let's create this network in the corresponding tab. We copy its name from the template, open the network settings in a new tab, add the network, specify its name, and do not change any other settings.
The network is created, and we can return to the stack settings page for the proxy manager. We start the stack and wait for the installation.
The proxy manager stack is also installed. Once inside, we can see the container that is available on the ports we specified earlier: 443, 80, and 81.
To access the admin panel and set up redirection from our domain to the N8N service, we go to the address of our server followed by :81. We are greeted by a login window where we enter the standard email and password, which will also be in the description of the video.
We have entered the service, which asks us to provide our full name, nickname within the service, and email address. We fill this in and save.
The next step will be to set our password. We enter the current password and set our own, then save. After this, we find ourselves inside the proxy manager as super administrators, where we can configure the redirection from our domain to the N8N service.
[Music]
Before continuing with the setup, you need to configure your domain with the registrar from whom you purchased the domain name. In the DNS settings, create an A record with the IP address of your server. There should be no other records with the A prefix.
If your main domain already hosts a website, you can create a third-level domain and set up DNS redirection to our server. After applying the settings, depending on the provider, it may take some time for the settings to take effect.
In the case of Bget, it takes no more than a couple of minutes.
Next, in the proxy manager window, we go to the dashboard, proxy hosts, and add a new host. Here, in the first parameter, we specify the domain name we will connect. In the connection scheme, we leave HTTP. This is the internal connection scheme of the proxy manager to the N8N service.
In the IP parameter, we specify the address of our server, and in the port parameter, we specify the port of the N8N service, which is 5678. In addition, we enable WebSocket support and exploit blocking.
We go to the SSL tab and generate a certificate for our domain name, activating the first four settings available on this tab. We specify our administrator email address if it is not already indicated and agree to the terms of use.
Now we will issue an SSL security certificate for our domain. After the SSL certificate request has been made, the settings are applied, and in the list of proxy hosts, we will see our domain connected on port 5678 with an online status.
Now we can proceed to our domain for direct access to the N8N service. Upon the first connection, we will be asked to provide the superuser name and password that we specified in the configuration during installation.
If necessary, you can return to the stack, open its settings, and see what parameters you specified. We enter the superuser login and password and see the N8N service administrator setup window.
We specify our email, name, and set a password. Congratulations! The N8N service is successfully installed on your own server under your domain.
After installation, N8N immediately creates a workflow for us and opens it on the canvas, which we can see on this screen. In the left menu, we see tabs for workflows, templates, access keys, global variables, and a list of already executed workflows and their results.
Below are help and tips, our account, and settings. The workflow itself is entirely configured on the working canvas. Here, we can specify its name, assign a tag or several tags, edit the workflow, or view all its runs and results from here.
We can enable and disable the workflow, as well as save it. The share button for the workflow is not available on our plan, but in the additional dropdown menu, we can also duplicate the workflow, download it as JSON, import it via a link, or import it from a file.
That is, we cannot share the configured workflow via a link, but we can download it as a file and import it into another N8N installation.
I should note that if any access keys to services are involved in the workflow, they will not be transferred; they need to be recreated in the other installation and connected in the workflow. We will also look at how to do this later.
We have already mentioned that workflows here are built from nodes. They can be added using the plus icon in the upper right corner of the screen.
From here, it is also possible to create notes. It is not always convenient to keep in mind or store information about what exactly a specific step in the workflow does. You can add a note to the canvas and write all this information on it.
Workflows in N8N are divided into groups. The first group is the largest; it contains actions in external services and applications. You can view all available pre-configured services in the list or find the one you need by name.
Next are the trigger actions that can start your workflow. These include scheduled launches that you configure, launches via webhooks that you specify, manual launches, and launches from another workflow within your N8N installation.
There is also another folder with actions that did not fit into any of the previous sections.
From experience, I can say the following: even if a service is listed here, it may not always have pre-configured nodes. It may happen that many settings or specific parameters that you need to accept or pass through it are lacking.
All of this can be similarly replaced through a request node, which is most often used in external calls. But there are also nodes that are very rich in functionality, such as the Telegram node. Through it, you can receive and send almost all parameters you may encounter while working with this messenger.
Let's add a node for calling a workflow via a webhook as an example to see what settings it has. When hovering over the node, we can delete it, pause it so that it does not execute and the workflow skips it, duplicate the node with all settings, or execute the workflow only up to this node.
Let's open it to view the settings. The node settings window is divided into three parts: the left part contains the data that comes into the node, the middle part is where we configure their processing, variables, parameters, and everything that the node can include, and on the right, we see the data that is obtained after processing by the node.
When working with different cases, different data comes in and goes out. We never know what form of display will be convenient. Depending on the node, you can resize all three parts of this node's settings window.
Also, each node contains a documentation tab. By going there, you can see the official documentation for configuring this node and examples of its use.
For the webhook node, we see the following parameters: the webhook URL, a test URL for production, which differ from each other with an internal parameter to prevent accidentally exposing the webhook in production, and vice versa, when you want to test some new settings but already have data flowing through this webhook.
Until you save the workflow, these settings will not go to production.
Next, you can configure the authentication method that will be used for authorization when making requests to this node, the method by which it should be called, and the path to the webhook. Here, you can write any text you like, as long as it does not duplicate.
In the case of the webhook node, if you have 20 identical names, they will all be called when triggered. You can also configure the response type for the webhook, either instantaneously when the last node in the workflow is executed or use a separate response node for the webhook, where you can also pass certain parameters in the response body.
You can enable either a 200 code, which means success, or a dynamic expression that is needed in your scenario.
Also, almost every node has additional settings that differ from node to node. Very important parameters are hidden in the gear icon on each node. Here, we can specify the following parameters: for example, continue the workflow even if the node received nothing and processed no data.
We can specify that this node should only be called once, which is convenient when you receive an array of data. For example, it should only be called once and not for each item in that array.
You can enable the retry parameter for working with this node, even if an error occurred during its operation, and enable the continuation of the workflow even if the work in the node with the data failed.
After the node is configured, you can connect the next one to it using the corresponding icon next to it.
Since our canvas already has one action node, the number of triggers has increased. Here, we can find actions for working with tabular data, manipulations for splitting and merging data sets, conditions, and selections based on certain criteria, and much more.
Once our workflow is configured, we can name it, activate it, and it will automatically save, or we can save it manually. After that, it will be available for display in the list of workflows.
From here, we can also search for workflows by name, sort them, and filter them.
In the templates tab, you can see workflows created by other developers who have shared them with external participants. Here, you can find data processing workflows that send data to various CRM systems, process notifications and send them to Telegram, as well as combine services like Notion, Airtable, and other popular services.
In the access keys tab, we can save data for services we plan to use in different workflows, whether they are CRM systems or any other service. By entering the login, password, or any other authentication method here once, we can simply select it later in the workflow and immediately authenticate with this data.
The global variables tab is not available on the Community plan, but it allows you to easily pass data between workflows and store them in the application's persistent memory.
The executed workflows tab will allow us to track the results of each executed workflow in the service. I should note that in each workflow, you can manually specify which results to save.
For example, if the process completed successfully, you may not need to save it, but if there was an error, it should be kept in history.
It is also very common that collecting results from a workflow takes much more resources from your server than executing the workflow itself. If a workflow ends with an error, try disabling the saving of its results, and it will likely execute successfully.
Not every result of a workflow needs to be stored in N8N's memory. If in your case it is suitable not to store this data, feel free to disable this parameter; it will save you from many errors.
In the settings tab, we have not so many options available on this tariff plan. Here, we can see the number of workflows from the unlimited ones available to us, as well as generate an API key for accessing our N8N installation from the outside.
After watching this video and performing all the actions, you will have a fully ready environment for working with and processing your data.
If you liked this format of the review and the detailed installation instructions for using a service like N8N, give this video a thumbs up and subscribe to us on social media. Leave comments on this video and send requests for reviews of other services in private messages.
This was the Assistant project. See you in the next videos.
[Music]