Newman report and its application to API testing

The development of robust and secure Application Programming Interfaces (APIs) is essential for building modern and scalable applications. In this context, carrying out appropriate tests is a critical step to guarantee API quality.

One of the main resources used to automate and organise these tests is Newman, a command-line tool from Postman (a platform for building and using APIs), which allows you to run collections of API tests.

This article explains what the Newman report is, how it can be integrated into the testing process and the importance of its reports in analysing results and optimising the development cycle. We'll see how this tool can be applied to generate detailed reports, making it easier to identify weaknesses and contributing to the continuous improvement of APIs.

What is Newman?

Newman is an open-source command line tool that allows you to run Postman test collections. Essentially, Newman acts as an ‘executor’ of collections, allowing the execution of API tests in an automated way, without the need for a graphical interface.

This is particularly useful in continuous integration (CI/CD) scenarios, where we want to automate the API testing process as part of a build and release pipeline.

Reporting and analysing results

Newman provides a wide range of options for generating reports on API tests. You can choose to generate reports in HTML, JSON (JavaScript Object Notation) or CLI (Command Line Interface) format, with detailed information about each request, the tests performed, the execution time and the status of each test.

The reports generated by Newman are crucial for analysing test performance. By analysing these reports, we can identify errors, performance bottlenecks, areas for improvement and ensure the overall quality of the API.

Let's delve a little deeper into each report format:

Setting up the test environment

Setting up the test environment to use Newman is a relatively simple process:

1. Install Node.js, which is Newman's execution platform.
2. Install Newman using the npm package manager. To do this, simply run the command ‘npm install -g Newman’ in the terminal.
3. Make sure that Postman is installed and that your test collections are configured accordingly.

Writing API tests with Postman

Well, we've seen what Newman is, its configuration and a bit about Postman. Let’s get down to business.

To start writing API tests with Postman, we need to create a collection. A collection is a group of related API requests. In each request, we can define the necessary information, such as the HTTP method (GET, POST, PUT, DELETE), the URL, the headers and the request body. Within each request, we can also define specific tests to check that the API response is in line with expectations. These tests can check the response status code, the response content, or other relevant aspects of the API.

Let’s go step by step:

1. Click on “Create Collection”:

2. Then click on “New”:

3. In this case, we'll use the HTTP format, but it's up to everyone's judgement:

4. Set up the request as follows:

• Method: GET
• URL: https://www.alter-solutions.com

5. Next, we'll do some validations to ensure that the API has returned what we expected. To do this, go to the ‘Scripts’ > ‘Post-response’ tab and insert the following code:

6. Now simply apply ‘ctrl’+‘s’ to save the request and export the collection:

Running API tests with Newman

Running API tests with Newman is as simple as executing a command in the terminal. The basic command for running a collection of tests is ‘newman run .json’.

We can customise this command according to our needs, adding options to specify environment variables, reporting options or other parameters. For example, we can use the ‘-e’ option to set environment variables for the collection. We can also use the ‘-r’ option to generate reports in HTML, JSON or CLI format.

Let's run it:

• Open Git Bash (it can be installed via this link).
• Enter the command ‘npm install -g newman’.
• Enter the command ‘npm install -g newman-reporter-htmlextra’.
• Go to the folder where the file is located.
• Enter the command:
  
  • ‘.json -r htmlextra’ (if we want to generate an HTML report)
    or
  • ‘newman run .json’ (to generate the report in the command line itself)

Note: if we choose to generate the report in HTML, a folder with the name ‘newman’ will be created in the location where it is tested. It will contain a file similar to this one:

Benefits of using Newman

• Test automation
  It automates the execution of API tests, freeing up time for other tasks and guaranteeing consistency in each execution.
  
  
• Continuous integration
  It allows API tests to be integrated into CI/CD pipelines, making the development process more efficient and robust.
  
  
• Scalability
  It makes it easy to run tests in various environments, such as development, staging and production, with just a few configuration changes.
  
  
• Detailed reports
  It generates complete reports on test performance, helping to identify and solve problems more quickly.

Conclusion

Newman is a powerful tool for automating API tests, improving the efficiency of the development process and guaranteeing API quality. By integrating Newman into the workflow, we can carry out tests consistently and efficiently, obtaining detailed information on the performance of our APIs.

I encourage you to explore Newman's various features, such as running tests in parallel, using environment variables and customising reports, to further optimise your testing processes.