Running collections on the command line with Newman

Newman is a command line Collection Runner for Postman. It allows you to run and test a Postman Collection directly from the command line. It is built with extensibility in mind so that you can easily integrate it with your continuous integration servers and build systems.

Newman maintains feature parity with Postman and allows you to run collections the way they are executed inside the collection runner in Postman.

Newman resides in the NPM registry and on GitHub.

We'll cover these topics to learn command line integration with Newman:

• Getting Started
• Options
• Example collection with failing tests
• Using Newman with CI/CD
• File uploads
• Library
• Custom reporters

Getting Started

Newman is built on Node.js. To run Newman, make sure you have Node.js installed.

You can download and install Node.js on Linux, Windows, and Mac OSX.

After you install Node.js, Newman is just a command away. Install Newman from npm globally on your system, which allows you to run it from anywhere.

$ npm install -g newman

The easiest way to run Newman is to run it with a collection. You can run any collection file from your file system.

You can export a collection to share as a file.

$ newman run mycollection.json

You can also pass a collection as a URL by sharing it.

Your collection probably uses environment variables. To provide an accompanying set of environment variables, export the template from Postman and run them with the -e flag.

$ newman run https://www.postman.com/collections/cb208e7e64056f5294e5 -e dev_environment.json

Options

Newman provides a rich set of options to customize a run. You can retrieve a list of options by running it with the -h flag.

$ newman run -h

Utility

Basic setup

Request options

Misc.

Use the -n option to set the number of iterations to run the collection.

$ newman run mycollection.json -n 10  # runs the collection 10 times

To provide a different set of data, such as variables for each iteration, you can use the -d to specify a JSON or CSV file.

For example, a data file such as the one shown below runs 2 iterations, with each iteration using a set of variables.

[{
    "url": "http://127.0.0.1:5000",
    "user_id": "1",
    "id": "1",
    "token_id": "123123",
},
{
    "url": "http://postman-echo.com",
    "user_id": "2",
    "id": "2",
    "token_id": "899899",
}]

$ newman run mycollection.json -d data.json

Here's an example of the CSV file for the above set of variables:

url, user_id, id, token_id
http://127.0.0.1:5000, 1, 1, 123123123
http://postman-echo.com, 2, 2, 899899

Newman, by default, exits with a status code of 0 if everything runs well, such as without any exceptions.

Continuous integration tools respond to these exit codes and correspondingly pass or fail a build.

You can use -x or --suppress-exit-code to override the default exit code for the current run.

You can use the --bail flag to tell Newman to halt on a test case error with a status code of 1, which can then be picked up by a CI tool or build system.

$ newman run PostmanCollection.json -e environment.json --bail newman

Example collection with failing tests

→ Status Code Test
  GET https://echo.getpostman.com/status/404 [404 Not Found, 534B, 1551ms]
  1\. response code is 200

┌─────────────────────────┬──────────┬──────────┐
│                         │ executed │   failed │
├─────────────────────────┼──────────┼──────────┤
│              iterations │        1 │        0 │
├─────────────────────────┼──────────┼──────────┤
│                requests │        1 │        0 │
├─────────────────────────┼──────────┼──────────┤
│            test-scripts │        1 │        0 │
├─────────────────────────┼──────────┼──────────┤
│      prerequest-scripts │        0 │        0 │
├─────────────────────────┼──────────┼──────────┤
│              assertions │        1 │        1 │
├─────────────────────────┴──────────┴──────────┤
│ total run duration: 1917ms                    │
├───────────────────────────────────────────────┤
│ total data received: 14B (approx)             │
├───────────────────────────────────────────────┤
│ average response time: 1411ms                 │
└───────────────────────────────────────────────┘

  #  failure        detail

 1\.  AssertionFai…  response code is 200
                    at assertion:1 in test-script
                    inside "Status Code Test" of "Example Collection with
                    Failing Tests"

The results of all tests and requests can be exported into a file and later imported into Postman for further analysis. Use the JSON reporter and a file name to save the runner output into a file.

$ newman run mycollection.json --reporters cli,json --reporter-json-export outputfile.json

Note: Newman allows you to use all libraries and objects that Postman supports to run tests and pre-request scripts.

Using Newman with CI/CD

By default, Newman exits with a status code of 0 if everything runs as expected with no exceptions. You can configure your continuous integration tools to respond to Newman's exit codes and correspondingly pass or fail a build. You can also utilize the --bail flag to make Newman stop the run if it encounters a test case error with a status code of 1, which can then be picked up by your CI tool or build system.

File uploads

Newman also supports file uploads. For this to work correctly, upload the file in the relative location specified in the collection. For instance, review this collection:

{
    "variables": [],
    "info": {
        "name": "file-upload",
        "_postman_id": "9dbfcf22-fdf4-f328-e440-95dbd8e4cfbb",
        "description": "A set of `POST` requests to upload files as form data fields",
        "schema": "https://schema.getpostman.com/json/collection/v2.0.0/collection.json"
    },
    "item": [
        {
            "name": "Form data upload",
            "event": [
                {
                    "listen": "test",
                    "script": {
                        "type": "text/javascript",
                        "exec": [
                            "var response = JSON.parse(responseBody).files[\"sample-file.txt\"];",
                            "",
                            "tests[\"Status code is 200\"] = responseCode.code === 200;",
                            "tests[\"File was uploaded correctly\"] = /^data:application\\/octet-stream;base64/.test(response);",
                            ""
                        ]
                    }
                }
            ],
            "request": {
                "url": "https://echo.getpostman.com/post",
                "method": "POST",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": [
                        {
                            "key": "file",
                            "type": "file",
                            "enabled": true,
                            "src": "sample-file.txt"
                        }
                    ]
                },
                "description": "Uploads a file as a form data field to `https://echo.getpostman.com/post` via a `POST` request."
            },
            "response": []
        }
    ]
}

The file sample-file.txt must be present in the current working directory as the collection. Run this collection as usual.

$ newman run file-upload.postman_collection.json

Library

Newman has been built as a library from the ground up. It can be extended and used in various ways. You can use it as follows in your Node.js code:

var newman = require('newman'); // require newman in your project

// call newman.run to pass `options` object and wait for callback
newman.run({
    collection: require('./sample-collection.json'),
    reporters: 'cli'
}, function (err) {
    if (err) { throw err; }
    console.log('collection run complete!');
});

Custom reporters

Custom reporters come in handy when one would want to generate collection run reports that cater to very specific use cases. For instance, logging out the response body when a request (or it's tests) fail, and so on.

Building custom reporters

A custom reporter is a Node module with a name of the form newman-reporter-<name>. To create a custom reporter:

1. Navigate to a directory of your choice, and create a blank npm package with npm init.
2. Add an index.js file, that exports a function of the following form:

function (emitter, reporterOptions, collectionRunOptions) {
  // emitter is is an event emitter that triggers the following events: https://github.com/postmanlabs/newman#newmanrunevents
  // reporterOptions is an object of the reporter specific options. See usage examples below for more details.
  // collectionRunOptions is an object of all the collection run options:
  // https://github.com/postmanlabs/newman#newmanrunoptions-object--callback-function--run-eventemitter
};

1. Publish your reporter using npm publish, or use your reporter locally [see usage instructions][7].

Scoped reporter package names like @myorg/newman-reporter-<name> are also supported. You can find working reporter examples in [working reporter examples][8].

Using custom reporters

In order to use the custom reporter, it will have to be installed first. For instance, to use the [Newman teamcity reporter][9]:

Install the reporter package.

npm install newman-reporter-teamcity

Note that the name of the package is of the form newman-reporter-<name>, where <name> is the actual name of the reporter. The installation should be global if Newman is installed globally, local otherwise. Run npm install ... with the -g flag for a global installation.

To use local (non-published) reporters, run the command npm install <path/to/local-reporter-directory> instead.

Use the installed reporter, either via the CLI, or programmatically. Here, the newman-reporter prefix is not required while specifying the reporter name in the options.

Scoped reporter packages must be specified with the scope prefix. For instance, if your package name is @myorg/newman-reporter-name, you must specify the reporter with @myorg/name.

CLI:

newman run /path/to/collection.json -r myreporter --reporter-myreporter-<option-name> <option-value> # The option is optional

Programmatically:

var newman = require('newman');

newman.run({
   collection: '/path/to/collection.json',
   reporters: 'myreporter',
   reporter: {
     myreporter: {
       'option-name': 'option-value' // this is optional
     }
   }
}, function (err, summary) {
  if (err) { throw err; }
  console.info('collection run complete!');
});

In both cases above, the reporter options are optional.

For the complete documentation, see the Newman README.

For more information about collection runs, see:

• Using the Collection Runner
• Working with data files
• Building workflows
• Integration with Jenkins
• Integration with Travis CI
• Newman with Docker