Start a Test Run

Prerequisites

1. Install/update ht-cli npm package

To Install and use HyperTest's ht-cli package, we will do the following steps:

1. Create PAT for the user - Hypertest Team will do this for you

2. Add NPM repo login credential in ~/.npmrc

3. Install ht-cli package globally using the following command

Add the following content in your ~/.npmrc.

1st line in this file has authToken for pulling packages from NPM registry.

2nd line indicates to use https://npm.pkg.github.com registry to pull packages scoped with @hypertestco.

.npmrc

//npm.pkg.github.com/:_authToken=<PAT>
@hypertestco:registry=https://npm.pkg.github.com/ 

npm install -g @hypertestco/ht-cli --save-exact

Verify htcli has been installed using the below command:

htcli --help

Getting Started:

1. Create a Test Configuration File

Available Configurations:

.htTestConf.js

const requestTypes = {
  HTTP: 'HTTP',
  GRAPHQL: 'GRAPHQL',
  KAFKA: 'KAFKA',
  GRPC: 'GRPC',
  AMQP: 'AMQP',
};

module.exports = {
  htBackendBaseUrl: "", // URL of HyperTest server (Required)
  serviceIdentifier: "", // UUID for the service (Required)
  requestTypesToTest: [requestTypes.HTTP], // What kind of requests to include in the test
  httpCandidateUrl: "", // HTTP URL of App under test (Optional)
  // graphqlCandidateUrl: "", // GraphQL URL of App under test (Optional)
  appStartCommand: "", // Command to start the app (Required)
  appStartCommandArgs: ["<Argument 1>", "<Argument 2>"],  // App start command arguments (Required)
  appWorkingDirectory: __dirname, // Working directory for the app (default: current working dir) (Optional)
  appStartTimeoutSec: 30, // Timeout in seconds for the start command (default: 10) (Optional)
  showAppLogs: true, // Whether to show app logs (default: false) (Optional)
  shouldReportHeaderDiffs: false, // Whether to report differences in headers (default: false) (Optional)
  testBatchSize: 50, // Number of concurrent test requests (default: 50) (Optional)
  //testRequestsLimit": 10, // Number requests to test (Optional)
  //httpReqFiltersArr: [], // "<GET /users>", "<ANY REGEX:^/payments>" (Optional)
  htExtraHeaders: { // Object containing additional headers for HyperTest server requests (Optional)
    // authorization: 'Bearer xyz'
  },
  fastMode: true, // Default false. (aggregate requests only on the basis of request input and output schema - ignoring mock schemas)
  // httpReqsToTest: [], // specific http requests to be tested can be mentioned. Request Id can be taken from "All Requests" page in dashboard.
  // graphqlReqsToTest: [], // specific graphql requests to be tested can be mentioned.
  // grpcReqsToTest: [], // specific grpc requests to be tested can be mentioned.
  // kafkaReqsToTest: [], // specific kafka requests to be tested can be mentioned.
  // amqpReqsToTest: [], // specific amqp requests to be tested can be mentioned.
  // tags: [{name: '', value: ''}], // requests which contain the mentioned tags will be tested. Refer Tags under "User Guides/Node.js SDK" for more information.
  // shouldIgnoreErrStackTraceDiffs: true, // Stack trace differences are ignored in errors.(default: true) (Optional)
  
  // filterFunctionToIgnoreMockDiffs:({ mockDiff, currentMock, requestObj }) => { 
  //  // if false is returned then the diff will be ignored
  //  if(mockDiff?.originalValue?.langType === 'Date') return false;
  //  if(mockDiff?.evaluatedPath?.at(-1) === "url" || mockDiff?.evaluatedPath?.at(-2) === "headers") return false;
  //  if(mockDiff?.evaluatedPath?.at(-1) === "host") return false;
  //  return true;
  //},
  
  // filterFunctionToIgnoreResponseDiffs: ({ responseDiff, requestObj }) => { // Param Types are mentioned in Type References page
  // // if false is returned then the response difference will be ignored  
  //  if(responseDiff?.evaluatedPath?.at(-1) === "url" || evaluatedPath?.evaluatedPath?.at(-2) === "headers") return false;
  //  if(responseDiff?.evaluatedPath?.at(-1) === "host") return false;
  //  return true;
  //},
  // exitCodeSetter({ testResult }) {
  //  console.log('==test results==')
  //  console.log(testResult)
  //  return 0;
  //},
  // initialTimestamp: "", // Initial timestamp in ISO format (Optional)
  // finalTimestamp: "", // Final timestamp in ISO format (Optional)
  // cliServerHost: "", // HT CLI server Host to be Used by Clients(server ignores this) (default: localhost) (Optional)
  // sdkServerHost: "", // HT SDK server Host to be Used by Clients(server ignores this) (default: localhost) (Optional)
  // autoAcceptChangesInCaseOnlyNoiseDetected: false,
  // shouldIgnoreErrStackTraceDiffs: true,
  // exclusionStringsForDifferences: [], // e.g., ['01\.02\.03\.04', 'HyPeRtEsT'],
  // reservedAppPorts: [], // Ports used by the host application e.g., [3001,4001]
};

2. CLI token generation

Please follow the link given below to complete token generation if test is being run in CI env.

CLI Login

3. Start new test

Start a new test by running this command.

htcli start-new-test --config-file-path ./.htTestConf.js

Please complete login for CLI token generation when prompted by cli during test progress on local. For more information click here.

Open the help to list all possible options available

htcli start-new-test --help

4. Test reports

After the test is completed. You can see the results on the dashboard under Test Results