Start a Test Run

Prerequisites

1. Install/update ht-cli npm package

To Install and use HyperTest's ht-cli package, we will do the following steps:

1. Create PAT for the user - Hypertest Team will do this for you

2. Add NPM repo login credential in ~/.npmrc

3. Install ht-cli package globally using the following command

Add the following content in your ~/.npmrc.

1st line in this file has authToken for pulling packages from NPM registry.

2nd line indicates to use https://npm.pkg.github.com registry to pull packages scoped with @hypertestco.

.npmrc

//npm.pkg.github.com/:_authToken=<PAT>
@hypertestco:registry=https://npm.pkg.github.com/ 

npm install -g @hypertestco/ht-cli --save-exact

Verify htcli has been installed using the below command:

htcli --help

Getting Started:

1. Generate a CLI auth token

Go to your accounts page

Click to create a new CLI token

Copy the newly generated CLI token to use in the test configuration file in the next step

2. Create a Test Configuration File

Available Configurations:

.htTestConf.js

const requestTypes = {
  HTTP: 'HTTP',
  GRAPHQL: 'GRAPHQL',
  KAFKA: 'KAFKA',
  GRPC: 'GRPC',
  AMQP: 'AMQP',
};

module.exports = {
  htBackendBaseUrl: "", // URL of HyperTest server (Required)
  htCliRefreshToken: "",  // Auth token for the CLI (Required)
  serviceIdentifier: "", // UUID for the service (Required)
  requestTypesToTest: [requestTypes.HTTP], // What kind of requests to include in the test
  httpCandidateUrl: "", // HTTP URL of App under test (Optional)
  // graphqlCandidateUrl: "", // GraphQL URL of App under test (Optional)
  appStartCommand: "", // Command to start the app (Required)
  appStartCommandArgs: ["<Argument 1>", "<Argument 2>"],  // App start command arguments (Required)
  appWorkingDirectory: __dirname, // Working directory for the app (default: current working dir) (Optional)
  appStartTimeoutSec: 30, // Timeout in seconds for the start command (default: 10) (Optional)
  showAppLogs: true, // Whether to show app logs (default: false) (Optional)
  shouldReportHeaderDiffs: false, // Whether to report differences in headers (default: false) (Optional)
  testBatchSize: 50, // Number of concurrent test requests (default: 50) (Optional)
  //testRequestsLimit": 10, // Number requests to test (Optional)
  //httpReqFiltersArr: [], // "<GET /users>", "<ANY REGEX:^/payments>" (Optional)
  htExtraHeaders: { // Object containing additional headers for HyperTest server requests (Optional)
    // authorization: 'Bearer xyz'
  },
  fastMode: true, // Default false. (aggregate requests only on the basis of request input and output schema - ignoring mock schemas)
  // httpReqsToTest: [], // specific http requests to be tested can be mentioned. Request Id can be taken from "All Requests" page in dashboard.
  // graphqlReqsToTest: [], // specific graphql requests to be tested can be mentioned.
  // grpcReqsToTest: [], // specific grpc requests to be tested can be mentioned.
  // kafkaReqsToTest: [], // specific kafka requests to be tested can be mentioned.
  // amqpReqsToTest: [], // specific amqp requests to be tested can be mentioned.
  // tags: [{name: '', value: ''}], // requests which contain the mentioned tags will be tested. Refer Tags under "User Guides/Node.js SDK" for more information.
  // shouldIgnoreErrStackTraceDiffs: true, // Stack trace differences are ignored in errors.(default: true) (Optional)
  
  // filterFunctionToIgnoreMockDiffs:({ mockDiff, currentMock, requestObj }) => { 
  //  // if false is returned then the diff will be ignored
  //  if(mockDiff?.originalValue?.langType === 'Date') return false;
  //  if(mockDiff?.evaluatedPath?.at(-1) === "url" || mockDiff?.evaluatedPath?.at(-2) === "headers") return false;
  //  if(mockDiff?.evaluatedPath?.at(-1) === "host") return false;
  //  return true;
  //},
  
  // filterFunctionToIgnoreResponseDiffs: ({ responseDiff, requestObj }) => { // Param Types are mentioned in Type References page
  // // if false is returned then the response difference will be ignored  
  //  if(responseDiff?.evaluatedPath?.at(-1) === "url" || evaluatedPath?.evaluatedPath?.at(-2) === "headers") return false;
  //  if(responseDiff?.evaluatedPath?.at(-1) === "host") return false;
  //  return true;
  //},
  // exitCodeSetter({ testResult }) {
  //  console.log('==test results==')
  //  console.log(testResult)
  //  return 0;
  //},
  // initialTimestamp: "", // Initial timestamp in ISO format (Optional)
  // finalTimestamp: "", // Final timestamp in ISO format (Optional)
  // cliServerHost: "", // HT CLI server Host to be Used by Clients(server ignores this) (default: localhost) (Optional)
  // sdkServerHost: "", // HT SDK server Host to be Used by Clients(server ignores this) (default: localhost) (Optional)
  // autoAcceptChangesInCaseOnlyNoiseDetected: false,
  // shouldIgnoreErrStackTraceDiffs: true,
  // exclusionStringsForDifferences: [], // e.g., ['01\.02\.03\.04', 'HyPeRtEsT'],
  // reservedAppPorts: [], // Ports used by the host application e.g., [3001,4001]
};

3. Start new test

Start a new test by running this command.

htcli start-new-test --config-file-path ./.htTestConf.js

Open the help to list all possible options available

htcli start-new-test --help

4. Test reports

After the test is completed. You can see the results on the dashboard under Test Results