When trying to generate coverage from yml or json file the coverage is 0, also the coveredApiCount.

[CopilotMe]

Steps to reproduce:
Case 1 - json working with proper test Get pet by status

psc.swaggerJsonUrl = "https://petstore.swagger.io/v2/swagger.json";

Output 1 - No tests:

{
  "basePath": "/v2",
  "coverage": 0,
  "coveredApiCount": 0,
  "missedApiCount": 14,
  "totalApiCount": 14,
  "coveredApiList": [],
  "missedApiList": [
    "/v2/pet/{petId}/uploadImage",
    "/v2/pet",
    "/v2/pet/findByStatus",
    "/v2/pet/findByTags",
    "/v2/pet/{petId}",
    "/v2/store/order",
    "/v2/store/order/{orderId}",
    "/v2/store/inventory",
    "/v2/user/createWithArray",
    "/v2/user/createWithList",
    "/v2/user/{username}",
    "/v2/user/login",
    "/v2/user/logout",
    "/v2/user"
  ]
}

Output 2 - ok with 1 test executed on /v2/pet/findByStatus

{
  "basePath": "/v2",
  "coverage": 0.21,
  "coveredApiCount": 3,
  "missedApiCount": 11,
  "totalApiCount": 14,
  "coveredApiList": [
    "/v2/pet",
    "/v2/pet/findByStatus",
    "/v2/pet/{petId}"
  ],
  "missedApiList": [
    "/v2/pet/{petId}/uploadImage",
    "/v2/pet/findByTags",
    "/v2/store/order",
    "/v2/store/order/{orderId}",
    "/v2/store/inventory",
    "/v2/user/createWithArray",
    "/v2/user/createWithList",
    "/v2/user/{username}",
    "/v2/user/login",
    "/v2/user/logout",
    "/v2/user"
  ]
}

Test used:

      await spec()
        .get('https://petstore.swagger.io/v2/pet/findByStatus')
        .withPathParams("status", 'available')
        .expectStatus(200)

Case 2 - yml file

1. Set path to yml file:

psc.swaggerYamlPath = 'C:\\Users\\Copilotme\\Downloads\\openapi3-sample.yml'

2. Output:

{
  "basePath": "http://petstore.swagger.io/v1",
  "coverage": 0,
  "coveredApiCount": 0,
  "missedApiCount": 2,
  "totalApiCount": 2,
  "coveredApiList": [],
  "missedApiList": [
    "http://petstore.swagger.io/v1/pets",
    "http://petstore.swagger.io/v1/pets/{petId}"
  ]
}

3. Sample yml is not attached but pasted as non-supported to upload

openapi: "3.0.0"
info:
  version: 1.0.0
  title: Swagger Petstore
  license:
    name: MIT
servers:
  - url: http://petstore.swagger.io/v1
paths:
  /pets:
    get:
      summary: List all pets
      operationId: listPets
      tags:
        - pets
      parameters:
        - name: limit
          in: query
          description: How many items to return at one time (max 100)
          required: false
          schema:
            type: integer
            format: int32
      responses:
        200:
          description: An paged array of pets
          headers:
            x-next:
              description: A link to the next page of responses
              schema:
                type: string
          content:
            application/json:    
              schema:
                $ref: "#/components/schemas/Pets"
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
    post:
      summary: Create a pet
      operationId: createPets
      tags:
        - pets
      responses:
        201:
          description: Null response
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
  /pets/{petId}:
    get:
      summary: Info for a specific pet
      operationId: showPetById
      tags:
        - pets
      parameters:
        - name: petId
          in: path
          required: true
          description: The id of the pet to retrieve
          schema:
            type: string
      responses:
        200:
          description: Expected response to a valid request
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Pets"
        default:
          description: unexpected error
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Error"
components:
  schemas:
    Pet:
      required:
        - id
        - name
      properties:
        id:
          type: integer
          format: int64
        name:
          type: string
        tag:
          type: string
    Pets:
      type: array
      items:
        $ref: "#/components/schemas/Pet"
    Error:
      required:
        - code
        - message
      properties:
        code:
          type: integer
          format: int32
        message:
          type: string

package.json

"dependencies": {
    "pactum": "^3.5.1"
  },
  "devDependencies": {
    "pactum-swagger-coverage": "^2.0.0"
  },

[leelaprasadv]

@CopilotMe Could you also share below

• Sample test spec/setup that generated this output.
• Test runner used in your setup.
• NodeJS version.

Thanks!

[CopilotMe]

@leelaprasadv Sure. Here's it is:

• v16.13.2
• mocha runner for pactum tests: "mocha": "^10.2.0"
• The desired output is as in Case 1 - Output 2
• Disregard Case 2 petstore v1 as it is not available (404).
• Here are another examples for private api:

Case 3: Serve the json on localhost
(Case 4 - same but using local yml file + overriden basepath produces same result - 0 coverage)

psc.swaggerJsonUrl = 'http://localhost:3333/openapi-person.json'

Expected Output:

{
"coverage": 1,
"coveredApiCount": 1,
"missedApiCount": 0,
"totalApiCount": 1,
"coveredApiList": [
    "http://localhost:3333/api/person",
],
"missedApiList": [

  ]
}

Actual Output:

{
"coverage": 0,
"coveredApiCount": 0,
"missedApiCount": 1,
"totalApiCount": 1,
"coveredApiList": [],
"missedApiList": [
    "http://localhost:3333/api/person",
  ]
}

Script and output:

npx mocha test/api/person.spec.js


  Person
    ✔ should get person (767ms)
    1) "after all" hook for "should get person"


  1 passing (4s)
  1 failing

  1) Person
       "after all" hook for "should get person":
     TypeError: Cannot convert undefined or null to object
      at Function.keys (<anonymous>)
      at getApiPaths (node_modules\pactum-swagger-coverage\src\lib\core.js:51:27)
      at Object.getSwaggerCoverage (node_modules\pactum-swagger-coverage\src\lib\core.js:67:20)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)
      at async Object.end (node_modules\pactum-swagger-coverage\src\index.js:32:22)
      at async Object.end (node_modules\pactum\src\exports\reporter.js:32:23)

[CopilotMe]

Hey,
I've managed to make it work when using swagger structure as in petstore example.
But it is not working for me when I use "openapi": "3.0.1" (which is used in most of our projects) with json structure:

{
    "openapi": "3.0.1",
    "info":  {
        "title": "API",
        "description": "API",
        "version": "1.0"
    },
    "servers":  [
        {
            "url": "https://",
            "description": "Generated server url"
        }
    ],
    "paths": {
        "/api/v1/xxx": {
            "get": { ... }
            "post": { ... }
        }
    },
    "components": { ... }
}

in which we are missing host, basePath, etc.

Do you support openapi 3.0.1?
What is essential in json structure so the coverage to work properly?

Another important question:
As I have split covering of endpoints in groups (admin, mobile, etc.) in separate test suites how to get combined coverage for them?
Example: Total endpoints to be covered: 9, and I have covered in each spec next number of endpoints:

• spec 1: 2 -> both covered with tests and psc is 2/9 = 0.22
• spec 2: 3 -> 1 covered and 2 missed so psc is 1/9 = 0.11
• spec 3: 4 -> 2 covered and 2 missed so psc is 2/9 = 0.22
• Total: 5 covered, 4 missed -> psc of 5/9 = 0.55
• How we can combine the 3 psc reports generated for the 3 specs so to have the combined coverage,coveredApiCount, missedApiCount and the updated lists for coveredApiList and missedApiList? (with the assumption that totalApiCount will be the same for each measurement as we'll be using same swagger json)

Thanks in advance.