Skip to content

bcgov/jag-traffic-courts-online

Repository files navigation

Maintainability Test Coverage

img Cucumber Tests

jag-traffic-courts-online

Welcome to Traffic Courts Online

Project Structure

    ├── .docker
    │   └── keycloak-realms
    ├── .github
    │   └── workflows
    ├── .gitops
    │   ├── azure
    │   └── charts
    │       ├── jaeger-aio
    │       └── traffic-court-online
    ├── docs
    ├── infrastructure
    │   ├── certificates
    │   ├── keycloak
    │   └── openshift
    ├── splunk-dash-board
    ├── src
    │   ├── backend
    │   │   ├── TrafficCourts
    │   │   └── oracle-data-api
    │   └── frontend
    │       ├── citizen-portal
    │       └── staff-portal
    └── tools
    │   ├── form-recognizer
    │   ├── minio
    │   ├── network-tester
    │   ├── performance-testing
    │   └── ticket-generator
    ├── COMPLIANCE.yaml
    ├── CONTRIBUTING.md
    ├── docker-compose.yml
    ├── LICENSE                                  # Apache License
    └── README.md                                # This file

Configuration

Configuration Sources

The dotnet based projects can read and combine configuration from multiple sources. The configuration uses the default Configuration in ASP.NET Core and is summarized below. The configuration for each app is provided in the following order, from highest to lowest priority:

  1. Command-line arguments using the Command-line configuration provider.
  2. Non-prefixed environment variables using the Non-prefixed environment variables configuration provider.
  3. User secrets when the app runs in the Development environment.
  4. appsettings.{Environment}.json using the JSON configuration provider. For example, appsettings.Production.json and appsettings.Development.json.
  5. appsettings.json using the JSON configuration provider.

Translating configuration between providers

While the rest of this document will showcase configuration examples in environment variable format, the same configuration can be expressed via any of the other configuration sources. For example, the serilog logging configuration in environment variable format can be expressed as:

Serilog__Using__0=Serilog.Sinks.Splunk
Serilog__WriteTo__0__Name=EventCollector
Serilog__WriteTo__0__Args__eventCollectorToken=00000000-0000-0000-0000-000000000000
Serilog__WriteTo__0__Args__splunkHost=http://localhost:8000

Environment variables use __(double underscore) as the hierarchical separator.

The same configuration can be expressed via structured json format in user secrets or appsettings.json files as,

{
    "Serilog": {
        "Using": [
          "Serilog.Sinks.Splunk"
        ],
        "WriteTo": [
            { 
                "Name": "EventCollector",
                "Args": [
                    "eventCollectorToken": "00000000-0000-0000-0000-000000000000",
                    "splunkHost": "http://localhost:8000"
                ]
            }
        ]
    }
}

or a flat json format,

{
    "Serilog__Using__0": "Serilog.Sinks.Splunk",
    "Serilog__WriteTo__0__Name": "EventCollector",
    "Serilog__WriteTo__0__Args__eventCollectorToken": "00000000-0000-0000-0000-000000000000",
    "Serilog__WriteTo__0__Args__splunkHost": "http://localhost:8000"
}

arc-dispute-api

Key Secret Default Description
ASPNETCORE_ENVIRONMENT Production
ASPNETCORE_URLS http://*:8080
OTEL_EXPORTER_JAEGER_ENDPOINT http://jaeger-collector:14268/api/traces
OTEL_EXPORTER_JAEGER_PROTOCOL http/thrift.binary
Sftp__Host Yes
Sftp__SshPrivateKey Yes
Sftp__Username Yes
Serilog__Using__0 Serilog.Sinks.Splunk
Serilog__WriteTo__0__Args__eventCollectorToken Yes
Serilog__WriteTo__0__Args__splunkHost Yes
Serilog__WriteTo__0__Name EventCollector
Swagger__Enabled

citizen-api

Key Secret Default Description
ASPNETCORE_ENVIRONMENT Production
ASPNETCORE_URLS http://*:8080
FormRecognizer__ApiVersion
FormRecognizer__ApiKey Yes
FormRecognizer__Endpoint Yes
FormRecognizer__ModelId
ObjectStorage__AccessKey Yes
ObjectStorage__BucketName Yes
ObjectStorage__Endpoint Yes
ObjectStorage__SecretKey Yes
RabbitMQ__Host
RabbitMQ__Password Yes
RabbitMQ__Port
RabbitMQ__Username Yes
RabbitMQ__VirtualHost /
Redis__ConnectionString
OTEL_EXPORTER_JAEGER_ENDPOINT http://jaeger-collector:14268/api/traces
OTEL_EXPORTER_JAEGER_PROTOCOL http/thrift.binary
Serilog__Using__0 Serilog.Sinks.Splunk
Serilog__WriteTo__0__Args__eventCollectorToken Yes
Serilog__WriteTo__0__Args__splunkHost Yes
Serilog__WriteTo__0__Name EventCollector
Swagger__Enabled false
MassTransit__Transport RabbitMq MassTransit transport. If not configured, will use RabbitMq

citizen-web

app.config.json

oracle-data-api

Key Secret Default Description
CODETABLE_REFRESH_CRON
CODETABLE_REFRESH_ENABLED
H2_DATASOURCE
JAVA_OPTS
OTEL_EXPORTER_JAEGER_ENDPOINT
REDIS_HOST
REDIS_PORT
SPLUNK_TOKEN Yes
SPLUNK_URL Yes
UNASSIGN_DISPUTES_CRON
UNASSIGN_DISPUTES_ENABLED

staff-api

Key Secret Default Description
ASPNETCORE_ENVIRONMENT Production
ASPNETCORE_URLS http://*:8080
ObjectStorage__AccessKey Yes
ObjectStorage__BucketName Yes
ObjectStorage__Endpoint Yes
ObjectStorage__SecretKey Yes
OracleDataApi__BaseUrl
OTEL_EXPORTER_JAEGER_ENDPOINT http://jaeger-collector:14268/api/traces
OTEL_EXPORTER_JAEGER_PROTOCOL http/thrift.binary
Serilog__Using__0 Serilog.Sinks.Splunk
Serilog__WriteTo__0__Args__eventCollectorToken Yes
Serilog__WriteTo__0__Args__splunkHost Yes
Serilog__WriteTo__0__Name EventCollector
Swagger__Enabled
TicketStorage__Type
MassTransit__Transport
Jwt__Audience
Jwt__Authority
KeycloakAdminApi__BaseUri
KeycloakAdminApi__Realm
OAuth__TokenUri
OAuth__ClientId
OAuth__ClientSecret Yes

|

staff-web

keycloak.config.json

workflow-service

Key Secret Default Description
ASPNETCORE_ENVIRONMENT Production
ASPNETCORE_URLS http://*:8080
OracleDataApi__BaseUrl
OTEL_EXPORTER_JAEGER_ENDPOINT http://jaeger-collector:14268/api/traces
OTEL_EXPORTER_JAEGER_PROTOCOL http/thrift.binary
RabbitMQ__ClientProvidedName
RabbitMQ__Host
RabbitMQ__Password Yes
RabbitMQ__Port
RabbitMQ__Username Yes
RabbitMQ__VirtualHost
Serilog__Using__0 Serilog.Sinks.Splunk
Serilog__WriteTo__0__Args__eventCollectorToken Yes
Serilog__WriteTo__0__Args__splunkHost Yes
Serilog__WriteTo__0__Name EventCollector
Swagger__Enabled
MassTransit__Transport

Docker

Download and install Docker

Run docker-compose

Copy the .env.template to .env and then run docker-compose up. Add the configuration for token and password for splunk. Default user is admin. Password is what is configured in .env

REDIS__HOST will be redis which is the service name.

docker-compose up

Examples

Develop the citizen-portal. Run the associated API, citizen-api. This starts the required services:

  • citizen-api
  • rabbitmq
docker-compose -f docker-compose.yml up -d citizen-api

Run citizen-api and have the backend logs go to local Splunk

Note, this is currently getting an error: "Error response from daemon: network ... not found"

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.splunk.yml up -d citizen-api

To stop when running Splunk,

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.splunk.yml down

The frontend app citizen-portal will be accessible in the browser at http://localhost:8080

To remove services run (all services and networking)

docker-compose down

Application Services

User Interfaces Ports: 8xxx

APIs Ports: 5xxx

Host Port: This is the port the developer can access from their computer. The developer will use localhost:port, ie http://localhost:8080/

Container Port: This is the port that is available inside the Docker network. The connection will use service-name:port, for example, http://citizen-api:8080

Service Host Port Container Port Notes
arc-dispute-api 5030 8080
citizen-api 5080 8080
citizen-portal 8080 8080
oracle-data-api 5010 8080
staff-api 5090 8080
staff-portal 8090 8080
workflow-service 5020 8080

Infrastructure Services

Host Port: This is the port the developer can access from their computer. The developer will use localhost:port, ie http://localhost:3000/

Container Port: This is the port that is available inside the Docker network. The connection will use service-name:port, for example, http://coms:3000

Whenever possible, the forwarded ports for infrastructure services should match their standard container exposed ports.

Service Host Port Container Port Notes
coms 3000 3000
coms-db 5432 5432 Postgres
form-recognizer 5200 5200
jaeger 14250 14250 accept model.proto directly from clients
jaeger 14268 14268 accept jaeger.thrift directly from clients
jaeger 16686 16686 Jaeger UI
minio 1 9100 9000 API Server (coms)
minio 9101 9001 Web Console
rabbitmq 5672 5672 AMQP 0-9-1 and AMQP 1.0 clients (masstransit clients)
rabbitmq 15672 15672 HTTP API clients, management UI and rabbitmqadmin
redis 6379 6379 cache clients
redis-ui 8082 8082 redis-commander
sftp 22000 22 Port 22 is protected and requires admin, avoid low port numbers
splunk 8000 8000 Web Console
splunk 8088 8088 HTTP Event Collector
virus-scan-service 5040 8080
smtp-server 5125 8080 Web Application
smtp-server 5025 5025 SMTP Server

1 ZSATunnel uses port 9000. Minio host ports are 100 more than the container ports

Logging

Developers can choose how logging is configured in the running apps. Developer can choose Splunk or Seq.

Splunk

The default docker-compose.yaml file does NOT contain Splunk configuration. To use/test with Splunk, you can include a docker-compose override to configure Splunk for the requested services, ie,

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.splunk.yml up

Open Local Splunk to view logs. Login with username: admin, password: password.

Splunk

A custom configuration file, ./.docker/splunk-dev-config.yaml is used to adjust the default settings. A key setting is disabling the SSL on the HEC endpoint.

See Splunk Docker examples for more information.

Seq

Seq is an alternative logging source that is more developer friendly, especially those unfamilar with Splunk.

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.seq.yml up

Open Local Seq to view logs.

Redis

By default, redis runs in Standalone mode (a single container). To run the project where redis is configured to run in sentinel mode (a high-availability failover configuration), specify the redis override file and run:

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.redis.yml up -d

Form Recognizer

A local containerized instance of Azure's Form Recognizer is available, consisting of 4 containers:

  • azure-cognitive-service-proxy (an nginx router to the below services)
  • azure-cognitive-service-custom-api (main api to the service)
  • azure-cognitive-service-custom-layout (the container that does all the work)
  • azure-cognitive-service-custom-supervised (tracks billing, submits to Azure)
docker-compose -f docker-compose.yml -f .docker/docker-compose-ocr.yml up -d

Configuration

Environment variable Description Default
FORMRECOGNIZER__APIVERSION One of "2.1" or "2022-06-30-preview" 2022-06-30-preview
FORMRECOGNIZER__APIKEY Billing key of Azure Form Recognizer (32 character GUID, excluding hypens)
FORMRECOGNIZER__BILLING_URL Azure endpoint for billing purposes (where the apikey is used)
FORMRECOGNIZER__ENDPOINT API endpoint
FORMRECOGNIZER__MODELID If API version is 2.1, this is the 36 character GUID of the model id (including hypens). If API version is 2022-06-30-preview, this is the model name, ie. ViolationTicket

Use 2.1 as the FORMRECOGNIZER__APIVERSION to target Form Recognizer running locally in Docker (or in OpenShift). Use 2022-06-30-preview to target Form Recognizer running in Azure cloud (which runs the latest api version, not 2.1).

The API key and Billing URL are found in the Azure Portal. Either Key 1 or Key 2 can be used for the API key.

 Form Recognizer

See Form Recognizer Containers (2.1) and Form Recognizer Containers (3.0) for information.

Debugging

A Developer can debug the application by attaching debugging instances to the processes running inside the docker containers.

Oracle Data API

To debug a running java application inside a docker container:

  1. Ensure the codebase is the same in Eclipse as what is running in the Docker container.

  2. For security reasons, java applications do not run in debug mode by default. To launch the oracle-data-api in debug mode (a mode that can accept debugging sessions), set the necessary -X flags on the container's entrypoint. This can be accomplish by simply having the JAVA_OPTS environment variable set in the .env file when lauching docker-compose. (The .env file is a file for custom env variables and should be next to the project's docker-compose.yaml file).

    JAVA_OPTS=-Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=0.0.0.0:8000,server=y,suspend=n -Dlogging.level.ca.bc.gov.open.jag.tco.oracledataapi=DEBUG

    The JAVA_OPTS environment variable is passed to the oracle-data-api Dockerfile when launching the container. The above configuration will enable the jvm to listen on port 8000 for debugging sessions. If suspend=y, the application will pause before launching for any attaching debug instances so a developer can debug the application's startup process. If suspend=n, the application will launch as normal and one can attach a debugging instance later.

  3. Create a Remote Java Application launch task in Eclipse. Run -> Debug Configurations -> add Remote Java Application Project: oracle-data-api Connection Type: Standard (Socket Attach) Connection Properties: Host: localhost Port: 8000

  4. Done. The application should suspend at any breakpoints in the project.

.NET projects (staff-api, citizen-api, workflow-service, etc)

To debug a running .NET application inside a docker container:

  1. Ensure the codebase is the same in Visual Studio as what is running in the Docker container.
  2. Navigate to Debug -> Attach to Process ...
  3. Connection type: Docker (Linux Container) Connection target: staff-api Click Attach
  4. Done. The application should suspend at any breakpoints in the project.