jag-traffic-courts-online

Welcome to Traffic Courts Online

Project Structure

    ├── .docker
    │   └── keycloak-realms
    ├── .github
    │   └── workflows
    ├── .gitops
    │   ├── azure
    │   └── charts
    │       ├── jaeger-aio
    │       └── traffic-court-online
    ├── docs
    ├── infrastructure
    │   ├── certificates
    │   ├── keycloak
    │   └── openshift
    ├── splunk-dash-board
    ├── src
    │   ├── backend
    │   │   ├── TrafficCourts
    │   │   └── oracle-data-api
    │   └── frontend
    │       ├── citizen-portal
    │       └── staff-portal
    └── tools
    │   ├── form-recognizer
    │   ├── minio
    │   ├── network-tester
    │   ├── performance-testing
    │   └── ticket-generator
    ├── COMPLIANCE.yaml
    ├── CONTRIBUTING.md
    ├── docker-compose.yml
    ├── LICENSE                                  # Apache License
    └── README.md                                # This file

Configuration

Configuration Sources

The dotnet based projects can read and combine configuration from multiple sources. The configuration uses the default Configuration in ASP.NET Core and is summarized below. The configuration for each app is provided in the following order, from highest to lowest priority:

Command-line arguments using the Command-line configuration provider.
Non-prefixed environment variables using the Non-prefixed environment variables configuration provider.
User secrets when the app runs in the Development environment.
appsettings.{Environment}.json using the JSON configuration provider. For example, appsettings.Production.json and appsettings.Development.json.
appsettings.json using the JSON configuration provider.

Translating configuration between providers

While the rest of this document will showcase configuration examples in environment variable format, the same configuration can be expressed via any of the other configuration sources. For example, the serilog logging configuration in environment variable format can be expressed as:

Serilog__Using__0=Serilog.Sinks.Splunk
Serilog__WriteTo__0__Name=EventCollector
Serilog__WriteTo__0__Args__eventCollectorToken=00000000-0000-0000-0000-000000000000
Serilog__WriteTo__0__Args__splunkHost=http://localhost:8000

Environment variables use __(double underscore) as the hierarchical separator.

The same configuration can be expressed via structured json format in user secrets or appsettings.json files as,

{
    "Serilog": {
        "Using": [
          "Serilog.Sinks.Splunk"
        ],
        "WriteTo": [
            { 
                "Name": "EventCollector",
                "Args": [
                    "eventCollectorToken": "00000000-0000-0000-0000-000000000000",
                    "splunkHost": "http://localhost:8000"
                ]
            }
        ]
    }
}

or a flat json format,

{
    "Serilog__Using__0": "Serilog.Sinks.Splunk",
    "Serilog__WriteTo__0__Name": "EventCollector",
    "Serilog__WriteTo__0__Args__eventCollectorToken": "00000000-0000-0000-0000-000000000000",
    "Serilog__WriteTo__0__Args__splunkHost": "http://localhost:8000"
}

arc-dispute-api

Key	Secret	Default
ASPNETCORE_ENVIRONMENT		Production
ASPNETCORE_URLS		http://*:8080
OTEL_EXPORTER_JAEGER_ENDPOINT		http://jaeger-collector:14268/api/traces
OTEL_EXPORTER_JAEGER_PROTOCOL		http/thrift.binary
Sftp__Host	Yes
Sftp__SshPrivateKey	Yes
Sftp__Username	Yes
Serilog__Using__0		Serilog.Sinks.Splunk
Serilog__WriteTo__0__Args__eventCollectorToken	Yes
Serilog__WriteTo__0__Args__splunkHost	Yes
Serilog__WriteTo__0__Name		EventCollector
Swagger__Enabled

citizen-api

Key	Secret	Default	Description
ASPNETCORE_ENVIRONMENT		Production
ASPNETCORE_URLS		http://*:8080
FormRecognizer__ApiVersion
FormRecognizer__ApiKey	Yes
FormRecognizer__Endpoint	Yes
FormRecognizer__ModelId
ObjectStorage__AccessKey	Yes
ObjectStorage__BucketName	Yes
ObjectStorage__Endpoint	Yes
ObjectStorage__SecretKey	Yes
RabbitMQ__Host
RabbitMQ__Password	Yes
RabbitMQ__Port
RabbitMQ__Username	Yes
RabbitMQ__VirtualHost		/
Redis__ConnectionString
OTEL_EXPORTER_JAEGER_ENDPOINT		http://jaeger-collector:14268/api/traces
OTEL_EXPORTER_JAEGER_PROTOCOL		http/thrift.binary
Serilog__Using__0		Serilog.Sinks.Splunk
Serilog__WriteTo__0__Args__eventCollectorToken	Yes
Serilog__WriteTo__0__Args__splunkHost	Yes
Serilog__WriteTo__0__Name		EventCollector
Swagger__Enabled		false
MassTransit__Transport		RabbitMq	MassTransit transport. If not configured, will use RabbitMq

citizen-web

app.config.json

oracle-data-api

Key	Secret	Default	Description
CODETABLE_REFRESH_CRON
CODETABLE_REFRESH_ENABLED
H2_DATASOURCE
JAVA_OPTS
OTEL_EXPORTER_JAEGER_ENDPOINT
REDIS_HOST
REDIS_PORT
SPLUNK_TOKEN	Yes
SPLUNK_URL	Yes
UNASSIGN_DISPUTES_CRON
UNASSIGN_DISPUTES_ENABLED

staff-api

Key	Secret	Default
ASPNETCORE_ENVIRONMENT		Production
ASPNETCORE_URLS		http://*:8080
ObjectStorage__AccessKey	Yes
ObjectStorage__BucketName	Yes
ObjectStorage__Endpoint	Yes
ObjectStorage__SecretKey	Yes
OracleDataApi__BaseUrl
OTEL_EXPORTER_JAEGER_ENDPOINT		http://jaeger-collector:14268/api/traces
OTEL_EXPORTER_JAEGER_PROTOCOL		http/thrift.binary
Serilog__Using__0		Serilog.Sinks.Splunk
Serilog__WriteTo__0__Args__eventCollectorToken	Yes
Serilog__WriteTo__0__Args__splunkHost	Yes
Serilog__WriteTo__0__Name		EventCollector
Swagger__Enabled
TicketStorage__Type
MassTransit__Transport
Jwt__Audience
Jwt__Authority
KeycloakAdminApi__BaseUri
KeycloakAdminApi__Realm
OAuth__TokenUri
OAuth__ClientId
OAuth__ClientSecret	Yes

|

staff-web

keycloak.config.json

workflow-service

Key	Secret	Default
ASPNETCORE_ENVIRONMENT		Production
ASPNETCORE_URLS		http://*:8080
OracleDataApi__BaseUrl
OTEL_EXPORTER_JAEGER_ENDPOINT		http://jaeger-collector:14268/api/traces
OTEL_EXPORTER_JAEGER_PROTOCOL		http/thrift.binary
RabbitMQ__ClientProvidedName
RabbitMQ__Host
RabbitMQ__Password	Yes
RabbitMQ__Port
RabbitMQ__Username	Yes
RabbitMQ__VirtualHost
Serilog__Using__0		Serilog.Sinks.Splunk
Serilog__WriteTo__0__Args__eventCollectorToken	Yes
Serilog__WriteTo__0__Args__splunkHost	Yes
Serilog__WriteTo__0__Name		EventCollector
Swagger__Enabled
MassTransit__Transport

Docker

Download and install Docker

Run docker-compose

Copy the .env.template to .env and then run docker-compose up. Add the configuration for token and password for splunk. Default user is admin. Password is what is configured in .env

REDIS__HOST will be redis which is the service name.

docker-compose up

Examples

Develop the citizen-portal. Run the associated API, citizen-api. This starts the required services:

citizen-api
rabbitmq

docker-compose -f docker-compose.yml up -d citizen-api

Run citizen-api and have the backend logs go to local Splunk

Note, this is currently getting an error: "Error response from daemon: network ... not found"

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.splunk.yml up -d citizen-api

To stop when running Splunk,

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.splunk.yml down

The frontend app citizen-portal will be accessible in the browser at http://localhost:8080

To remove services run (all services and networking)

docker-compose down

Application Services

User Interfaces Ports: 8xxx

APIs Ports: 5xxx

Host Port: This is the port the developer can access from their computer. The developer will use localhost:port, ie http://localhost:8080/

Container Port: This is the port that is available inside the Docker network. The connection will use service-name:port, for example, http://citizen-api:8080

Service	Host Port	Container Port
arc-dispute-api	5030	8080
citizen-api	5080	8080
citizen-portal	8080	8080
oracle-data-api	5010	8080
staff-api	5090	8080
staff-portal	8090	8080
workflow-service	5020	8080

Infrastructure Services

Host Port: This is the port the developer can access from their computer. The developer will use localhost:port, ie http://localhost:3000/

Container Port: This is the port that is available inside the Docker network. The connection will use service-name:port, for example, http://coms:3000

Whenever possible, the forwarded ports for infrastructure services should match their standard container exposed ports.

Service	Host Port	Container Port	Notes
coms	3000	3000
coms-db	5432	5432	Postgres
form-recognizer	5200	5200
jaeger	14250	14250	accept model.proto directly from clients
jaeger	14268	14268	accept jaeger.thrift directly from clients
jaeger	16686	16686	Jaeger UI
minio ¹	9100	9000	API Server (coms)
minio	9101	9001	Web Console
rabbitmq	5672	5672	AMQP 0-9-1 and AMQP 1.0 clients (masstransit clients)
rabbitmq	15672	15672	HTTP API clients, management UI and rabbitmqadmin
redis	6379	6379	cache clients
redis-ui	8082	8082	redis-commander
sftp	22000	22	Port 22 is protected and requires admin, avoid low port numbers
splunk	8000	8000	Web Console
splunk	8088	8088	HTTP Event Collector
virus-scan-service	5040	8080
smtp-server	5125	8080	Web Application
smtp-server	5025	5025	SMTP Server

¹ ZSATunnel uses port 9000. Minio host ports are 100 more than the container ports

Logging

Developers can choose how logging is configured in the running apps. Developer can choose Splunk or Seq.

Splunk

The default docker-compose.yaml file does NOT contain Splunk configuration. To use/test with Splunk, you can include a docker-compose override to configure Splunk for the requested services, ie,

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.splunk.yml up

Open Local Splunk to view logs. Login with username: admin, password: password.

Splunk

A custom configuration file, ./.docker/splunk-dev-config.yaml is used to adjust the default settings. A key setting is disabling the SSL on the HEC endpoint.

See Splunk Docker examples for more information.

Seq

Seq is an alternative logging source that is more developer friendly, especially those unfamilar with Splunk.

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.seq.yml up

Open Local Seq to view logs.

Redis

By default, redis runs in Standalone mode (a single container). To run the project where redis is configured to run in sentinel mode (a high-availability failover configuration), specify the redis override file and run:

docker-compose -f docker-compose.yml -f ./.docker/docker-compose.redis.yml up -d

Form Recognizer

A local containerized instance of Azure's Form Recognizer is available, consisting of 4 containers:

azure-cognitive-service-proxy (an nginx router to the below services)
azure-cognitive-service-custom-api (main api to the service)
azure-cognitive-service-custom-layout (the container that does all the work)
azure-cognitive-service-custom-supervised (tracks billing, submits to Azure)

docker-compose -f docker-compose.yml -f .docker/docker-compose-ocr.yml up -d

Configuration

Environment variable	Description	Default
FORMRECOGNIZER__APIVERSION	One of "2.1" or "2022-06-30-preview"	2022-06-30-preview
FORMRECOGNIZER__APIKEY	Billing key of Azure Form Recognizer (32 character GUID, excluding hypens)
FORMRECOGNIZER__BILLING_URL	Azure endpoint for billing purposes (where the apikey is used)
FORMRECOGNIZER__ENDPOINT	API endpoint
FORMRECOGNIZER__MODELID	If API version is 2.1, this is the 36 character GUID of the model id (including hypens). If API version is 2022-06-30-preview, this is the model name, ie. ViolationTicket

Use 2.1 as the FORMRECOGNIZER__APIVERSION to target Form Recognizer running locally in Docker (or in OpenShift). Use 2022-06-30-preview to target Form Recognizer running in Azure cloud (which runs the latest api version, not 2.1).

The API key and Billing URL are found in the Azure Portal. Either Key 1 or Key 2 can be used for the API key.

See Form Recognizer Containers (2.1) and Form Recognizer Containers (3.0) for information.

Debugging

A Developer can debug the application by attaching debugging instances to the processes running inside the docker containers.

Oracle Data API

To debug a running java application inside a docker container:

Ensure the codebase is the same in Eclipse as what is running in the Docker container.
For security reasons, java applications do not run in debug mode by default. To launch the oracle-data-api in debug mode (a mode that can accept debugging sessions), set the necessary -X flags on the container's entrypoint. This can be accomplish by simply having the JAVA_OPTS environment variable set in the .env file when lauching docker-compose. (The .env file is a file for custom env variables and should be next to the project's docker-compose.yaml file).

JAVA_OPTS=-Xdebug -Xrunjdwp:server=y,transport=dt_socket,address=0.0.0.0:8000,server=y,suspend=n -Dlogging.level.ca.bc.gov.open.jag.tco.oracledataapi=DEBUG

The JAVA_OPTS environment variable is passed to the oracle-data-api Dockerfile when launching the container. The above configuration will enable the jvm to listen on port 8000 for debugging sessions. If suspend=y, the application will pause before launching for any attaching debug instances so a developer can debug the application's startup process. If suspend=n, the application will launch as normal and one can attach a debugging instance later.
Create a Remote Java Application launch task in Eclipse. Run -> Debug Configurations -> add Remote Java Application Project: oracle-data-api Connection Type: Standard (Socket Attach) Connection Properties: Host: localhost Port: 8000
Done. The application should suspend at any breakpoints in the project.

.NET projects (staff-api, citizen-api, workflow-service, etc)

To debug a running .NET application inside a docker container:

Ensure the codebase is the same in Visual Studio as what is running in the Docker container.
Navigate to Debug -> Attach to Process ...
Connection type: Docker (Linux Container) Connection target: staff-api Click Attach
Done. The application should suspend at any breakpoints in the project.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

jag-traffic-courts-online

Project Structure

Configuration

Configuration Sources

Translating configuration between providers

arc-dispute-api

citizen-api

citizen-web

oracle-data-api

staff-api

staff-web

workflow-service

Docker

Run docker-compose

Examples

Application Services

Infrastructure Services

Logging

Splunk

Splunk

Seq

Redis

Form Recognizer

Configuration

Debugging

Oracle Data API

.NET projects (staff-api, citizen-api, workflow-service, etc)

Files

README.md

Latest commit

History

README.md

File metadata and controls

jag-traffic-courts-online

Project Structure

Configuration

Configuration Sources

Translating configuration between providers

arc-dispute-api

citizen-api

citizen-web

oracle-data-api

staff-api

staff-web

workflow-service

Docker

Run docker-compose

Examples

Application Services

Infrastructure Services

Logging

Splunk

Splunk

Seq

Redis

Form Recognizer

Configuration

Debugging

Oracle Data API

.NET projects (staff-api, citizen-api, workflow-service, etc)