Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build an initial "risk assessment" worker #74

Open
brianok77 opened this issue Apr 29, 2020 · 1 comment
Open

Build an initial "risk assessment" worker #74

brianok77 opened this issue Apr 29, 2020 · 1 comment

Comments

@brianok77
Copy link

brianok77 commented Apr 29, 2020

A "risk assessment" worker is a Kotlin service running in a container that listens to an SQS queue for a "risk assessment" payload. When one is received, it launches a thread for the task that does the following:

  1. loads the proper risk assessment scoring algorithm measure from a library of selections (by an identifier in the payload)
  2. executes the risk assessment measure on a single device node (by an identifier in the payload), outputting a risk score
  3. saves the score to a DynamoDb table, where partition key is the device id and the sort key is the "{risk assessment measure id}#{timestamp}" of the assessment. The table contains the score as a column (see Create a DynamoDb table to store risk assessment results #77 for full details)
  4. triggers an SNS notification that the work was completed.
  5. either terminates itself or indicates to the manager that it is available for a new task.

NOTE: We should have a parameter to set a max "worker thread" per container because each measure might have different memory requirements and/or different computational complexity. Long term, we should have a cloudwatch alarm on the SQS topic that will spin up more of these containers should it back up.

NOTE 2: We need to make sure the SQS topic visibility timeout is sufficiently long. The default of 30 seconds may not be long enough, which will result in the workers doing repeat work.

@senthilmk
Copy link

@brianok77 Response to Note#2: Have responded to task #75.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants