support callback request subscribers and job notification refactoring

[fmigneault]

Changes

• Add Job subscribers support to define OGC-compliant callback URLs where HTTP(S) requests will be sent upon
  reaching certain Job status milestones (fixes [Feature] Support and validate Job notification callback URL #230).
• Add email notification support to the new subscribers definition (extension over OGC minimal requirements).
• Deprecate Job notification_email in the OpenAPI specification in favor of subscribers, but preserve
  parsing of its value if provided in the JSON body during Job submission for backward compatibility support of
  existing servers. The Job.notification_email attribute is removed to avoid duplicate references.
• Add notification email for Job started status, only available through the subscribers property.
• Add {PROCESS_ID}/{STATUS}.mako template detection under the weaver.wps_email_notify_template_dir location
  to allow per-Process and per-Job status email customization.
• Refactor weaver/notify.py and weaver/processes/execution.py to avoid mixed references to the
  encryption/decryption logic employed for notification emails. All notifications including emails and
  callback requests are now completely handled and contained in the weaver/notify.py module.
• Remove partially duplicate Mako Template definition as hardcoded string and separate file for email notification.

References

• Relates to Clarify what is expected for callback request/response opengeospatial/ogcapi-processes#366
• Relates to fix notification email encrypt/decrypt implementation #571
• Relates to Weaver tries to send e-mail to hashed notification_email #568
• Fixes [Feature] Support and validate Job notification callback URL #230

[codecov]

Codecov Report

Merging #572 (fe55014) into master (1572fd0) will increase coverage by 0.16%.
The diff coverage is 95.48%.

@@            Coverage Diff             @@
##           master     #572      +/-   ##
==========================================
+ Coverage   84.71%   84.87%   +0.16%     
==========================================
  Files          79       79              
  Lines       17828    17910      +82     
  Branches     2708     2716       +8     
==========================================
+ Hits        15103    15202      +99     
+ Misses       1992     1980      -12     
+ Partials      733      728       -5     

Files	Coverage Δ
weaver/notify.py	92.19% <100.00%> (+11.55%)	⬆️
weaver/store/base.py	100.00% <ø> (ø)
weaver/store/mongodb.py	78.31% <ø> (ø)
weaver/wps_restapi/api.py	78.97% <ø> (ø)
weaver/wps_restapi/jobs/jobs.py	89.63% <100.00%> (ø)
weaver/wps_restapi/jobs/utils.py	77.94% <ø> (ø)
weaver/wps_restapi/swagger_definitions.py	99.82% <100.00%> (+<0.01%)	⬆️
weaver/processes/execution.py	86.28% <80.00%> (+1.95%)	⬆️
weaver/cli.py	86.30% <94.28%> (+0.39%)	⬆️
weaver/datatype.py	79.08% <62.50%> (ø)

[perronld]

Works well with Finch!
( using notification_email)

[ChaamC]

Minor fixes on outputs/comments/variable names, the rest looks good to me.
Good job!

[Nazim-crim]

LGTM, if I understood this correctly, multiple subscribers are only supported for different job status. Do you think it could be interesting to allow multiple emails notification for each status ?

[fmigneault]

LGTM, if I understood this correctly, multiple subscribers are only supported for different job status.

Yes

Do you think it could be interesting to allow multiple emails notification for each status ?

Could be, but I think we can work around it with a mailing list for the moment without changing the code.
If an actual use case to support multiple emails arises, it could be added later on.