Update query version #398
Merged
Update query version #398
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lilyyangyi301
pushed a commit
that referenced
this pull request
Jan 30, 2024
lilyyangyi301
added a commit
that referenced
this pull request
Feb 7, 2024
* Minio container healthcheck fix.
* Candig data portal container healthcheck fix.
* Switched the healthcheck to vault-runner.
* Update query version (#398)
* DIG-1382, DIG-1384: query uses a service token with htsget (#393)
* clean should remove images
* Update requirements.txt
* query uses service store
* restart container after create_service_store
* pass in KEYCLOAK_URL for testing
* update relevant submodules
* Add a test for query info in beacon response
* update all requirements to authx@v2.1.0
* Merge branch 'develop' into daisieh/query-token
* updating submodules (#400)
Co-authored-by: github-actions <github-actions@github.com>
* htsget healthcheck added
* candig-ingest healthcheck added
* katsu healthcheck added
* postgres healthcheck added
* Healthcheck modifed for federation
* healthcheck command updated for opa-runner
* submodules updates for federation and opa
* added KEYCLOAK_PUBLIC_URL under env
* pull the commit from opa submodule
* opa submodule update for healthcheck.py
* federation submodule update for healthcheck.py
* other submodule updates
* resolve conflicts
* Add integration test changes to be hg38, and also query integration tests (#403)
* DIG-1382, DIG-1384: query uses a service token with htsget (#393)
* clean should remove images
* Update requirements.txt
* query uses service store
* restart container after create_service_store
* pass in KEYCLOAK_URL for testing
* update relevant submodules
* Add a test for query info in beacon response
* update all requirements to authx@v2.1.0
* Merge branch 'develop' into daisieh/query-token
* other submodule updates
* Add integration test changes to be hg38, and also query integration tests (#403)
* DIG-1382, DIG-1384: query uses a service token with htsget (#393)
* clean should remove images
* Update requirements.txt
* query uses service store
* restart container after create_service_store
* pass in KEYCLOAK_URL for testing
* update relevant submodules
* Add a test for query info in beacon response
* update all requirements to authx@v2.1.0
* Merge branch 'develop' into daisieh/query-token
* Add integration test changes to be hg38, and also query integration tests (#403)
* Added FEDERATION_SERVICE_URL in docker-compose
* federation healthcheck.py url change
* Add CANDIG_VERSION for front-end dynamic update (#402)
* Add CANDIG_VERSION for front-end dynamic update
* Add to example.env
* Remove added file
* Addition from Marion for linking version in footer
* Updated version to match current release version
Signed-off-by: Courtney Gosselin <courtney@gosselin.io>
---------
Signed-off-by: Courtney Gosselin <courtney@gosselin.io>
* merge stable changes into develop (#410)
* v1.2.0 release (#150)
* v2.0.0 stable candidate: MoH data model (#268)
* bridge-net instead of bridge (#79)
* bump toil to 5.3.1, following 5.3.x release (#80)
* Initial jenkins setup (#82)
* set a default value for WORKING_DIR
* activate conda on login
* working dir
* oops
* Update example.env
* initial commit for setup_jenkins
* test
* disable toil-docker for now
* update setup_jenkins
* try editing env vars
* try editing env vars
* try editing env vars
* don't push toil modules either
* working_dir is just the wd
* change location for progress.txt
* try using conda activate as the test
* touch logfile
* touch logfile
* touch logfile
* move log file
* update cancogen-dashboard (#81)
* create Jenkinsfile (#83)
* Pin version of alpine to 3.13 (#84)
* add pinned version for ALPINE_VERSION
* pin alpine_version
* test
* test
* test
* test
* test
* update htsget submodule (#85)
* update htsget submodule
* update again to stable
* add GitHub credentials (#86)
* Switch back to Dockerhub registry (#87)
* instead of overriding DOCKER_REGISTRY in the env var directly, override as a make argument
* check out the code branch that matches the Jenkins-UI one
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* add options for which registry
* update to new stable commit for htsget (#89)
* update datasets submodule to point to develop branch (#90)
* update datasets submodule to point to develop branch
* update datasets submodule
* Switch for registries in Jenkins (#88)
* how it would ideally work
one parameter to pass in to the pipeline
* Fix syntax
* add defaultValue
* first choice is default choice
* actually last choice is the default choice
* log value of REGISTRY_URL at top of console log
* log value of REGISTRY_URL at top of console log
* bump datasets version
* Reorganize submodules (#92)
* move Dockerfile to within repo
* move Dockerfile to within repo
* move dockerfile for datasets submodule
* remove Dockerfile from cnv-service
* remove Dockerfile from cnv-service
* moving Dockerfile into submodule
* updating templates
* update context in template docker-compose
* Hotfix: update htsget app pointer (#93)
* update htsget_app pointer
* update htsget_app pointer
* Move submodules to candig forks (#94)
* update htsget_app pointer
* update drs-server to candig fork
* change repo for chord_metadata_service to candig fork
* move Dockerfile to chord_drs submodule; adjust links
* Hotfix/submodules redux (#95)
* update htsget_app pointer
* forgot to commit actual gitmodules file
* Hotfix/submodules redux redux (#96)
* update htsget_app pointer
* forgot to commit actual gitmodules file
* update links for chord_metadata
* quick fix: correct name of htsget repo
* update htsget-server to stable
* update htsget-server to stable (#98)
* update chord-metadata to use secrets file for password (#100)
* update htsget-server to stable
* fix typo in chord-metadata/docker-compose
* add shared-data to lib/compose/docker-compose
* add env POSTGRES_PASSWORD_FILE
* Post Auth Merge Fixes (#91)
* fixing authx-down command
* container_name cleanup
* DIG-515 : authentication refactoring
* DIG-515: authz
* DIG-512
* DIG-513
* DIG-510
* DIG-511 + external compose volumes
* update (conda): settings that allow for conda env setup without intervention
DIG-633
DIG-633
* refactor (conda): use common variable for CONDA path
* Add Authentication Tools - Tyk and Keycloak (#99)
* feature (vagrant): add IP address option to Vagrantfile
* feature (authx): add keycloak to the setup launch
* feature (authx): minor formatting for keycloak scripts
* refactor (authx): KEYCLOAK_SERVICE* to KEYCLOAK*
* feature (authx): CHECKPOINT in case of fire
* feature (authx): add + as exclusion in makefile secret generator
* feature (authx): fix tyk confs url
* feature (authx): remove candig-server from authx makefile because it is already launched; add image removal in cleanup; DIG-633
* feature (authx): fix tyk redirect uri instead of candig server in keycloak client redirect uri settings
* feature (authx): Tyk api redirect works
* docs (authx): document steps, and a todo
* refactor (authx): renames TEMP_KEYCLOAK.. to KEYCLOACL...PROD because thats the purpose of that URL
* feature (authx): analytics for tyk
* feature (authx): remove check for local idp for now
* feature (authx): add warning comments
* feature (authx): add directory cleanup for tyk tmp
* feature (authx): add directory cleanup for tyk tmp
* docs (authx): adds new api section because we need to convey that
* docs (authx): steps to add new api
* feature (authx): add `tee` to logfile
* feature (authx): fix failing incorrect health checks for containers
* chore (authx): bumps up version of tyk and redis
* fix (authx): remove repeat line
* fix (authx): indentation should be tabs, not 4 spaces in Make
* fix (authx): remove arbiter
Co-authored-by: Brennan Brouillette <brennan.brouillette.culliford@gmail.com>
Co-authored-by: Amanjeev Sethi <aj@amanjeev.com>
* Forgot to tee to the logfile on a couple of lines (#103)
* Update of submodules (#102)
* submodules are up to date
* bridge-net creation error handling
* update htsget_app to v0.1.5
* update chord-drs to v0.4.0
* update katsu, cancogen-dashboard versions
Co-authored-by: daisie_local <daisieh@gmail.com>
* Tyk add new API feature + New API for Katsu/Chord Metadata (#104)
* fixing authx-down command
* container name patches
* container_name cleanup
* DIG-515 : authentication refactoring
* DIG-515: authz
* DIG-512
* DIG-513
* DIG-510
* begin authx setup skipping local idp
* DIG-511 + external compose volumes
* update (conda): settings that allow for conda env setup without intervention
DIG-633
* add (vagrant): libvirt section to launch using stuff like QEMU
DIG-633
* refactor (authx): reorganizing structure; WIP; DIG-633
* chore (README): spelling
* chore (gitignore): add .idea directory
* feature (vagrant): add IP address option to Vagrantfile
* feature (authx): add keycloak to the setup launch
* feature (authx): minor formatting for keycloak scripts
DIG-633
* feature (authx): minor formatting for keycloak scripts
DIG-633
* feature (authx): WIP tyk service, simplifying setup
DIG-633
* feature (authx): WIP tyk service, simplifying setup
moving tmp inside lib/tyk alleviates this pain for now but
this is not a good solution as it breaks the repo convention.
DIG-633
* refactor (authx): KEYCLOAK_SERVICE* to KEYCLOAK*
DIG-633
* feature (authx): CHECKPOINT in case of fire
DIG-633
* feature (authx): add + as exclusion in makefile secret generator
DIG-633
* feature (authx): fix tyk confs url
DIG-633
* feature (authx): remove candig-server from authx makefile because it is already launched; add image removal in cleanup; DIG-633
* feature (authx): refactor variables in keycloak script to remove global and rename locals; DIG-633
* feature (authx): formatting; DIG-633
* feature (authx): fix tyk redirect uri instead of candig server in keycloak client redirect uri settings; DIG-633
* feature (authx): add security TODO warning; DIG-633
* feature (authx): CHECKPOINT in case of fire, working on tyk;
DIG-633
* feature (authx): Tyk api redirect works
DIG-633
* docs (authx): document steps, and a todo
DIG-633
* refactor (authx): renames TEMP_KEYCLOAK.. to KEYCLOACL...PROD because thats the purpose of that URL
adds the variable to environment
DIG-633
* feature (authx): analytics for tyk
DIG-633
* feature (authx): remove check for local idp for now
DIG-633
* feature (authx): add warning comments
DIG-633
* feature (authx): add directory cleanup for tyk tmp
DIG-633
* feature (authx): add directory cleanup for tyk tmp
DIG-633
* docs (authx): adds new api section because we need to convey that
right now the tyk setup is adhoc at best, it deploys fine with single
api (candig) but it is not enough. this section documents how to achieve
this in a hacky way. it is rather sad but it is also need of the hour.
DIG-633
* docs (authx): steps to add new api
* update (conda): settings that allow for conda env setup without intervention
DIG-633
* add (vagrant): libvirt section to launch using stuff like QEMU
DIG-633
* fix (conda): removes hard-coded instances of CONDA, uses single CONDA to avoid edge case DIG-633
* refactor (conda): use common variable for CONDA path
* feature (authx): resolve conflicts because I clearly cannot read; DIG-633
* feature (authx): add `tee` to logfile
Suggestion at https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r736020301
DIG-633
* feature (authx): add a way to add new api to tyk
* feature (authx): fix failing incorrect health checks for containers
DIG-633
* chore (authx): bumps up version of tyk and redis
DIG-633
* chore (authx): remove test example from policies
DIG-633
* CHECKPOINT
DIG-652
* fix (authx): remove repeat line
https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r746730247
DIG-633
* fix (authx): indentation should be tabs, not 4 spaces in Make
https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r746732718
DIG-633
* fix (authx): missing new line
https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r746745183
DIG-633
* fix (authx): remove arbiter
https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r746992537
DIG-633
* chore (authx): remove tabs from template file
* chore (authx): comma fix
* fix (authx): keycloak public key needs to be saved
DIG-633
DIG-652
DIG-653
* docs (authx): add usage comments to make recipes
DIG-633
DIG-652
DIG-653
* fix (authx): better docker image deletion
DIG-633
DIG-652
DIG-653
* fix (authx): better consolidation of keycloak setup inside the script
DIG-633
DIG-652
DIG-653
* fix (authx): use proper segments in SESSION_ENDPOINTS for proper login redirect
DIG-656
Co-authored-by: Brennan Brouillette <brennan.brouillette.culliford@gmail.com>
Co-authored-by: Shaikh Farhan Rashid <sk.f.rashid@gmail.com>
* Add CanDIG Data Portal to the stack (#107)
* feature (candig-data-server): add git submodule for the candig-data-server service
* feature (candig-data-server): add candig-data-portal service DIG-650
* feature (candig-data-server): add candig-data-portal service; add to example env; DIG-650
* docs: update README links to template, adds candig-data-portal in the list; DIG-650
* feature (candig-data-server): add health checks
DIG-650
* CanDIG Data Portal - bugfixes (#108)
* feature (candig-data-server): add git submodule for the candig-data-server service
* feature (candig-data-server): add candig-data-portal service DIG-650
* feature (candig-data-server): add candig-data-portal service; add to example env; DIG-650
* docs: update README links to template, adds candig-data-portal in the list; DIG-650
* feature (candig-data-server): add health checks
DIG-650
* feature (candig-data-portal): fixes after PR #107
DIG-650
DIG-651
* feature (candig-data-portal): fixes after PR #107
CANDIG_MODULES order fix
DIG-650
DIG-651
* Pushing htsget app jenkins script (#105)
* update submodule to catch up (#110)
* move submodule for katsu
* move submodule for katsu (#111)
* update submod for katsu (#113)
* move submodule for katsu
* update katsu submod to v1.4.1
* bump version for CHORD_METADATA_VERSION to v1.4.1
* CanDIG Data Portal and Katsu API + New Tyk Middlewares (#112)
* initial commit of new auth middleware
* feature (candig-data-server): add git submodule for the candig-data-server service
* feature (candig-data-server): add candig-data-portal service DIG-650
* feature (candig-data-server): add candig-data-portal service; add to example env; DIG-650
* docs: update README links to template, adds candig-data-portal in the list; DIG-650
* feature (candig-data-server): add health checks
DIG-650
* feature (candig-data-portal): fixes after PR #107
DIG-650
DIG-651
* feature (candig-data-portal): fixes after PR #107
CANDIG_MODULES order fix
DIG-650
DIG-651
* fix (traefik): version value is three digits now
* fix (keycloak): script to add client in keycloak must use the name and not base64 of the name
* fix (tyk): script needs some time for redis to come up
If this fails in the future, add a more robust test
DIG-766
* feature (data-portal): add tyk api for data-portal
* doc (authmiddleware): comment for new middlware files
* remove: not needed config templates from data-portal
* add (tyk): new middleware from Jimmy and use them in data-portal and katsu
* fix (data-portal): bug in Dockerfile to envsubst missing templates
DIG-651
* fix (candig-server): removes front-end capacity from candig-server
Adds backendAuthMiddleware because candig-server will only be or should
only be used as the api/backend.
DIG-651
* fix (chord-metadata): adds image name back in docker-compose.yml
DIG-651
* update make target (#114)
Co-authored-by: Jimmy Li <lihaoyuan1996@gmail.com>
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* Add a note about updating hosts
* Fix names of secrets files in compose/docker-compose (#115)
* move submodule for katsu
* fix names of secrets files
* Update htsget submodule (#116)
* move submodule for katsu
* update htsget submodule
* DIG-772: GraphQL Integration with CanDIGv2 stack - Part 2 (#117)
* DIG-772: Initial Commit for GraphQL integration with CanDIGv2 stack
* Renamed Docker Compose File & Updated Branch of GraphQL Submodule
* Added extra config variables
* Fixed Configuration for GraphQL Interface
* Modified Authx Makefile & Updated Submodule
* Removed unneeded vault addition to Makefile
* Changed Formatting
* Changes to logging stack and updates to GQL-i
* Modified Logging Config
* Fluentd Logging Changes for GraphQL-interface
* Logging Changes due to Formatting
* Submodule Updates
* Changes to Fluentd Logging and Submodule Updates
* Fixed Fluentd Logging Regex & Submodule Updates
* Fluentd Configuration Changes
* Config Changes
* Submodule changes
* Submodule Updates and Config Changes
* Integrating OPA into the stack (#119)
* build vault and opa
* tweaks to catch up with current infrastructure
* variables and opa setup tweaks
* Opa doesn't need self certs
* Opa doesn't need self certs
* Opa doesn't need self certs
* vault setup tweaks
* making submodule for opa
* update katsu
* add second keycloak user on setup
* rename rego_dev_playground to candig-opa
* set proper secrets for client-secret for opa
* update keycloak_setup to add client-scopes and mappers
* having opa in compose prevents multi-service compose
* remove unnecessary ssl-cert
* best to compose before init-auth
* don't build traefik
* update to new integrated candig_opa
* add cleanup tweaks for clean-authx
* env vars for vault_setup
* fetch keys and restart opa after build
* set test user 1 to trusted_researcher
* use env var in a few more places
* opa submodule tweak
* remove old auth stuff from candig-server
* move a bunch of modules to not be default
* corresponding move for https://github.com/CanDIG/candig-opa/pull/1701
* Move Opa datasets permissions to separate file (#120)
* corresponding move for https://github.com/CanDIG/candig-opa/pull/1701
* remove SERVICE lines
* pass env vars in docker-compose
* pass CANDIG_AUTHORIZATION in to Dockerfile
* Update candig-server deployment to use Opa (#122)
* corresponding move for https://github.com/CanDIG/candig-opa/pull/1701
* Move Opa datasets permissions to separate file (#120)
* corresponding move for https://github.com/CanDIG/candig-opa/pull/1701
* remove SERVICE lines
* use config file
* tweaks
* update dockerfile for candig-server
* Update opa
* bump candig-server-version to 1.5.0
* HTSGET uses Opa to authorize user access to datasets (#121)
* corresponding move for https://github.com/CanDIG/candig-opa/pull/1701
* pass env vars in docker-compose
* pass CANDIG_AUTHORIZATION in to Dockerfile
* this pr needs the htsget changes
* fixes for keycloak container port, candig-server build disable (#123)
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
* changes to match opa tweaks (#124)
* corresponding move for https://github.com/CanDIG/candig-opa/pull/1701
* pass env vars in docker-compose
* pass CANDIG_AUTHORIZATION in to Dockerfile
* don't specify the server address
* load paths.json
* update submodule
* fix opa_url
* oops, didn't mean to comment this out
* double-quotes causing a parsing error in tyk
* remove quotes from policies.json.tpl (#125)
* clean up the way we start opa
* clean up opa startup call (#126)
* corresponding move for https://github.com/CanDIG/candig-opa/pull/1701
* pass env vars in docker-compose
* pass CANDIG_AUTHORIZATION in to Dockerfile
* clean up the way we start opa
* template fixes
* candig-data-portal docker fixes
* another docker fix
* opa_runner uses internal ip addresses
* straighten out uses of internal and external urls
* portal port
* Update opa
* pass in IDP to env in opa-runner
* move script exec
* short internal container name for vault
* Consolidate to just API_SLUG instead of separate API_NAME
* move to a variable set in .env
* Update htsget_app
* Update opa
* Update candig-data-portal
* Building with Vagrant on VirtualBox no longer works (#127)
The base image in use was too old (debian buster), and docker-compose was not installed
* update opa
* add quotes back
* Miscellaneous changes to server deployment settings (#128)
* corresponding move for https://github.com/CanDIG/candig-opa/pull/1701
* pass env vars in docker-compose
* pass CANDIG_AUTHORIZATION in to Dockerfile
* clean up the way we start opa
* template fixes
* candig-data-portal docker fixes
* another docker fix
* opa_runner uses internal ip addresses
* straighten out uses of internal and external urls
* portal port
* Update opa
* pass in IDP to env in opa-runner
* move script exec
* short internal container name for vault
* Consolidate to just API_SLUG instead of separate API_NAME
* move to a variable set in .env
* Update htsget_app
* Update opa
* Update candig-data-portal
* update opa
* add quotes back
* Update htsget_app
* Update htsget_app
* tiny setting tweaks (#130)
* update urls for portal
* don't bother with test datasets in opa
* Build the auth containers as well on VirtualBox (#136)
* Updated list of module names in the example (#133)
* More tweaks (#131)
* update urls for portal
* don't bother with test datasets in opa
* Add an ingest URL for bypassing tyk
* inside katsu, the opa url should be the internal one
* Update chord_metadata_service
* add ssl-cert as a dependency for init-docker
* grab container name from docker ps
* DIG-828: HTSget should be behind Tyk (#138)
* add htsget to tyk policy/api
* Add htsget to tyk
* add htsget stuff to .env
* Update example.env
* Revert "Update example.env"
This reverts commit ae612ec7a3e6edee8092d98c17a2405087848dd5.
* Assign site-admin credentials to user2 (#139)
* create credential for site_admin
* assign site_admin to user2
* need to pass opa_site_admin_key to opa's Dockerfile
* DIG-663 & DIG-763 - Adding Vault Helper Tool to Candigv2 & Documentation for Running VHT (#137)
* added VHT as git submodule
* modified gitmodules
* deleted submodule
* moved submodule
* adding docs file for testing guide
* Remove Vault Helper tool from git submodules
Co-authored-by: shaikh-rashid <44211165+shaikh-rashid@users.noreply.github.com>
* remove directory for VHT
* bump candig-data-portalversion ⬆
* Changes required in AuthX stack, bug fixes and tweaks 🐛🚀 (#143)
* Changes required in AuthX stack, bug fixes and tweaks 🐛🚀
* update keycloak url in opa docker-compose
* bump candig-data-portalversion
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
Co-authored-by: Debian <debian@candig-dev.novalocal>
* Update minio container (#140)
* update urls for portal
* don't bother with test datasets in opa
* Add an ingest URL for bypassing tyk
* inside katsu, the opa url should be the internal one
* Update chord_metadata_service
* add a site_admin user with user attribute
* allow minio console access
* Update ssl certs stuff
* update minio setup ports etc
* add ssl support to minio
* Revert "add a site_admin user with user attribute"
This reverts commit 78b52080fdbc73282e45177b2c01bc61c4ff4c54.
* Update alt_names.txt
* add minio keys to htsget
* DIG-828: HTSget should be behind Tyk (#138)
* add htsget to tyk policy/api
* Add htsget to tyk
* add htsget stuff to .env
* Update example.env
* Revert "Update example.env"
This reverts commit ae612ec7a3e6edee8092d98c17a2405087848dd5.
* minio and ssl
* remove redundant settings.py file
* don't change bucket name
* update to correct commit of htsget
* don't need to redo ssl-certs in minio-secrets
* touch up seds and alt_names.txt
* add MINIO_SELF_CERT flag
* pass in MINIO_SELF_CERT to minio-runner
* only set up certs if MINIO_SELF_CERT is 1
* Update docker-compose.yml
* Opa behind Tyk (#141)
* update urls for portal
* don't bother with test datasets in opa
* Add an ingest URL for bypassing tyk
* inside katsu, the opa url should be the internal one
* Update chord_metadata_service
* add a site_admin user with user attribute
* allow minio console access
* Update ssl certs stuff
* update minio setup ports etc
* add ssl support to minio
* Revert "add a site_admin user with user attribute"
This reverts commit 78b52080fdbc73282e45177b2c01bc61c4ff4c54.
* Update alt_names.txt
* add minio keys to htsget
* DIG-828: HTSget should be behind Tyk (#138)
* add htsget to tyk policy/api
* Add htsget to tyk
* add htsget stuff to .env
* Update example.env
* Revert "Update example.env"
This reverts commit ae612ec7a3e6edee8092d98c17a2405087848dd5.
* minio and ssl
* remove redundant settings.py file
* move vault keys to the standard tmp location
* add opa to tyk
* don't change bucket name
* ha, forgot opa tyk api
* Update example.env
* forgot to update the opa commit
* pick up changes
* pick up changes
* Update docker-compose.yml
* new release: (#146)
- bump htsget-server to v0.1.6 🧬
- bump federation-service to v0.5.2 🌎
- bump katsu to v1.4.3 📜
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
* pin python version and alpine version for katsu📌 (#149)
* pin python version and alpine version for katsu📌
* disabled toil from docker pull
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
* Vault stores s3 secrets (#142)
* add a site_admin user with user attribute
* Update ssl certs stuff
* update minio setup ports etc
* add htsget to tyk policy/api
* Update example.env
* minio and ssl
* add vault_s3_token
* reorg vault so that we can do more active token refreshing
* allow vault to be accessed via tyk
* Update candig-data-portal
* crontab for vault
* TYK_USE_SSL flag for prod
* katsu needs OPA_SITE_ADMIN_KEY var
* bump candig-server to version 1.6.0
* remove unneeded args
* bump version and submodule for federation-service and data-portal 🆙
* federation-service needs to be started manually
* minor syntax fixes
* Small fixes (#151)
* v1.2.0 release (#150)
* v1.2.1 patch
* htsget uses opa_private_url
* add external volume for htsget-data
* add debug flag
* add db_path
* pick up paths in htsget
* Update htsget_app
* Update htsget_app
* Update htsget_app
* bump htsget version
Co-authored-by: shaikh-rashid <44211165+shaikh-rashid@users.noreply.github.com>
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
* in case we're uploading things to our own minio
* bump htsget_app to v0.1.6
* candig-data-portal v0.1.2 📜
* in case we're uploading things to our own minio (#152)
* OPA secrets as docker secrets (#153)
* secrets instead of env vars
* Secrets should have fewer unpredictable chars
* load secrets
* Update opa
* Delete test.yml
* Add switches to Makefile to download M1 binaries (#154)
* add arm64mac flag
* add switches for arm64 macs
* remove kubernetes targets
* remove tabs
* try again
* fix target for traefik
* update traefik version in example.env for M1 support
* add sed backup suffix for cross-platform usage; fix miniconda copy-paste error
* add note about location of example env file
* better separate instructions for docker-compose and docker swarm
Co-authored-by: Karen Cranston <karen.cranston@gmail.com>
* Update python and pip version to Apple Silicon (#155)
* add arm64mac flag
* add switches for arm64 macs
* remove kubernetes targets
* remove tabs
* try again
* update python and pip for apple silicon
- Python bump from 3.7 to 3.9
- Pip bump from 20.2.4 to 21.2.2
This should resolve the error packages not available in conda channels.
Co-authored-by: Daisie Huang <daisieh@gmail.com>
Co-authored-by: Karen Cranston <karen.cranston@gmail.com>
* quick patch for federation_service 🩹
* bump candig-data-portalto v0.1.3
* Update submodules (#158)
* update submodules
* Update example.env
* fix: add compatibility (#160)
For Docker Desktop 1.x use _ but 2.x use - when naming. This will option retain the compose compatibility
* pass container name into chord_metadata (#162)
* Feature/federation behind tyk api (#163)
* env, templates and scripts update
* patched api federation strip listen path
* env, templates and scripts update
* patched api federation strip listen path
* reverted opa and vault command changes
* update branch chord-metadata service
* no symbols at all in random secrets (#164)
Co-authored-by: Daisie Huang <daisieh@gmail.com>
Co-authored-by: Brennan Brouillette <brennan.brouillette@computationalgenomics.ca>
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* Sonchau/install docker m1 (#161)
* docs: update docker for m1
* docs: wording
docs: typo and styling
docs: wording
docs: wording
* docs: update title
* fix: update insall-docker.md
* install-docker docs patch
- c3g arm64-keycloak image
Co-authored-by: Brennan Brouillette <brennan.brouillette@computationalgenomics.ca>
* pass in env var for HTSGET_URL (#166)
* pass in env var for HTSGET_URL
* actually, igv is going to need public urls
* Update keycloak_setup.sh (#165)
* Update keycloak_setup.sh
Set ${OPA_SITE_ADMIN_KEY} as a role and assign it to test user 2
* Update keycloak_setup.sh
* Updates for docker build of candig-data-portal (#167)
* moved Dockerfile inside repo
* update versions for python and alpine
* pass in env vars
* Sonchau/install docker m1 (#170)
* docs: update docker for m1
* docs: wording
docs: typo and styling
docs: wording
docs: wording
* docs: update title
* fix: update insall-docker.md
* install-docker docs patch
- c3g arm64-keycloak image
* update md with docker.localhost
no longer use host.docker.internal
* docs: update mac m1 readme
no longer use host.docker.internal
Co-authored-by: Brennan Brouillette <brennan.brouillette@computationalgenomics.ca>
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* Remove chord-drs/drs-server from stack (#168)
* Tiny fixes (#171)
* no base name for data portal
* token cookie can't be httponly
* I could've sworn I turned crond on...
* Refresh token rotation (#173)
* Update submods (#169)
* Update submods
* bump
* Update candig-data-portal
* Update chord_metadata_service
* Update federation_service
* Update htsget_app
* update CANDIG_MODULES to include federation-service
* remove vars for cancogen_dashboard
* Update htsget_app
* Update chord_metadata_service
* Update candig-data-portal
* Update example.env
* Update htsget
* Update candig-data-portal
* bump htsget version
* Update candig-data-portal
* Update opa
* Update htsget docker-compose's DB_PATH
* remove unused env vars and secrets
* Update htsget_app
* Update candig-data-portal
* Update federation_service
* Update versions for submodules
* Tyk federation fixes (#175)
* disable vault permissionsStoreMiddleware for federated apis
* Documentation for federation-service, candig-prod chagnes
* Update candig-data-portal
* Update chord_metadata_service
* Update federation_service
* Update htsget_app
* Update example.env
* Update opa
* Update htsget docker-compose's DB_PATH
* remove unused env vars and secrets
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* DIG-895: integrating candig-authx module (#177)
* update env vars to match candigv2-authx
* update katsu to match candigv2-authx env vars
* Update chord_metadata_service
* Update htsget_app
* Update htsget_app
* bump htsget version
* bump katsu version
* fix reversion
* set emails for fake users (#178)
* Update opa
* post-deployment party instruction cleanup (#179)
* clean up all instructions for m1
* highlight docker deployment guide
* add note about location of M1 instructions
* stub of testing instructions
* explicit mention of env file
* documentation of module configuration
* remove outdated architecture diagram
* update project structure
* [Documentation] Add in further Host-editing documentation
* Add WSL instructions
Signed-off-by: Courtney Gosselin courtney@gosselin.io
* updated hosts/firewall docs
* add ingest instructions
* changes from PR review
* one more note about hosts
* update email user2 example
* add federation service instruction
* fix the copy path to katsu
* Update README.md
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
* Update docs/ingest-and-test.md
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
* Update docs/ingest-and-test.md
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
Signed-off-by: Courtney Gosselin courtney@gosselin.io
Co-authored-by: Karen Cranston <karen.cranston@gmail.com>
Co-authored-by: fnguyen <OrdinaryNeumann@gmail.com>
Co-authored-by: Courtney Gosselin <courtney@gosselin.io>
Co-authored-by: yavyx <javiercastilloar@gmail.com>
* DIG-931: Vault aws policy needs update permissions (#180)
* aws policy needs update permissions
* env var in case it's needed
* Remove candig server and update module list (#181)
* remove candig-server from default module list
* update minimal and prod modules in readme
* Add documentation for Docker and submodules (#182)
* Add documentation for submodules
Signed-off-by: Courtney Gosselin <courtney@gosselin.io>
* Add docker and submodule documentation
Signed-off-by: Courtney Gosselin <courtney@gosselin.io>
* add links to post-install docs
* add documentation for finding module name
Signed-off-by: Courtney Gosselin <courtney@gosselin.io>
Co-authored-by: Karen Cranston <karen.cranston@gmail.com>
* Update install-docker.md
Fix links to other files in docs dir.
* Documentation for WSL federation configuration (#183)
* Documentation for WSL federation configuration
* Adding dropdown to WSL section
* Change wording
* Add WSL information to only one file
Signed-off-by: Courtney Gosselin <courtney@gosselin.io>
* cleanup of docs (#186)
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
* Update docs (#187)
* cleanup of docs
* fix doc links
* fix doc links
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
* Module cleanup (#184)
* removed unused modules
* remove swarm, kubernetes, and tox related make commands
* remove modules from env
* fixed make compose methods
* Remove vargrant, tox, and traefik from modules
* remove graphql module
* cleanup of docs
* change minio to bind to local address
* start opa container even if exited
* more specific grepping for container names
* fix doc links
* fix for network issues
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* Move settings.py here (#190)
* settings.py is a convenient script
* add minio vars
* move settings.py here, not in ingest
* DIG-996: Automate /etc/hosts step during init-authx (#188)
* DIG-996: Automate /etc/hosts step during init-authx
* DIG-996: Vault startup commands fix for MacOS
* Fix documentation on hosts editing
* DIG-996 doc
* DIG-996: Move the hosts setup to outside of the vault step, into its own step as part of init-docker
Also skip the step if it is not required, and warn the user if multiple
IP addresses were detected
* Swap tab with whitespace
* [Misc] Fix bug where echo was removing newlines
* Move init-hosts-file into its own step
* add instructions to mohccn-data and integration testing (#191)
* cleanup of conda related scripts that are no longer needed (#189)
* cleanup of conda related scripts that are no longer needed
* pipenv use instead of conda
* update docs
* cleanup of conda related scripts that are no longer needed
* require python-dev
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
Co-authored-by: Daisie Huang <daisieh@gmail.com>
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
* Sonchau/docs mac m1 (#194)
* Update install-docker.md
update for mac m1
* Update install-docker.md
* minor doc changes for M1
Co-authored-by: Karen Cranston <karen.cranston@gmail.com>
* Keycloak m1 patch (#192)
* update docs
* keycloak base_image env
* update docker-compose.yml
* VENV_OS is available for .env if needed
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
* Add no-cache build instructions (#196)
Add documentation on how BUILD_OPTS='--no-cache' works to the install instructions
* DIG-1071: Revert "cleanup of conda related scripts that are no longer needed" (#197)
* Revert "cleanup of conda related scripts that are no longer needed (#189)"
This reverts commit dc209a53b8b1e9289b137ab6cc03002e7a3522d7.
* Missing a line to actually activate the newly made conda environment
* DIG-1071: Revert pipenv changes in favour of conda
Add WSL debugging steps as found by @CourtneyGosselin
* DIG-1071: Revert pipenv changes in favour of Conda
Fix the location of the CONDA_BASE change
* add instructions for local conda
* harmonize conda_base and install dir
---------
Co-authored-by: Karen Cranston <karen.cranston@gmail.com>
* Update katsu docker-compose.yml (#199)
* Update docker-compose.yml
This update clean up unused variables in the docker compose and use custom entrypoint in Dockerfile to fix the bug katsu doesn't run migration when database is not ready
* Update docker-compose.yml
better naming to not confused with CANDIG_AUTHORIZATION
* Conda path fixes (#200)
* fix conda paths; add zsh init for M1
* update M1 mods
* Sonchau/edit chord metadata compose (#204)
* Update docker-compose.yml
This update clean up unused variables in the docker compose and use custom entrypoint in Dockerfile to fix the bug katsu doesn't run migration when database is not ready
* Update docker-compose.yml
better naming to not confused with CANDIG_AUTHORIZATION
* Update docker-compose.yml
add django env settings to docker
* Update docker-compose.yml
add posgres database name
* Update chord_metadata_service
this commit bring katsu up to date with develop branch
* Bake the `make init_hosts_file` step so it is no longer needed + Add a `make build-all` command (#203)
* Change the default value of federation-service to one that will work out of the box
* Bake hosts: Replace the make init_hosts_file step with Docker's extra_hosts
* bake_hosts: Fix a few errors with the new setup_hosts script
* [Misc] Fix missing newline
* bake_hosts: Remove old init_hosts_file command
* Add the docker.localhost redirect to Tyk as well
* bake_hosts: Add a sample LOCAL_IP_ADDR for people to redefine their local IP address
* bake hosts: Apply code review suggestions by @daiseh
* bake_hosts: Fix the documentation when no IP address can be found
* docker compose instead of docker-compose
* Add extra-hosts to all docker-composes
* bake_hosts: Add missing LOCAL_IP_ADDR step to the make compose command
* bake-hosts: add missing LOCAL_IP_ADDR step to clean-compose
* bake_hosts: Prevent an error while doing cleanup
* bake_hosts: Fix a typo in keycloak's extra_hosts
* bake_hosts: Update documentation with the new method + loopback address
* Update docs with LOCAL_IP_ADDR
---------
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* Fix a bug where make compose-% was missing the IP address (#206)
* Update chord_metadata_service (#209)
update to latest katsu from PR 27
* DIG-999: Automate integration tests (#207)
* DIG-1018, DIG-1019, DIG-1020, DIG-1021: fix make clean targets (#208)
* fix clean-compose
* remove only candigv2 images
* remove only volumes labeled candigv2
* remove only secrets labeled candigv2
* clean-authx first
* Update htsget to add label
* Update chord_metadata_service
* Update candig-data-portal
* Update federation_service
* Update opa
* Add candigv2 label to tyk Dockerfile
* Add candigv2 label to vault Dockerfile
* add candigv2 label to wes-server Dockerfile
* don't stop on error if volume doesn't exist
* remove dangling volumes
* remove all unused, dangling images (-a)
* make sure that source env.sh is in the shell that is running pytest
* Add a label onto the container spawned by postgresql (#210)
* Catching up (#211)
* remove clean-conda from clean-all
* update vault version and image
* add federation-service back to main modules
* move build-all to be near the other build targets
* remove whitespace
* rename make images to make build-images
* move build-images with other build- targets
* add option to build-images to docs
* remove make clean-conda from docs
* explicitly build images in build-all
* Update install-docker.md
* Update Makefile
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
---------
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
* Update opa (#212)
* Test htsget inside its container (#213)
* Update htsget_app
* DIG-1133: changing CANDIG_DOMAIN from docker.localhost to candig.docker.internal (#214)
* update docs
* Update example.env
* Update setup_hosts.sh
* update docker-compose extra-hosts
* Update requirements.txt
* add init-conda to build-all
* small HTSGet updates (#215)
* Update htsget_app
* add explicit value for SERVER_LOCAL_DATA in docker-compose
* add LGPL-3.0 license (#219)
* DIG-1028: standardize preflight/setup files in compose targets (#216)
* Fix an issue with pre-build-check which caused it to fail on certain MacOS (#217)
systems
* Diff the .env file, fix typo (#220)
* correct typo in dscacheutil line
* check diff
* Remove diff flag not available on all versions (#221)
* Remove diff flag not available on all versions
* y/n
* Replace all instances of Chord/Chord-metadata with Katsu (#195)
* Rename Chord_Metadata to Katsu
* Katsu json rename
* Missed a submodule
* [Katsu rename] Fix the submodule pull location to be the same as the
rest of the gitmodules
* [Katsu_rename] Reset the gitmodules using git submodules command instead
* tweaks
* change module name
* Update katsu_service
* Update configure-federation.md
Fix minor typo in the docs
* remove Katsu DRS
* these weren't meant to get merged back in
---------
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* Rename htsget-server/htsget-app to htsget (#222)
* Update opa version (#223)
* Add opa-runner image info
* bump OPA_VERSION to tagged
* Bump htsget version (#224)
* Bump versions and submodules (#226)
* katsu update (#225)
* katsu update
- update python to 3.11
- update alpine to 3.17
- bump katsu version to 2.0.0
- update katsu_service to part_31
* Update docker-compose.yml
change using python and alpine from .env
* Add pre-build-checks to prevent the problems found during BCGSC test server deployment
* Add check in case $CANDIG_DOMAIN isn't set
* Add pre-build-checks to prevent the problems found during BCGSC test server deployment (#227)
* copy things first to /app, then copy to /vault in entrypoint (#233)
* Remove explicit use of PWD from Makefiles (#230)
* DIG-1131: Federation refactoring (#231)
* Rename htsget-server folder to htsget
* Federation: Add Tyk .tpl template editing script
* Delete setup_containers.sh
* add more env stuff
* federation is part of CANDIG_AUTH_MODULES
* simplify init-authx
* more env stuff
* initialize services and our own server
* pass in env vars
* remove specific entrypoint in docker-compose
* services don't need to be via tyk
* compose tyk only
* pass some secrets and env vars into federation
* Tyk reloads apis with filenames = api_id
* self-server is only one item
* remove old files
* pass in env vars
* only one server, get a token
* use non-tyk urls
* clean up tyk templates
* pass in TYK_FEDERATION_API_ID
* strip listen path for federation
* don't need tmp/federation folder
* don't need to go through tyk
* move federation
* rename setup
* rename module
* Update ingest-and-test.md
* Update federation_setup.sh
* rename federation_service to federation
* strip_listen_path is false
* update paths
* add in CONFIG_DIR
* add federation integration tests
* add some comments
* explain about the define/endef thing
* bump opa
* Update federation
* Delete configure-federation.md
* Update federation
* don't import dotenv_values directly
* Delete insert-domain.py
* Federation setup runs after keycloak setup, so need to regenerate env.sh
* update federation
---------
Co-authored-by: fnguyen <OrdinaryNeumann@gmail.com>
* DIG-1172: Build validation (#235)
* DIG-1172: Post-build script container validation
* Restore updated Makefile from #230
* Log stderr into file
* DIG-1172: Add error logging from stderr/stdout to build process
* Makefile/post build script fixes
* Update post_build.sh
Add shebang
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
* Show relevant error logs in post build script
* Filter logs in post build script instead of Makefile
* Print all relevant error logs & Makefile clean directive
* Update description of post_build.sh
* Store error log location in .env
* Return to plain Docker output
* Always display error logs after builds
---------
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
* DIG-1143: test SampleDrsObjects (#234)
* fix up some self_uri stuff for htsget
* add a second dataset SYNTHETIC-2
* test adding samples + genomic
* Bump htsget
* Update issue templates
Adding a template based on https://candig.atlassian.net/wiki/spaces/CA/pages/730038273/Troubleshooting+checklist for failures of integration tests
* Sonchau/katsu update (#232)
* katsu update
- update python to 3.11
- update alpine to 3.17
- bump katsu version to 2.0.0
- update katsu_service to part_31
* Update docker-compose.yml
change using python and alpine from .env
* Update test_integration.py
update katsu tests to use v2
* Update test_integration.py
shorten katsu/v2 to v2
* Update test_integration.py
fix opa katsu path
* Update katsu_service
update katsu to part 32
* Update opa
update opa to use katsu v2
* Clean federation during `make clean-authx` (#240)
* Post build fixes (#239)
* when assertions fail, print response.text
* Make post build script compatible with OSX/Bash 3
* Stop 'Build started at' spam
* Update issue templates
Adding a template based on https://candig.atlassian.net/wiki/spaces/CA/pages/730038273/Troubleshooting+checklist for failures of integration tests
* Clean federation during `make clean-authx` (#240)
---------
Co-authored-by: Daisie Huang <daisieh@gmail.com>
Co-authored-by: OrdiNeu <OrdinaryNeumann@gmail.com>
* Update ingest-and-test.md (#241)
* Update ingest-and-test.md
update ingest instructions for clinical data and genomic data
* Update ingest-and-test.md
- add test-integration
- remove confluence page since it's not public
* Fix the "Websocket error: Could not connect to ws:// <x>" error (#243)
* Add issue template for deployment errors
* small fixes for Vault setup (#247)
* remove unnecessary $PWDs
* move the copying of vault-config.json
* create token as part of setup
* GitHub Actions for CanDIGv2 (#248)
* Test actions (#246)
* Display error logs in action testing
* Save docker container logs as artifact
* fix path for vault-config.json
* Update test_integration.py
* Update entrypoint.sh
* Update vault_setup.sh
* Cache CanDIG conda environment
* Workflow dispatch
* Run on PRs
* Remove print statement in integration tests
* Update entrypoint.sh
---------
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* Conda hotfix (#249)
* Conda hotfix
* Update install-docker.md
* Sonchau/add katsu secret (#250)
* Create docker-compose.prod.yml
* Update docker-compose.yml
add katsu secret key
* Update Makefile
add katsu secret with 50 chars limit, instead of 16 like password
* Update docker-compose.yml
add persistent connection time out for dev
* Update docker-compose.prod.yml
add persistent connection time out for prod
* Update docker-compose.yml
* Update docker-compose.yml
* Update docker-compose.prod.yml
* Update docker-compose.prod.yml
* Update test_integration.py
add test to clean up and delete to katsu
* Update docker-compose.prod.yml
update katsu port to prevent 3rd access
* Create stable_pr_template.md
* Update katsu, fix integration tests (#253)
* Update the candig-data-portal ref (#251)
* Update the candig-data-portal ref
* Also increment the version of data-portal
* Update katsu, fix integration tests (#253)
* Update the candig-data-portal ref
* Also increment the version of data-portal
---------
Co-authored-by: Son Chau <soncqb@gmail.com>
Co-authored-by: Daisie Huang <daisieh@gmail.com>
* Updating all submodules to latest stable release (#254)
* Updating federation
* Update opa
* Update htsget
* Updating federation
* Update opa
* Update htsget
* Update candig-data-portal
* Update stable_pr_template.md
* Prod changes that still work on local installs (#271)
* Prod changes that still work on local installs
* Use FEDERATION_PORT instead of hardcoded port numbers in federation initialization
* fix formatting strings
* retrigger check
* DIG-1282: Fix docker-compose dependency issues & GitHub Actions (#272)
* Conda hotfix attempt
* Update candig-testing.yml
* Conda location change
* Remove docker-compose package
* Update install-docker.md
* Remove actions-hotfix branch from actions
---------
Co-authored-by: Daisie Huang <daisieh@gmail.com>
Co-authored-by: Justin <justin106@protonmail.com>
---------
Signed-off-by: Courtney Gosselin courtney@gosselin.io
Signed-off-by: Courtney Gosselin <courtney@gosselin.io>
Co-authored-by: shaikh-rashid <44211165+shaikh-rashid@users.noreply.github.com>
Co-authored-by: Shaikh Farhan Rashid <sk.f.rashid@gmail.com>
Co-authored-by: Brennan Brouillette <brennan.brouillette.culliford@gmail.com>
Co-authored-by: Amanjeev Sethi <aj@amanjeev.com>
Co-authored-by: Jagdeep Sason <85504663+jagdeepsason2021@users.noreply.github.com>
Co-authored-by: Jimmy Li <lihaoyuan1996@gmail.com>
Co-authored-by: AliRZ-02 <alirzaidi02@gmail.com>
Co-authored-by: Shaikh Rashid <shaikh.rashid@uhn.ca>
Co-authored-by: Sergiu Dumitriu <sdumitriu@users.noreply.github.com>
Co-authored-by: Laiba Zaman <77404125+Zamm178@users.noreply.github.com>
Co-authored-by: Debian <debian@candig-dev.novalocal>
Co-authored-by: Karen Cranston <karen.cranston@gmail.com>
Co-authored-by: Son Chau <soncqb@gmail.com>
Co-authored-by: Brennan Brouillette <brennan.brouillette@computationalgenomics.ca>
Co-authored-by: fnguyen <OrdinaryNeumann@gmail.com>
Co-authored-by: Courtney Gosselin <courtney@gosselin.io>
Co-authored-by: yavyx <javiercastilloar@gmail.com>
Co-authored-by: Courtney <37649170+CourtneyGosselin@users.noreply.github.com>
Co-authored-by: Justin <justinyoung106@gmail.com>
Co-authored-by: Justin <justin106@protonmail.com>
* v2.1.0: Ingest and Query microservice, Debian, Model Changes (#328)
* Switch for registries in Jenkins (#88)
* how it would ideally work
one parameter to pass in to the pipeline
* Fix syntax
* add defaultValue
* first choice is default choice
* actually last choice is the default choice
* log value of REGISTRY_URL at top of console log
* log value of REGISTRY_URL at top of console log
* bump datasets version
* Reorganize submodules (#92)
* move Dockerfile to within repo
* move Dockerfile to within repo
* move dockerfile for datasets submodule
* remove Dockerfile from cnv-service
* remove Dockerfile from cnv-service
* moving Dockerfile into submodule
* updating templates
* update context in template docker-compose
* Hotfix: update htsget app pointer (#93)
* update htsget_app pointer
* update htsget_app pointer
* Move submodules to candig forks (#94)
* update htsget_app pointer
* update drs-server to candig fork
* change repo for chord_metadata_service to candig fork
* move Dockerfile to chord_drs submodule; adjust links
* Hotfix/submodules redux (#95)
* update htsget_app pointer
* forgot to commit actual gitmodules file
* Hotfix/submodules redux redux (#96)
* update htsget_app pointer
* forgot to commit actual gitmodules file
* update links for chord_metadata
* quick fix: correct name of htsget repo
* update htsget-server to stable
* update htsget-server to stable (#98)
* update chord-metadata to use secrets file for password (#100)
* update htsget-server to stable
* fix typo in chord-metadata/docker-compose
* add shared-data to lib/compose/docker-compose
* add env POSTGRES_PASSWORD_FILE
* Post Auth Merge Fixes (#91)
* fixing authx-down command
* container_name cleanup
* DIG-515 : authentication refactoring
* DIG-515: authz
* DIG-512
* DIG-513
* DIG-510
* DIG-511 + external compose volumes
* update (conda): settings that allow for conda env setup without intervention
DIG-633
DIG-633
* refactor (conda): use common variable for CONDA path
* Add Authentication Tools - Tyk and Keycloak (#99)
* feature (vagrant): add IP address option to Vagrantfile
* feature (authx): add keycloak to the setup launch
* feature (authx): minor formatting for keycloak scripts
* refactor (authx): KEYCLOAK_SERVICE* to KEYCLOAK*
* feature (authx): CHECKPOINT in case of fire
* feature (authx): add + as exclusion in makefile secret generator
* feature (authx): fix tyk confs url
* feature (authx): remove candig-server from authx makefile because it is already launched; add image removal in cleanup; DIG-633
* feature (authx): fix tyk redirect uri instead of candig server in keycloak client redirect uri settings
* feature (authx): Tyk api redirect works
* docs (authx): document steps, and a todo
* refactor (authx): renames TEMP_KEYCLOAK.. to KEYCLOACL...PROD because thats the purpose of that URL
* feature (authx): analytics for tyk
* feature (authx): remove check for local idp for now
* feature (authx): add warning comments
* feature (authx): add directory cleanup for tyk tmp
* feature (authx): add directory cleanup for tyk tmp
* docs (authx): adds new api section because we need to convey that
* docs (authx): steps to add new api
* feature (authx): add `tee` to logfile
* feature (authx): fix failing incorrect health checks for containers
* chore (authx): bumps up version of tyk and redis
* fix (authx): remove repeat line
* fix (authx): indentation should be tabs, not 4 spaces in Make
* fix (authx): remove arbiter
Co-authored-by: Brennan Brouillette <brennan.brouillette.culliford@gmail.com>
Co-authored-by: Amanjeev Sethi <aj@amanjeev.com>
* Forgot to tee to the logfile on a couple of lines (#103)
* Update of submodules (#102)
* submodules are up to date
* bridge-net creation error handling
* update htsget_app to v0.1.5
* update chord-drs to v0.4.0
* update katsu, cancogen-dashboard versions
Co-authored-by: daisie_local <daisieh@gmail.com>
* Tyk add new API feature + New API for Katsu/Chord Metadata (#104)
* fixing authx-down command
* container name patches
* container_name cleanup
* DIG-515 : authentication refactoring
* DIG-515: authz
* DIG-512
* DIG-513
* DIG-510
* begin authx setup skipping local idp
* DIG-511 + external compose volumes
* update (conda): settings that allow for conda env setup without intervention
DIG-633
* add (vagrant): libvirt section to launch using stuff like QEMU
DIG-633
* refactor (authx): reorganizing structure; WIP; DIG-633
* chore (README): spelling
* chore (gitignore): add .idea directory
* feature (vagrant): add IP address option to Vagrantfile
* feature (authx): add keycloak to the setup launch
* feature (authx): minor formatting for keycloak scripts
DIG-633
* feature (authx): minor formatting for keycloak scripts
DIG-633
* feature (authx): WIP tyk service, simplifying setup
DIG-633
* feature (authx): WIP tyk service, simplifying setup
moving tmp inside lib/tyk alleviates this pain for now but
this is not a good solution as it breaks the repo convention.
DIG-633
* refactor (authx): KEYCLOAK_SERVICE* to KEYCLOAK*
DIG-633
* feature (authx): CHECKPOINT in case of fire
DIG-633
* feature (authx): add + as exclusion in makefile secret generator
DIG-633
* feature (authx): fix tyk confs url
DIG-633
* feature (authx): remove candig-server from authx makefile because it is already launched; add image removal in cleanup; DIG-633
* feature (authx): refactor variables in keycloak script to remove global and rename locals; DIG-633
* feature (authx): formatting; DIG-633
* feature (authx): fix tyk redirect uri instead of candig server in keycloak client redirect uri settings; DIG-633
* feature (authx): add security TODO warning; DIG-633
* feature (authx): CHECKPOINT in case of fire, working on tyk;
DIG-633
* feature (authx): Tyk api redirect works
DIG-633
* docs (authx): document steps, and a todo
DIG-633
* refactor (authx): renames TEMP_KEYCLOAK.. to KEYCLOACL...PROD because thats the purpose of that URL
adds the variable to environment
DIG-633
* feature (authx): analytics for tyk
DIG-633
* feature (authx): remove check for local idp for now
DIG-633
* feature (authx): add warning comments
DIG-633
* feature (authx): add directory cleanup for tyk tmp
DIG-633
* feature (authx): add directory cleanup for tyk tmp
DIG-633
* docs (authx): adds new api section because we need to convey that
right now the tyk setup is adhoc at best, it deploys fine with single
api (candig) but it is not enough. this section documents how to achieve
this in a hacky way. it is rather sad but it is also need of the hour.
DIG-633
* docs (authx): steps to add new api
* update (conda): settings that allow for conda env setup without intervention
DIG-633
* add (vagrant): libvirt section to launch using stuff like QEMU
DIG-633
* fix (conda): removes hard-coded instances of CONDA, uses single CONDA to avoid edge case DIG-633
* refactor (conda): use common variable for CONDA path
* feature (authx): resolve conflicts because I clearly cannot read; DIG-633
* feature (authx): add `tee` to logfile
Suggestion at https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r736020301
DIG-633
* feature (authx): add a way to add new api to tyk
* feature (authx): fix failing incorrect health checks for containers
DIG-633
* chore (authx): bumps up version of tyk and redis
DIG-633
* chore (authx): remove test example from policies
DIG-633
* CHECKPOINT
DIG-652
* fix (authx): remove repeat line
https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r746730247
DIG-633
* fix (authx): indentation should be tabs, not 4 spaces in Make
https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r746732718
DIG-633
* fix (authx): missing new line
https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r746745183
DIG-633
* fix (authx): remove arbiter
https://github.com/CanDIG/CanDIGv2/pull/99#discussion_r746992537
DIG-633
* chore (authx): remove tabs from template file
* chore (authx): comma fix
* fix (authx): keycloak public key needs to be saved
DIG-633
DIG-652
DIG-653
* docs (authx): add usage comments to make recipes
DIG-633
DIG-652
DIG-653
* fix (authx): better docker image deletion
DIG-633
DIG-652
DIG-653
* fix (authx): better consolidation of keycloak setup inside the script
DIG-633
DIG-652
DIG-653
* fix (authx): use proper segments in SESSION_ENDPOINTS for proper login redirect
DIG-656
Co-authored-by: Brennan Brouillette <brennan.brouillette.culliford@gmail.com>
Co-authored-by: Shaikh Farhan Rashid <sk.f.rashid@gmail.com>
* Add CanDIG Data Portal to the stack (#107)
* feature (candig-data-server): add git submodule for the candig-data-server service
* feature (candig-data-server): add candig-data-portal service DIG-650
* feature (candig-data-server): add candig-data-portal service; add to example env; DIG-650
* docs: update README links to template, adds candig-data-portal in the list; DIG-650
* feature (candig-data-server): add health checks
DIG-650
* CanDIG Data Portal - bugfixes (#108)
* feature (candig-data-server): add git submodule for the candig-data-server service
* feature (candig-data-server): add candig-data-portal service DIG-650
* feature (candig-data-server): add candig-data-portal service; add to example env; DIG-650
* docs: update README links to template, adds candig-data-portal in the list; DIG-650
* feature (candig-data-server): add health checks
DIG-650
* feature (candig-data-portal): fixes after PR #107
DIG-650
DIG-651
* feature (candig-data-portal): fixes after PR #107
CANDIG_MODULES order fix
DIG-650
DIG-651
* Pushing htsget app jenkins script (#105)
* update submodule to catch up (#110)
* move submodule for katsu
* move submodule for katsu (#111)
* update submod for katsu (#113)
* move submodule for katsu
* update katsu submod to v1.4.1
* bump version for CHORD_METADATA_VERSION to v1.4.1
* CanDIG Data Portal and Katsu API + New Tyk Middlewares (#112)
* initial commit of new auth middleware
* feature (candig-data-server): add git submodule for the candig-data-server service
* feature (candig-data-server): add candig-data-portal service DIG-650
* feature (candig-data-server): add candig-data-portal service; add to example env; DIG-650
* docs: update README links to template, adds candig-data-portal in the list; DIG-650
* feature (candig-data-server): add health checks
DIG-650
* feature (candig-data-portal): fixes after PR #107
DIG-650
DIG-651
* feature (candig-data-portal): fixes after PR #107
CANDIG_MODULES order fix
DIG-650
DIG-651
* fix (traefik): version value is three digits now
* fix (keycloak): script to add client in keycloak must use the name and not base64 of the name
* fix (tyk): script needs some time for redis to come up
If this fails in the future, add a more robust test
DIG-766
* feature (data-portal): add tyk api for data-portal
* doc (authmiddleware): comment for new middlware files
* remove: not needed config templates from data-portal
* add (tyk): new middleware from Jimmy and use them in data-portal and katsu
* fix (data-portal): bug in Dockerfile to envsubst missing templates
DIG-651
* fix (candig-server): removes front-end capacity from candig-server
Adds backendAuthMiddleware because candig-server will …
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See comments here: CanDIG/candigv2-query#12