Lock down Airflow and OGC API endpoints

[jpl-btlunsfo]

Purpose

• Locking down k8s ingress to newly created SG, with jpl-local rules and a specific rule (and check) for venue-services proxy

Proposed Changes

• ADD new security group with JPL-local ingress rules
• ADD a specific ingress rule for the venue-services proxy
  • and add a check to disable its creation if the venue-services proxy does not exist for the current project/venue
• ADD an annotation to lock down the kubernetes ingress to the above security group

Issues

• Implement shared services to dev proxy for SPS airflow endpoints unity-cs#429

Testing

• Tested on personal instance
  • Ensured security group created, and access locked down to JPL-local addresses

[jpl-btlunsfo]

It looks like the pre-commit check is failing due to... having ingress rules allowing traffic from (a portion of) the public internet. Given that's the point, I think it's ignorable. Will move this PR from draft after testing.

[drewm-jpl]

Hi Brad,

I realize this is still a Draft PR, but I have a few questions for you:

• Can we obtain the values for cidr_ipv4 dynamically using a data source in Terraform.
• Can we utilize a for_each to remove the need to create each ingress rule individually?
• Can we move the port number of the applications (5000, 5001) to local variables so that the values are only specified once?

[jpl-btlunsfo]

Thanks- I didn't realize I could collapse that down nicely (the for_each).

Can we obtain the values for cidr_ipv4 dynamically using a data source in Terraform.

Are you referring to ["128.149.0.0/16", "137.78.0.0/16", "137.79.0.0/16"]? I don't think there's a dynamic place to get that from (it's not in any SSM var, as far as I know). I could make it a local?

[drewm-jpl]

Thanks- I didn't realize I could collapse that down nicely (the for_each).

Can we obtain the values for cidr_ipv4 dynamically using a data source in Terraform.

Are you referring to ["128.149.0.0/16", "137.78.0.0/16", "137.79.0.0/16"]? I don't think there's a dynamic place to get that from (it's not in any SSM var, as far as I know). I could make it a local?

Can those values be parsed out of the VPC data source or the VPC subnet data source?

How did you find those values manually?

[jpl-btlunsfo]

module.unity-sps-airflow.aws_security_group.airflow_ingress_sg: Creating...
module.unity-sps-airflow.aws_security_group.airflow_ingress_sg: Creation complete after 2s [id=sg-027a4acf481b63924]
module.unity-sps-airflow.aws_vpc_security_group_ingress_rule.airflow_ingress_sg_jpl_rule["128.149.0.0/16"]: Creating...
module.unity-sps-airflow.aws_vpc_security_group_ingress_rule.airflow_ingress_sg_jpl_rule["128.149.0.0/16"]: Creation complete after 0s [id=sgr-037a89f6d68081af9]
module.unity-sps-airflow.aws_vpc_security_group_ingress_rule.airflow_ingress_sg_jpl_rule["137.79.0.0/16"]: Creating...
module.unity-sps-airflow.aws_vpc_security_group_ingress_rule.airflow_ingress_sg_jpl_rule["137.79.0.0/16"]: Creation complete after 0s [id=sgr-0ddcc0f0c348f0172]
module.unity-sps-airflow.aws_vpc_security_group_ingress_rule.airflow_ingress_sg_jpl_rule["137.78.0.0/16"]: Creating...
module.unity-sps-airflow.aws_vpc_security_group_ingress_rule.airflow_ingress_sg_jpl_rule["137.78.0.0/16"]: Creation complete after 0s [id=sgr-09176d37e74c39423]

confirmed security group and ingresses created in test deployment! However there's something up with the loadbalancer connections, as it seems to be returning a constant gateway timeout (504).

I'm taking a look at the loadbalancer-controller documentation, but I think this I left out a required annotation. Testing that now.

[jpl-btlunsfo]

Yep, that was it. Can confirm security group set up, and can confirm traffic blocked from non-JPL addresses:

btlunsfo@MT-306763 ~ % curl -I http://k8s-sps-airflowi-1d96512cab-569953460.us-west-2.elb.amazonaws.com:5000
HTTP/1.1 302 FOUND
Date: Wed, 18 Sep 2024 01:12:51 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 197
Connection: keep-alive
Server: gunicorn
Location: /home
Cache-Control: no-store
X-Robots-Tag: noindex, nofollow
Set-Cookie: session=5115b2a3-bb6a-48ac-ad8c-b6d10457343a.fEQGLU6jKginFeeVAWFbnVQPjKo; Expires=Fri, 18 Oct 2024 01:12:51 GMT; HttpOnly; Path=/; SameSite=Lax

btlunsfo@MT-306763 ~ % curl -I http://k8s-sps-airflowi-1d96512cab-569953460.us-west-2.elb.amazonaws.com:5000
curl: (28) Failed to connect to k8s-sps-airflowi-1d96512cab-569953460.us-west-2.elb.amazonaws.com port 5000 after 150036 ms: Couldn't connect to server

[jpl-btlunsfo]

@LucaCinquini @drewm-jpl I think this is finally ready to go, unless you have any further changes?

[drewm-jpl]

Nice work! How did you manage to track down that missing annotation, that seems like it would've been tricky.

Can the OGC API still interact with the Airflow API? I imagine the answer should be yes, but could you try testing that?

[jpl-btlunsfo]

How did you manage to track down that missing annotation, that seems like it would've been tricky.

The second footnote, here, clued me in.

Can the OGC API still interact with the Airflow API? I imagine the answer should be yes, but could you try testing that?

I'll give it a try tomorrow (I want to doublecheck the venue-services proxy doesn't have issue with these changes either). Based on the container spec here, the ogc container's only configured reference to the airflow API is an internal kubernetes fqdn- a .svc.cluster.local address, which I assume means everything travels within the node security group.

[drewm-jpl]

@jpl-btlunsfo, you can add the following comment above the new VPC SG ingress rules to have tfsec ignore the issues it flagged which we have deemed as safe.

#tfsec:ignore:AVD-AWS-0107
resource "aws_vpc_security_group_ingress_rule" "ogc_ingress_sg_proxy_rule" {

[LucaCinquini]

I was able to install this branch and verified the access from within the JPL network, and no access outside. I noticed that the SGs are tagged as "U-CS", should they be tagged as "U-SPS"?
Thanks for all the work.

[jpl-btlunsfo]

The ones here?

data "aws_security_groups" "venue_proxy_sg" {
  filter {
    name   = "group-name"
    values = ["${var.project}-${var.venue}-ecs_service_sg"]
  }
  tags = {
    Service = "U-CS"
  }
}

Those are actually data-source lookups searching for the existing ECS venue proxy service, so yeah, they're U-CS because they're CS's.

[LucaCinquini]

Ah ok. Then I am good with the PR, thanks.

[jpl-btlunsfo]

Have confirmed the OGC api on my deployed instance doesn't have any issues deploying to airflow/API works. Going to go ahead and merge 🚀