feature(terraform): init terraform module

[SollyzDev]

This PR creates a terraform module the creates the Forwarder lambda and allows the user to pass a list of log group names to create the needed Subscription filters.

[SollyzDev]

hey @pecigonzalo regarding your comment previously:

Code wise/module wise, I dont think we should be doing the log-group subscriptions from Terraform. I think I mentioned the same in CF. Given we have Lambdas that can do the subscription backfill and dynamic addition of new log-groups, I don't think static resources (tf and cf) are suitable for handling dynamic resources like log-groups.

The problem with Subscriber/Unsubscriber module is that it created resources that Clouformation is not aware of. I wanted to avoid this pattern with terraform, and make it easier to find all the created subscription logs and easily maintain a list of log group names. I agree with you there should be a better way to handle the dynamic log groups. Wondering if current setup + listener only would fit, wdyt?

[pecigonzalo]

The problem with Subscriber/Unsubscriber module is that it created resources that Clouformation is not aware of. I wanted to avoid this pattern with terraform, and make it easier to find all the created subscription logs and easily maintain a list of log group names. I agree with you there should be a better way to handle the dynamic log groups. Wondering if current setup + listener only would fit, wdyt?

I understand the reasoning. The challenge is that the dynamic nature of resource like cloudwatch log groups does not fit well with Terraform rather static and stateful nature. If AWS supported a single global "subscription config" for all CWLG (CloudWatch Log Group) I would certainly prefer to configure this via Terraform.

For example, as a "Cloud Engineer", as soon as a new CWLG is created I want to ensure its logs are propagated the logging pipeline.
If this is done via a static terraform config for which I have to populate names, I have to either preemptively create the group and subscription or do it after it was created and I notice my logs are missing.
Furthermore, this is a challenge because its likely that the code that provisions the resource that creates the CWLG and the place in which I configure this are different.

I think the idea of having a CWLG controller, that takes care of configuring them all (kind of what we did with the lambda, and something I've implemented myself at other places) is more maintainable, fit for porpouse and works well in most environments.
The way I generally configure this is with a single controller lambda that has multiple triggers:

• CWLG creation event -> Subscribe and set retention
• Periodic -> Check all CWLG and reconcile any drift
• Manual event -> Check a named CWLG and reconcile

So instead of having two ways of doing this, one that is a bit more subpar than the other (terraform), I would advocate for a single recommended approach. Particularly when anyone needing to configure this via Terraform only needs 1 or 2 resources at most, so its not something for which we add a ton of value in our module or they might get more value out of the settings of the module they might using already (be that a dedicated CWLG module or a Lambda or RDS module that already contains some CWLG settings).

[SollyzDev]

@a-khaledf this is ready to be released. Let's make sure to squash those commits into one.