Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1216: AWS SSM->new data model #1217

Merged
merged 5 commits into from
Aug 2, 2023
Merged

#1216: AWS SSM->new data model #1217

merged 5 commits into from
Aug 2, 2023

Conversation

achantavy
Copy link
Contributor

Refactors AWS SSM module to use cartography's data model.

@@ -321,10 +321,6 @@ CREATE INDEX IF NOT EXISTS FOR (n:SpotlightVulnerability) ON (n.host_info_local_
CREATE INDEX IF NOT EXISTS FOR (n:SpotlightVulnerability) ON (n.lastupdated);
CREATE INDEX IF NOT EXISTS FOR (n:SQSQueue) ON (n.id);
CREATE INDEX IF NOT EXISTS FOR (n:SQSQueue) ON (n.lastupdated);
CREATE INDEX IF NOT EXISTS FOR (n:SSMInstanceInformation) ON (n.id);
Copy link
Contributor Author

@achantavy achantavy Jul 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer need this in the file since these are now automatically handled

@@ -1,25 +0,0 @@
{
"statements": [
Copy link
Contributor Author

@achantavy achantavy Jul 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer need these since they're now automatically handled

@achantavy achantavy requested a review from ryan-lane July 15, 2023 16:04
@achantavy achantavy linked an issue Jul 15, 2023 that may be closed by this pull request
@@ -65,6 +81,16 @@ def get_instance_patches(
return instance_patches


def transform_instance_patches(data_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
for p in data_list:
p["Id"] = f"{p['_instance_id']}-{p['Title']}"
Copy link
Contributor

@ramonpetgrave64 ramonpetgrave64 Jul 24, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nonblocker:
The title can have hyphen and spaces. I think we should use the kbid instead.

Suggested change
p["Id"] = f"{p['_instance_id']}-{p['Title']}"
p["Id"] = f"{p['_instance_id']}/{p['KBId']}"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocker: we could move the somewhat out-of-band transformation on line 79 to a loop within sync().

  • patch["_instance_id"] = instance_id

to something like

...
for instance_id in instance_ids:
  data = transform_instance_patches(instance_id, data)
  load_instance_patches(neo4j_session, data, region, current_aws_account_id, update_tag)
...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The old way of forming an ID used this way. Let's keep it the same so that this is a refactor that doesn't change something important like an id.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the out-of-band thing, good idea, we can include that later on

Copy link
Contributor

@ramonpetgrave64 ramonpetgrave64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to change the way id is composed, but otherwise it's good.

ryan-lane
ryan-lane previously approved these changes Aug 1, 2023
Copy link
Collaborator

@ryan-lane ryan-lane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I really love the new schema-based approach for loading/transforming data!

ryan-lane and others added 2 commits August 1, 2023 11:30
Co-authored-by: Ramon Petgrave <32398091+ramonpetgrave64@users.noreply.github.com>
@achantavy achantavy enabled auto-merge (squash) August 2, 2023 16:27
@achantavy achantavy merged commit 361fc5d into master Aug 2, 2023
4 checks passed
@achantavy achantavy deleted the standardizeec2instarn branch August 2, 2023 17:06
chandanchowdhury pushed a commit to juju4/cartography that referenced this pull request Jun 26, 2024
Refactors AWS SSM module to use cartography's data model.

---------

Co-authored-by: Ryan Lane <ryan.lane@paypay-corp.co.jp>
Co-authored-by: Ramon Petgrave <32398091+ramonpetgrave64@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Refactor AWS SSM to new data model
3 participants