-
Notifications
You must be signed in to change notification settings - Fork 330
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
#1216: AWS SSM->new data model #1217
Conversation
@@ -321,10 +321,6 @@ CREATE INDEX IF NOT EXISTS FOR (n:SpotlightVulnerability) ON (n.host_info_local_ | |||
CREATE INDEX IF NOT EXISTS FOR (n:SpotlightVulnerability) ON (n.lastupdated); | |||
CREATE INDEX IF NOT EXISTS FOR (n:SQSQueue) ON (n.id); | |||
CREATE INDEX IF NOT EXISTS FOR (n:SQSQueue) ON (n.lastupdated); | |||
CREATE INDEX IF NOT EXISTS FOR (n:SSMInstanceInformation) ON (n.id); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No longer need this in the file since these are now automatically handled
@@ -1,25 +0,0 @@ | |||
{ | |||
"statements": [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No longer need these since they're now automatically handled
@@ -65,6 +81,16 @@ def get_instance_patches( | |||
return instance_patches | |||
|
|||
|
|||
def transform_instance_patches(data_list: List[Dict[str, Any]]) -> List[Dict[str, Any]]: | |||
for p in data_list: | |||
p["Id"] = f"{p['_instance_id']}-{p['Title']}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nonblocker:
The title can have hyphen and spaces. I think we should use the kbid instead.
p["Id"] = f"{p['_instance_id']}-{p['Title']}" | |
p["Id"] = f"{p['_instance_id']}/{p['KBId']}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Non-blocker: we could move the somewhat out-of-band transformation on line 79 to a loop within sync()
.
- patch["_instance_id"] = instance_id
to something like
...
for instance_id in instance_ids:
data = transform_instance_patches(instance_id, data)
load_instance_patches(neo4j_session, data, region, current_aws_account_id, update_tag)
...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The old way of forming an ID used this way. Let's keep it the same so that this is a refactor that doesn't change something important like an id.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For the out-of-band thing, good idea, we can include that later on
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may want to change the way id is composed, but otherwise it's good.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. I really love the new schema-based approach for loading/transforming data!
Co-authored-by: Ramon Petgrave <32398091+ramonpetgrave64@users.noreply.github.com>
3c250d5
Refactors AWS SSM module to use cartography's data model. --------- Co-authored-by: Ryan Lane <ryan.lane@paypay-corp.co.jp> Co-authored-by: Ramon Petgrave <32398091+ramonpetgrave64@users.noreply.github.com>
Refactors AWS SSM module to use cartography's data model.