Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1210: EBSVolume => new data model, Allow node attr updates from multiple intel modules #1214

Merged
merged 4 commits into from
Jul 14, 2023

Conversation

achantavy
Copy link
Contributor

@achantavy achantavy commented Jul 14, 2023

See #1210 for full context.

#1154 tried to solve this problem by updating the querybuilder but this was too complex and would not generalize well.

This solution is simpler where we use different property classes for each API response so that we don't overwrite properties on a node set by another sync job.

This PR can be reviewed commit-by-commit:

  • c0d9ac4 shows a repro of the error with a failing integration test.
  • facb63b shows the solution using multiple classes.

Copy link
Contributor

@ramonpetgrave64 ramonpetgrave64 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great



@dataclass(frozen=True)
class EBSVolumeInstanceProperties(CartographyNodeProperties):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the sample DESCRIBE_INSTANCES, the only properties about Volumes that are returned are just volumeid and deleteontermination, right?

'Ebs': {
'AttachTime': datetime.datetime(2018, 10, 14, 16, 30, 26),
'DeleteOnTermination': True,
'Status': 'attached',
'VolumeId': 'vol-04',
},

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, will update

(r['n1.id'], r['n2.id']) for r in result

# Assert that the account to volume rels exist
assert check_rels(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should maybe do this check after syncing instances, but before syncing the volumes

Co-authored-by: Ramon Petgrave <32398091+ramonpetgrave64@users.noreply.github.com>
@achantavy achantavy merged commit fb3247a into master Jul 14, 2023
4 checks passed
@achantavy achantavy deleted the ebsbug branch July 14, 2023 20:16
ramonpetgrave64 added a commit that referenced this pull request Aug 3, 2023
…iple intel modules (#1214)

See #1210 for full context.

#1154 tried to solve this problem by updating the querybuilder but this
was too complex and would not generalize well.

This solution is simpler where we use different property classes for
each API response so that we don't overwrite properties on a node set by
another sync job.

This PR can be reviewed commit-by-commit:
- c0d9ac4 shows a repro of the error
with a failing integration test.
- facb63b shows the solution using
multiple classes.

---------

Co-authored-by: Ramon Petgrave <32398091+ramonpetgrave64@users.noreply.github.com>
achantavy added a commit that referenced this pull request Sep 20, 2023
Refactors EC2 Network Interfaces and Private IPs to the cartography data
model. Fixes #1152 since the data model includes automatic cleanup jobs.

- Uses same technique as #1214 to represent Network Interfaces as known
by EC2 instances vs Network Interfaces known by
describe-network-interfaces.
- Also clearly marks that the Private IPs ingested here are known by
Network Interfaces and not another source.
chandanchowdhury pushed a commit to juju4/cartography that referenced this pull request Jun 26, 2024
…multiple intel modules (lyft#1214)

See lyft#1210 for full context.

lyft#1154 tried to solve this problem by updating the querybuilder but this
was too complex and would not generalize well.

This solution is simpler where we use different property classes for
each API response so that we don't overwrite properties on a node set by
another sync job.

This PR can be reviewed commit-by-commit:
- c0d9ac4 shows a repro of the error
with a failing integration test.
- facb63b shows the solution using
multiple classes.

---------

Co-authored-by: Ramon Petgrave <32398091+ramonpetgrave64@users.noreply.github.com>
chandanchowdhury pushed a commit to juju4/cartography that referenced this pull request Jun 26, 2024
lyft#1219)

Refactors EC2 Network Interfaces and Private IPs to the cartography data
model. Fixes lyft#1152 since the data model includes automatic cleanup jobs.

- Uses same technique as lyft#1214 to represent Network Interfaces as known
by EC2 instances vs Network Interfaces known by
describe-network-interfaces.
- Also clearly marks that the Private IPs ingested here are known by
Network Interfaces and not another source.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants