Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(PUP-11685) Update total fact count #9150

Merged
merged 1 commit into from
Nov 15, 2023

Conversation

mhashizume
Copy link
Contributor

@mhashizume mhashizume commented Nov 9, 2023

Puppet calculates the total number of facts on an agent node to warn the user if it will cause performance issues with PuppetDB. PuppetDB performance can be impacted when the leaf count of a factset is too high.

Prior to this commit, Puppet inconsistently calculated facts when generating a warning. This did not count individual array elements; for example, the processor.models fact with a fact per processor core would only be counted once. Additionally, hash keys were incorrectly each counted as individual facts when they would not represent separate leaf counts in PuppetDB.

These behaviors lead to a slight discrepancy between what Puppet would report to the user and what the total fact count ended up being in PuppetDB.

This commit updates Puppet to more accurately count facts to reflect a factset's leaf count in PuppetDB.

Puppet calculates the total number of facts on an agent node to warn
the user if it will cause performance issues with PuppetDB. PuppetDB
performance can be impacted when the leaf count of a factset is too
high.

Prior to this commit, Puppet inconsistently calculated facts when
generating a warning. This did not count individual array elements; for
example, the processor.models fact with a fact per processor core would
only be counted once. Additionally, hash keys were incorrectly each
counted as individual facts when they would not represent separate leaf
counts in PuppetDB.

These behaviors lead to a slight discrepancy between what Puppet would
report to the user and what the total fact count ended up being in
PuppetDB.

This commit updates Puppet to more accurately count facts to reflect
a factset's leaf count in PuppetDB.
@mhashizume
Copy link
Contributor Author

Here are some examples from my testing for this PR.

This is Puppet's current behavior when counting total facts (using Puppet 7.27.0 with legacy facts omitted):

root@ethical-stuff:~# puppet agent -t --debug | grep 'The total number of facts registered is'
Debug: The total number of facts registered is 360

This is the updated behavior using the code in this PR:

root@ethical-stuff:~# puppet agent -t --debug | grep 'The total number of facts registered is'
Debug: The total number of facts registered is 352

This is what a PuppetDB query returns:

root@younger-crepe:~# curl -sX GET http://localhost:8080/pdb/query/v4/fact-contents --data-urlencode 'query=["=", "certname", "ethical-stuff.delivery.puppetlabs.net"]' | jq '. | length'
356

The updated behavior is more accurate to what is reported to PuppetDB. The difference of four facts between the two counts is due to the presence of trusted facts in PuppetDB.

Overall while this new count is more accurate, it seems like generally there won't be a massive difference from the old behavior. The old methodology overcounted some things (hash keys) but undercounted others (array elements), so it seems like it overall kind of balanced out.

@mhashizume
Copy link
Contributor Author

@Sharpie @rbrw thanks for your earlier feedback on this issue. Would you be able to review this when you have the chance?

Thank you!

@joshcooper joshcooper added the bug Something isn't working label Nov 15, 2023
@mhashizume mhashizume marked this pull request as ready for review November 15, 2023 19:43
@mhashizume mhashizume requested a review from a team as a code owner November 15, 2023 19:43
@cthorn42 cthorn42 merged commit f33de24 into puppetlabs:7.x Nov 15, 2023
12 checks passed
@mhashizume mhashizume deleted the PUP-11685/7.x/fact-count branch December 1, 2023 22:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants