Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PMM-8019 Support partial updates of settings and other entities #2705

Merged
merged 24 commits into from
Dec 27, 2023
Merged

PMM-8019 Support partial updates of settings and other entities #2705

merged 24 commits into from
Dec 27, 2023

Conversation

artemgavrilov
Copy link
Contributor

PMM-8019

Link to the Feature Build: SUBMODULES-0

If this PR adds or removes or alters one or more API endpoints, please review and add or update the relevant API documents as well:

  • API Docs updated

If this PR is related to some other PRs in this or other repositories, please provide links to those PRs:

  • Links to related pull requests (optional).

@codecov Codecov
Copy link

codecov bot commented Dec 18, 2023 •
edited
Loading

Codecov Report

Attention: 92 lines in your changes are missing coverage. Please review.

Comparison is base (5ee16cf) 43.36% compared to head (ca6219a) 43.37%.

Files Patch % Lines
managed/services/server/server.go 53.65% 11 Missing and 8 partials ⚠️
managed/utils/envvars/parser.go 61.53% 15 Missing ⚠️
managed/services/platform/platform.go 0.00% 8 Missing ⚠️
managed/models/settings_helpers.go 75.86% 3 Missing and 4 partials ⚠️
managed/services/management/alerting/service.go 0.00% 4 Missing and 3 partials ⚠️
managed/services/management/checks.go 12.50% 6 Missing and 1 partial ⚠️
managed/models/user_flags_helpers.go 0.00% 6 Missing ⚠️
managed/models/settings.go 87.50% 4 Missing and 1 partial ⚠️
...aged/services/management/backup/backups_service.go 60.00% 2 Missing and 2 partials ⚠️
managed/services/management/role.go 60.00% 3 Missing and 1 partial ⚠️
... and 8 more
Additional details and impacted files 
@@                   Coverage Diff                    @@
##           v3-api-breaking-changes    #2705   +/-   ##
========================================================
  Coverage                    43.36%   43.37%           
========================================================
  Files                          361      361           
  Lines                        42431    42410   -21     
========================================================
- Hits                         18401    18396    -5     
+ Misses                       22462    22451   -11     
+ Partials                      1568     1563    -5
Flag Coverage Δ
admin 11.42% <ø> (-0.05%) ⬇️
agent 53.37% <ø> (ø)
managed 44.73% <57.20%> (+0.02%) ⬆️
vmproxy 68.57% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

percona-robot
percona-robot reviewed Dec 18, 2023
View reviewed changes
api/common/common.proto Show resolved Hide resolved
api/server/v1/server.proto Outdated Show resolved Hide resolved
api/server/v1/server.proto Outdated Show resolved Hide resolved
percona-robot
percona-robot reviewed Dec 26, 2023
View reviewed changes
managed/services/user/user.go Show resolved Hide resolved
managed/services/user/user.go Show resolved Hide resolved
managed/services/server/server.go Show resolved Hide resolved
managed/services/server/server.go Show resolved Hide resolved
managed/services/server/server.go Show resolved Hide resolved
managed/services/server/server.go Show resolved Hide resolved
managed/services/server/server.go Show resolved Hide resolved
managed/services/server/server.go Show resolved Hide resolved
managed/services/server/server.go Show resolved Hide resolved
managed/services/server/server.go Show resolved Hide resolved
@artemgavrilov artemgavrilov marked this pull request as ready for review December 26, 2023 13:29
@artemgavrilov artemgavrilov requested review from ademidoff and JiriCtvrtka and removed request for a team December 26, 2023 13:29
@artemgavrilov artemgavrilov changed the title PMM-8019 Get rid of enable/disable flags in settings PMM-8019 Support partial updates of settings and other entities Dec 26, 2023
ademidoff
ademidoff approved these changes Dec 26, 2023
View reviewed changes
Copy link
Member

@ademidoff ademidoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, that's awesome!

@artemgavrilov artemgavrilov merged commit 32ffd53 into v3-api-breaking-changes Dec 27, 2023
30 of 31 checks passed
@artemgavrilov artemgavrilov deleted the PMM-8019-fix-bools-in-api branch December 27, 2023 09:37
@ademidoff ademidoff mentioned this pull request Apr 30, 2024
Open
ademidoff added a commit that referenced this pull request Jul 18, 2024
@artemgavrilov @ademidoff
@BupycHuk @idoqo @catalinaadam
PMM-8641 PMM v3 API breaking changes (#2635)
ee86153 
* PMM-12624 Single inventory service add endpoint (#2591)

* PMM-12529 provide one AddService for all types of services

* PMM-12529 use context in transactions

* PMM-12529 remove rpc AddMySQLService

* PMM-12624 remove rpc AddMongoDBService

* PMM-12624 remove rpc AddPostgreSQLService

* PMM-12624 remove rpc AddProxySQLService

* PMM-12624 remove rpc AddHAProxyService

* PMM-12624 remove rpc AddExternalService

* PMM-12624 final cleanup

* PMM-12624 fix the API tests

* PMM-12624 add and update the API docs

* PMM-12624 fix duplicate entry in the sample payload

* PMM-12624 return straight away

* PMM-12624 rename the messages to ...Params

* PMM-12624 reorder the message fields

* Update docs/api/inventory/add-node.md

Co-authored-by: Nurlan Moldomurov <nurlan.moldomurov@percona.com>

---------

Co-authored-by: Nurlan Moldomurov <nurlan.moldomurov@percona.com>

* PMM-12681 remove deprecated node add endpoints (#2632)

* PMM-12681 remove deprecated Node/Add... endpoints

* PMM-12681 restore the Makefile

* PMM-12681 trigger CI

* PMM-12681 fix the test

* PMM-12681 rename NodeRequest to NodeParams

* PMM-12681 fix the error messages in tests

* PMM-12529 regen descriptors

* PMM-12529 take descriptors from the parent branch

* PMM-12529 regen descriptors

* PMM-12529 regen descriptors

* PMM-12673 Protobuf linters (#2612)

* PMM-12673 Replace _INVALID suffix in enums with _UNSPECIFIED

* PMM-12673 Remove unused imports

* Refactor all protobuf enums

* PMM-12673 Fix

* PMM-12673 Fix qan tests

* PMM-12673 Add service suffix to all grpc service names

* PMM-12673 Fix QAN GetReport request/response naming

* PMM-12673 Fix

* PMM-12673 Fix

* PMM-12673 Fix tests

* PMM-12673 Fix tests

* PMM-12673 Fix test

* PMM-12673 Refactor user API requests and responses

* PMM-12673 Fix tests

* PMM-12673 Fix tests

* PMM-12673 Refactor requests/responses names

* PMM-12673 Refactoring

* PMM-12673 Fix

* PMM-12673 Fix

* PMM-12673 Fix

* PMM-12673 Refactoring

* PMM-12673 Restructure

* PMM-12673 Restructure

* PMM-12673 Restructure

* PMM-12673 Restructure

* PMM-12673 Restructure

* PMM-12673 Fix

* PMM-12673 Restructure

* PMM-12673 Restructure

* PMM-12673 Restructure

* PMM-12673 Restructure

* PMM-12673 Restructure

* PMM-12673 Fix

* PMM-12673 Configure buf lint, setup CI action

* PMM-12673 Cleanup

* PMM-12673 Fix CI

* PMM-12673 Rename

* PMM-12673 Trigger API lint warning

* PMM-12673 Remove redundant go_package option

* PMM-12673 Refactoring

* PMM-12673 Refactoring

* PMM-12673 Refactoring

* PMM-12673 Refactoring

* PMM-12673 Refactoring

* PMM-12673 Refactoring

* PMM-12673 Refactoring

* PMM-12673 Refactoring

* PMM-12673 Refactoring

* PMM-12673 Fix buf lint in CI

* PMM-12673 Fix

* PMM-12673 add debug data

* PMM-12673 Add debug data [skip ci]

* PMM-12673 Trigger CI

* PMM-12673 Debug

* PMM-12673 Remove debug data

* PMM-12673 Refactoring

* PMM-12673 Fix some linter warnings

* PMM-12673 Fix test

* Update docs/managed/update-process.md

* Update api/buf.yaml

Co-authored-by: Alex Demidoff <alexander.demidoff@percona.com>

* PMM-12673 Fix API tests

* Fix import allias

* PMM-12673 Post-merge fix

---------

Co-authored-by: Alex Demidoff <alexander.demidoff@percona.com>

* PMM-12646 Exclude inventory API from public spec (#2644)

* PMM-12646 Exclude inventory API from public API spec

* PMM-12646 Trigger CI

* PMM-6722 Preserve empty JSON fields in API responses (#2645)

* PMM-6722 enable 'EmitUnpopulated' serialization option

* PMM-6722 Add API test

* PMM-6722 Add license header

* PMM-6722 Fix linter

* PMM-6722 Fix API tests

* PMM-6722 Fix API test

* PMM-6722 Fix API test

* PMM-6722 Fix API tests

* Post-merge fixes

* Cleanup

* ci: trigger the build

* Fix after merge

* Fix after merge

* PMM-12624 single inventory agents add endpoint (#2722)

* PMM-12624 migrate to a single exporter/add endpoint

* PMM-12624 update descriptors to silence errors

* PMM-12624 upgrade mongo to at least 4.4

* PMM-12624 revert proto fields renumbering

* PMM-12624 rename AddExporter to AddAgent

* PMM-12624 add Agents to refactoring

* PMM-12624 make gen

* PMM-12624 rename addExporter to addAgent

* PMM-12624 add the documentation

* PMM-12624 make grpc methods private

* PMM-8019 Support partial updates of settings and other entities (#2705)

* PMM-8019 Get rid of enable/disable flags in settings

* PMM-8019 Fixes

* PMM-8019 Fix

* PMM-8019 Cleanup

* PMM-8019 Fix tests

* PMM-8019 Fixes

* PMM-8019 Fix migrations order

* PMM-8019 Fix DB migrations

* PMM-8019 Fix tests

* PMM-8019 Update docs

* PMM-8019 Fix

* PMM-8019 Make check enabling toogle optional

* PMM-8019 Make role update params optional

* PMM-8019 Fix API tests

* PMM-8019 Use optional fields for tour flags

* PMM-8019 Use optionals instead of wrappers in API where it's possible

* PMM-8019 Comments

* PMM-8019 Fix tests

* PMM-8019 Fix linters

* PMM-8019 Fix

* PMM-12678 Reset protobuf field nums, remove all deprecated stuff from API (#2733)

* PMM-12678 Reorder protobuf fields

* PMM-12678 Fix tests

* PMM-12678 Mark some fileds as reserved

* PMM-12678 Cleanup

* PMM-12678 Fix tests

* PMM-12678 Fix tests

* PMM-12678 Cleanup

* PMM-12678 Cleanup

* PMM-12678 Cleanup

* PMM-12678 Cleanup

* PMM-12678 Fix mocks

* PMM-12678 Fix proto fields numbering

* PMM-12677 remove STT naming (#2779)

* PMM-12677: rename all stt usages to advisors

* rename stt in query

* PMM-12677 fix migrations (#2812)

* add null check to values before re-use

We try to re-use existing values for disabled checks and check
intervals. But it is possible these values does not exist, leading to
malformed JSON. This adds a null check, and only re-use these existing
settings if they exist.

* Update managed/models/database.go

---------

Co-authored-by: Alex Demidoff <alexander.demidoff@percona.com>

* PMM-12811 single agents change endpoint (#2777)

* PMM-12811 fix a typo

* PMM-12811 fix tests

* PMM-12811 modify agents.proto

* PMM-12811 modify agents_server and tests

* PMM-12811 fix API tests

* PMM-12811 fix linter warning

* PMM-12811 increase container wait time

* PMM-12811 print docker logs on failure

* PMM-12811 increase the container wait time again

* PMM-12811 try exit-code param for container launch

* PMM-12811 check the container exit code

* PMM-12811 launch pmm w/o waiting

* PMM-12811 launch the container manually

* PMM-12811 simplify, remove redundant methods

* PMM-12811 rename exporter to agent in proto

* PMM-12811 document the change

* PMM-12811 fix API tests

* PMM-12811 fix platform_test.go

* PMM-12811 prefer a single enable boolean vs paired

* PMM-12811 make custom_labels map optional

* PMM-12811 fix Advisor params

* PMM-12811 get rid of remove_custom_labels field

* PMM-12811 remove a duplicate test

* PMM-12811 follow up on review

* PMM-12811 pass context to changeAgent

* PMM-12811 move tableCount to MysqldExporter

* PMM-12811 pass context around, fix tests

* PMM-12811 add an annotation about one param

* PMM-12811 fix Azure test

* PMM-12677 stt naming cleanup (#2805)

* update tests

* rename security checks to advisor checks

* update test files

* update route

* PMM-12836 restructure management API (#2814)

* PMM-12836 restructure alerting api

* PMM-12836 restucture api-tests

* PMM-12836 modify alerting for swagger

* PMM-12836 restructure backup api

* PMM-12836 make gen after rebase

* PMM-12836 restructure bump api

* PMM-12836 remove outdated json specs

* PMM-12836 regenerate proto specs

* PMM-12836 restructure role api

* PMM-12836 restructure advisors api

* PMM-12836 restructure qan api, promote to v1

* PMM-12836 add qan.v1 to the main swagger spec

* PMM-12836 restructure azure api

* PMM-12836 restructure actions api

* PMM-12836 fix lint errors

* PMM-12836 fix api-tests

* PMM-12836 regenerate code

* PMM-12836 remove grpc wrappers for node & mysql

* PMM-12836 fix the comment

* PMM-12836 remove grpc wrappers for mongo, pg and proxysql services

* PMM-12836 make platform API lowercase

* PMM-12836 remove grpc wrapper for annotation service

* PMM-12836 remove the comments

* PMM-12836 fix lint error

* PMM-12836 revert QAN migration to one service

* chore: run make gen

* chore: fix linter errors

* chore: fix tests

* chore: fix linter errors

* chore: fix more linter errors

* chore: fix more linter errors

* PMM-12913 Use regular http verbs (#2828)

* PMM-12913 fix logrus' field name for dump

* PMM-12913 refactor empty proto messages

* PMM-12913 make /updates and /settings lowercase

* PMM-12913 fix linter errors

* PMM-12913 merge all QAN endpoints into one service

* PMM-12913 update QAN endpoints with summary and description

* PMM-12913 fix linter errors

* PMM-12913 merge mongodb to management/service

* PMM-12913 merge proxysql to management/service

* PMM-12913 merge postgresql & mysql to management/service

* PMM-12913 merge external, haproxy & rds to management/service

* PMM-12913 merge anotation & node to management/service

* PMM-12913 merge MgmtNode to MgmtService

* PMM-12913 merge MgmtAgent to MgmtService

* PMM-12913 merge Azure to MgmtService

* PMM-12913 rename Service to ManagementService

* PMM-12913 provide a migration doc

* PMM-12913 rename MgmtService to ManagementServiceV1Beta

* Revert "PMM-12913 refactor empty proto messages"

This reverts commit a4e9545.

* PMM-12913 revert the addition of google.protobuf.Empty

* PMM-12913 remodel some APIs

* PMM-12913 move table_count to the beginning of rds.proto

* PMM-12913 rename RestoreHistoryService to RestoreService

* PMM-12913 fix the message name

* PMM-12913 move /v1/backup/Backups/Restore  to RestoreService

* PMM-12913 merge changes from parent

* PMM-12913 fix the linter error

* PMM-12913 remove table_count from rds.proto

* PMM-12913 fix null pointer ref in supervisord.go

* chore: remove unused imports in proto files

* chore: run make gen

* test

* Minor readibility improvements

* Linguistic review

* Linguistic review

* Merge branch 'v3' into v3-api-breaking-changes

* PMM-13090: Sync environment variables with v3-api-breaking-changes (#2982)

* PMM-12619: Make PMM environment variables more consistent (#2857)

* refactored environment variables

* update migration doc

* replace test line

* drop no lint rule

* fix parser tests

* re-order table values

* remove deprecated env variables

* fix tests

* use singluar advisor

* introduce PMM_DEV_ prefix for dev-related variables

* clean up environment variables

* drop printf

* drop unused variable

* update tests

* add notes for removed variable

* update oauth variables

* update changed variable list

* update variable check validation

* refactor tests

* surpress linter error

* fix docker readme

* restore error checks

* include backup management variable

* Update for grammar

* PMM-13020 Drop identifier prefixes (#2970)

* PMM-13020 Add a db migration

* PMM-13020 strip agent_id, node_id, action_id

* PMM-13020 drop service_id

* PMM-13020 drop machine_id

* PMM-13020 drop location_id

* Update agent/agents/mysql/slowlog/slowlog_test.go

Co-authored-by: Nurlan Moldomurov <nurlan.moldomurov@percona.com>

* PMM-13020 fix wrong strip-offs

* PMM-13020 fix text output spacing

* PMM-13020 fix text output spacing

* PMM-13020 drop role_id migration

* PMM-13020 fix text output spacing

* PMM-13020 fix wrong agent_id

* PMM-13020 fix slowlog jsons

* PMM-13020 fix scrape config and rds tests

* PMM-13020 fix scrape config test

* PMM-13020 fix victoriametrics test

* PMM-13020 fix node_helpers test

* PMM-13020 fix grafana test (name was too long)

* PMM-13020 strip prefixes in qan fixtures

* PMM-13020 fix QAN tests

* PMM-13020 cut the prefix when reading params

* PMM-13020 fix the job_name in vm tests

* PMM-13020 update the docs

---------

Co-authored-by: Nurlan Moldomurov <nurlan.moldomurov@percona.com>

* Merge branch 'v3' into v3-api-breaking-changes

* fix: node.proto and node_test

* fix: linter errors

* PMM-12913 Migrate api endpoints to REST (#2880)

* PMM-12913 Add a resource name to endpoints ending with /v1

* PMM-12913 fix the linter error (goconst)

* PMM-12913 migrate /v1/version

* PMM-12913 migrate /v1/readyz

* PMM-12913 all endpoints begin with `/v1`

* PMM-12913 make gen

* PMM-12913 migrate /v1/settings/Get

* PMM-12913 fix unit test fo /v1/server/settings

* PMM-12913 migrate /v1/settings/Change

* PMM-12913 migrate /v1/AWSInstanceCheck

* PMM-12913 fix alerting api tests

* PMM-12913 fix ManagementService being disabled

* PMM-12913 fix backup API tests

* PMM-12913 temporarily allow /v1/version

* PMM-12913 update admins' readme

* PMM-12913 fix the Agent connect path in auth

* PMM-12913 fix wrong grpc paths

* PMM-12913 fix the port in readme

* PMM-12913 fix some grpc paths

* PMM-12913 add check-agent-setup script

* PMM-12913 migrate /v1/server/leaderHealthCheck endpoint

* PMM-12913 migrate /v1/settings/Get

* PMM-12913 fix the api test

* PMM-12913 migrate /v1/updates/Start

* PMM-12913 fix actions

* PMM-12913 migrate /logs.zip

* PMM-12913  fix auth_server_test

* PMM-12913 migrate /v1/updates/Status

* PMM-12913 migrate /v1/users

* PMM-12913 migrate /v1/inventory/Agents

* PMM-12913 fix logs_test

* PMM-12913 use an interface for commonAgentParams

* PMM-12913 re-add `/agent_id/`

* PMM-12913 migrate /v1/inventory/Agents/Get

* PMM-12913 migrate /v1/inventory/Agents/Remove

* PMM-12913 migrate /v1/inventory/Agents/GetLogs

* PMM-12913 migrate /v1/inventory/Nodes

* PMM-12913 migrate /v1/inventory/Services

* PMM-12913 fix tests for /v1/inventory/services

* PMM-12913 migrate /v1/management/Service

* PMM-12913 migrate /v1/actions

* PMM-12899 migrate /v1/alerting endpoints

* PMM-12899 migrate /v1/advisors emdpoints

* PMM-12899 rename BackupsService to BackupService

* PMM-12899 migrate /v1/backup/Artifacts

* PMM-12899 migrate /v1/backup/Backups

* PMM-12913 refactor /v1/backups/locations

* PMM-12913 refactor /v1/backups/restores

* PMM-12913 refactor /v1/dumps

* PMM-12913 move dump and role API to v1beta1 folder

* PMM-12913 refactor /v1/role, rename tp /v1/accesscontrol

* PMM-12913 refactor /v1/management/nodes {Get,LIst}

* PMM-12913 refactor /v1/management/services/List

* PMM-12913 refactor /v1/management/services/azure

* PMM-12913 refactor /v1/management/agent/List

* PMM-12913 migrate /v1/platform

* PMM-12913 migrate /v1/qan

* PMM-12913 refactor the primitive types in alerting.proto

* PMM-12913 fix a few linter errors

* PMM-12913 do not show CollectorService in swagger

* PMM-12913 split /backup/GetLogs up to two endpoints

* PMM-12913 fix restore tests

* PMM-12913 update migration examples

* PMM-12913 fix /v1/qan/metrics:getFilters being unavailable

* PMM-12913 fix the linter error

* PMM-12913 fix /v1/qan/metrics:getFilters test

* PMM-12913 update migration examples

* PMM-12913 fix an iteration bug

* PMM-12913 fix an issue with QAN failing to persist data

* PMM-12913 remove debug output

* PMM-12913 auth_server fixes for QAN

* PMM-12913 fix agent API tests

* PMM-12913 fix the QAN URL routing issue

* PMM-12913 remove the unnecessary PMM_NICER_API

* PMM-12913 add the routing error handler

* PMM-12913 minor syntax fixes

* PMM-12913 remove PMM_LESS_LOG_NOISE envvar

* PMM-12913 remove ID normalizers

* PMM-12913 update the migration doc

* PMM-12913 better error handling

* PMM-12913 fix /v1/backups/artifacts/{artifact_id} delete endpoint

* PMM-12913 simplify setup.py

* PMM-12913 fix a linter error

* PMM-12913 merge ArtifactService with BackupService

* Revert "PMM-12913 better error handling"

This reverts commit add2d3f.

* PMM-12913: regenerate code, fix Makefile.include

* PMM-12913: fix node_test

* PMM-12913: fix UnregisterNode method name

* PMM-12857 fix API tests

* PMM-12857 fix agent tests

* PMM-12857 fix auth_test and clean up readyz_test

* PMM-12857 fix settings_test

* PMM-12857 fix auth_test

* PMM-13054 fix the backup test

* PMM-13054 fix advisor tests

* PMM-13054 fix auth_server tests again

* PMM-13054 remove redundant functionality

* PMM-13054 fix actions tests

* PMM-13054 minor cleanup

* PMM-13054 disable platform tests until the env is ready

* PMM-13054 fix advisors_test

* PMM-13054 try better error msg to debug failures

* PMM-13054 fix alerting_test

* Revert "PMM-13054 try better error msg to debug failures"

This reverts commit 862ec3b.

---------

Co-authored-by: Alex Demidoff <alexander.demidoff@percona.com>
Co-authored-by: Nurlan Moldomurov <nurlan.moldomurov@percona.com>
Co-authored-by: Michael Okoko <10512379+idoqo@users.noreply.github.com>
Co-authored-by: Catalina A <94133018+catalinaadam@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@percona-robot percona-robot percona-robot left review comments

@ademidoff ademidoff ademidoff approved these changes

@idoqo idoqo Awaiting requested review from idoqo

@BupycHuk BupycHuk Awaiting requested review from BupycHuk

@JiriCtvrtka JiriCtvrtka Awaiting requested review from JiriCtvrtka JiriCtvrtka was automatically assigned from percona/pmm-review-be

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@artemgavrilov @ademidoff @percona-robot