Remove emergency alerts db schema #4202
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
this is a data migration (so no downgrade) that deletes broadcast services and all associated data - this includes obvious things like broadcast events and templates, but also everything connected to those services like user permissions, inbound senders, ft_billing rows, etc Although some of these theoretically shouldn't exist for broadcast services (eg returned_letters), issue the deletes out of an abundance of caution. These migrations need to pass on preview and staging as well as production, and for example there was a broadcast service on staging with an inbound sms number This has been tested against a copy of the staging database.
leohemsted
force-pushed
the
remove-emergency-alerts-db-schema
branch
from
ef27085
to
67a99d0
Compare
no longer used. there might be additional cleanup we can do of other utils functions that dont have "alert" or "broadcast" in the name that are no longer used
the schema was originally created so that it could not show content. However, then it needed content conditionally for broadcast messages so content was added back in. Now that broadcasts no longer exist we can revert that - however, for a template that has a specific list of expected keys, it's nicer to just list those rather than maintain a huge list of exceptions that needs to be modified every time a field changes on the template model
now that there's no data that depends on these, we can delete the tables completely. this was important to do as separate steps, or we could end up in a case where have have fragments of broadcast services sitting in our database but without the broadcast-specific tables, which would leave us unsure as to what those services were.
the enum dance - rename the enum to have a temp name, create a new type with the proper name that doesn't have the unwanted values, and then alter the column on the permissions table to change the type (casting the old value from old_enum -> text -> new_enum), then we drop the old type with the temp name.
leohemsted
force-pushed
the
remove-emergency-alerts-db-schema
branch
from
67a99d0
to
35c650f
Compare
This was referenced Oct 1, 2024
|
the squawk failures are:
i think these are okay. we've done the work to remove reference to these tables so that we can drop them Full output
/tmp/upgrade.sql:96:2: warning: ban-drop-table
96 | -- Running upgrade 0465_delete_broadcast_data -> 0466_delete_broadcast_tables
97 |
98 | DROP TABLE service_broadcast_settings;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:100:2: warning: ban-drop-table
100 | DROP TABLE broadcast_channel_types;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:102:2: warning: ban-drop-table
102 | DROP TABLE broadcast_provider_message_number;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:104:2: warning: ban-drop-table
104 | DROP TABLE broadcast_provider_message;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:106:2: warning: ban-drop-table
106 | DROP TABLE broadcast_event;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:108:2: warning: ban-drop-table
108 | DROP TABLE broadcast_provider_message_status_type;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:110:2: warning: ban-drop-table
110 | DROP TABLE broadcast_message;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:112:2: warning: ban-drop-table
112 | DROP TABLE broadcast_provider_types;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:114:2: warning: ban-drop-table
114 | DROP TABLE service_broadcast_provider_restriction;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:116:2: warning: ban-drop-table
116 | DROP TABLE broadcast_status_type;
note: Dropping a table may break existing clients.
/tmp/upgrade.sql:118:2: warning: ban-drop-column
118 | ALTER TABLE templates DROP COLUMN broadcast_data;
note: Dropping a column may break existing clients.
/tmp/upgrade.sql:120:2: warning: ban-drop-column
120 | ALTER TABLE templates_history DROP COLUMN broadcast_data;
note: Dropping a column may break existing clients.
/tmp/upgrade.sql:136:2: warning: changing-column-type
136 | ALTER TABLE permissions ALTER COLUMN permission TYPE permission_types USING permission::text::permission_types;
note: Requires an ACCESS EXCLUSIVE lock on the table which blocks reads.
note: Changing the type may break existing clients.
/tmp/upgrade.sql:164:2: warning: changing-column-type
164 | ALTER TABLE templates ALTER COLUMN template_type TYPE template_type USING template_type::text::template_type;
note: Requires an ACCESS EXCLUSIVE lock on the table which blocks reads.
note: Changing the type may break existing clients.
/tmp/upgrade.sql:166:2: warning: changing-column-type
166 | ALTER TABLE templates_history ALTER COLUMN template_type TYPE template_type USING template_type::text::template_type;
note: Requires an ACCESS EXCLUSIVE lock on the table which blocks reads.
note: Changing the type may break existing clients.
/tmp/upgrade.sql:168:2: warning: changing-column-type
168 | ALTER TABLE service_contact_list ALTER COLUMN template_type TYPE template_type USING template_type::text::template_type;
note: Requires an ACCESS EXCLUSIVE lock on the table which blocks reads.
note: Changing the type may break existing clients. |
we can't _just_ do the same process as 0467 because there are check constraints that depend on these. When you say "alter column template_type set type = new_type using old_type::text::new_type", it takes that "using" statement as a way of translating the prev enum into the new enum, in our case just casting to a str as an intermediate step. however, the template_type column has several check constraints against it (to say only letters can have a letter_attachment for example). Postgres isn't smart enough to take the `using` clause and apply it to these constraints. There are two ways around this: * You could create a comparator function that tells postgres "if you have to compare old_type with new_type, here's a custom function"[^1] * You can drop the old constraints and recreate them after the migration is done We've gone with the second option as it's a bit simpler and I could copy and paste from our previous scripts [^1]: https://stackoverflow.com/a/57952345 was very helpful for this
leohemsted
force-pushed
the
remove-emergency-alerts-db-schema
branch
from
35c650f
to
8781f8a
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Lots to read in the commits
I've tested this against a backup of staging - both upgrades and downgrades - i'll chat about that in a tech catchup soon