Misc Features and Functionalities
- Bot Channel Manager
- Why no Staging Guild?
- How Help Messages are Deleted
- How Logging is Done
- How Errors are handled
- Config Variables
- Helper Commands/Classes
There are 2 types of text channels that the bot keeps track of/interacts with:
- Log Channels
- Functionality Channels [for lack of a better word]
The logs channels are the ones under the category WALL-E LOGS on the CSSS Discord Guild. these are created by the create_or_get_channel_id_for_service
Functionality Channels are ones that are used for any other reason, from creating the channel where reminders are sent to channels where role-commands are limited to, etc. These are created by the create_or_get_channel_id
The major difference between those 2 functions are just that the log channels are created under the WALL-E LOGS category while the functional category creates the channel with no logic for where to place it. It is up to the moderator of the guild to place the functional channels bot channels in the correct place.
As you may have noticed, help messages auto-delete after a period of time. I implemented this auto-delete as there's really no reason for them to stick around after the user has learnt what they need to learn and pollute the channel.
There were 2 ways to implement the auto-delete.
- use the python
sleepcommand to wait a period of time and then delete within the same function that sends off the help message. This would have been done right after the function that returns themsgvariable in pieces of code like this [there are multiple instances of that kind of logic in that file]. It would have replaced the code that starts withawait HelpMessage.insert_record(. There is one problem with this approach however, and that is that if someone called the help command and then before the delete kicks in, if wall_e is restarted for any reason, then the help message will be left behind and someone has to manually go in there and delete it. To address this problem, I went with option 2. - A way to address being able to delete an existing help message even after a restart is to persist the details of the help message. And this is where the database comes in. As you can see from the code block here, I save the necessary details about the help message to the database. The
message_idandchannel_idare necessary for actually deleting the message.time_createdis necessary to know when to delete the message.channel_nameis really just in case debugging was ever needed.
How does the help message actually get deleted?
discord.py thankfully has a mechanism perfectly suited for this kind of background task, called appropriately tasks. I started the task here and the task that does the deletion is here
Every cog class makes the following calls to setup_logger.py which does the logging setup.
There are 2 functions that sets up the logging:
-
sys's redirector setup function_setup_sys_logger: since Python has a great feature where you can pretty easily direct even python errors to a logger, I have a function that does that setup early inglobal_vars.pyso that any errors will be caught by the logger as early as possible. - service's logger setup function
_setup_logger: each cog class [amongst other places] initializes theirlogger in their constructor. Anything that goes into the service logger will also get caught by thesyslogger. Sosyslogger will for the most part capture logs that have already been printed by a service logger. except in the case of python exception/errors, those will always only be caught bysyslogger. The only error caught by a service's logger is things the developer purposefully logged at the error level, like this
First off, why are log files uploaded to discord?
cause it's easy to look at them quickly by opening discord then it is by sshing into the wall_e server.
Taking, Misc by example, the upload_ functions calls start_file_uploading which calls a bunch of functions, one of them being making a background task for log_channel.py, what this does it just continuously read the log file created by the logger object and anytime there is any new lines in the file, they will be read and then written to the correct text channel.
What happens if the discord.py library experiences an error?
the default error handlers are overwritten with the error handlers in error_handlers.py.
This gives us the option to handle some errors more gracefully, like send the user a message back on discord with details of the error so they know how to correctly call a command or allows us to just log the error and move on. Or if all else fails and an unexpected and unknown exception is encountered, then the full stack-trace is printed
In addition to uploading errors to discord, I also had the bot create issues on the wall_e GitHub repo. why?
- better error tracking
- I personally go to discord maybe once a week and if it's only reported there, I'd be rather late in fixing it.
How does it work?
the same upload_ functions also creates a task for each log file for each cog.
the error_reporter.py will run just on the debug file and through some logic I had to play around with, where I tried my best to ensure that a stack-trace for a particular error get reported to github issue only once.
What needs to happen after an issue the bot experienced has been fixed?
- the GitHub issue needs to be closed
- the text channels need to be cleared.
If you want to clear a _warn or _error channel, you can do that with emoji reactions.
if you want to delete all the messages in the channel, you can just emoji react with :arrow_up: and that will trigger the reaction_detected function that was registered here to happen whenever an emoji reaction is detected to a message.
If the up arrow is detected, that function will delete all the messages in the channel that the up arrow was detected, starting from the message that was reacted to and up in the conversation history. Any messages after the message with the reaction will be left.
However, if you want to delete a stack-trace but not the ones above it, you need to first react to the top-most message to delete with the :arrow_down: emoji before using the :arrow_up:
Like any respectable code-base, wall_e has a need for config variables that dictate anything from the names of the channels to use for certain purposes to the database connection.
You can see the list of config variables that wall_e pulls in the local.ini
the wall_e code-base configuration variable setup is configured to use the following concepts:
the class that handles the setup and initialization of the configuration variables that wall_e needs.
The first piece of logic that uses the WallEConfig logic is actually the script that sets up the Django connection. It needs the environment variable to get the necessary credentials for setting up a connection to the database.
Right after the Django connection is established, then wall_e itself uses the WallEConfig logic. That wall_e_config is used in multiple files and classes as it's the entry-point to getting the value of a config variables from inside the code.
There is some complexity here that I will attempt to break down.
The first thing that WallEConfig does is to read in the specified ini file that it will use to create the general structure of the config object self.config.
Then it will iterate through each key/value in self.config.
If any of the values it read from the .ini have a corresponding environment entry, the .ini value will be replaced with the environment entry.
As you may notice, local.ini contains no actual values.
This is because I wanted the .local.ini to be a representative only of the possible variables that wall_e will read. Not an indicator of the values themselves. This was done for a two-fold reason:
- I found myself often changing the values of certain variables that wall_e would need to read via configuration and given that
local.iniis a file in the repo, I didn't like have to constantly mess with git as I changed the values - I wanted to give fellow developers the freedom to set the variables to whatever they wanted and not have to be defaulted to the values I prefer.
As such, how the values are read is:
- an
.envfile is created by the .run_walle.py at the following location with the necessary variables - Then when running wall_e in PyCharm, I have setup my Run Configuration to set the environment variables using that file.
As you can see here, the embed objects on discord have limits to how many characters each field can take. As well as the number of fields that an embed can contain.
Having people make their own embeds in the code and send it obviously can be done, but given that embeds are commonly created with dynamic strings where the content is determined at run-time, there is not often a guarantee that the limitations won't be exceeded. And if limitations are exceeded, what happens is the bot does not respond to the command and instead throws out a stack trace error and the user will most likely not know what happened as it's not intuitive for most of the CSSS Discord Guild users to know to check the logs if the bot acts weird.
So the best way to handle this kind of situation imo was to setup a centralized helper method for creating embeds that will check each given input to make sure it fits within the limits set by discord, and if not, instead of a completed embed being returned, False is returned and the it's on the developer of the particular logic being implemented to decide what to do if what they are using to create an embed exceeds the limits.
As you can see here: https://github.com/CSSS/wall_e/blob/125a6aea87fed3afccaabda1b910a94ff279a3db/wall_e/extensions/reminders.py#L151-L174
A very basic functionality of the bot is just the ability to paginate. the way that navigation works is that the bot adds certain reactions to an embed message in order to provide navigation, specifically, the forward [⏩], backward [⏪] or done [✅] emoji. Then when someone clicks on one of the emojis, there is a loop running that will detect that reaction and change the page variables [prev_page and current_page ] accordingly: https://github.com/CSSS/wall_e/blob/125a6aea87fed3afccaabda1b910a94ff279a3db/wall_e/utilities/paginate.py#L100-L141
We used to use a pagination helper function that did just pagination of a text message and nothing special, the paginate function. However that was last used back when the role and Role commands used it [they now use the embed paginator too]. Now the pagination function use exclusively is the paginate_embed function which is capable of paginating through embed messages as the name suggests.
Given that the bot sometimes sends message that can be long in length and therefore will hit the limitation of <2000 characters, there was a need for a function that can take in the desired attributes of a message [content, tts, embed, file, etc] and be capable of automatically breaking down the contents into multiple messages that are each 2000 characters in length at max.
This was particularly useful for the .exc command that admins can run, as ls -l can have a very big output.
As you may have noticed if you have looked online for examples of discord bots and also when trying to troubleshoot an issue, that the most common way to initialize a bot object is by doing
bot = commands.Bot(command_prefix='.', intents=intents, help_command=EmbedHelpCommand())but we have a custom python class called WalleBot that we initialize instead with
bot = WalleBot()First things first, what is going on with WalleBot it's relationship with commands.Bot is something called inheritance, this video is a good introduction to that concept [feel free to look for more videos if you need help understanding]
So, why I implemented a custom bot subclass? the bot class itself is the basic essentials that a discord bot needs to be able to initialize and connect to the discord API.
However, our bot has several things it needs on top of that, from a custom initialization code [for setting the BotChannelManager to the command_prefix to the help_command] to updating the run command so that the custom log_handler is automatically specified, to all the custom listeners that are added in the setup_hook.
Basically, there are several custom configurations that the CSSS Discord Guild Bot need to function correctly and therefore making it appropriate to create a custom bot to setup those custom initializations while setting up the general connection to discord's API.
To see a more clear representation of the benefits of the custom class, take a look at the setup function in each of the extensions folders. As you may notice, each cog is available only on the CSSS discord guild [and not as a global command] despite the fact that when doing bot.add_cog, we are not specifying any discord guild and that usually leads to the commands being global commands.
This is because in wall_e_bot.py, we are overriding the Bot.add_cog method to ensure that the guild object is specified before calling the Bot.add_cog method that is being overridden [aka the superclass]. This way we are ensuring that regardless of whether or not the developer of an extension remembered to include the guild in the call to add_cog, that parameter will still be used in the call to Bot.add_cog.
This centralization of code logic is a benefit of the custom WalleBot class.