The following is a description of the steps required to build a full CPAN Testers server. As we've had to do this several times over the years, it will help whenever we have to do this again.
Please consider this a living document, so as things change, please update.
Having got a machine to build, the current OS used for a CPAN Testers server is Debian.
The current primary user account is 'barbie'. This runs all the cronjobs, as well as holds permissions for web and backend processing directories. For security, it would make more sense for the cronjobs to run under a 'cron' user, and ensure it has permissions to write to appropriate directories.
ProFTP uses the 'ftp' user, but this should be created by the system. If not, please add the user.
Most other user accounts should be disabled for login. /etc/passwd shells should be set to '/usr/sbin/nologin' or similar.
To help prevent ssh attacks, it is recommended that 'fail2ban' is also installed. See 'http://www.fail2ban.org/wiki/index.php/Main_Page' for additional information.
Depending on how the filesystem is mounted, entries in /etc/fstab are needed to ensure the script and web paths all work. See fstab.txt to see the current extras added to cpantesters3.
Note that the mounted SATA disks are:
/media/backend (contains MySQL data files, and CPAN Testers backend scripts) /media/web (contains all web files, including CPAN and BACKPAN directories)
After this point we will assume that web sites will be available in '/var/www', backend files will use the path '/opt/projects', and mount points for ProFTP are in '/home/ftp'.
In order to have a working system, a number of system installs and CPAN installs are needed. Run server-installs.sh as root to install apps and libraries.
The websites themselves are not straight installs from CPAN, but offsite builds, from the ./vhost directory in the base distribution, then loading Labyrinth and required plugins locally. As such different websites may be built against different versions of Labyrinth, due to the various sites being upgraded at different times.
The list of the associated websites are:
path: /var/www/reports distribution: CPAN-Testers-WWW-Reports GitHub: https://github.com/barbie/cpan-testers-www-reports
This includes the Static site (/var/www/reports/html/static), and the Pass Matrix (/var/www/reports/html/stats) site.
path: /var/www/cpanadmin distribution: CPAN-Testers-WWW-Admin GitHub: https://github.com/barbie/cpan-testers-www-admin
path: /var/www/cpanprefs distribution: CPAN-Testers-WWW-Preferences GitHub: https://github.com/barbie/cpan-testers-www-preferences
path: /var/www/cpanstats distribution: CPAN-Testers-WWW-Statistics GitHub: https://github.com/barbie/cpan-testers-www-statistics
The site is built from scripts and files run from and stored in:
/opt/projects/cpantesters/cpanstats
path: /var/www/cpandevel distribution: CPAN-Testers-WWW-Development GitHub: https://github.com/barbie/cpan-testers-www-development
The site is built from scripts and files run from and stored in:
/opt/projects/cpantesters/cpandevel
The websites require additional scripts and processes to maintain the databases. These scripts are again installed via the CPAN distributions.
path: /opt/projects/cpantesters/generate distribution: CPAN-Testers-Data-Generator GitHub: https://github.com/barbie/cpan-testers-data-generator
Retrieves the reports data feed from the Metabase server, and parses the report metadata into the cpanstats database.
path: /opt/projects/cpantesters/uploads distribution: CPAN-Testers-Data-Uploads GitHub: https://github.com/barbie/cpan-testers-data-uploads
Parses the CPAN and BACKPAN directories for new and deleted files, and updates the cpanstats database accordingly.
path: /opt/projects/cpantesters/cpanstats-excel distribution: CPAN-Testers-WWW-Statistics-Excel GitHub: https://github.com/barbie/cpan-testers-www-statistics-excel
Processes the statistis data into Excel spreadsheet files for download.
path: /opt/projects/cpantesters/reports-mailer distribution: CPAN-Testers-WWW-Reports-Mailer GitHub: https://github.com/barbie/cpan-testers-www-reports-mailer
Mails authors links to their reports.
path: /opt/projects/cpantesters/generate distribution: CPAN-Testers-WWW-Generator-Mailer GitHub: not available
Sends error reports, in the event a faulty report is detected.
path: /opt/projects/cpantesters/uploads-mailer distribution: CPAN-Testers-Data-Uploads-Mailer GitHub: https://github.com/barbie/cpan-testers-data-uploads-mailer
Sends authors emails, if a distribution doesn't adhere to the approved naming convention. Also welcomes new authors, and provides an introduction to CPAN Testers.
There are several databases in MySQL, several for the dynamic websites, and others to maintain the reports and statistics.
Database schema can be built from the schemas of the various CPAN distributions. See the above distributions for details.
TODO: Provide a consolidated schema file for each database.
Where possible build the database from existing backups. It will take over a month to rebuild otherwise!
'/var/www/reports/toolkit/dbfix.pl' is a handy script used to verify the database is in a healthy state. It will run ANALYZE TABLE, and REPAIR TABLE as required. Should only be run when other processes have been stopped to prevent the script from being unnecessarily blocked.
When the databases were created many years ago, the database engine MyISAM was used. In retrospect this should have been InnoDB, and a migration to this engine should be considered.
A few modern tables have UTF-8 enabled, however all should be migrated to UTF-8. Although distribution names are currently ASCII, it is only a matter of time before PAUSE can fully support Unicode.
To Be Completed.....