FETcher

GitHub

This repository contains two Python scripts. The first, fetch.py, performs FilE Transfer for the purposes of CATcher. It takes from the command line

• either the location of a CSV file containing student usernames (see fetcher/data.csv for how it should be structured), or a regular file containing one username per line to transfer
• a destination repository in the form owner/name
• a credentials file, containing four lines of author name, email address, GitHub username and access token, e.g.

Derpy Hooves
derpy@equestria.net
muffinsmuffins
4f17daf987b3028dd4367f8bef6ec39e929203a8

The script then clones the students' repositories, transfers their files to a central folder in the destination repository called files, commits and pushes. Multiple processes (specified by the -p option, defaults to the number of cores on the running machine) can be used in parallel to speed up repository cloning.

An interrupted transfer process can be resumed by passing remaining instead of the CSV file as the first argument to fetch.py. The file remaining is automatically updated with a list of usernames still to process.

Example times for transferring the CS2103/T AY1920S1 cohort (332 students) with 4 processes:

real    6m10.815s
user    1m59.742s
sys     0m10.230s

The second script, fetch-issues.py, takes from the command line

• a source and destination repository, both in the form owner/name
• a credentials file as in fetch.py

It then

• downloads the title, body and labels of all issues in the source repository using the GraphQL API and saves them to issues.json as a backup
• creates issues with the same data in the destination repository using the REST API

Example times for transferring the issues found in the practical exam dry run by the CS2103/T AY1920S1 cohort (3052 issues):

real    29m10.705s
user    2m25.138s
sys     0m8.460s

Issues continued to be created for six minutes after the program completed.

---

Fet is also the name of a Norwegian municipality, quite close to Akershus Fortress.