iTC Administrator Tools Overview

Acknowledgements

This is designed for an admin to setup the listed tools and to manage the cPP process.

This document intends to communicate to iTC admins how to setup tools and some basic automation to manage the cPP creation process. The scope covers both client side tools necessary for workflow, operating systems, and backup strategies. Finally, focus is heavily on FOSS (Free Open Source Software) because cost should not be a barrier to iTC participation in the development of a cPP.

Audience

The intended audience for this overview are those people who will manage the technical processes of the iTC. This may be the chair or it may be someone else (ideally there should be multiple people providing this support).

Ideally the person who is responsible for GitHub should be someone familiar with it. While this guide provides what is needed to get an iTC up and running it is not meant to be an exhaustive guide for GitHub and all its capabilities.

Resources

1. Introduction

The administrators of the iTC will need to have some level of technical skills to manage the cPP creation process. This is true regardless of the tools in use, but is more explicitly laid out here because the tools here are not "sending Word documents around" but require the use of specific development tools to provide a foundation for the long term health of the cPP process.

This site is laid out to focus on the administration of the two primary tools recommended for use: GitHub and Asciidoctor. There are other suggested tools to help the iTC fill out the workflow process, but these are the two primary tools that are used in this process.

2. GitHub

GitHub is a well known source code repository site that also provides free accounts and usage (perfect for our purposes). While it may seem strange to use a source code repository site for creating and managing the cPP creation process, there are several good reasons for doing this:

It provides a central repository of information that anyone can access
Revision control system with access control, branching, and submission approval built-in
Discussions on issues are easily facilitated in browser
Wiki can be used to communicate meeting minutes, document development status, etc.

By using GitHub as the central repository, many of the problems with collaboration are solved, from multiple concurrent editors to discussion/decision history. It does bring its own problems for our purposes, which will be discussed in the Asciidoctor section.

2.1. GitHub Setup

The steps here assume that the person who will manage the GitHub repository already has a GitHub account (a free account is sufficient).

2.1.1. Organization Setup

To setup the iTC organization, do the following:

Login to https://github.com with your account
Choose what the iTC group’s Organization and Repositories should be named
Click the down arrow in the upper right corner of the page next to the user icon
Click "New Organization", and enter the agreed on Organization name
1. Select the "My personal account" option for the owner of the new Organization

2.1.2. Organization Owners

Once the Organization has been created, the first thing that should be done is to assign at least one more Owner for the organization.

This assumes that the other users who will be owners have already created GitHub accounts for themselves.

To set another Owner for the organization, go to the Organization page and click on the People tab at the top.

Click the Invite button on the top right
Enter the username of the person who will become another Owner
Choose Owner as the role in the organization and click Send Invitation

Once the user accepts the invitation, they will automatically be added as an Owner.

If the user already exists within the list of People in the Organization, then clicking the Gear icon to the right of their name and selecting "Change role…" will let you change their role accordingly.

2.2. GitHub Repositories

Within a GitHub organization, repositories are project spaces. Each repository has independent tracking of issues and changes. This section focuses on the setup of the repositories within the organization.

2.2.1. Repository Setup

The number of repositories needed within the organization may depend on where you are in the process. Three template repositories have been created to help create the structure and populate the initial templates you need.

Table 1. Template Repositories
Name	Description	URL
New-iTC-Admin-template	This contains templates for the iTC such as the ESR, ToR and other governing documents.	https://github.com/itc-wgtools/New-iTC-Admin-template
New-iTC-Template	This contains templates for the cPP, SD and PP-Configuration (and PP-Module if needed)	https://github.com/itc-wgtools/New-iTC-Template
New-iTC-website-template	This contains a template for the github.io website that can be setup for the iTC	https://github.com/itc-wgtools/New-iTC-website-template

All three templates should be used when setting up the iTC, though it is possible that some may be used at different times (see iTC Workflow).

Each template uses the same steps to do the initial setup of your new repository.

To create the new repository, follow these steps:

Go to the URL for the type of repository you are going to create from the Template Repositories table
Click the green "Use this template" button
Choose the Organization as the owner (it will default to your account) and enter the Repository name, add a description if necessary, and finally click Create repository from template. The rest can be left as default
The next screen after creation is useful for getting up and running. Read the Quick setup steps, commands used, etc.

2.2.1.1. Alternate Repository Setup (Import)

It is also possible to copy the contents of the template repository into a blank repository. If you already have a blank repository, it is possible to import the contents of the template repository to achieve the same result.

To import the contents to an existing blank repository:

Open the repository page
At the bottom of the page, click the "Import code" button
Enter the URL for the type of repository you are going to create from the Template Repositories table and click Begin Import

This only works if you created a repository but have no content in it. Once there is content you will not be able to import the templates.

2.2.2. iTC Template Repository Configuration

The iTC Template repository would be the primary work location for the cPP and SD for the iTC. As such there are a few additional steps that should be followed to provide some additional capabilities to the iTC in the management of the work of maintaining the documents over the long term. While these can be used on other repositories, they are laid out with a focus on the cPP/SD work.

2.2.2.1. repo-labels script

To support some level of standardization, a script has been created to edit the labels in the repository to a common set. These are then used for all Issues and Pull Requests in a standard way to support standard instructions/capabilities and provide a basis for potential future tools over time.

The script and instructions for running it can be found at the repo-labels repository.

While the repository will work fine without this script, some of the features provided here in terms of templates for managing Issues and Pull Requests will not work correctly without these labels.

The repository assumes the user has installed Ruby (such as by following Installing Ruby).

2.2.3. Repository Projects

GitHub Projects allow you to track the development process (in our case, the documentation process) to the completion of the project. Projects can be very powerful, but can also be a little bit of work to setup and maintain (and GitHub doesn’t provide a large amount of capabilities natively).

More general information aboud GitHub Projects can be found here.

It is recommended to have generally, 2 projects at any one time. The first project would be the "Next Release" project (called Maintenance, here), while the second project would be for the Interpretation Team.

2.2.3.1. Installing project-bot

To help automate some aspects of tracking projects, we are going to use project-bot. This will let movement of tasks between steps be automated.

To install project-bot:

Go to the install page
Click Install
Select the iTC organization
Select All repositories (the default) and click Install

Once the installation is complete you should see the Settings page for the organization.

The project-bot site provides instructions on how to create cards in the Project that can automate some tasks.

2.2.3.2. Creating Projects

It is possible to copy projects into the repository from the New-iTC-Template repository. This will copy the outline (the columns) of the project and built-in automations, but it will not copy the contents of the project (in this case the project-bot cards).

To create the projects, follow these steps for each project in the New-iTC-Template repository:

Go to the Projects tab
Select the project to be copied
Click the Menu icon in the upper right-hand corner of the Project display
Click the 3 dots (…) and select Copy
For the Owner, choose the repository for the cPP/SD documents
Enter the name (generally remove the [COPY] text) and the description
Make sure Copy automation settings is checked
Click Copy Project

This will create 2 blank projects with the columns and a few basic automations laid out.

2.2.3.3. Setting the Project Automation

Once the projects have been created, the project-bot configuration needs to be added. There is no automated way to handle this, but it is a simple copy paste from the examples below to fill it out.

In all but the last column of each project there is a single "card". Cards can be anything (like notes), but the project-bot can use specially formatted cards to run automations.

There are two provided project templates, one for Maintenance and one for the Interpretation Team. Most of these are common, but there are some differences. The examples below are noted for which project they are applicable to (Maintenance, Interpretation Team or both).

To create the card, do the following for each column that is noted:

Copy ALL the text in the card example (below)
Click the + in the upper right hand corner of the matching column in the new project
Paste the contents into the note and click Add

Once all the cards have been copied, the Projects should be complete.

It is expected that you will make multiple copies of the Maintenance project to manage ongoing updates.

2.2.3.3.1. To Do (Both)

###### Automation Rules

<!-- Documentation: https://github.com/philschatz/project-bot -->

- `added_label` **Interpretation**
- `new_issue`

2.2.3.3.2. Awaiting Priority (Both)

###### Automation Rules

<!-- Documentation: https://github.com/philschatz/project-bot -->

- `added_label` **Awaiting Priority**

2.2.3.3.3. Awaiting Review (Both)

###### Automation Rules

<!-- Documentation: https://github.com/philschatz/project-bot -->

- `added_label` **Awaiting Review**

2.2.3.3.4. In Progress (Maintenance only)

###### Automation Rules

<!-- Documentation: https://github.com/philschatz/project-bot -->

- `added_label` **In Progress**
- `new_pullrequest`

2.2.3.3.5. In Progress (Interpretation Team only)

###### Automation Rules

<!-- Documentation: https://github.com/philschatz/project-bot -->

- `added_label` **In Progess**

2.2.4. Issue Templates

Within the template repository for the cPP/SD are Issue Templates to help users with filling out the information needed when creating a new issue. While these may not be needed for everyone, they will likely be very helpful for users who do not participate often in the group and so may not be completely familiar with the information needed to respond to their concerns. In addition it is possible to assign labels to classify the issue as well as users to review.

While the default information should be sufficient overall, to help make sure new issues are properly reviewed, it is useful to assign users to review the incoming issues.

Table 2. Issue Templates
Name	Should be Assigned	Project	Description
cPP change request	Editor Team	Maintenance	Issue specifically for the cPP (requirements, etc, but not the SD)
PP-Configuration change request	Editor Team	Maintenance	Issue specifically for a PP-Configuration document
Request for interpretation	Interpretation Team	Interpretation	Issue specifically related to a request for a Technical Decision/Interpretation of the current published cPP/SD
SD change request	Editor Team	Maintenance	Issue specifically for the SD (evaluation activities, etc, but not the cPP requirements directly)

The Issue Templates should be assigned to the members of the teams, but they must be assigned individually. The Teams cannot be used for assigning issues, only individual users.

To assign the team members to the Issue Templates (do this for each template):

Open the Repository Settings
On the Options tab, under Features, click Set up templates
Click Preview and edit
Click the pencil icon next to the title
Click the Assignees gear symbol
Select the members of the Editors team (up to 10) to be assigned to the issue when created
Click Close Preview

Once all the templates have been edited with the proper Assignees:

Click Propose changes
Select Commit directly to the Master branch (this should be the default branch)
Follow the steps to commit, approve and merge the changes (this may require an Administrator override at this point)

2.3. Repository Branches

One of the most important aspects of working in GitHub is the use of Branches. This is how you manage the process of creating the documents. When created, a repository has a single branch, Master. This should be reserved for the current "published" content. Ongoing work though, should be done in different branches, which would then be merged into the Master on some periodic basis.

Here two new branches should be created for different purposes: Working and Interpretation.

To create the new branches for the repository (do this for each branch):

Go to the Code tab
Click the Branch: master menu button
Enter the name of the new branch in the dialog box and then click the Create branch at the bottom of the window

The Working branch is where everyone should be submitting work to, while the Master would be considered the current "published" document.

The Interpretation branch is specifically for the Interpretation Team to work in for changes to the existing Master, and will be limited to only members of the IT being able to merge changes.

2.4. GitHub Repository Collaborators

GitHub public repositories are visible to anyone with a GitHub account. This means that anyone can submit issues or make comments. The limitation here though is that only collaborators (members) of the repository can have permissions which let them actually edit the documents contained in the repository. In GitHub, the most common permission used for collaborators is Write.

To add Collaborators, go to the Repository:

Click the Settings tab
Select the Collaborators & teams from the left side
Scroll to the bottom and enter the username of the person who will be working in the repository and click Add collaborator

They will be added to the list of collaborators. If the permission does not show Write, select the menu item and choose Write.

2.5. GitHub Teams

In addition to having collaborators in the repository, the organization can also create Teams of collaborators. Generally teams have two purposes: to control access to a branch and to request reviews on a Pull Request. Additionally, teams also have their own specific GitHub pages where discussions outside of Issues or Pull Requests can happen within the team.

More information about teams can be found here.

While an organization can create teams to suit their needs, there should be at least 2 specific Teams that are used in the cPP/SD repository.

Teams are specific to a repository, so to have the same team in multiple repositories will require creating that team in each separate repository.

To create a new team:

Go to the Repository Settings
Choose the Collaborators & teams menu
Click +Create new team
Enter a team name (IT, Editor, etc) and a description
Click Create team

Once the team has been created, you will be taken to the Team discussion page.

Back on the Collaborators & team settings page, you can assign the permissions to the repository for the team. The [Organization Teams] table shows the recommended Teams.

Table 3. Organization Teams
Team	Branch	Permission	Purpose
Editors	Master	Maintain	To triage and prioritize Issues to be worked on. To manage the Master/Published versions of the cPP/SD and related docs (only users with Write access to the Master branch).
IT	Interpretation	Maintain	To handle incoming Interpretation requests. To manage the Interpretation branch for changes related to requests (only users with Write access to the Interpretation branch).

2.5.1. Issue Triage and Teams

One important consideration with the use of teams is that they cannot be used everywhere in GitHub. The main point is that Issues cannot be assigned to a Team. When a new issue is created, it isn’t possible to specify an entire team to be notified of the issue, only up to 10 possible users.

To simplify incoming issue triage, it is recommended that a small team be created (this does not have to be a formal GitHub Team) that would be assigned (for Issue Templates) and would also regularly check for new issues that may have been created without having been assigned. The purpose of this group would be to assign the issue to a proper Project for review, not to necessarily issue any judgement on the contents of the issue.

2.6. Branch Protection Rules

Once the branches, users and teams have been setup, rules should be configured to protect the branches. This protection forces changes to be reviewed before they can be merged into the document, thereby preventing accidental changes and also forcing review of the changes before they become part of the document.

While each branch should have protection, the amount of protection is usually different. In general, each branch should be protected based on its importance. For example, the Master is the "official" version, and so should have a higher bar of approval as well as more limited access to the approval process to be changed, while the Working branch is where work on the next iteration is being done, and so should be more easily updated.

The [Branch Protection] table shows some examples of the levels to set the approvals at. What these settings mean is how many users must approve a change before it can be merged into the specified branch.

Table 4. Branch Protection
Branch	Ratio	Example
Master	50%	2 (for 3-4 members of the Editor Team)
Working	25%	2 or 3 (for 10 members)
Interpretation	75%	3 or 4 (for 4-5 members of the IT)

The numbers should be set to something workable for the iTC. While there may be 50 members in an iTC, maybe only 14 may be active (obviously this will vary), so setting approval requirement to 25 could prevent ever being able to make changes. Generally it would be recommended to base these numbers on the active collaborators within the iTC.

These numbers can be changed at any time, so start low and work up based on the number of active collaborators.

In addition to the total number of approvals needed to make a change, is the layer of permissions about who can make these changes. This is where the Editor and IT teams come in, where only members of these teams will be allowed to make changes (i.e. merge Pull Requests) to specific branches.

To set a rule:

Click the Settings tab
Select Branches from the left side
Click the Add rule button on the right hand side
Type in the name of the branch to be protected in the Branch name pattern
Check the box in Require pull request reviews before merging
Select the number in the Required approving reviews dropdown
Check the box Include administrators
Check the box in Restrict who can push to matching branches
In the dialog type in the name of the appropriate team
Click Create

Do this for both the Master and the Interpretation branches. The Working branch can be created without the Restrict who can push to matching branches setting being enabled.

The setting to Include administrators is specifically to ensure that administrators follow the same rules on branches as everyone else. There may be times when this is a problem, and can be adjusted as needed by the organization.

A good reference for additional information about protecting branches can be found here.

2.7. GitHub Backups

One of the most important thing to do is have a backup of the Organization repository. While online services are usually safe, occasionally there have been reports about repositories being lost in GitHub (and not when someone actually deleted it).

While there are simple measures for making sure all the branches are backed up, the concern here is to have the entirety of the repository backed up, from the branches to the Issues, Pull Requests, Wiki, etc.

The following steps require someone with familiarity or knowledge of Python.

Automatic download and archival of this GitHub content can be facilitated by using a script and cron on a Linux system to pull the data.

Python can be installed and run in automated manners on Windows systems as well, but this example uses a Linux system.

Perform the following:

Use pip to install the software:
```
sudo su && pip install github-backup
```
Generate an access Token
1. Login to GitHub
2. Click the user avatar icon, and click Settings.
3. Click Developer settings in the left hand list.
4. Click Personal access tokens.
5. Click Generate new token in the upper right quadrant.
6. Define the permissions for the access token. It’s recommended to avoid adding permissions to access GPG keys, or similar security tokens. It’s unnecessary for the purpose of backups.
7. Click Generate token at the bottom of the page.
8. Once generated then copy it to a safe place, because you’ll never see the code again.
Create a crontab entry with your desired download schedule, and designate a location to store the data.

To backup one repository (and accompanying data):

github-backup USERNAME -P -t ACCESS_TOKEN -o BACKUP_DIR --all -O -R REPOSITORY

Replace the items in all caps with your respective information.

If multiple repositories are desired, or other syntactical adjustments are necessary, read the help pages for github-backup:

github-backup --help

3. Asciidoctor

Asciidoctor is a fast, open source text processor and publishing toolchain for converting AsciiDoc content to HTML5, DocBook, PDF, and other formats. Asciidoctor is written in Ruby and runs on all major operating systems. The Asciidoctor project is hosted on GitHub.

— Asciidoctor website

3.1. Background

The second major tool being used here is Asciidoctor. The reason for this is because of how GitHub works. Git functionality (and hence GitHub) is centered around the use of text files. This makes a lot of sense in the programming world where the code is eventually compiled into binaries that run things. The downside of course is that we expect nice output such as that available from Word or similar applications in the creation of our documentation, and these all use binary formats. The problem here is that the binary formats do not allow for the easy comparisons or multi-editor usage, but these are built-in capabilities of Git and GitHub.

So the question then is how to use GitHub while also being able to have rich output text, and this is where Asciidoctor comes in. Asciidoctor uses text files as the input, basically as a lightweight markup language (think HTML but with a LOT less bracketing), and provides the ability to create rich output such as HTML or PDF (the two main concerns for us). By utilizing Asciidoctor within GitHub, you get the benefits of GitHub while making the document editing hopefully not completely painful.

3.2. Asciidoctor Setup

While documents are written in the Asciidoctor syntax, the Asciidoctor toolchain must be installed to process the Asciidoctor files into HTML or PDF output. Asciidoctor is distributed as a RubyGem, and so can be installed on any system where a late version of Ruby (2.5.5 or later recommended) can be installed.

3.2.1. Installing Ruby

A Ruby for Windows installer can be found here. Pick one with the devkit (I’m not sure if this is strictly needed, but prefer to have everything to make sure there are no problems).

This is not the only way to install Ruby, but is being used here as a reference.

3.2.2. Installing the Asciidoctor Toolchain

Once Ruby has been installed, start "Command prompt with Ruby". The execute the following commands:

gem install asciidoctor
gem install asciidoctor-pdf --pre
gem install asciidoctor-diagram

This will install the components of the Asciidoctor toolchain.

3.2.3. Installing CCUF Custom Components

The following components need to be installed:

itc_gentools Output Generation package

3.2.3.1. itc_gentools Output Generation

This particular package can be found here. The site has instructions to download the Gem file needed to install the package.

3.2.4. Test the Toolchain

To quickly test the full toolchain, pick a file and then run this command (fill in your own Asciidoctor file):

itc_gentools both <sample.adoc>

If there are no errors or prompts that something went wrong, there should be an HTML and a PDF files in the folder.