ArunodhayamBy default, GitHub offers hosted runners to run our workloads inside. For their Linux runners, GitHub offers runners with specs: 2-core CPU, 7 GB of RAM, 14 GB of SSD disk space.
Some of your CI/CD workloads may require more powerful hardware than GitHub-hosted runners could provide. In such case, you can configure an EC2 instance to act as your GitHub runner instead.
For example, you may run a c5.4xlarge instance type for some of your workloads, or even a r5.xlarge for workloads that process large data sets in-memory.
Let us begin:
#1. Create a GitHub personal access token
Go to your GitHub Settings
Click the
Developer settings
- Click
Personal access tokento create one
- Create a new GitHub personal access token with the
reposcope in the name ofGH_PERSONAL_ACCESS_TOKEN. The action will use the token for self-hosted runners management in the GitHub account on the repository level.
#2. Create an AMI image
- Create a new EC2 instance based on a Linux distribution of your choice
- Connect to the instance ```sh
ssh -i user_name@IP_ADDRESS
- Install packages and configure the instance according to your workflow requirements
- After that, land on the EC2 console, select your instance, right-click on it to find `Image and templates` , hover to see `Create image`, click it
- Enter the `Image name`, `Image description`, and check `No reboot`, and at last , hit `Create`
- The AMI should show up in the AMI console
- You can safely terminate the instance as it's of no use any more, of course, you could just keep it running if you plan on making any changes to it, however
## #3. Create VPC with subnet and security group
- Create a new [VPC and Subnet](https://docs.aws.amazon.com/vpc/latest/userguide/working-with-vpcs.html) or use an existing
- Create a new [Security group](https://docs.aws.amazon.com/cli/latest/userguide/cli-services-ec2-sg.html) for the runners in the VPC and only allow outbound traffic to port 443, for pulling jobs from GitHub. No port opening for inbound traffic is necessay.
## #4. Configure the environment secrets
- The following environment variables must be set for this action
| Secrets | Description |
| ------- | ----------- |
| `GH_PERSONAL_ACCESS_TOKEN` | Add the GitHub pat token in secret |
| `AWS_REGION` | Add the region in secret |
| `AWS_ACCESS_KEY_ID` | Add the access key in secret |
| `AWS_SECRET_ACCESS_KEY` | Add the secret key in secret |
| `AMI` | Add the EC2 AMI ID in secret |
| `INSTANCE_TYPE` | Provide the instance type like t2.medium, t2.micro |
| `SUBNET` | Add the subnet ID in secret |
| `SECURITY_GROUP` | Add the security group ID in secret |
- Go to your project `repo settings` to add the above Secrets
- Expand the `Secrets` drop-down and click `Actions`
- In the _Actions secrets_ tab click `New repository secret`
- Name your secrets and add their values to be consumed by the action
- A complete listing of the required secrets should look something like
## #5. Configure the GitHub workflow
- Create a new GitHub Action with the below example workflow
- Please don't forget to set up a job for removing the EC2 instance at the end of the workflow execution.
Otherwise, the EC2 instance won't be removed and continue to run even after the workflow execution is finished.
Now you're ready to go!
### Inputs
| Name | Required | Description |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mode` | Always required. | Specify here which mode you want to use: <br> - `start` - to start a new runner; <br> - `stop` - to stop the previously created runner. |
| `GitHub-token` | Always required. | GitHub Personal Access Token with the `repo` scope assigned. |
| `ec2-image-id` | Required if you use the `start` mode. | EC2 Image Id (AMI). <br><br> The new runner will be launched from this image. <br><br> The action is compatible with Amazon Linux 2 images. |
| `ec2-instance-type` | Required if you use the `start` mode. | EC2 Instance Type. |
| `subnet-id` | Required if you use the `start` mode. | VPC Subnet Id. <br><br> The subnet should belong to the same VPC as the specified security group. |
| `security-group-id` | Required if you use the `start` mode. | EC2 Security Group Id. <br><br> The security group should belong to the same VPC as the specified subnet. <br><br> Only the outbound traffic for port 443 should be allowed. No inbound traffic is required. |
| `label` | Required if you use the `stop` mode. | Name of the unique label assigned to the runner. <br><br> The label is provided by the output of the action in the `start` mode. <br><br> The label is used to remove the runner from GitHub when the runner is not needed anymore. |
| `ec2-instance-id` | Required if you use the `stop` mode. | EC2 Instance Id of the created runner. <br><br> The id is provided by the output of the action in the `start` mode. <br><br> The id is used to terminate the EC2 instance when the runner is not needed anymore. |
| `iam-role-name` | Optional. Used only with the `start` mode. | IAM role name to attach to the created EC2 runner. <br><br> This allows the runner to have permission to run additional actions within the AWS account, without having to manage additional GitHub secrets and AWS users. <br><br> Setting this requires additional AWS permissions for the role launching the instance (see above). |
| `aws-resource-tags` | Optional. Used only with the `start` mode. | Specifies tags to add to the EC2 instance and any attached storage. <br><br> This field is a stringified JSON array of tag objects, each containing a `Key` and `Value` field (see example below). <br><br> Setting this requires additional AWS permissions for the role launching the instance (see above). |
| `runner-home-dir` | Optional. Used only with the `start` mode. | Specifies a directory where pre-installed actions-runner software and scripts are located.<br><br> |
### Outputs
| Name | Description |
| ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `label` | Name of the unique label assigned to the runner. <br><br> The label is used in two cases: <br> - to use as the input of `runs-on` property for the following jobs; <br> - to remove the runner from GitHub when it is not needed anymore. |
| `ec2-instance-id` | EC2 Instance Id of the created runner. <br><br> The id is used to terminate the EC2 instance when the runner is not needed anymore. |
### Example workflow
`ci.yml`
```yml
name: ci
on: pull_request
jobs:
start-runner:
name: Start self-hosted EC2 runner
runs-on: ubuntu-latest
outputs:
label: ${{ steps.start-ec2-runner.outputs.label }}
ec2-instance-id: ${{ steps.start-ec2-runner.outputs.ec2-instance-id }}
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Start EC2 runner
id: start-ec2-runner
uses: machulav/ec2-github-runner@v2
with:
mode: start
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
ec2-image-id: ${{ secrets.AMI }}
ec2-instance-type: ${{ secrets.INSTANCE_TYPE }}
subnet-id: ${{ secrets.SUBNET }}
security-group-id: ${{ secrets.SECURITY_GROUP }}
iam-role-name: my-role-name # optional, requires additional permissions
aws-resource-tags: > # optional, requires additional permissions
[
{"Key": "Name", "Value": "ec2-github-runner"},
{"Key": "GitHubRepository", "Value": "${{ github.repository }}"}
]
do-the-job:
name: Do the job on the runner
needs: start-runner # required to start the main job when the runner is ready
runs-on: ${{ needs.start-runner.outputs.label }} # run the job on the newly created runner
steps:
- name: Hello World
run: echo 'Hello World!'
stop-runner:
name: Stop self-hosted EC2 runner
needs:
- start-runner # required to get output from the start-runner job
- do-the-job # required to wait until the main job is done
runs-on: ubuntu-latest
if: ${{ always() }} # required to stop the runner even if the error happened in the previous jobs
steps:
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ secrets.AWS_REGION }}
- name: Stop EC2 runner
uses: machulav/ec2-github-runner@v2
with:
mode: stop
github-token: ${{ secrets.GH_PERSONAL_ACCESS_TOKEN }}
label: ${{ needs.start-runner.outputs.label }}
ec2-instance-id: ${{ needs.start-runner.outputs.ec2-instance-id }}
After a successful first workflow execution, you should see this
