☑️ Why Include Accessibility Testing in your Workflow?

SvelteKit accessibility testing does not have to be complicated — we’ see that in this post. The need for accessibility testing in your apps is clear, and there is no point rehearsing the motivation for accessibility testing itself here. However, a valid question is why include accessibility testing in the continuous integration process? In other words, once you have built your site and improved the accessibility, why do you need to continue running tests?

The simple answer is that it is very easy, as you update a site, to break its accessibility. SvelteKit tooling goes a long way to help you make your site accessible, with helpful accessibility integration in the VSCode plugin. Not to mention handy hints from the svelte-check tool. However, these don’t currently check how the new tint of your brand colour impacts the colour contrast ratio. Nor if you accidentally reused a previously taken id on a new element which you just added.

In short, your initial accessibility tests help to set you on the right track, while automated integration tests run with each commit or push, help make sure your site stays accessible. Of course these are automated tests and have limitations, so to cover all bases you may also want to get the site audited by an accessibility professional. That said the automated tests are still a handy tool to keep in your tool belt.

🧱 What we're Building

In this post we'll see how you can use Playwright to run integration tests as you develop or each time you build your site. Playwright is an automated testing tool like Cypress. We just use it for accessibility testing here, but you can also use it for more complete end-to-end and integration testing. On top we level up the testing using the pa11y tool, running axe and htmlcs tests. Then we add polish with Husky, merging the tests into your continuous integration process. If that sounds like something you can get excited about, then why don’t we get cracking?

⚙️ SvelteKit Accessibility Testing: Setup

We will run through using the SvelteKit MDsveX starter but if you already have a SvelteKit site, it might make more since for you to follow along but working on a test branch of your own site. If you are using the starter, let’s get going by cloning it locally:

git clone https://github.com/rodneylab/sveltekit-blog-mdx.git sveltekit-accessibility-testing
cd sveltekit-accessibility-testing
pnpm install

Next we’ll install the extra packages needed (do this whether you're working on the starter or your own site):

pnpm add -D @playwright/test pa11y pa11y-ci playwright-core
pnpx install playwright

Playwright can run tests in various browsers and the second step above installs all the browser packages Playwright needs. This can take a few minutes, depending on your internet connection, so you might want to brew a coffee once you start that step running. You will also need to rerun that step each time you update Playwright.

playwright.config.js

Once you have all of that installed, let's move on to adding a Playwright config file. Create playwright.config.js in your project’s root folder and add the following content:

import { devices } from '@playwright/test';

/** @type {import('@playwright/test').PlaywrightTestConfig} */
const config = {
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  use: {
    trace: 'on-first-retry',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    {
      name: 'firefox',
      use: { ...devices['Desktop Firefox'] },
    },
    {
      name: 'webkit',
      use: { ...devices['Desktop Safari'] },
    },
  ],
  // webServer: {
  //   command: 'pnpm run build && pnpm run preview',
  //   port: 3000,
  // },
};
export default config;

There’s a couple of ways you can run the tests. If you want to run tests with a dev server spinning in the background, then the config above, as it is, is what you need. Alternatively, once you’re ready to commit, you can stop your dev server and have Playwright automatically build and test the site. In this case uncomment lines 24–27 (also change the port in line 26 to suit your needs).

The final piece of setup it so add an environment variable to make things a little more configurable. If you are working on your own site, you will need to install the dotenv package. Either way, update the .env file to include the following line:

SITE_TEST_URL="http://localhost:3000"

Once again change the port if needed.

🏡 Home Page Test

We will place all of our test files in a new tests folder in the project root. Create a pages folder within tests then add an index.spec.mjs file (if you prefer TypeScript create index.spec.ts). Add the following content:

import { expect, test } from '@playwright/test';
import 'dotenv/config';
import pa11y from 'pa11y';

const path = '/';
const url = `${process.env.SITE_TEST_URL}${path}`;

test.describe('it has no accessbility issues', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto(url);
  });
  test('it has no issues identified by htmlcs or axe', async () => {
    const results = await pa11y(url, {
      runners: ['axe', 'htmlcs'],
    });
    if (results.issues) {
      results.issues.forEach(({ code, message, runner, selector }) => {
        console.log({ code, message, runner, selector });
      });
    }
    expect(results.issues.length).toBe(0);
  });
});

This is a basic Playwright test for our home page. It does not look too different to test files from other testing tools, so we won't spend too much time on it. If there is anything that needs clarification, please drop a comment below so I can update the post.

In line 14 you see we run both axe and HTML Code Sniffer tests. Together these two runners will find 35% of accessibility issues. I have noticed that the axe tests can fail on contrast ratio when you use background gradients (even when manual testing shows contrast ratios are good). You will probably want to tweak configuration on such pages, if you run into this issue.

You can test the home page now, from the Terminal:

pnpx pa11y http://localhost:3000/

We will look at some scripts and integrated testing in a moment. Before that, try breaking accessibility to reassure yourself the tests work. As an example, you could temporarily remove the title in the HTML head. In the starter, this is injected in src/lib/components/SEO/index.svelte. Just comment out the <title>{pageTitle}</title> at line 27 to test.

You can create a similar test file, to the one above, for other pages. In the starter, we have a contact page so copy the contents above to tests/pages/contact.spec.mjs and just update line 4 in the new file:

import { expect, test } from '@playwright/test';
import pa11y from 'pa11y';

const path = '/contact';
const url = `${process.env.SITE_TEST_URL}${path}`;

Next let's see what we can do for blog posts.

🖋 SvelteKit Blog Post Accessibility Testing

We will create a single test file for all blog posts, so you can see how to parameterise Playwright tests. This pattern will save you creating new test files as you write fresh blog posts. Make a new folder: tests/posts and in there create index.spec.mjs:

import { expect, test } from '@playwright/test';
import 'dotenv/config';
import { existsSync, lstatSync, readdirSync } from 'node:fs';
import { join, resolve } from 'node:path';
import pa11y from 'pa11y';

export const BLOG_PATH = 'src/content/blog';
const __dirname = resolve();
export function getSlugs(location) {
  const directories = readdirSync(location).filter((element) =>
    lstatSync(`${location}/${element}`).isDirectory(),
  );
  const articles = [];

  directories.forEach((element) => {
    const contentPath = `${location}/${element}/index.md`;
    if (existsSync(contentPath)) {
      articles.push(element);
    }
  });
  return articles;
}

const postLocation = join(__dirname, BLOG_PATH);
const slugs = getSlugs(postLocation);

test.describe('it has no accessbility issues', () => {
  for (const slug of slugs) {
    const url = `${process.env.SITE_TEST_URL}/${slug}`;
    test.beforeEach(async ({ page }) => {
      await page.goto(url);
    });
    test(`${slug}: it has no issues identified by htmlcs or axe`, async () => {
      const results = await pa11y(url, {
        runners: ['axe', 'htmlcs'],
      });
      if (results.issues) {
        results.issues.forEach(({ code, message, runner, selector }) => {
          console.log({ code, message, runner, selector, url });
        });
      }
      expect(results.issues.length).toBe(0);
    });
  }
});

Here at the top we have a function to generate a list of all the slugs for blog posts. You might need to adapt this to work on your own site. Then in line 28 you see the for loop which runs the tests for each slug. If you update this code for your own site, be cognisant that the existing for loop contains async code so you will need to be careful if you opt for using the forEach array method. In the test name at line 33 we include the slug parameter to give each test a unique name.

Try testing a blog post or two. Next we will look at some integration scripts. Add more test files if you have different types of pages in your own site.

Husky Continuous Integration Scripts

We are almost done. Tinker with the following steps to suit your own taste. First add two new testing scripts to package.json:

{
  "name": "sveltekit-blog-mdx",
  "version": "2.0.0",
  "scripts": {
    // ...TRUNCATED
    "test:a11y": "pa11y-ci --sitemap 'http://localhost:3000/sitemap.xml' --sitemap-find 'https://example.com' --sitemap-replace 'http://localhost:3000'",
    "test": "playwright test"
  },

Update the ports and URLs here depending on your site and local setup. I would run the test script with each commit, to check my work as I go along (or even between commits, depending on what I am working on). This will run all of the test files in the tests folder. Then I would do a double-check before pushing, which involves running test:a11y on all pages in the sitemap. I would build the site and run it in preview mode, making sure the sitemap is up-to-date first. With the starter, you can update the sitemap by running pnpm run generate:sitemap.

With those scripts defined, let’s add Husky scripts. These will run when we commit and push the code. If you are running on your own site, install husky to get this going.

When you are ready, update .husky/pre-commit and .husky/pre-push:

#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

pnpm run prettier:check && pnpm run lint:scss && pnpm run test

#!/bin/sh
. "$(dirname "$0")/_/husky.sh"

pnpm run check && pnpm run test:a11y

Then finally to complete setup in the Terminal run:

pnpx husky install

That’s it. I hope it has all worked well for you. Let me know if you had any issues or if anything needs more clarification.

🙌🏽 SvelteKit Accessibility Testing: Wrapping Up

In this post we have seen a nice way of adding automated a11y testing to your Svelte project. In particular we have seen:

• how to configure Playwright with SvelteKit,
• a way to use pa11y with axe and htmlcs tests in Playwright,
• how to parameterise Playwright tests.

The full code for the app is available in the repo on Rodney Lab GitHub.

I hope you found this article useful and am keen to hear how you plan to use the new tests in your Svelte projects.

🙏🏽 SvelteKit Accessibility Testing: Feedback

Have you found the post useful? Would you prefer to see posts on another topic instead? Get in touch with ideas for new posts. Also if you like my writing style, get in touch if I can write some posts for your company site on a consultancy basis. Read on to find ways to get in touch, further below. If you want to support posts similar to this one and can spare a few dollars, euros or pounds, please consider supporting me through Buy me a Coffee.

Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on Twitter and also askRodney on Telegram. Also, see further ways to get in touch with Rodney Lab. I post regularly on Astro as well as SvelteKit. Also subscribe to the newsletter to keep up-to-date with our latest projects.