Hello Everyone, I hope you all are rocking in your digital world. So, Here is my first blog, I will be sharing my experience and what i learned while working on end-to-end testing for Litmus-Portal. we will be going through how to get started with End-to-End testing with Cypress in any of your project and how litmus-Portal is using Cypress for testing different scenarios.Stay tuned till the end, you will get to know many awesome things.

Litmus-Portal provides console and UI experience for managing, monitoring, and events around chaos workflows. Chaos workflows consist of a sequence of experiments run together to achieve the objective of introducing some kind of fault into an application or the Kubernetes platform.Using Litmus-Portal, you can make your projects or products more resilient.

For doing all this and providing resiliency to your product, Litmus-Portal has to be resilient itself.That's where Cypress comes which helps us to test Litmus-Portal in different scenarios and makes it more resilient.

Cypress

Cypress is a modern frontend End-to-End testing tool using which we can write our tests in javascript as well as Typescript. It simplifies how we write our tests, makes our tests less flaky, and helps us in reducing maintenance cost of our project.

Why Cypress?

Well, we could have used some other framework for our purpose but, we wanted the one, which is easy to setup and reliable.There are many advantages of using Cypress -

• Easy to setup, documentation is more than sufficient.
• It helps us in adapting best practices for testing with it's documentation.
• As Cypress shows all logs side by side to AUT (Application Under Test), it is very easy to debug our projects.
• A plugins catalog provided by Cypress and it's community, which is very helpful to test different scenarios.
• It is very easy to adapt, as it build on top of Mocha, Chai, chai-jQuery and many other libraries.

Installing Cypress

Cypress is an NPM package. We can install Cypress as a development dependency like

npm install cypress --save-dev

We can use Cypress in two modes -

• Browser mode

For using Cypress in Browser Mode, we can make use of this command -

npx cypress open

This will open a browser for you, showing different default-test scripts. We can click on different scripts to execute them.

• Headless mode

For using Cypress in Headless Mode, we can make use of this command -

npx cypress run

This will open a terminal for you, and start to execute the tests present in the Test scripts path (By default, Integration directory).

After executing this command,you will observe that some predefined directories and files have been added in your project -

-cypress
  |_fixtures
  |_integration
  |_plugins
  |_support
-cypress.json

Here, cypress is the directory that contains everything required for testing using Cypress.

• fixtures/ - This directory contains all the static data (data you want to use for setting your Databases between tests or you want to input on your different screens) to be used while testing in form of JSON files.

An example of User.json (fixture file for users) is given below -

{
    "projectname":"litmus",
    "AdminName":"John",
    "AdminPassword":"admin1234",
    "AdminEmail":"admin@gmail.com",
    "NewName":"John",
    "NewPassword":"John123",
    "NewEmail":"John@gmail.com"
}

• integration/ - This directory contains all the test scripts. We can configure a different location for storing our test scripts in cypress.json.

• plugins/index.js - This file can contain configuration for all plugins installed.

• support/commands.js - It will contain all the custom functions that we might need while writing our test so that we don't repeat ourselves.

• support/index.js - This file contains any configurations for test cases. for e.g. by default cookies are not preserved between tests. They can be preserved add following code in this file -
  

// For preserving cookies between tests.
Cypress.Cookies.defaults({
    preserve: 'token'
});

cypress.json - This is a configuration file for Cypress.

An example of cypress.json is given below -

{
  "baseUrl": "http://localhost:3001",
  "experimentalFetchPolyfill": true,
  "viewportWidth": 1800,
  "viewportHeight": 1200,
  "defaultCommandTimeout": 10000,
  "chromeWebSecurity": false,
  "video": false,
  "experimentalNetworkStubbing":true,
  "env": {
    "authURL" : "http://localhost:3000",

  }
}

You will get to know more about them, as we proceed with testing different scenarios.

Let's have something in our pocket before moving on further -

DOM Selector - It is a selector that is used for selecting different objects in DOM for testing or automation. A Selector can be any CSS property, Ids, and Classes. But let me tell you when you make a product your CSS properties, ids, and classes keep changing, which might break our tests.

The best practice is to use a unique identifier which is not much subjected to change also which is unique on a particular page for identifying an element. While using Cypress, we have the support to use data-* attributes with our elements.

As a best-practice. we enforce everyone working on Litmus-Portal to use data-cy=<selector> selector to every different component, so that it can targeted easily by Cypress while testing. This practice is also preferred by Cypress test runner

In Cypress, We can query an element on DOM using the command cy.get(<Your_Selector>)

In Cypress, data-* are given high priority while querying, so it is a bit fast as well.

for example, If there is button like this,

<button class="btn1">Save</button>

We can inject a unique identifier like this

<button class="btn1" data-cy="save">Save</button>

Now, we can access this button like this

cy.get('[data-cy=save]');

Okay, now I think we are good with testing and Cypress, we will go deep while working with live scripts, but let's get our hands dirty a bit.

Firstly, for testing an App, We need an endpoint of our WebApp.

Let's say the endpoint of our web app is https://localhost:3001

In Cypress, We can visit this link by using the function visit()

cy.visit("https://localhost:3001");

But being a lazy person, we don't want to write this bigger link again and again in every test or even in different test scripts.

You will be happy to know, we can also set the endpoint in cypress.json for universal use in test scripts.

In cypress.json,

{
   "baseUrl":"https://localhost:3001"
}

Now, anywhere in the test script, we want to visit the link, we can just do

cy.visit("/");

Well, this setup will work well in local setup. But when we are working on different CI's, we won't be able to use it because every time we setup the full stack web app in CI, A dynamic Link will be getting generated.

As Litmus-Portal is a cloud native web application, we have to deploy it on kubernetes while testing on different CI's. Everytime we generate a new dynamic link using loadbalancer for accessing the frontend.So, for this we needed a better approach as we can't provide access link before deploying Litmus-Portal.

But Hurray, I have something for you, We can provide the link as an environment variable to cypress while starting testing using command -

CYPRESS_BASE_URL=$link npx cypress run

So, Cypress will use this URL as BaseURL while executing our test scripts.

Now, as we know how to query an element and how to open our web app to be tested, the Next thing is how we write tests for our app.

We will be using Litmus-portal as our web app for Testing.

Starting with login Page for Litmus-Portal.

While writing tests for the Login Page, we have to consider all scenarios including positive and negative tests.

A Positive scenario can be like this -

1. Visit the login page.
2. Find the input for the name and type our correct name in it.
3. Find the input for the password and type our correct password in it.
4. Click on the login button.
5. Check if we are landing on welcome modal after clicking on the login button.

A Negative scenario can be like this -

1. Visit the login page.
2. Find the input for the name and type a wrong name in it.
3. Find the input for the password and type a wrong password in it.
4. Click on the login button.
5. Check if we are prompted with the error "Wrong Credentials".

Let me give you a small script for login page testing,

This was what I wrote on my first try. So don't laugh.

describe("Checking functionality of Login Page",()=>{

    it("Testing the only single input sign in [ Should not be possible ]",()=>{
        cy.visit("/");
        cy.get('[data-cy=inputName] input').type(" ");
        cy.get('[data-cy=inputPassword] input').type("John123");
        cy.get('[data-cy=loginButton]').click();
        cy.contains("Wrong Credentials").should('be.visible');
    })

    it("Testing with wrong details [ Should not be possible ]",()=>{
        cy.visit("/");
        cy.get('[data-cy=inputName] input').type("Johnce");
        cy.get('[data-cy=inputPassword] input').type("John123");
        cy.get('[data-cy=loginButton]').click();
        cy.url().should('include','/login');
        cy.contains("Wrong Credentials").should('be.visible');
    })

    it("Testing with Correct details [ Must redirect to Welcome modal ]",()=>{
        cy.visit("/");
        cy.get('[data-cy=inputName] input').type("John");
        cy.get('[data-cy=inputPassword] input').type("John123");
        cy.get('[data-cy=loginButton]').click(); //Correct Details
        cy.contains("Welcome to Portal");
    })
})

If you have worked with Mocha before, then you must be knowing that it's BDD interface provide describe(), it(), beforeEach(), afterEach() and many other functions. If you don't, don't worry we will discuss all of them as we use them.

Here, describe() is mainly used for collecting all tests under a single umbrella (Test-suite) which are related to each other or similar in objective.

it() is the function which signifies one test or one scenario.

Also, you must be seeing some new interactive functions as well.
Let me explain what we are doing here.

cy.visit("/"); visits the login page.
cy.get("[data-cy=inputName]") finds the input field for name and type(<data_to_be_typed>) is used to type in the selected input field.
cy.contains(<text>) is used to search text on page.

You must be seeing that we are visiting the login page again and again and also writing the same functions many times.

Let's refactor it a bit with one more BDD function i.e. beforeEach().

beforeEach() is used when we want to do something before every test scenario or if something is common before every scenario.

describe("Checking functionality of Login Page",()=>{

    beforeEach(Visiting the login Page,()=>{
        cy.visit("/");
    });

    it("Testing the only single input sign in [ Should not be possible ]",()=>{
        cy.get('[data-cy=inputName] input').type(" ");
        cy.get('[data-cy=inputPassword] input').type("John123");
        cy.get('[data-cy=loginButton]').click();
        cy.contains("Wrong Credentials").should('be.visible');
    })

    it("Testing with wrong details [ Should not be possible ]",()=>{
        cy.get('[data-cy=inputName] input').type("Johnce");
        cy.get('[data-cy=inputPassword] input').type("John123");
        cy.get('[data-cy=loginButton]').click();
        cy.url().should('include','/login');
        cy.contains("Wrong Credentials").should('be.visible');
    })

    it("Testing with Correct details [ Must redirect to Welcome modal ]",()=>{
        cy.get('[data-cy=inputName] input').type("John");
        cy.get('[data-cy=inputPassword] input').type("John123");
        cy.get('[data-cy=loginButton]').click(); //Correct Details
        cy.contains("Welcome to Portal");
    })
})

One problem is solved, but we are still writing functions for logging in many times which are the same, just the value provided is different.

So here, we take the help of the Custom Commands facility provided by Cypress.

Now, we will create a custom function which will take username and password as arguments and login the user. We can add that function in commands.js inside Support directory.

In your support/commands.js file inside the support folder,

// Custom login function for logging In which takes username and password as parameters.
Cypress.Commands.add('login',(Username,Password)=>{
    cy.get('[data-cy=inputName] input').type(Username);
    cy.get('[data-cy=inputPassword] input').type(Password);
    cy.get('[data-cy=loginButton]').click();
})

Your test script will look like this,

// Here in the script, we can just call the custom login function that we made just by using cy.login(username, password). 
describe("Checking functionality of Login Page",()=>{

    beforeEach(Visiting the login Page,()=>{
        cy.visit("/");
    });

    it("Testing the only single input sign in [ Should not be possible ]",()=>{
        cy.login("John"," ");
        cy.contains("Wrong Credentials").should('be.visible');
    })

    it("Testing with wrong details [ Should not be possible ]",()=>{
        cy.login("Vedant","1234");
        cy.url().should('include','/login');
        cy.contains("Wrong Credentials").should('be.visible');
    })

    it("Testing with Correct details [ Must redirect to Welcome modal ]",()=>{
        cy.login("John","John123");
        cy.contains("Welcome to Portal");
    })
})

Currently, the above script will work fine if we are testing locally, but when we work on a production server or CI, There might be delays in response from backend server, Cypress might get timed out waiting for homepage to load.

For dealing with this situation, we can use one command i.e,

cy.wait(8000);

Here, cy.wait() will wait make the test to wait for constant time given as argument to wait() function.

But this will make our test slower when we wait for constant time.

So, the better approach here is to use aliases for waiting for request to get resolved.
Here is an example -

// Custom login function for logging In which takes username and password as parameters and also waits for data from server.
Cypress.Commands.add('login',(Username,Password)=>{
    cy.server();
    cy.route("POST", Cypress.env('apiURL')+"/query").as("detailsResponse");
    cy.get('[data-cy=inputName] input').type(Username);
    cy.get('[data-cy=inputPassword] input').type(Password);
    cy.get('[data-cy=loginButton]').click();
    cy.wait("@detailsResponse").its("status").should("eq", 200); //Request Done.
})

In above example, cy.server() will start a mock server and will be intercepting all requests from frontend. cy.route() will tell Cypress to intercept a request going on a particular route. We can mock response, status and many other parameters while intercepting a request using cy.route().

Now, for waiting for a request to get resolved, we have to make alias for that route using as(). as() makes alias of any route with given name, which the Cypress will remember for us.

cy.route("POST", Cypress.env('apiURL')+"/query").as("detailsResponse");

Now, we can wait for this request using cy.wait() by giving alias name to it and checking it's status property using its() function after executing our steps.

cy.wait("@detailsResponse").its("status").should("eq", 200); //Request Done.

Thanks for staying with me till here, In my next article, we will be discussing more about how to test other scenarios that we face in a project.Till then, you can always check their documentation here

If you are interested in learning more about different test scenarios in Litmus-Portal, Checkout our Litmus-Portal-E2E Repository here

Conclusion

Feel free to check out our ongoing project - Litmus Portal and do let us know if you have any suggestions or feedback regarding the same. You can always submit a PR if you find any required changes.

Make sure to reach out to us if you have any feedback or queries. Hope you found the blog informative!

If chaos engineering is something that excites you or if you want to know more about cloud-native chaos engineering, don’t forget to check out our Litmus website, ChaosHub, and the Litmus repo. Do leave a star if you find it insightful. 😊

I would love to invite you to our community to stay connected with us and get your Chaos Engineering doubts cleared.
To join our slack please follow the following steps!

Step 1: Join the Kubernetes slack using the following link: https://slack.k8s.io/

Step 2: Join the #litmus channel on the Kubernetes slack or use this link after joining the Kubernetes slack: https://slack.litmuschaos.io/

Cheers!