While at Offerup.com I had to use Cucumber to run Gherkin style tests. I had not used this style of test writing before. While it seems easy and straight forward at first, I quickly realized that its flexibility would allow the wording and structure of tests to quickly diverge and become unmanageable.

Here is my approach to writing decent tests, Gherkin best practices, and how to name elements. These guidelines are designed to make the tests as understandable, concise, and flexible while reducing duplicated code. A lot of ideas were taken from this Automation Panda post, which is good reading.

General

• Describing behavior in a concise manner is the most important thing
• You should strive to use short phrases to accurately convey what each line is doing
• What you write should describe what you are doing, not how or why
• Feature titles should follow the Scenario rules below
• Do not use the Rules keyword. More than one is not supported by cucumber.js, which makes it very awkward
• Try to not use the Background keyword. Since it is common setup, it can render the Given sections of tests irrelevant. Leaving out the Given then makes the flow odd.
  • You could use Background then make a step definition for “Given The Background task passed” and have it do nothing. This would just serve to create decent Gherkin in the spec file.

Example

  Scenario: Create a new account using valid information
    Given The user starts on the create account screen
    When The user enters valid account information
      And presses the "Create Account > Create Account" button
    Then The user arrives at the "Secure Your Account" screen

The wording is concise and describes what is being done so anyone can easily see what this test does. It is also worded in a way that allows for flexibility with dynamic items (phrases in quotes).

Embedded Test Data

• Don't use scenario outlines or data tables to execute extra cases you don’t need just because you can
  • This increases the execution time and increases the change of a random failure
  • Most of the time one element of a behavioral set is fine to verify the behavior of all. For example, you don’t need to open up every search category if they all look the same, but you may want to do one of one type and one of another if they can result in different results.

Example

  Scenario: The first message page contains the proper elements
    Given The user is logged in
    When The user taps the "Item > Inquire" button
    Then The user sees these elements on the "Send Message" page
      | Send A Message Heading |
      | User Profile           |
      | Item Details           |
      | New Message Input      |

A data table is appropriate here since we want to verify a number of elements are present. One may be sufficient if we just wanted to know if we were on that page, but the Scenario clearly says we want to verify all proper elements.

  Scenario Outline: Send a first message with each suggested message
    Given The user is logged in
    When The user taps the "Send A Message > <suggested response>" button
    Then The first sent message is "<suggested response>"

    Examples:
      | index | suggested response             |
      | 0     | Hi, is this still available?   |
      | 1     | Hi, I want to buy this         |
      | 2     | Hi, can you meet?              |
      | 3     | Will you ship via Frobozz?     |

Like the previous example this makes sense since we want to test each option presented to the user and each results in a different behavior.

Scenario Text

• Scenario titles should be one line, any additional info can go in the description
  • Watch out for usage of "and" and "or" which can indicate the need to split into multiple scenarios
  • "Because", "since", and "so" portions can be left out since they describe why you are doing this, not what
  • No "verify" or "should" since that is about assertions, not describing behavior. This belongs in the steps.

Examples

  Scenario Outline: Send a first message with each suggested message

Not bad, but the "suggested message" part could probably be cut or moved to the Description section.

  Scenario: Selecting the email login method takes you to that landing page

The "takes you to that landing page" part has an implicit "should" in front of it and should probably be removed. This phrase describes a result, and that should be covered by a subsequent Then statement.

Scenario Content

• Scenarios should be 10 lines or less
• Try to have only one Given/When/Then flow per scenario. This is the idea of one scenario equals one behavior.
• Lengthy sections can be encapsulated into a single step definition. This could also enable reuse of behavior since it is done outside of the spec file.
• It may be possible to skip long setup sections by directly navigating to the start of the actual test. This is most appropriate for UI tests.

Examples

        Then The user sees the "Account > Public Profile Name" element
            And sees the "Account > Public Profile Ratings" element
            And sees the "Account > Public Profile Bought Count" element
            etc...

All of these conditions could be collapsed into a single step definition and replaced with something like:

     Then The Account page contains all expected elements

Sometimes you may want to break the single Given/When/Then block in the scenario: If a test depends on the state of a previous test and there is not an easier way to set up those conditions atomically. You could make a step definition and use it twice in two tests. However, this could greatly increase run time since you are executing a long section of behavior twice.

Instead, you could do something like this:

Scenario: Scrollable items can be scrolled
    Given The user is on the "Home" screen
    When The user swipes "left" on the "Home > Category Bar" section
    Then The user does not see the "Home > All Categories" button

    Given The previous block passed
    When The user scrolls the "Home > Items Grid" section "down"
    Then The user does not see the "Home > Main Search Bar" element

The step “Given The previous block passed” actually does nothing in it’s step definition code. It is just here to provide proper Gherkin by saying what is going on. This allows you to build off previous functionality, but still break things up into individual Given/When/Then blocks that are atomic.

The drawback is this is really multiple tests per scenario, and will be reported as such in the results. But the time savings and code reuse is worth it.

Steps

• Each step should be less than 120 characters
• Reading each line by itself should tell you exactly what it is doing. Use the form: subject, action, predicate
• Don't spell out content that is implied, such as pressing Submit after doing something
• Try to not depend on exact data if it doesn't matter (ex: if you just care the list has content, look for a first item, not "item equals X" in your data)
• Hide data in step definitions instead of polluting the feature with it, unless it is critical to having the step text make sense
• Like scenario titles, if you have “and” or “or” in the text, split it into multiple steps
• If you have more than one or two Ands or Buts, you probably need to encapsulate them into a new keyword with a more descriptive name

Example

    Given The user is logged in as "Test Seller"
      And has at least one message
    When The user opens message number 1

Here the steps are short and declarative. There is some data included, but it is needed to specify an exact object acted upon.

The And clause is a separate behavior and split off like this. It also is made more generic since we just care that we have a message, not what it is or any other details.