Demystifying Strapi's Populated & Filtering

Paul Bratslavsky - Aug 22 '23 - - Dev Community

Introduction

Hello everyone! This discussion is based on Richard's Strapi Conf talk on demystifying Strapi's populated filtering. You can find the whole video here

This blog post serves as a quick refresher on effectively consuming data in Strapi using field selection, population, and filtering.

Why is This Discussion Necessary?

Over the years, Strapi has evolved significantly, and the most noticeable change has been in how we interact with our content. Content consumption is a crucial part of the data interaction experience. This post aims to help you become comfortable with the tools you need to interact with your data efficiently.

You can learn about these topics in our documentation. Check out the following sections, populate and filtering, as well as, this video where we discuss the topic in more details.

Getting Started with Population and Field Selection

Let's dive right into the concepts of population and field selection.

not-on-list.png

As you can see above, you can only get in if your name is on the list. Strapi operates this way with population and field selection.

By default, Strapi does not automatically populate or select your relations for you – you must do it manually.

This approach saves bandwidth from the database queries Strapi generates. As you interact with your data, for instance, through the REST API, you need to enable permissions for specific operations.

For instance, if you have a user with an articles (a relation), you can't populate this relation unless you enable the find permission on the collection type, as shown below.

permission-example.png

So here are some things to consider when working with populate and filtering in Strapi.

keep-in-mind@0.5x.png

Note: Remember the depth of your population and selection, as complex requests can take longer to execute.

Exploring Population and Field Selection with Examples

Let's go through some examples. I have created a repl environment that uses the qs library to generate LHS Bracket syntax strings that we can use to test out our queries in the browser or using Insomnia.

You can find the repl here with a basic example.

But for simplicity, I will be using Strapi's Query Builder tool, which you can find here.

Here is a helpful Chrome extension to make your JSON more beautiful when viewing in the browser. JSON Viewer

In my Strapi application, I have created several collection types and relations, which I will explain as we proceed. We have top-level relations, top-level fields, dynamic zones, and components within those dynamic zones.

2023-08-16_12-00-28 1.png

Let's take a look at some examples of population and field selection.

Populate Everything with the Wildcard Operator

Before looking at the wildcard * operator, let's look at what we get as a response when querying our articles without passing and populating or filtering options.

You can run this GET query in Insomnia, but we will just look at it in the browser for brevity.

You can download this chrome extension to make your JSON readable.

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles

Click to see the response

In our response below, we can see that we only return the top-level fields and nothing else. None of the other relations, components, or dynamic zones.

2023-08-16_12-15-31.png

If you want to bring back as much information as possible, use the wildcard * operator.

{
  populate: "*",
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate=*

Click to see the response

Remember that some details may not be visible, such as avatar and articles within the authorsBio relation, due to limited population depth.

2023-08-16_12-50-28.png

When using the wildcard * operator, you will only populate relations, dynamic zones, and components to a depth of one.

You can learn more in our documentation on the wildcard operator here.

Below, we have additional relations avatar and articles; let's look to see how we can populate those fields next.

2023-08-16_12-24-43.png

Populate Specific Relations

To populate specific relations, we must specify which fields we want to populate in Strapi. Let's take a look at the following example.

Instead of using the wildcard, we will build out the query manually. Here is what our current query looks like without any options being passed.

2023-08-16_12-15-31.png

Let's add this populate query. We can use this array populate notation to specify which fields we want to populate. The following will get the same data as when we used the wildcard and populated one level deep for all the specified items.

{
  populate: [
    "cover",
    "category",
    "blocks",
    "authorsBio",
    "seo"
  ],
}
Enter fullscreen mode Exit fullscreen mode

or using the object notation

{
  populate: {
    cover: {
      populate: true,
    },
    category: {
      populate: true,
    },
    blocks: {
      populate: true,
    },
    authorsBio: {
      populate: true
    },
    seo: {
      populate: true
    }
  },
}
Enter fullscreen mode Exit fullscreen mode

In most complex queries, object notation is preferred. We will see why later in the post.

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate[0]=cover&populate[1]=category&populate[2]=blocks&populate[3]=authorsBio&populate[4]=seo

Click to see the response

Screenshot 2023-08-16 at 1.14.31 PM.png

Before looking at how we can populate nested relations, let's look at how we can populate specific fields.

Populate Nested Relations

Let's look at how we can populate our nested relationships. We will use the above query as a starting point.

We will look at how to populate additional relations in our authorsBio relation and seo component.

{
  populate: [
    "cover",
    "category",
    "blocks",
    "authorsBio.avatar",
    "authorsBio.articles",
    "seo.shareImage"
  ],
}
Enter fullscreen mode Exit fullscreen mode

or using the object notation

{
  populate: {
    cover: {
      populate: true,
    },
    category: {
      populate: true,
    },
    blocks: {
      populate: true,
    },
    authorsBio: {
      populate: ["avatar", "articles"]
    },
    seo: {
      populate: ["shareImage"]
    }
  },
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate[0]=cover&populate[1]=category&populate[2]=blocks&populate[3]=authorsBio.avatar&populate[4]=authorsBio.articles&populate[5]=seo.shareImage

Click to see the response

In the response below, we can see the populated nested relationships.

2023-08-16_13-47-41.png

Now, let's look at the avatar response within authorsBio looks like.

Screenshot 2023-08-16 at 1.49.50 PM.png

There are a lot of fields. What if we wanted to populate selected fields only. That is exactly what I will show you in the next section.

Populate and Field Select

We will now see how you can tell Strapi only to return specific fields. We will also see where the object notation is useful since array notation does not allow you to do this.

Array notation example:

{
  populate: [
    "cover",
    "category",
    "blocks",
    "authorsBio.avatar",
    "authorsBio.articles",
    "seo.shareImage"
  ],
}
Enter fullscreen mode Exit fullscreen mode

Object notation example:

{
  populate: {
    cover: {
      populate: true,
    },
    category: {
      populate: true,
    },
    blocks: {
      populate: true,
    },
    authorsBio: {
      populate: ["avatar", "articles"]
    },
    seo: {
      populate: ["shareImage"]
    }
  },
}
Enter fullscreen mode Exit fullscreen mode

Let's refactor our object notation example to see how to populate only specific fields. We will focus on the authorsBio avatar field.

This is what our original response looked like.

Screenshot 2023-08-16 at 1.49.50 PM.png

Let's now only return the name, alternativeText, caption, and url.

To accomplish this, let's make the following changes to our query.

{
  populate: {
    cover: {
      populate: true,
    },
    category: {
      populate: true,
    },
    blocks: {
      populate: true,
    },
    authorsBio: {
      populate: {
        avatar: {
          fields: [
            "name",
            "alternativeText",
            "caption",
            "url"
          ]
        },
        articles: {
          populate: true,
  }
      }
    },
    seo: {
      populate: ["shareImage"]
    }
  },
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate[cover][populate]=true&populate[category][populate]=true&populate[blocks][populate]=true&populate[authorsBio][populate][avatar][fields][0]=name&populate[authorsBio][populate][avatar][fields][1]=alternativeText&populate[authorsBio][populate][avatar][fields][2]=caption&populate[authorsBio][populate][avatar][fields][3]=url&populate[authorsBio][populate][articles][populate]=true&populate[seo][populate][0]=shareImage

Click to see the response

Notice in the response below we returned all of our data from before and only the specified fields for our avatar.

2023-08-16_14-01-50.png

In the next section, we will look at complex population and **field selection.

Complex Population and Field Selection

In this example, instead of getting our articles, we will look to populate our home page. We will look first find our page by slug. We will populate our dynamic zone and look only to get the data for our testimonials group component.

2023-08-16_19-14-10 1.png

We will build this query in steps.

First, let's find our page via slug. We will use filters to filter where the slug is equal to home. We will look at filters in more detail in the next section.

{
  filters: {
    slug: "home";
  }
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug]=home

Click to see the response

We get the following response.

Screenshot 2023-08-16 at 11.59.33 PM.png

We can populate our dynamic zones with what we already learned. Let's modify our query with the following.

{
  filters: {
    slug: "home"
  },
  populate: {
    contentSections: {
      populate: true,
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug]=home&populate[contentSections][populate]=true

Click to see the response

2023-08-17_00-07-07.png

The above response shows all of the contentSections items populated to the first depth.

From what we learned above, we can add to our query to populate nested relations and fields.

But instead, I want to show you how to populate individual dynamic zones, components, and their fields using the on option.

You can see our docs for additional information here.

Let's make a query to populate the following dynamic zone.

2023-08-16_19-14-10 1.png

We will focus on only returning the testimonials-group component from our dynamic zone with the title field and the nested testimonials component with the text and authorName fields.

Let's make the following changes.

{
  filters: {
    slug: "home"
  },
  populate: {
    contentSections: {
      on: {
      "sections.testimonials-group": {
        fields: ["title"],
        populate: {
        testimonials: {
          fields: ["text", "authorName"]
        }
        }
      }
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug]=home&populate[contentSections][on][sections.testimonials-group][fields][0]=title&populate[contentSections][on][sections.testimonials-group][populate][testimonials][fiedls][0]=text&populate[contentSections][on][sections.testimonials-group][populate][testimonials][fiedls][1]=authorName

Click to see the response

As you can see in the response, we only return the testimonials-group component from the dynamic zone with the selected fields and the nested testimonials component with the specified fields.

2023-08-17_00-22-00.png
2023-08-17_00-17-09.png

Now that we know how to populate individual components from our dynamic zones, we will look at how to apply filtering to our query in the next section.

Filtering Data

Filtering allows you to retrieve specific information from your application. However, note that deep filtering is not available in dynamic zones, and complex filters may cause performance issues. You can learn more about filtering here.

Screenshot 2023-08-17 at 12.44.33 AM@0.5x.png

Here are some filtering examples:

Top-level Filtering Example

We already saw this example when we filtered our pages by slug.

{
  filters: {
    slug: "home";
  }
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug]=home

Click to see the response

By default, the above notation uses the $eq operator; we can also write this with the following.

{
  filters: {
    slug: {
      '$eq': "home"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[slug][$eq]=home

Click to see the response

Here are some examples of common operators.

Operator Description
$eq Equal
$ne Not equal
$lt Less than
$gt Greater than
$in Included in an array
$notIn Not included in an array
$contains Contains
$notContains Does not contain
$null Is null
$notNull Is not null
$or Joins the filters in an "or" expression
$and Joins the filters in an "and" expression
$not Joins the filters in an "not" expression

You can see the complete list here

Let's look for a page with the shortName field set to null.

Screenshot 2023-08-17 at 1.10.24 AM.png

We can run the following query.

{
  filters: {
    shortName: {
      "$null": true
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/pages?filters[shortName][$null]=true

Click to see the response

Filtering by Relation Example

You can filter based on relations, like listing all the articles written by a specific author. We will filter our articles based on the name in the authorsBio.

2023-08-17_01-16-45 1.png

{
  populate: {
   authorsBio: {
     fields: ["id", "name", "email"]
   }
  },
  filters: {
    authorsBio: {
      name: {
      "$eq": "Megan"
      },
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?populate[authorsBio][fields][0]=id&populate[authorsBio][fields][1]=name&populate[authorsBio][fields][2]=email&filters[authorsBio][name][$eq]=Megan

Click to see the response

In the response below, we can see all the articles Megan wrote.

Screenshot 2023-08-17 at 1.36.24 AM.png

Complex Filtering

You can use complex filters, like combining multiple conditions using the "and" operator. Let's refactor the above query to see this in action.

In the following query, we would only like to return articles by Megan and a category named Strapi using our $and operator. Notice how it takes an array as an argument.

{
  fields: ["title"],
  populate: {
   authorsBio: {
     fields: ["name"]
   },
   category: {
     fields: ["name"]
   },
  },
  filters: {
    $and:
      [ { authorsBio: {  name: { "$eq": "Megan" } } }, { category: { name: { "$eq": "strapi" } } }],
    }
}
Enter fullscreen mode Exit fullscreen mode

LHS Notation: https://young-spirit-57ca23986d.strapiapp.com/api/articles?fields[0]=title&populate[authorsBio][fields][0]=name&populate[category][fields][0]=name&filters[$and][0][authorsBio][name][$eq]=Megan&filters[$and][1][category][name][$eq]=strapi

Click to see the response

In the following response, you can see that we only return posts that were written by Meagan and belong to Strapi category.

{
  "data": [
    {
      "id": 1,
      "attributes": {
        "title": "The Benefits of a Headless CMS Like Strapi",
        "authorsBio": {
          "data": { "id": 2, "attributes": { "name": "Megan" } }
        },
        "category": { "data": { "id": 1, "attributes": { "name": "strapi" } } }
      }
    },
    {
      "id": 2,
      "attributes": {
        "title": "Unleashing the Power of Customization with Strapi CMS",
        "authorsBio": {
          "data": { "id": 2, "attributes": { "name": "Megan" } }
        },
        "category": { "data": { "id": 1, "attributes": { "name": "strapi" } } }
      }
    }
  ],
  "meta": {
    "pagination": { "page": 1, "pageSize": 25, "pageCount": 1, "total": 2 }
  }
}
Enter fullscreen mode Exit fullscreen mode

Conclusion

To conclude, today, we demystified populate and filtering in Strapi.

We strongly recommend that you are the one who writes the populate and filtering logic in your app and not rely on external plugins such as Populate Deep.

Yes, it is an easy solution, but it comes at the cost of performance since it will populate all the data to the depth specified.

From what we covered above, Strapi's populate and filtering gives you granular control of what data you want to return from your api.

This allows you to only get the data that you need. You can also add the populate and filtering inside a route middleware, allowing you to write less code on your front end.

Here is an excellent article written by Kellen Bolger that you should check out here.

Hope you enjoyed this article. The API we have been using to test our queries is hosted on Strapi Cloud. You can learn more about it here.

As always, thank you for stopping by, and remember to join the Strapi community on Discord to discuss and learn more.

We have daily open office hours Monday - Friday from 12:30 PM CST to 1:30 PM CST.

Thanks for reading!

Image
15 Shocking Facts About Bifold Door Glass Replacement Cost That You Never Known
Image
Uneven multi-task focus system pertaining to men's prostate bed segmentation inside computed tomography pictures.
Image
Discovering Christopher Hildebrant: His Impact on Cincinnati's Community
Image
How to Improve Test Coverage of Your Testing Program

test, automation
Image
Strengthening North Texas Businesses: How ASG IT Consulting Boosts Cybersecurity and IT Efficiency

cybersecurity
Image
How AI-Enhanced Depalletizing is Revolutionizing Logistics?
Image
What is HTTP 413 Error? (Payload Too Large)

http
Image
DOCK2 Promotes Pleural Fibrosis through Modulating Mesothelial to Mesenchymal Changeover.
Image
KB3H8: an environment-friendly reagent for that picky reduction of aldehydes as well as ketones to be able to alcohols.
Image
Singled out keratinising cornael ocular surface area squamous neoplasia together with multifocal repeat.
Image
A state review assessing practitioners' employ and perceived electricity involving well-designed review.
Image
Immunotherapy-induced coeliac ailment inside preventive lung cancer.
Image
Hardship threshold and also posttraumatic tension problem: an organized evaluate and meta-analysis.
Image
Kutzneria chonburiensis sp. late., separated via garden soil.
Image
Mastering IELTS: Unlocking Global Opportunities Through Language Proficiency
Image
Real-Time Genomic Monitoring pertaining to SARS-CoV-2 Variants or worry, Uruguay.
Image
Whatever you always aspired to find out about gastric MALT-lymphoma: a focus upon the latest developments.
Image
Continuous sprinting load- and force-velocity profiling: Review and also probable software.
Image
Within Situ Tuning in the Charge-Carrier Polarity in Imidazole-Linked Single-Molecule Junctions.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .