Katie McLaughlinWhen using the Google Cloud command-line tool (gcloud cli), you often get various forms of output, depending on the command called.
For example, a general command, like listing your current gcloud configuration:
$ gcloud config list
[core]
account = katie@glasnt.com
disable_usage_reporting = True
project = glasnt-unicodex
[run]
region = us-central1
Or something more specific, like which GCP services are currently enabled for your project:
$ gcloud services list --enabled
NAME TITLE
bigquery-json.googleapis.com BigQuery API
cloudapis.googleapis.com Google Cloud APIs
cloudbuild.googleapis.com Cloud Build API
...
But there are ways to alter this output in order to get consistent, more context useful results. In just these two examples, we had two very different formats being presented by default.
If you're after a consistent, specific format, or are only interested in a subsection of the output, using a --format option could be useful for you!
Each API is different, but using these steps will help you introspect each on the fly, and help you build a command that will help you quickly.
For all output here, we'll be using output from the most up-to-date version of gcloud. Our terminal input will be prefaced with $, and output will be as is, except where specifically truncated for clarity, which will be marked with .... For example:
$ gcloud --version
Google Cloud SDK 257.0.0
alpha 2019.05.17
beta 2019.05.17
...
Let's start with that services listing from earlier.
$ gcloud services list
NAME TITLE
bigquery-json.googleapis.com BigQuery API
cloudapis.googleapis.com Google Cloud APIs
cloudbuild.googleapis.com Cloud Build API
...
What is this output, anyway? --help can give us more insight.
$ gcloud services list --help
NAME
gcloud services list - list services for a project
SYNOPSIS
gcloud services list [--available | --enabled] [--filter=EXPRESSION]
[--limit=LIMIT] [--page-size=PAGE_SIZE] [--sort-by=[FIELD,...]]
[GCLOUD_WIDE_FLAG ...]
DESCRIPTION
This command lists the services that are enabled or available to be enabled
by a project. You can choose the mode in which the command will list
services by using exactly one of the --enabled or --available flags.
--enabled is the default.
FLAGS
At most one of these may be specified:
--available
Return the services available to the project to enable. This list
will include any services that the project has already enabled.
--enabled
(DEFAULT) Return the services which the project has enabled.
LIST COMMAND FLAGS
--filter=EXPRESSION
Apply a Boolean filter EXPRESSION to each resource item to be listed.
If the expression evaluates True, then that item is listed. For more
details and examples of filter expressions, run $ gcloud topic filters.
This flag interacts with other flags that are applied in this order:
--flatten, --sort-by, --filter, --limit.
--limit=LIMIT
Maximum number of resources to list. The default is unlimited. This
flag interacts with other flags that are applied in this order:
--flatten, --sort-by, --filter, --limit.
--page-size=PAGE_SIZE
Some services group resource list output into pages. This flag
specifies the maximum number of resources per page. The default is
determined by the service if it supports paging, otherwise it is
unlimited (no paging). Paging may be applied before or after --filter
and --limit depending on the service.
--sort-by=[FIELD,...]
Comma-separated list of resource field key names to sort by. The
default order is ascending. Prefix a field with ``~'' for descending
order on that field. This flag interacts with other flags that are
applied in this order: --flatten, --sort-by, --filter, --limit.
GCLOUD WIDE FLAGS
These flags are available to all commands: --account, --billing-project,
--configuration, --flags-file, --flatten, --format, --help,
--impersonate-service-account, --log-http, --project, --quiet,
--trace-token, --user-output-enabled, --verbosity. Run $ gcloud help for
details.
EXAMPLES
To list the services the current project has enabled for consumption, run:
$ gcloud services list --enabled
To list the services the current project can enable for consumption, run:
$ gcloud services list --available
NOTES
These variants are also available:
$ gcloud alpha services list
$ gcloud beta services list
Yes, this is a long output. But let's step through it.
The FLAGS section here is super useful as it shows us that the DEFAULT for the gcloud services list command is to show us only enabled output.
If we ask it for all available services, we get a very different list.
$ gcloud services list --available
NAME TITLE
abusiveexperiencereport.googleapis.com Abusive Experience Report API
acceleratedmobilepageurl.googleapis.com Accelerated Mobile Pages (AMP) URL API
...
This list is really long. How long? We can check by piping the command through utility that will count the number of lines.
$ gcloud services list --available | wc -l
244
What would be useful here is a list of services showing what is enabled and what is not enabled. At the moment we can only get either: all enabled services, or all available services. It would be useful to see all available services, and if they are enabled or disabled.
We can see from earlier that we have a series of GCLOUD WIDE FLAGS, and the one we're going to focus on today is --format.
$ gcloud help
...
--format=FORMAT
Set the format for printing command output resources. The default is a
command-specific human-friendly output format. The supported formats
are: config, csv, default, diff, disable, flattened, get, json, list,
multi, none, object, table, text, value, yaml. For more details run $
gcloud topic formats.
...
Depending on which data format is most readable to you, you can use a format that gives you a deep look at the data structure for the command in question, which we can then introspect to get the values we want to focus on (in our case, service name, and it's state (status? state? Let's find out!))
$ gcloud services list --available --format=json
[
{
"config": {
"authentication": {},
"documentation": {
"summary": "A data platform for customers to create, manage, share and query data."
},
"name": "bigquery-json.googleapis.com",
"quota": {},
"title": "BigQuery API",
"usage": {
"requirements": [
"serviceusage.googleapis.com/tos/cloud"
]
}
},
"name": "projects/1009201337834/services/bigquery-json.googleapis.com",
"state": "ENABLED"
},
{
"config": {
"authentication": {},
"documentation": {
"summary": "This is a meta service for Google Cloud APIs for convenience. Enabling this service enables all commonly used Google Cloud APIs for the project. By default, it is enabled for all projects created through Google Cloud Console and Google Cloud SDK, and should be manually enabled for all other projects that intend to use Google Cloud APIs. Note: disabling this service has no effect on other services.\n"
},
"name": "cloudapis.googleapis.com",
"quota": {},
"title": "Google Cloud APIs",
"usage": {
"requirements": [
"serviceusage.googleapis.com/tos/cloud"
]
}
},
"name": "projects/1009201337834/services/cloudapis.googleapis.com",
"state": "ENABLED"
},
...
We can see that we have an array of objects (a list of dictionaries, a number of rich objects; your descriptive terms may vary). We can see, though, the fields that will be useful for us: title, and state.
One of the formatting options we have is "value", a tab-separated list of values. This is good as a short-form for debugging our query, as it's easily read in the terminal.
It's also useful to read up the Formats section of Formats (as suggested earlier in the --format tag description).
$ gcloud topic formats
...
A format expression has 3 parts:
NAME
name
ATTRIBUTES
[ [no-]attribute-name[=value] [, ... ] ]
PROJECTION
( resource-key [, ...] )
NAME is required, ATTRIBUTES are optional, and PROJECTIONS may be required
for some formats. Unknown attribute names are silently ignored.
...
The value format type has an attribute "separator", which we need to specify in the following format:
--format='value[separator=","](thing,anotherthing...)'
Important note here: the parameter we pass to --format is a complex string, so it must be wrapped in quotation marks. If we don't do this, our terminal will get very confused with the parentheses and brackets going on. And since we have to declare the comma as the separator, we need to use another set of quotation marks. We could also escape our quotation marks, but this version looks cleaner.
So if we want title and state, we need to specify the fields by their full names:
- config.title, and
- state
$ gcloud services list --available --format="value[separator=' → '](config.title,state)"
Abusive Experience Report API → DISABLED
Accelerated Mobile Pages (AMP) URL API → DISABLED
...
BigQuery API → ENABLED
...
In this last example, I used an unusual separator just to emphasise the breadth of what you could use.
The output format you want is greatly going to depend on what is going to be consuming the output.
A shell script? Maybe it should be --format=csv with limited values.
An internal README? --format=table can be quite cool.
$ gcloud services list --available \
--format="table[box,margin=3,title='✨ Enabled Services ✨'](config.title,state)"
┌───────────────────────────────────────────────────────────────┐
│ ✨ Enabled Services ✨ │
├────────────────────────────────────────────────────┬──────────┤
│ TITLE │ STATE │
├────────────────────────────────────────────────────┼──────────┤
│ Abusive Experience Report API │ DISABLED │
│ Accelerated Mobile Pages (AMP) URL API │ DISABLED │
...
│ BigQuery API │ ENABLED │
...
│ Zync Render API │ DISABLED │
└────────────────────────────────────────────────────┴──────────┘
The gcloud topic formats help also gives a number of EXAMPLES of useful calls. For example:
List the project authenticated user email address:
$ gcloud info --format="value(config.account)"
List the URIs for all compute instances:
$ gcloud compute instances list --format="value(uri())"
Hopefully using these discovery tips can help you introspect your system with relative and just-in-time-discovery ease!
You can read more about using gcloud --format on the GCP Blog.
