# Metadata Queries - Google App Engine - Google Code

English ▼
Skip to page content Skip to main navigation
e.g. "templates" or "datastore"

# Metadata Queries

Metadata queries return information about the nature of the data your application stores in the Google App Engine Datastore. Such metadata is commonly used to support metaprogramming, implement backend administrative functions, and similar purposes; you can use it, for instance, to build a custom administration console for your application. The metadata available includes information on the namespaces, entity kinds, and properties your application uses, as well as the property representations associated with each property.

You can also find metadata about your application in the Datastore Statistics tab of the App Engine Administration Console, but the data displayed there differs in some important respects from that returned by metadata queries:

- Freshness. Metadata queries return data that is current as of the time of the query. Data in the Datastore Statistics tab is updated only once daily.
- Comprehensiveness. The data in the Datastore Statistics tab is more extensive than what is available via metadata queries.
- Speed. Metadata queries are billed in the same way as ordinary queries, but are generally slower to execute. As a rule of thumb, expect a metadata query that returns N entities to take about the same time as N ordinary queries each returning a single entity. Furthermore, property representation queries (non-keys-only property queries) are slower than keys-only property queries.

This page describes the following aspects of metadata queries:

- Metadata Queries in Java
- Namespace Queries
- Kind Queries
- Property Queries

- Property Queries: Keys-Only
- Property Queries: Non-Keys-Only (Property Representation)

## Metadata Queries in Java

The Java class `Query`, defined in the `com.google.appengine.api.datastore` package, provides three special entity kinds that are reserved for metadata queries. They are denoted by static constants of the `Query` class:

Static constant | Entity kind
--- | ---
`Query.NAMESPACE_METADATA_KIND` | `__namespace__`
`Query.KIND_METADATA_KIND` | `__kind__`
`Query.PROPERTY_METADATA_KIND` | `__property__`

These kinds will not conflict with others of the same names that may already exist in your application. By querying on these special kinds, you can retrieve entities containing the desired metadata.

The entities returned by metadata queries are generated dynamically, based on the current state of the Datastore. While you can create local `Entity` objects of kinds `__namespace__`, `__kind__`, or `__property__`, any attempt to store them in the Datastore will fail with an `IllegalArgumentException`.

The easiest way to issue metadata queries is with the low-level Datastore API. The following example prints the names of all namespaces in an application:

    import com.google.appengine.api.datastore.DatastoreService;
    import com.google.appengine.api.datastore.Entity;
    import com.google.appengine.api.datastore.Query;

    void printAllNamespaces(DatastoreService ds, PrintWriter writer) {

      Query q = new Query(Query.NAMESPACE_METADATA_KIND);

      for (Entity e : ds.prepare(q).asIterable()) {
        // A nonzero numeric id denotes the default namespace;
        // see Namespace Queries, below
        if (e.getKey().getId() != 0) {
          writer.println("<default>");
        } else {
          writer.println(e.getKey().getName());
        }
      }

## Namespace Queries

If your application uses the Namespaces API, you can use a namespace query to find all namespaces used in the application's entities. This allows you to perform activities such as administrative functions across multiple namespaces.

Namespace queries return entities of the special kind `__namespace__` whose key name is the name of a namespace. (An exception is the default namespace designated by the empty string `""`: since the empty string is not a valid key name, this namespace is keyed with the numeric ID `1` instead.) Queries of this type support filtering only for ranges over the special pseudoproperty `__key__`, whose value is the entity's key. The results can be sorted by ascending (but not descending) `__key__` value. Because `__namespace__` entities have no properties, both keys-only and non-keys-only queries return the same information.

The following example returns a list of an application's namespaces in the range between two specified names, `start` and `end`:

    import com.google.appengine.api.datastore.DatastoreService;
    import com.google.appengine.api.datastore.Entity;
    import com.google.appengine.api.datastore.Key;
    import com.google.appengine.api.datastore.KeyFactory;
    import com.google.appengine.api.datastore.Query;

    import java.util.ArrayList;
    import java.util.List;

    // Helper function to make key from namespace name
    Key makeNamespaceKey(String namespace) {
      return KeyFactory.createKey(Query.NAMESPACE_METADATA_KIND, namespace);
    }

    List<String> getNamespaces(DatastoreService ds, String start, String end) {

      // Start with unrestricted namespace query
      Query q = new Query(Query.NAMESPACE_METADATA_KIND);

      // Limit to specified range, if any
      if (start != null) {
        q.addFilter(Entity.KEY_RESERVED_PROPERTY,
                    Query.FilterOperator.GREATER_THAN_OR_EQUAL,
                    makeNamespaceKey(start));
      }
      if (end != null) {
        q.addFilter(Entity.KEY_RESERVED_PROPERTY,
                    Query.FilterOperator.LESS_THAN_OR_EQUAL,
                    makeNamespaceKey(end));
      }

      // Initialize result list
      List<String> results = new ArrayList<String>();

      // Build list of query results
      for (Entity e : ds.prepare(q).asIterable()) {
        if (e.getKey().getId() != 0) {
          results.add("");      // Nonzero numeric ID denotes the default namespace
        } else {
          results.add(e.getKey().getName());
        }
      }

      // Return result list
      return results;
    }

## Kind Queries

Kind queries return entities of kind `__kind__` whose key name is the name of an entity kind. Queries of this type are implicitly restricted to the current namespace and support filtering only for ranges over the `__key__` pseudoproperty. The results can be sorted by ascending (but not descending) `__key__` value. Because `__kind__` entities have no properties, both keys-only and non-keys-only queries return the same information.

The following example prints all kinds whose names start with a lowercase letter:

    import com.google.appengine.api.datastore.DatastoreService;
    import com.google.appengine.api.datastore.Entity;
    import com.google.appengine.api.datastore.Key;
    import com.google.appengine.api.datastore.KeyFactory;
    import com.google.appengine.api.datastore.Query;

    import java.io.PrintWriter;

    // Helper function to make key from kind name
    Key makeKindKey(String kind) {
      return KeyFactory.createKey(Query.KIND_METADATA_KIND, kind);
    }

    void printLowercaseKinds(DatastoreService ds, PrintWriter writer) {

      // Start with unrestricted kind query
      Query q = new Query(Query.KIND_METADATA_KIND);

      // Limit to lowercase initial letters
      q.addFilter(Entity.KEY_RESERVED_PROPERTY,
                  Query.FilterOperator.GREATER_THAN_OR_EQUAL,
                  makeKindKey("a"));
      String endChar = Character.toString('z' + 1);           // Character after 'z'
      q.addFilter(Entity.KEY_RESERVED_PROPERTY,
                  Query.FilterOperator.LESS_THAN,
                  makeKindKey(endChar));

      // Print heading
      writer.println("Lowercase kinds:");

      // Print query results
      for (Entity e : ds.prepare(q).asIterable()) {
        writer.println("  " + e.getKey().getName());
      }
    }

## Property Queries

Property queries return entities of kind `__property__` denoting the properties associated with an entity kind. The entity representing property P of kind K is built as follows:

- The entity's key has kind `__property__` and key name P.
- The parent entity's key has kind `__kind__` and key name K.

The behavior of a property query depends on whether it is a keys-only or a non-keys-only (property representation) query, as detailed in the subsections below.

### Property Queries: Keys-Only

Keys-only property queries return a key for each indexed property of a specified entity kind. (Unindexed properties are not included.) The following example prints the names of all of an application's entity kinds and the properties associated with each:

    import com.google.appengine.api.datastore.DatastoreService;
    import com.google.appengine.api.datastore.Entity;
    import com.google.appengine.api.datastore.Key;
    import com.google.appengine.api.datastore.KeyFactory;
    import com.google.appengine.api.datastore.Query;

    import java.io.PrintWriter;

    void printProperties(DatastoreService ds, PrintWriter writer) {

      // Create unrestricted keys-only property query
      q = new Query(Query.PROPERTY_METADATA_KIND).setKeysOnly();

      // Print query results
      for (Entity e : ds.prepare(q).asIterable()) {
        writer.println(e.getKey().getParent().getName()
                         + ": "
                         + e.getKey().getName());
      }
    }

Queries of this type are implicitly restricted to the current namespace and support filtering only for ranges over the pseudoproperty `__key__`, where the keys denote either `__kind__` or `__property__` entities. The results can be sorted by ascending (but not descending) `__key__` value. Filtering is applied to kind-property pairs, ordered first by kind and second by property: for instance, if you have

- kind `Account` with properties
  - `balance`
  - `company`
- kind `Employee` with properties
  - `name`
  - `ssn`
- kind `Invoice` with properties
  - `date`
  - `amount`
- kind `Manager` with properties
  - `name`
  - `title`
- kind `Product` with properties
  - `description`
  - `price`

then the query

    import com.google.appengine.api.datastore.DatastoreService;
    import com.google.appengine.api.datastore.Entity;
    import com.google.appengine.api.datastore.Key;
    import com.google.appengine.api.datastore.KeyFactory;
    import com.google.appengine.api.datastore.Query;

    import java.io.PrintWriter;

    // Helper function to make key from kind name
    Key makeKindKey(String kind) {
      return KeyFactory.createKey(Query.KIND_METADATA_KIND, kind);
    }

    // Helper function to make property key from kind and property names
    Key makePropertyKey(String kind, String property) {
      Key kindKey = makeKindKey(kind);
      return KeyFactory.createKey(kindKey, Query.PROPERTY_METADATA_KIND, property);
    }

    void printPropertyRange(DatastoreService ds, PrintWriter writer) {

      // Start with unrestricted keys-only property query
      q = new Query(Query.PROPERTY_METADATA_KIND).setKeysOnly();

      // Limit range
      q.addFilter(Entity.KEY_RESERVED_PROPERTY,
                  Query.FilterOperator.GREATER_THAN_OR_EQUAL,
                  makePropertyKey("Employee", "salary"));
      q.addFilter(Entity.KEY_RESERVED_PROPERTY,
                  Query.FilterOperator.LESS_THAN_OR_EQUAL,
                  makePropertyKey("Manager", "salary"));

      // Print query results
      for (Entity e : ds.prepare(q).asIterable()) {
        writer.println(e.getKey().getParent().getName()
                         + ": "
                         + e.getKey().getName());
      }
    }

will print the following:

    Employee: ssn
    Invoice: date
    Invoice: amount
    Manager: name

Notice that the results do not include the `name` property of kind `Employee` and the `title` property of kind `Manager`, nor any properties of kinds `Account` and `Product`, because they fall outside the range specified for the query.

Note: Since the `__key__` kind filters on kind-property pairs, you can't use a single query for the same property name in different kinds. You can query for all properties in all kinds (or in a range of kinds), but you cannot construct a single query that retrieves, for example, all properties with names between m and q in all kinds (or in kinds A and B).

Property queries also support ancestor filtering on a `__kind__` or `__property__` key, to limit the query results to a single kind or property. You can use this, for instance, to get the properties associated with a given entity kind, as in the following example:

    import com.google.appengine.api.datastore.DatastoreService;
    import com.google.appengine.api.datastore.Entity;
    import com.google.appengine.api.datastore.Query;

    import java.util.ArrayList;
    import java.util.List;

    // Helper function to make key from kind name
    Key makeKindKey(String kind) {
      return KeyFactory.createKey(Query.KIND_METADATA_KIND, kind);
    }

    List<String> propertiesOfKind(DatastoreService ds, String kind) {

      // Start with unrestricted keys-only property query
      Query q = new Query(Query.PROPERTY_METADATA_KIND).setKeysOnly();

      // Limit to specified kind
      q.setAncestor(makeKindKey(kind));

      // Initialize result list
      ArrayList<String> results = new ArrayList<String>();

      //Build list of query results
      for (Entity e : ds.prepare(q).asIterable()) {
        results.add(e.getKey().getName());
      }

      // Return result list
      return results;
    }

### Property Queries: Non-Keys-Only (Property Representation)

Non-keys-only property queries, known as property representation queries, return additional information on the representations used by each kind-property pair. The entity returned for property P of kind K has the same key as for a corresponding keys-only query, along with an additional `property_representation` property returning the property's representations. The value of this property is an instance of class `java.util.Collection<String>` containing one string for each representation of property P found in any entity of kind K.

Note that representations are not the same as property classes; multiple property classes can map to the same representation. (For example, `java.lang.String` and `com.google.appengine.api.datastore.Blob` both use the `STRING` representation.) The following table maps from property classes to their representations:

Property class | Representation
--- | ---
`java.lang.Byte` | `INT64`
`java.lang.Short` | `INT64`
`java.lang.Integer` | `INT64`
`java.lang.Long` | `INT64`
`java.lang.Float` | `DOUBLE`
`java.lang.Double` | `DOUBLE`
`java.lang.Boolean` | `BOOLEAN`
`java.lang.String` | `STRING`
`com.google.appengine.api.datastore.Text` | `STRING`
`com.google.appengine.api.datastore.ShortBlob` | `STRING`
`com.google.appengine.api.datastore.Blob` | `STRING`
`java.util.Date` | `INT64`
`com.google.appengine.api.datastore.GeoPt` | `POINT`
`com.google.appengine.api.datastore.PostalAddress` | `STRING`
`com.google.appengine.api.datastore.PhoneNumber` | `STRING`
`com.google.appengine.api.datastore.Email` | `STRING`
`com.google.appengine.api.users.User` | `USER`
`com.google.appengine.api.datastore.IMHandle` | `STRING`
`com.google.appengine.api.datastore.Link` | `STRING`
`com.google.appengine.api.datastore.Category` | `STRING`
`com.google.appengine.api.datastore.Rating` | `INT64`
`com.google.appengine.api.datastore.Key` | `REFERENCE`
`com.google.appengine.api.blobstore.BlobKey` | `STRING`
`java.util.Collection<T>` | Representation of `T`

The following example finds all representations of a specified property for a given entity kind:

    import com.google.appengine.api.datastore.DatastoreService;
    import com.google.appengine.api.datastore.Entity;
    import com.google.appengine.api.datastore.Query;

    import java.util.Collection;

    // Helper function to make key from kind name
    Key makeKindKey(String kind) {
      return KeyFactory.createKey(Query.KIND_METADATA_KIND, kind);
    }

    // Helper function to make property key from kind and property names
    Key makePropertyKey(String kind, String property) {
      Key kindKey = makeKindKey(kind);
      return KeyFactory.createKey(kindKey, Query.PROPERTY_METADATA_KIND, property);
    }

    Collection<String> representationsOfProperty(DatastoreService ds,
                                                 String kind,
                                                 String property) {

      // Start with unrestricted non-keys-only property query
      Query q = new Query(Query.PROPERTY_METADATA_KIND);

      // Limit to specified kind and property
      q.setAncestor(makePropertyKey(kind, property));

      // Get query result
      Entity propInfo = ds.prepare(q).asSingleEntity();

      // Return collection of property representations
      return (Collection<String>) propInfo.getProperty("property_representation");
    }