Package org.apache.beam.sdk.schemas.transforms

Class Filter

java.lang.Object

org.apache.beam.sdk.schemas.transforms.Filter

@Experimental(SCHEMAS)
public class Filter
extends java.lang.Object

A PTransform for filtering a collection of schema types.

Separate Predicates can be registered for different schema fields, and the result is allowed to pass if all predicates return true. The output type is the same as the input type.

For example, consider the following schema type:

 public class Location {
   public double latitude;
   public double longitude;
 }
 

In order to examine only locations in south Manhattan, you would write:

 PCollection<Location> locations = readLocations();
 locations.apply(Filter
    .whereFieldName("latitude", lat -> lat < 40.720 && lat > 40.699)
    .whereFieldName("longitude", long -> long < -73.969 && long > -74.747));
 

Predicates that require examining multiple fields at once are also supported. For example, consider the following class representing a user account:

 class UserAccount {
   public double spendOnBooks;
   public double spendOnMovies;
       ...
 }
 

Say you want to examine only users whos total spend is above $100. You could write:

 PCollection<UserAccount> users = readUsers();
 users.apply(Filter
    .whereFieldNames(Lists.newArrayList("spendOnBooks", "spendOnMovies"),
        row -> return row.getDouble("spendOnBooks") + row.getDouble("spendOnMovies") > 100.00));
 

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	Filter.Inner<T>	Implementation of the filter.

Constructor Summary

Constructors

Constructor	Description
Filter()

Method Summary

Modifier and Type	Method	Description
static <T> Filter.Inner<T>	create()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Filter

public Filter()

Method Detail

create

public static <T> Filter.Inner<T> create()