Pipeline operators

A pipeline passes a Bag through a series of steps, read from left to right. These examples use this Def of sales:

def sales = {
  (category: "a", amount: 2)
  (category: "a", amount: 3)
  (category: "b", amount: 4)
  (category: "b", amount: 1)
}

Transform each Value

map runs for each occurrence. The expression after it is called its kernel. Inside it, . is the current Value:

def amounts = sales map .amount

amounts // {1, 2, 3, 4}

You can name that Value instead:

def doubled = sales map sale => sale.amount * 2

doubled // {2, 4, 6, 8}

Or name its parts. (category, amount) names two fields, and [x, y] names the positions of a Tuple:

def labels = sales map (category, amount) => "\{category}: \{amount}"

labels // {"a: 2", "a: 3", "b: 4", "b: 1"}

A kernel can give several Values, and map collects them all. Repeats in the input repeat the results:

{1, 2'2} map {., . + 10} // {1, 2'2, 11, 12'2}

Filter Values

if keeps Values that pass a test:

def larger_sales = sales if .amount > 1

larger_sales
// {
//   (amount: 2, category: "a")
//   (amount: 3, category: "a")
//   (amount: 4, category: "b")
// }

Transform the complete Bag

agg runs once on the whole Bag. Here .category reads every category and % removes repeats:

def categories = sales agg .category%

categories // {"a", "b"}

Choose for each Value

match chooses the first matching arm. Each arm lists Values, then ~> and a result. _ matches anything left:

def sizes = sales map .amount match {
  1 ~> "single"
  {2, 3} ~> "few"
  _ ~> "many"
}

sizes // {"few"'2, "many", "single"}

The arms must cover every possible input. Use then … else for a Boolean choice.

Build directional Maps

to uses the input as the key and the computed result as the value:

def successors = {1, 2} to . + 1

successors // {1 -> 2, 2 -> 3}

by uses the computed result as the key and the input as the value:

def sales_by_category = sales by .category

sales_by_category
// {
//   "a" -> (amount: 2, category: "a")
//   "a" -> (amount: 3, category: "a")
//   "b" -> (amount: 1, category: "b")
//   "b" -> (amount: 4, category: "b")
// }

Repeated keys stay separate. The next page groups them.

Chain steps

Each step takes the previous step’s result. Filtering before mapping keeps only the larger amounts:

sales if .amount > 1 map .amount * 2 // {4, 6, 8}

. belongs to the step it is inside. Here it is a sale in both steps; after map .amount, it would be a number.

Work on Map values

The v forms keep a Map’s key and work on its value. mapv changes each value, ifv filters Maps by their values, and aggv groups equal keys and runs once on each Bag of values. Inside these steps, . is the value, or the Bag of values in aggv. A header key -> value => names the key and the value, and _ leaves either one unnamed.

The next page, Grouping and summarizing, uses all three.

Quick reference

Form	Works on	Result
X map K	Each Value	Results of K
X if P	Each Value	Values that pass P
X agg K	The whole Bag	Result of K
X to K	Each Value	Input -> result
X by K	Each Value	Result -> input
X match {S ~> K}	Each Value	Result of the first matching arm
X mapv K	Each Map’s value	Key -> result
X ifv P	Each Map’s value	Maps whose values pass P
X aggv K	Each key’s value Bag	Key -> result