Week 4 · Lesson 1

Vectors

Until now every ALU op took scalars, and to touch many elements you wrote a loop by hand. This week values get a shape again: an ALU op can take two vectors and combine them element by element. Your compiler's job is to turn that back into loops, a pass called rangeify (lesson 4.3): it asks each vector for "your element at index i" and builds a loop around the answer.

Rank-1 values

A value of shape (n,) is a vector of n elements, all of one dtype. You've had them since week 1 (stack builds one) and week 2 (load of a whole buffer or a [k] view gives one). What's new is that ALU ops accept them:

%x = buffer 4 dtype=f32 slot=0 data=[1.0 2.0 3.0 4.0]
%y = buffer 4 dtype=f32 slot=1 data=[10.0 20.0 30.0 40.0]
%vx = load %x                # shape (4,)
%vy = load %y                # shape (4,)
%z  = add %vx %vy            # shape (4,): [11.0 22.0 33.0 44.0]

The rule is elementwise: element i of the result is the op applied to element i of each operand. Every ALU op works this way, including cmplt (a vector of bools), where (a vector condition picks per element) and cast. All operands must have the same shape, and the result has that shape too.

Nothing about the arithmetic changes. Each element is computed with week 1's rules: i32 wraps, f32 rounds to nearest-even after the op, max propagates NaN. A vector op is just n scalar ops that happen not to depend on each other, which is exactly why they can run in any order, or all at once.

No implicit broadcasting

NumPy and PyTorch let you write x + 1.0 and quietly stretch the scalar to x's shape. Here that's a type error:

%v = load %x               # shape (4,)
%z = add %v 1.0            # invalid: shapes (4,) and () don't match

To use a scalar with a vector, you expand it first, which lesson 4.2 covers properly:

%one = expand 1.0 shape=[4]   # [1.0 1.0 1.0 1.0]
%z   = add %v %one            # fine: both (4,)

Why make broadcasts explicit

tinygrad does the same thing: a broadcast is its own node in the graph, never a side effect of an add. That keeps every op's shape rule trivial (all inputs, one shape), and it puts the broadcast where the compiler can see it. When rangeify later asks "what is element i of this value", an expand answers "the scalar, whatever i is", and that's the whole reason a broadcast costs nothing in the generated loop.

An affine map, elementwise

Slot 0 holds five f32s x, and slot 1 holds two f32s a and b. Store y[i] = a * x[i] + b into slot 2, multiplying first, then adding. The starter forgets b.

Shapes the type checker enforces

program	result
`add` of two `(4,)` f32 vectors	f32 `(4,)`
`cmplt` of two `(3,)` i32 vectors	bool `(3,)`
`where` with a bool `(3,)` condition and two `(3,)` branches	the branches' dtype, `(3,)`
`add` of `(4,)` and `(3,)`	invalid
`add` of `(4,)` and a scalar	invalid: `expand` the scalar
`index` of an `(n,)` result	a scalar, as in week 1

Vectors don't mix with hand-written loops

A vector may not depend on an open range from a week 3 loop. "The element of a vector at iteration k" is something rangeify will build for you. Mixing the two by hand is an error.

Absolute difference

Slots 0 and 1 hold four i32s each. Store |x[i] - y[i]| into slot 2, with i32 wraparound (there's no subtract op, and no abs op either).

Check yourself

What does %i hold?

%x = stack 1 2 3
%y = stack 4 5 6
%z = mul %x %y
%i = index %z 1

Work out %z element by element, then pick element 1 (counting from 0).

%z is [4 10 18], and index 1 is 10.

%v is an f32 vector of shape (4,). What happens with %z = add %v 1.0?

Does any ALU op stretch a scalar for you?

There's no implicit broadcasting. Write %one = expand 1.0 shape=[4] and add that.

%x and %y are i32 vectors of shape (3,). What is cmplt %x %y?

Elementwise means the result has the operands' shape, and cmplt always returns bool.

Each element compares on its own, so you get three bools, shape (3,).

What does %w hold?

%c = stack true false true
%a = stack 1 2 3
%b = stack 9 9 9
%w = where %c %a %b

The condition picks a branch separately for each element.

Element 0 takes a (1), element 1 takes b (9), element 2 takes a (3): [1 9 3].

What does cast %v dtype=i32 give for %v = stack 1.5 -2.5 0.5?

Week 1's cast rule, applied to each element. Which way does f32 to i32 round?

f32 to i32 truncates toward zero, element by element: 1, -2, 0.

Why does the course (like tinygrad) make broadcasting an explicit expand node?

Think about what each op's type rule looks like with and without it.

With explicit broadcasts, "all inputs have one shape" is the whole rule for every ALU op. And the expand node tells rangeify that the element at any index is the same scalar, so it costs nothing in the loop. Implicit broadcasting compiles fine, but it hides that structure inside every op.