Week 5 test guide
A test failed and you don't know why? Find it here: what it checks, and the plausible bugs that are known to fail it. Then read the matching part of the week 5 spec.
How to use this
Run python tools/test.py --compiler PATH --week 5 -v, copy the failing test's name, and search below. "Bugs known to fail it" lists mistakes that were planted in the reference compiler and caught by this test, so one of them is a good first guess. It's not a complete list, and the debugging guide covers what to do when none fit.
| test | kind | what it checks | bugs known to fail it |
|---|---|---|---|
render/gemm_movement_3x4x5 | render | the spec's GEMM: reshape, permute, expand, mul, reduce pop=1 | render expand reads i render flat_of order render unflat stride |
render/pad_2d | render | pad a 2x3 with a ring of zeros: 4x5 | render flat_of order render pad mask off by one render unflat stride |
render/r_column_normalize | render | divide each column by its sum: reduce, reshape, expand, mul | render flat_of order render unflat stride |
render/r_conv1d_same | render | same-size conv1d with a 5-tap kernel | render expand reads i render pad mask off by one render shrink offset sign render stack chain off by one |
render/r_conv2d_16x16 | render | 3x3 same-size conv2d on 16x16 | render expand reads i render pad mask off by one render shrink offset sign render stack chain off by one |
render/r_elementwise_rank3 | render | elementwise mul and max on rank 3 | none recorded |
render/r_expand_scalar | render | broadcast a scalar to a matrix | none recorded |
render/r_expand_size1 | render | broadcast a column | none recorded |
render/r_matmul_1x8x5 | render | a row vector times a matrix | none recorded |
render/r_matmul_2x3x4 | render | matmul via movement ops | none recorded |
render/r_matmul_64x64 | render | 64x64 by 64x64: three nested loops in C, well under 20 KB | none recorded |
render/r_matmul_i32_wrap | render | i32 matmul with wraparound | none recorded |
render/r_pad_32x32 | render | pad a 32x32 image by one: 34x34 | none recorded |
render/r_pad_3d | render | pad a rank-3 value | render pad shift wrong side |
render/r_pad_bool | render | bool pad fills false | render pad shift wrong side |
render/r_pad_left_right | render | pad both sides of a vector | render pad shift wrong side |
render/r_pad_then_shrink | render | pad then an off-center window: some zeros, some data | render shrink offset sign |
render/r_permute_rank3 | render | a rank-3 permute | render permute inverse |
render/r_permute_rank4 | render | a rank-4 permute | render permute inverse |
render/r_reduce_pop1 | render | column sums of a matrix | none recorded |
render/r_reduce_pop2_64x64 | render | sum of a 64x64 matrix in row-major order | render reduce column-major |
render/r_reduce_pop2_rank3 | render | reduce the two leading axes of a rank-3 value | none recorded |
render/r_reduce_rowmajor_order | render | row-major fold order is visible in f32 bits | render reduce column-major |
render/r_reshape_permute_2d | render | transpose via permute, stored flat | none recorded |
render/r_reshape_permute_4096 | render | 4096 elements reshaped to 16x16x16 and reversed axis order | none recorded |
render/r_reshape_same_rank_then_transpose | render | reshape [2 3] to [3 2] is not a no-op: transposing afterwards shows the new row boundaries | render reshape same rank skipped |
render/r_shrink_symbolic | render | a run-time but provable offset: mod (max o 0) 4 with o = 13 is 1 | none recorded |
render/r_shrink_window | render | a value window | none recorded |
render/r_stack_index | render | index a stack of vectors | render index pops the end |
render/r_stack_matrices | render | stack of matrices rendered as a where chain | render stack chain off by one |
render/r_stage_rank2 | render | stage a transposed matrix, then use it twice | none recorded |
render/r_transpose_64x64 | render | 64x64 transpose: loops, not 4096 unrolled stores | none recorded |
run/assert_rank2_fails | run: assert_fail | assert compares every element of a rank-2 value | none recorded |
run/broadcast_column_sums_back | run: pass | reduce over rows, reshape to [1 4], expand back, add: the normalization shape | expand reads i not 0 reduce pops trailing axes reshape column-major |
run/conv1d_same | run: pass | same-size conv1d: pad, a stack of shifted shrinks, multiply by the broadcast kernel, reduce over taps | expand reads i not 0 pad fill not zero reduce pops trailing axes shrink offset sign |
run/conv2d_same | run: pass | 3x3 same-size conv2d on a 4x5 image via pad and nine shrinks | expand reads i not 0 pad fill not zero reduce pops trailing axes reshape column-major shrink offset sign |
run/elementwise_rank3 | run: pass | add, mul, cmplt, where and cast work elementwise on rank-3 values | none recorded |
run/err_expand_non1_axis | run: invalid | a size-2 axis cannot grow to 4 | expand grows non-1 axes |
run/err_expand_rank_mismatch | run: invalid | a non-scalar expand keeps the rank | none recorded |
run/err_index_runtime_oob | run: invalid | a computed index past the end is a run-time error | none recorded |
run/err_index_too_many | run: invalid | at most one index per axis | index count unchecked |
run/err_pad_length | run: invalid | pads has two numbers per axis | pads length unchecked |
run/err_pad_negative | run: invalid | pads are >= 0 (cut with shrink) | none recorded |
run/err_param_rank2 | run: invalid | params stay rank 0 or 1 | none recorded |
run/err_permute_duplicate | run: invalid | order must be a permutation | permute any order |
run/err_permute_out_of_range | run: invalid | order must name each axis once | permute any order |
run/err_permute_wrong_length | run: invalid | order has one entry per axis | none recorded |
run/err_reduce_pop_too_big | run: invalid | pop can drop at most every axis | none recorded |
run/err_reshape_count | run: invalid | reshape must keep the element count | reshape count unchecked |
run/err_reshape_zero_dim | run: invalid | shape sizes are positive | none recorded |
run/err_shrink_offset_count | run: invalid | one offset per axis | shrink offset count unchecked |
run/err_shrink_past_end | run: invalid | offset 3 plus 2 elements passes the end of 4 | shrink bounds unchecked |
run/err_shrink_unprovable | run: invalid | a raw loaded offset could be anything | shrink bounds unchecked |
run/err_stack_mixed_shape | run: invalid | stack operands share one shape | stack shapes unchecked |
run/expand_rank3_middle | run: pass | a middle size-1 axis broadcasts | reshape column-major |
run/expand_scalar_rank2 | run: pass | a scalar expands to any shape | none recorded |
run/expand_size1_axes | run: pass | only size-1 axes grow, and they repeat the single element | none recorded |
run/expand_then_reduce | run: pass | broadcast then reduce: every column gets the same column sum | none recorded |
run/gemm_movement_value | run: pass | C[2][4] computed independently with per-op f32 rounding | none recorded |
run/matmul_movement_1x5x3 | run: pass | (1x5) @ (5x3) with reshape, permute, expand, mul, reduce pop=1; sums over k in order | none recorded |
run/matmul_movement_2x3x2 | run: pass | (2x3) @ (3x2) with reshape, permute, expand, mul, reduce pop=1; sums over k in order | none recorded |
run/matmul_movement_3x3x1 | run: pass | (3x3) @ (3x1) with reshape, permute, expand, mul, reduce pop=1; sums over k in order | none recorded |
run/matmul_movement_4x1x2 | run: pass | (4x1) @ (1x2) with reshape, permute, expand, mul, reduce pop=1; sums over k in order | none recorded |
run/matmul_syllabus_form | run: pass | the syllabus GEMM: A stored transposed as K x M, reshape [K M 1], no permute | none recorded |
run/pad_2d_mixed | run: pass | pad rows on top and columns on the right, f32 fill 0.0 | pad fill not zero pad sides swapped |
run/pad_3d | run: pass | pads apply per axis, in axis order | pad sides swapped |
run/pad_bool_false | run: pass | bool pads with false | pad sides swapped |
run/pad_left_only | run: pass | pads=[2 0] adds two zeros on the left | none recorded |
run/pad_right_only | run: pass | pads=[0 3] adds three zeros on the right | none recorded |
run/pad_shrink_roundtrip_3d | run: pass | shrinking a pad at its left pads recovers the value | shrink offset sign |
run/pad_shrink_value | run: pass | pad then shrink back recovers the original, and an off-center window sees the zeros | none recorded |
run/pad_zero | run: pass | pads of zero change nothing | none recorded |
run/permute_identity | run: pass | the identity order changes nothing | none recorded |
run/permute_inverse_roundtrip | run: pass | order [1 2 0] then its inverse [2 0 1] gives x back | permute inverse |
run/permute_rank3_all_orders | run: pass | all six axis orders of a [2 3 4] value | permute inverse |
run/permute_rank4 | run: pass | shape[k] = x.shape[order[k]]: [2 3 1 2] with order [3 1 0 2] is [2 3 2 1] | permute inverse |
run/permute_then_reshape | run: pass | reshape after a transpose reads the transposed (row-major) order | none recorded |
run/permute_transpose | run: pass | permute [2 3] with order [1 0] is the transpose | none recorded |
run/reduce_last_axis_via_permute | run: pass | to reduce the last axis, permute it to the front (row sums of x) | none recorded |
run/reduce_max_nan_mul_wrap | run: pass | max over rows propagates NaN per column, and i32 mul wraps (65536 * 65536 = 0) | none recorded |
run/reduce_pop1_pop2 | run: pass | pop=1 sums over rows (one total per column), and pop=2 sums everything | none recorded |
run/reduce_rank3_pops | run: pass | pop=1, 2, 3 on a rank-3 value, and pop defaults to 1 | none recorded |
run/reduce_rowmajor_order | run: pass | pop=2 folds x00, x01, x10, x11 in row-major order: 1e8+1 rounds to 1e8, so the total is 1, not 2 | reduce folds column-major |
run/reshape_from_scalar | run: pass | a scalar reshapes to [1 1] and back down | none recorded |
run/reshape_rank_down | run: pass | reshape a rank-3 value flat keeps row-major order | none recorded |
run/reshape_rank_up | run: pass | reshape [12] to [3 4] is row-major: element (2, 1) is 9 | none recorded |
run/reshape_same_rank | run: pass | reshape [2 3] to [3 2] is not a transpose: same flat order | none recorded |
run/reshape_to_scalar | run: pass | reshape [1] to shape=[] gives a scalar | none recorded |
run/shrink_constant | run: pass | a [2 3] window starting at (1, 2) | none recorded |
run/shrink_of_shrink | run: pass | offsets of nested shrinks add up | none recorded |
run/shrink_symbolic_provable | run: pass | mod (max o 0) 2 is in [0, 1], so a 3-wide window of 4 is provably in range (o = 5 gives 1) | none recorded |
run/stack_matrices_runtime_index | run: pass | stack of matrices is rank 3; a run-time index picks one row | none recorded |
run/stack_vectors_and_index | run: pass | stack of vectors is [3 4]; index pops leading axes | none recorded |
run/stage_rank2 | run: pass | stage is the identity on a rank-2 value | none recorded |