Week 1 test guide
A test failed and you don't know why? Find it here: what it checks, and the plausible bugs that are known to fail it. Then read the matching part of the week 1 spec.
How to use this
Run python tools/test.py --compiler PATH --week 1 -v, copy the failing test's name, and search below. "Bugs known to fail it" lists mistakes that were planted in the reference compiler and caught by this test, so one of them is a good first guess. It's not a complete list, and the debugging guide covers what to do when none fit.
| test | kind | what it checks | bugs known to fail it |
|---|---|---|---|
run/basic_add | run: pass | the syllabus example | none recorded |
run/basic_chain | run: pass | names refer to earlier statements | none recorded |
run/call_basic | run: pass | the spec's first call example | evaluates everything |
run/call_closed_body | run: pass | substitution must not go inside a nested call's body: inner's param 0 is NOT outer's param 0 | evaluates everything nested body sees outer env |
run/call_composition | run: pass | f(f(x)) built from calls | evaluates everything global memo across calls nested body sees outer env |
run/call_extra_args | run: pass | extra args are ignored | none recorded |
run/call_f32_bool | run: pass | params of other dtypes | global memo across calls where branches swapped |
run/call_memo_per_call | run: pass | the same body called with different args gives different results (memoize per call, not globally) | global memo across calls |
run/call_no_params | run: pass | a body with no params is just a value | none recorded |
run/call_rank1_param | run: pass | a param can be rank 1 | none recorded |
run/call_runtime_error_inside | run: invalid | a runtime error inside a call body is still a runtime error | div by zero returns 0 |
run/call_shared_param_node | run: pass | two functions that use the same param node (hash consed) still mean different things | none recorded |
run/call_spec_inner_outer | run: pass | the spec's closed-body example | none recorded |
run/call_two_params | run: pass | arg i binds param i | none recorded |
run/cast_basic | run: pass | cast rules from the spec | bool->num always 1 f32->i32 rounds i32->f32 not rounded |
run/cast_i32_edges | run: pass | the largest f32 below 2^31 still fits in i32 | i32->f32 not rounded |
run/cmplt_basic | run: pass | cmplt is strict less-than and returns bool | cmplt is <= |
run/cmplt_nan_zero | run: pass | any NaN compare is false; -0.0 is not less than 0.0 | cmplt is <= |
run/comments_whitespace | run: pass | comments, blank lines, and extra spaces | none recorded |
run/const_dtype | run: pass | const with an explicit dtype | none recorded |
run/dag_sharing_300 | run: pass | 300 levels of doubling: a tree walk is 2^300 steps, a memoized DAG walk is 300 | add doesn't wrap no memoization |
run/dag_sharing_mix | run: pass | every level uses the previous one twice, so it needs memoization | no memoization |
run/empty_program | run: pass | a program with no asserts trivially passes | none recorded |
run/err_alu_rank1 | run: invalid | ALU ops only take scalars | implicit broadcast allowed |
run/err_arity | run: invalid | add takes exactly two operands | none recorded |
run/err_assert_shape | run: invalid | assert needs matching shapes | none recorded |
run/err_attr_before_operand | run: invalid | operands come before attributes | attributes before operands |
run/err_attr_duplicate | run: invalid | an attribute can be given once | duplicate attributes allowed |
run/err_attr_unknown | run: invalid | add takes no attributes. Unknown attributes are errors, not ignored | unknown attributes ignored |
run/err_call_missing_arg | run: invalid | every param in the body needs an arg | missing args not checked |
run/err_call_missing_arg_unused | run: invalid | arg checks are static: this call is never evaluated, but it's still a type error | missing args not checked |
run/err_call_shape_mismatch | run: invalid | arg shape must match the param | arg types not checked |
run/err_call_type_mismatch | run: invalid | arg dtype must match the param | arg types not checked |
run/err_call_type_unused | run: invalid | arg dtype checks are static too | arg types not checked |
run/err_call_void_body | run: invalid | a void body can't be called | void operands allowed |
run/err_cast_big_i32 | run: invalid | 2^31 doesn't fit in i32 | f32->i32 no range check |
run/err_cast_inf_i32 | run: invalid | inf can't become an i32 | none recorded |
run/err_cast_nan_i32 | run: invalid | NaN can't become an i32 | none recorded |
run/err_div_zero | run: invalid | div by zero is a runtime error | div by zero returns 0 |
run/err_index_f32 | run: invalid | the index must be i32 | none recorded |
run/err_index_neg | run: invalid | negative index | index allows negative |
run/err_index_oob | run: invalid | index past the end | none recorded |
run/err_int_too_big | run: invalid | integer literals must fit in i32 | int literals unbounded |
run/err_literal_hex | run: invalid | no hex literals | none recorded |
run/err_literal_infinity | run: invalid | the word is inf, not infinity | loose float literals |
run/err_literal_plus | run: invalid | a literal can't start with + | none recorded |
run/err_literal_underscore | run: invalid | no digit separators | none recorded |
run/err_mod_zero | run: invalid | mod by zero is a runtime error | div by zero returns 0 |
run/err_param_outside_call | run: invalid | a param evaluated outside any call is a runtime error | param outside call is 0 |
run/err_param_two_types | run: invalid | one slot can't be two types inside one body | none recorded |
run/err_param_two_types_b | run: invalid | same as err_param_two_types with the uses in the other order: check every use, not just the first or last | param slot types unchecked |
run/err_param_two_types_uncalled | run: invalid | one slot used with two types is a static error even if the body is never called | param slot types unchecked slot types checked on call only |
run/err_parse | run: invalid | a line that isn't a statement | none recorded |
run/err_redefine | run: invalid | a name can be defined once | redefinition allowed |
run/err_shape_rank2 | run: invalid | week 1 shapes are [k] with one k >= 1 | none recorded |
run/err_shape_zero | run: invalid | a shape of [0] isn't allowed in week 1 | none recorded |
run/err_shl_32 | run: invalid | shift amounts outside [0, 31] are runtime errors | shl allows 32 |
run/err_shl_neg | run: invalid | shift amounts outside [0, 31] are runtime errors | none recorded |
run/err_stack_mixed | run: invalid | stack operands must share a dtype | stack dtypes unchecked |
run/err_stack_mixed_shapes | run: invalid | stack operands must share one shape (stacking vectors is fine from week 5; mixing shapes never is) | none recorded |
run/err_type_assert | run: invalid | assert needs matching dtypes | none recorded |
run/err_type_bool_add | run: invalid | add is not defined on bool | none recorded |
run/err_type_cast_void | run: invalid | you can't cast to void | none recorded |
run/err_type_cmplt_bool | run: invalid | cmplt is defined on numbers only | none recorded |
run/err_type_div_f32 | run: invalid | div is i32 only | none recorded |
run/err_type_mix | run: invalid | no implicit conversion between i32 and f32 | implicit dtype mixing |
run/err_type_recip_i32 | run: invalid | recip is f32 only | none recorded |
run/err_type_static | run: invalid | type errors are static: this bad line is never evaluated, but the program is still invalid | implicit dtype mixing |
run/err_type_void_operand | run: invalid | an assert's value (void) can't be an operand | void operands allowed |
run/err_type_where_branches | run: invalid | where branches must have the same dtype | none recorded |
run/err_type_where_cond | run: invalid | where needs a bool condition | none recorded |
run/err_type_xor_f32 | run: invalid | xor is not defined on f32 | none recorded |
run/err_undefined | run: invalid | names must be defined before use | none recorded |
run/err_unknown_op | run: invalid | unknown op | none recorded |
run/err_use_before_def | run: invalid | names must be defined above their first use | none recorded |
run/err_where_strict | run: invalid | where is strict: an error in the untaken branch is still an error | none recorded |
run/error_before_fail | run: invalid | an error in an earlier assert wins over a later failure | asserts in reverse order |
run/f32_assert_equality | run: pass | assert compares f32 by value, and NaN counts as equal to NaN | -0.0 != 0.0 in assert NaN != NaN in assert |
run/f32_literals | run: pass | float literal forms | f32 consts not rounded |
run/f32_nan_sources | run: pass | inf - inf and inf * 0 are NaN | NaN != NaN in assert |
run/f32_overflow | run: pass | f32 overflow goes to inf | f32 ops not rounded |
run/f32_product_rounds | run: pass | every op rounds to f32, so 3 * (1/3) is exactly 1 but 1/3 is not 0.333333333 | cmplt is <= f32 consts not rounded f32 ops not rounded |
run/f32_recip | run: pass | recip, including signed zero and inf | recip 0 ignores sign |
run/f32_rounding_step | run: pass | f32 has 24 bits of mantissa: 2^24 + 1 rounds back to 2^24 | f32 ops not rounded |
run/f32_signed_zero_visible | run: pass | -0.0 and 0.0 compare equal but recip tells them apart | recip 0 ignores sign |
run/f32_tenths | run: pass | in f32, 0.1 + 0.2 rounds to exactly 0.3 (unlike f64) | f32 consts not rounded |
run/fail_basic | run: assert_fail | a failing assert exits 1 and names itself | none recorded |
run/fail_before_error | run: assert_fail | asserts run in file order, so this fails before the later div by zero is reached | asserts in reverse order |
run/fail_first_in_order | run: assert_fail | report the first failing assert in file order | asserts in reverse order |
run/fail_then_static_error | run: invalid | type checking happens before any assert runs, so a later type error beats an earlier failing assert | implicit dtype mixing |
run/hash_consing_same_node | run: pass | identical statements are one node. Nothing observable changes, but the program must still work | none recorded |
run/i32_div_intmin | run: pass | INT_MIN / -1 wraps to INT_MIN, and INT_MIN mod -1 is 0 | none recorded |
run/i32_div_trunc | run: pass | div truncates toward zero, like C (not floor like Python) | div floors like Python |
run/i32_mod_sign | run: pass | mod takes the sign of the dividend, like C | div floors like Python mod uses Python % |
run/i32_shl | run: pass | shl wraps | shl doesn't wrap |
run/i32_wrap_add | run: pass | i32 add wraps around (two's complement) | add doesn't wrap |
run/i32_wrap_mul | run: pass | i32 mul keeps the low 32 bits | mul doesn't wrap |
run/lazy_unused_runtime_error | run: pass | runtime errors only matter if an assert depends on them | none recorded |
run/literal_float_forms | run: pass | 1., .5, -.5, 1E5 and 1e+5 are all float literals | none recorded |
run/literal_float_out_of_range | run: pass | a float literal too big for f32 rounds to inf, and one too small rounds to 0 | none recorded |
run/max_basic | run: pass | max on i32 and f32 | none recorded |
run/max_nan | run: pass | max propagates NaN from either side | NaN != NaN in assert max ignores NaN |
run/max_signed_zero | run: pass | max(-0.0, 0.0) is +0.0 in either order, and recip tells the zeros apart | max(-0.0, 0.0) order-dependent |
run/stack_assert | run: pass | assert compares rank-1 values elementwise | none recorded |
run/stack_assert_fail | run: assert_fail | one element differs | none recorded |
run/stack_f32_bool | run: pass | stack works for any dtype | -0.0 != 0.0 in assert |
run/stack_index | run: pass | stack builds a rank-1 value, index reads it | none recorded |
run/where_basic | run: pass | where picks a branch | where branches swapped |
run/xor_i32_bool | run: pass | xor is bitwise on i32 and logical on bool | xor bool is equality |
simplify/add_negzero_f32 | simplify, max_ops=0 | x + -0.0 IS x on f32, for every x including -0.0 and nan | none recorded |
simplify/add_zero | simplify, max_ops=0 | x + 0 -> x on i32 | none recorded |
simplify/bounds_cmplt | simplify, max_ops=0 | max(x, 0) mod 8 is in [0, 7], so it is always < 8 | none recorded |
simplify/bounds_cmplt_false | simplify, max_ops=0 | max(x, 0) is never < 0 | none recorded |
simplify/bounds_div_elim | simplify, max_ops=0 | a value in [0, 7] div 8 is 0 | none recorded |
simplify/bounds_max | simplify, max_ops=0 | max(b, 5) where b is in [0, 1] is 5 | none recorded |
simplify/bounds_mod_elim | simplify, max_ops=1 | a value in [0, 1] mod 4 is itself | none recorded |
simplify/call_inline | simplify, max_ops=0 | inline the call, then simplify through it | no call inlining |
simplify/cast_same | simplify, max_ops=0 | a cast to the same dtype is a no-op | none recorded |
simplify/chain_of_identities | simplify, max_ops=0 | a pile of no-ops | none recorded |
simplify/cmplt_self | simplify, max_ops=0 | x < x -> false (also true for f32: nan < nan is false) | none recorded |
simplify/const_fold | simplify, max_ops=1 | fold the constant part | none recorded |
simplify/div_mul_bounded | simplify, max_ops=1 | (b * 2) / 2 -> b when bounds prove b * 2 can't wrap | div elim off by one no hash consing |
simplify/index_stack | simplify, max_ops=0 | index of a stack with a const index picks the element | none recorded |
simplify/mul_one_f32 | simplify, max_ops=0 | x * 1.0 -> x is exact on f32, even for nan and -0.0 | none recorded |
simplify/mul_zero_i32 | simplify, max_ops=0 | x * 0 -> 0 on i32 | none recorded |
simplify/reassoc_i32 | simplify, max_ops=1 | (1 + x) + 2 -> x + 3: i32 is a ring mod 2^32, so reassociation is safe even with wraparound | no const-right canonicalize |
simplify/shared_subexpr | simplify, max_ops=2 | hash consing: (x*x) + (x*x) is two ops, not three | no hash consing |
simplify/trap_add_zero_f32 | simplify, max_ops=1 | TRAP: x + 0.0 is NOT x on f32: -0.0 + 0.0 is +0.0, and the grader compares bit patterns | i32 rules applied to f32 x + 0.0 -> x on f32 |
simplify/trap_bounds_edge | simplify, max_ops=3 | TRAP: a value in [0, 7] is not always < 7 | cmplt bounds off by one |
simplify/trap_bounds_wrap | simplify, max_ops=3 | TRAP: max(x, 0) + 1 wraps to INT_MIN when x = INT_MAX, so it CAN be < 0 | bounds clamp instead of wrap |
simplify/trap_div_elim_edge | simplify, max_ops=3 | TRAP: a value in [0, 7] div 7 is not always 0 | div elim off by one |
simplify/trap_div_mul_wrap | simplify, max_ops=2 | TRAP: (x * 2) / 2 is NOT x on i32 (2^30 * 2 wraps) | (x*c)/c ignores wrap |
simplify/trap_f32_reassoc | simplify, max_ops=2 | TRAP: (x + 1e8) - 1e8 is NOT x on f32 (1.0 + 1e8 rounds to 1e8) | i32 rules applied to f32 |
simplify/trap_minus_one_less | simplify, max_ops=2 | TRAP: x - 1 < x is false at INT_MIN (it wraps to INT_MAX) | none recorded |
simplify/trap_mod_negative | simplify, max_ops=3 | TRAP: x mod 8 can be negative (sign follows x), so this can't fold | mod bounds ignore sign |
simplify/trap_mul_zero_f32 | simplify, max_ops=1 | TRAP: x * 0.0 is NOT 0.0 on f32 (nan * 0 = nan, inf * 0 = nan) | i32 rules applied to f32 x*0 -> 0 on f32 |
simplify/trap_nested_call | simplify, max_ops=1 | TRAP: inline correctly. The param 0 of inner is bound by outer's call, not by out's args | no call inlining top-down inline enters bodies |
simplify/trap_xor_two_bools | simplify, max_ops=1 | TRAP: xor of two different params can't fold | none recorded |
simplify/where_const_cond | simplify, max_ops=0 | the condition folds, then the where does | none recorded |
simplify/where_same | simplify, max_ops=0 | where c x x -> x | none recorded |
simplify/xor_self | simplify, max_ops=0 | x ^ x -> 0 | none recorded |
simplify/zero_add | simplify, max_ops=0 | 0 + x -> x: put consts on the right first | no const-right canonicalize |