whats this lesson about — Sharp AI

Skip to content

whats this lesson about

Lesson 10.2 in the Telemark curriculum is about creating structured, objective test plans and gathering reliability data to ensure robot mechanisms are truly ready for competition. According to the Telemark overview, casual testing often leads to competition failures, so teams must convert vague assumptions into actionable numbers by writing a five-part test plan before testing begins . As detailed in the section on what a test plan contains, this plan consists of a question, a success criterion, a testing method, a list of what to record, and the testing conditions .

To get honest reliability data, Telemark's guide on trial counts recommends running at least 25 trials to prove a mechanism is competition-ready or to confirm that a modification is an actual improvement rather than random noise . The lesson also advises having someone other than the builder, such as the driver, operate the mechanism to prevent unconscious bias . Additionally, the section on realistic conditions explains that tests must be run under match-like circumstances, including using field tiles, realistic battery voltages, and the robot's full match weight .

Rather than just tracking total numbers, Telemark teaches teams to record each failure individually along with its specific conditions to help diagnose underlying patterns and root causes . Finally, the lesson notes that any physical change or repair to a mechanism invalidates its previous reliability data, meaning teams must repeat their tests to verify that the mechanism still works as intended .

Was this right?

Sources16

1

Fill-in-the-Blank Practice

Telemark·Team owned

The success criterion must be written __________ the test is run, so the standard is not adjusted to fit the result.
Comparing whether a change improved a mechanism requires enough trials that the difference exceeds the __________.
Failures should be recorded __________ with their conditions, rather than only as a total count.

Show answers

before
noise (random variation)
individually

---

2

Overview

Telemark·Team owned

Lesson 10.2: Writing Test Plans and Collecting Reliability Data

---

"We tested it and it works" is the sentence that precedes most competition failures. It usually means the mechanism was operated a few times under favorable conditions by the person who built it.

A test plan converts that into a number you can act on.

---

3

Exercise

Telemark·Team owned

Write a full test plan for one mechanism using the five part format, then run it with 25 trials and someone other than the builder operating. Record each failure individually with conditions. Then look for a pattern in the failures.

4

What a Test Plan Contains

Telemark·Team owned

Five parts, written before the test rather than after:

QUESTION
Does the intake acquire a game element placed up to 2 in off
center, reliably enough for competition?

SUCCESS CRITERION
At least 23 of 25 attempts acquired within 2.0 s.

METHOD
Robot stationary on field tile. Element placed at 0, 1, and 2
in off center, alternating left and right. Driver commands
intake at full power. One attempt per placement, 25 total.

WHAT WE RECORD
Success or failure per attempt, time from contact to secured,
offset and side for each attempt, and a note on any failure.

CONDITIONS
Battery above 12.5 V at start. New compliant wheels. Field
tile swept before starting.

Writing the success criterion before the test is what keeps the result honest. Deciding afterward what counts as good enough is how teams talk themselves into shipping a mechanism that is not ready.

---

5

Record Failures Individually

Telemark·Team owned

A total is much less useful than a list.

Fail 4:  element 2 in left, rode up funnel wall, not acquired
Fail 11: element 2 in left, same behavior
Fail 19: element 1 in left, jammed at handoff

Three failures out of 25 is a number. That list is a diagnosis: two failures share a cause at 2 in left, which points at an asymmetry, and the third is a different problem entirely at the handoff.

Also record the conditions for each failure: battery voltage, which element, where on the tile, who was operating. Patterns hide in those columns.

---

6

Trial Counts and What They Buy You

Telemark·Team owned

More trials narrow the uncertainty. A rough guide for FTC decisions:

| Purpose | Trials | What it tells you | | --- | --- | --- | | Does it work at all | 3 to 5 | Basic feasibility | | Is it worth keeping | 10 | Rough reliability, obvious failure modes | | Is it competition ready | 25 or more | A reliability figure you can plan around | | Is a change an improvement | 25 before and after | Whether the difference is real |

That last row matters. A mechanism that goes from 8 out of 10 to 9 out of 10 has not demonstrably improved: with ten trials, that difference is easily chance. Comparing changes requires enough trials that the difference is larger than the noise.

The person who built the mechanism unconsciously operates it in the way that works: the right approach speed, the right angle, a small nudge. Have someone else run the trials, ideally the driver, and preferably someone who did not watch it being built.

---

7

Test in Realistic Conditions

Telemark·Team owned

Bench conditions flatter every mechanism. Test where it will be used:

On field tile, not a table
With the robot at match weight, including the battery and all mechanisms
With a battery at realistic charge, not only a fresh one
With the mechanism at the temperature it reaches after several matches
With the driver operating from the driver station position, not standing over the robot

A mechanism that passes only at 13 V with a cold motor and a builder operating it has not been tested for competition.

---

8

Repeat Tests After Changes

Telemark·Team owned

A test result belongs to the version of the mechanism that was tested. Any change, including a repair that put things back the way they were, invalidates the number.

This is not bureaucratic. Teams repeatedly discover that a mechanism reassembled after a repair behaves differently, because a spacer went back in a different order or a tension was set differently. Re-running an abbreviated version of the test after any significant change catches that in ten minutes.

---

9

Trial Counts and Honest Numbers

Telemark·Team owned

One trial is a demonstration. Ten trials is a measurement. The difference matters because mechanisms fail intermittently, and an intermittent failure is invisible in a single trial.

A practical minimum for FTC work:

| Question | Trials | | --- | --- | | Does this basically work at all | 3 | | Is this good enough to keep | 10 | | Is this reliable enough to compete with | 25 or more |

Record the failures individually, not as a total. "3 failures out of 12" tells you less than "failures on trials 4, 5, and 11, all with the element off center to the left," which points at an asymmetry you can go find.

When a trial fails because someone bumped the robot, write it down as a bumped trial rather than deleting it. Teams that discard inconvenient data end up with a notebook that shows better reliability than the robot has, which is exactly the surprise you do not want at a tournament.

---

10

Fill-in-the-Blank Practice

Telemark·Team owned

The four parts of a standard entry are Problem, What We Tried, Result, and __________.
Recording that a mechanism "worked better" is insufficient because it is an opinion rather than __________.
Determining whether a mechanism is reliable enough to compete with requires at least __________ trials.

Show answers

Next (the decision or next step)
data (a number with a trial count)
25 (or more)

---

11

Why Each Part Matters

Telemark·Team owned

Problem forces you to state what you were actually trying to fix. Entries that skip this read as activity logs: "worked on the intake today."

What we tried must include enough detail to reproduce it. Materials, dimensions, angles, and the test conditions. "Added walls" is not reproducible; "1/8 in polycarbonate at 20 degrees" is.

Result is where teams weaken most. It needs numbers and trial counts. "Worked better" is an opinion. "7 out of 12 at 2 inches off center" is data, and it is what makes the next entry meaningful when the number changes to 11 out of 12.

Next is what turns a log into a process. It shows the loop from Module 0 running.

Keep the notebook open on the workbench. Fill in Problem and What We Tried before you start testing, while you still know what you are doing and why. Fill in Result as the trials happen. Only Next gets written at the end, and by then it writes itself.

---

12

Exercise

Telemark·Team owned

Run one test on any current mechanism, using the four part format, with at least 10 trials. Record each failure individually with the conditions under which it happened. Then look at the failure list and see whether there is a pattern. There usually is.

13

Record It

Telemark·Team owned

A failure analysis entry is among the most valuable pages in the notebook. It should contain the symptom, the evidence including a photograph, the why chain, the identified root cause, the fix, and the verification that the fix worked.

That last part matters: after the fix, re-run the test from Lesson 10.2 and record the new number. A fix that was never verified is a hypothesis.

---

14

Four Things Every Requirement Needs

Telemark·Team owned

A statement alone is not enough. A requirement that the team can actually work from carries four things, and a row missing any one of them will cause an argument later.

| Field | Why it exists | | --- | --- | | An id | So a notebook entry, a design review, and a test result can all cite the same thing. "R2 failed" is precise; "the drive requirement" is not. | | A number and a unit | Without one, nothing can pass or fail. | | A verification method | How you will know. Test means run it repeatedly and count. Measure means one instrument reading. Inspect means check against a list. Demonstrate means show it once, end to end. | | An owner | A subsystem that answers for it. A requirement everyone owns is one everyone assumes someone else has. |

Plus a source, so a rule can be told from a preference.

Once R2 exists, a mechanism can be described as "satisfies R2", a test result can be recorded against R2, and a design review can ask to see R2 rather than asking whether the drivetrain feels fast enough. That thread from a decision to the evidence for it is what separates a team that can explain its robot from a team that built one.

---

15

Overview

Telemark·Team owned

Lesson 0.3: Writing Requirements and Separating Them From Constraints

---

"The intake should be fast" is not a requirement. It cannot be tested, it cannot be designed against, and two people reading it will picture different mechanisms. A requirement that cannot fail a test is not a requirement, it is a wish.

This lesson covers how to turn strategy into statements that a mechanism either satisfies or does not.

---

16

Fill-in-the-Blank Practice

Telemark·Team owned

A notebook entry describing a test should contain numbers rather than __________.
Judges consider a notebook that records only successes to be less credible because it lacks documented __________.
Every photograph in the notebook needs a __________ explaining what it shows and why it is there.

Show answers

adjectives (subjective descriptions such as "fast" or "reliable")
failures (or iteration, problems encountered)
caption

---