Frame 07Course projectRobotics perceptionUMD, spring 2024

Find the horizon by vote, then let the halt flags outrank the steering

A single ROS 2 node for the University of Maryland ENPM673 final project. It takes the TurtleBot3 camera feed, estimates the horizon once with a hand-written RANSAC, watches the floor below it for anything moving fast, checks each frame for a stop sign, and only then lets a homography lane follower steer. The README shows the same node driving a simulated waffle in Gazebo and a physical TurtleBot3.

Repo: ROS_2_TurtleBot_perception_project
Stack: ROS 2 Humble, rclpy, OpenCV, Ultralytics YOLO, PyTorch, NumPy, Gazebo, TurtleBot3 Waffle
Scope: Course project, UMD ENPM673, spring 2024. Sole author of the two Python files on the course scaffold: findhorizonline.py and enpm673_final_proj_main.py.
Ran on: The ENPM673 Gazebo world and a physical TurtleBot3, per the two README gifs

The 30-second version

Camera in, Twist out. Horizon by RANSAC, obstacle halt by Lucas-Kanade flow, stop-sign halt by YOLO, steering by a top-down homography and the nearest white lane contour.
The vanishing point is a vote: up to 400 random line pairs, a line is an inlier within 13 px, and the loop exits early once more than 93% of the lines agree (findhorizonline.py).
Reading it back for this page, the halt flag is one shared boolean that a low-confidence YOLO box can clear in the same frame. No test caught it, because there are none.

What could go wrong, and how we would know

A perception loop on a moving robot has three ways to be confidently wrong. The horizon can land on a floor seam instead of the wall line, and every downstream crop is then anchored to the wrong row. The obstacle detector can fire on one bad optical-flow track, and the robot freezes for nothing. The stop-sign detector can miss the sign, and the robot drives through it. I designed around the first two and accepted the third as the model's problem.

Why it holds up

The horizon is not a single Hough line; it is the intersection the most lines agree on. Each candidate is scored by counting lines within 13 px of it, and the search stops early only once more than 93% of the filtered lines agree. Near-vertical and near-horizontal lines are dropped first (theta kept between 0.4 and 1.4 rad or 1.7 and 2.8 rad), so only the converging edges vote. The obstacle halt needs a track moving more than 25 px per frame, counted more than ten times. The stop-sign halt needs a YOLO box above 0.8 confidence. Both halts write a zero Twist before the lane follower may publish anything.

What I built

On the first frame only, findhorizonline.py runs Canny at 60/150, HoughLines with an accumulator threshold of 130, and the angle filter above. RANSAC then samples two lines at a time, solves the 2x2 system for their intersection with np.linalg.solve, and measures every line's perpendicular distance to that point. The best point by inlier ratio becomes the vanishing point. The red line through it is the horizon the rest of the node uses, and it is never computed again.

Every later frame runs three steps in a fixed order. Optical flow: crop the rows below the horizon, pick up to 500 corners, track them with Lucas-Kanade in a 15x15 window (pyramid maxLevel 2), and count any feature that jumped more than 25 px. Past ten such counts the node publishes a zero Twist and starts a 50-frame wait with the halt flag raised. Stop sign: a custom-trained YOLO model (models/sto.pt) runs on the full frame and raises the same flag for any box above 0.8 confidence. Steering: warp the frame to a top-down view with cv2.findHomography, threshold at 253 so only the white lane patches survive, keep contours over 1,000 px of area, and take the centroid nearest the robot.

The control rule is deliberately dumb. If the halt flag is up, publish zero. Otherwise, if the centroid is within 20 px of the image centre, drive straight at 0.06 m/s; if not, creep at 0.003 m/s and turn at 0.05 rad/s toward it. No controller tuning, because the lane in this course world is a string of white patches and the goal was a robot that never drives through a stop sign, not one that corners smoothly.

Chose	Over	Because	Cost
RANSAC over Hough line intersections	Least squares over all intersections	Plank seams and wall edges are outlier lines; a mean drifts toward them	Random sampling, so the point moves a few pixels run to run
Estimate the horizon once	Re-estimate every frame	Stable crop for a camera at a fixed pitch, no per-frame Hough cost	A pitch change on the real robot is never noticed
Halt flags checked before steering	Blending stop and steer commands	A halt is absolute; the lane follower gets no vote when a sign is in view	One shared boolean written by two functions in order, the bug found below
Count fast tracks, halt past ten	Halt on the first fast track	One bad Lucas-Kanade match should not stop the robot	The count never resets until it fires, so it is a noise filter, not a velocity estimate
Homography plus a 253 threshold	Hough lane lines in the raw frame	The lane patches are pure white; a warped binary image gives a clean centroid	Tuned for Gazebo lighting; source points hard-coded for a 640x480 camera

Gazebo camera frame: brick wall above, wooden floor with white lane patches below, a red horizontal line at the estimated horizon and a green dot at the vanishing point. — Horizon and vanishing point on the first frame (Gazebo).

Gazebo frame with a can-shaped obstacle on the left; short optical-flow tracks cluster on the can and corner points mark the edges of a white lane patch. — Lucas-Kanade tracks below the horizon; the fast ones on the can trip the halt.

Gazebo frame: a red stop sign on a post inside a magenta bounding box labelled Stop sign, white lane patches on the floor ahead. — YOLO stop-sign box above 0.8 confidence (Gazebo).

Flow diagram: read image via cv_bridge, detect horizon line, then optical flow and stop-sign detection each set Stop to TRUE; Move Robot runs only when Stop is False. — System flow from the README. Its crop labels are swapped relative to the code: flow runs below the horizon and YOLO sees the full frame.

Try it

The demo re-runs the RANSAC scorer from findhorizonline.py on a synthetic road: lines that converge on a hidden vanishing point, plus outlier lines you can add.

Mirrors the loop in findhorizonline.py: two random lines, a 2x2 solve for their intersection, inliers counted within the threshold (default 13 px), early exit past a 0.93 inlier ratio. The road lines and outliers are synthetic and illustrative; the real node scores Hough lines from a camera frame.

Verification

There is no test suite. What verified this node was running it. The README shows the same code driving the waffle around the ENPM673 Gazebo world, then a physical TurtleBot3 following a curve of white paper sheets on a lab floor. Both gifs are too heavy to embed (20.7 MB and 18 MB), so they are linked: Gazebo run and real-robot run. The stop-sign model has no dataset, training script or metrics in the repo; only the weights are committed. Every pipeline number on this page is a threshold the code uses, not a measured accuracy.

What I found

Finding

The halt flag is one boolean written by two functions and read by a third, in a fixed order. perform_optical_flow sets it, detect_stop_sign overwrites it with the verdict of the last YOLO box it saw (true above 0.8, false otherwise), and control_movement reads it. A box at 0.5 confidence listed after a real stop sign in the same frame, or any box under 0.8 during an obstacle wait, clears the halt. Nothing in the loop reports this. It was found by reading, and a two-box synthetic frame in a unit test would have caught it in seconds.

Outcome, and what it does not prove

The node did what the project asked: it followed the lane and stopped for the sign and for a fast-moving object in simulation, and the same node followed a paper lane on the real robot. It does not prove the perception is robust. The horizon is estimated once, so a camera that pitches on a real floor keeps the crop from frame one. The lane threshold at 253 assumes Gazebo-white patches. The homography source points are hard-coded for a 640x480 camera. And the ten-count obstacle rule can be met by eleven fast features in one frame or by one fast feature across eleven frames, so it says "something moved a lot", not "an object is approaching".

What I would do differently

Re-estimate the horizon on a timer or whenever the inlier ratio drops, and keep the last good one. Give each halt source its own flag and OR them in control_movement. Split perception and cmd_vel into separate nodes, and use the ROS logger instead of print. Most of all, write the three synthetic tests this page found gaps for: two boxes in one frame, ten fast tracks in a single frame, and a first frame with only plank seams.

Honest notes

Course project on the UMD ENPM673 scaffold. The CMake, launch, worlds, models and C++ files are the course template; my work is the two Python files and the trained weights.
The horizon is estimated once on the first frame and never re-estimated.
stop_signal_detected is reset to false by any YOLO box under 0.8 confidence, so a second box can cancel a real detection or an obstacle wait in the same frame.
frame_count is incremented twice per callback, in image_callback and again at the end of perform_optical_flow.
wait_counter starts at 0 and the wait branch is not gated on fast movement, so the first 50 frames with a tracked feature also raise the halt flag; the break in that branch leaves the feature loop after the first feature.
No tests, no CI, no dataset or metrics for the stop-sign weights. The 22.5 MB weights file is committed twice and __pycache__ is in the tree.
Commits: 20 in May 2024 (the code, during the course) and 28 in January 2025 (README and media). The four stills above are Gazebo frames; the physical robot appears only in the linked gif, which I have not re-encoded for the site.

Sources findhorizonline.py enpm673_final_proj_main.py README commit history Scope: I (sole author of the Python; course scaffold otherwise) Verified 3 Sep 2026

Elsewhere

Robotics perceptionPallet detection and ground segmentation RoboticsPlanning, perception, control Production AIVerita AI: audit pipeline and platform