Core Modules

Graphical User Interface (GUI) - gui/

The GUI is built using PyQt6 and provides a user-friendly way to interact with the ADAS.

gui/main_window.py: * The main application window that houses all other UI pages using a QStackedWidget. * Manages navigation between different views (mode selection, configuration, ADAS view, logs, Q&A). * Connects signals from the ADAS backend (e.g., new frames, status updates) to GUI elements.
gui/mode_selection.py: * The initial page allowing the user to choose between “Real Driving Mode,” “Carla Simulation Mode,” or the “Q&A (Traffic Code Chatbot).”
gui/real_config.py & gui/carla_config.py: * Configuration pages for the selected mode. * Real Mode: Configure left/right camera sources and camera intrinsic/extrinsic parameters (focal length, principal point, baseline). Saves to config/real_config.yaml. * CARLA Mode: Configure CARLA host, port, and parameters for two virtual cameras (image size, FOV, location). Saves to config/carla_config.yaml.
gui/adas_view.py: * Displays the processed video feed from the ADAS system. * Shows visual alerts (e.g., “Collision Warning,” “Lane Departure”) overlaid on the video. * Handles keyboard input for controlling the ego vehicle in CARLA mode (W, A, S, D).
gui/logs_view.py: * Displays status messages and logs generated by the ADAS backend.
gui/qa_page.py: * Provides an interface for the Moroccan Traffic Code Q&A chatbot. Users can type questions and receive answers.
gui/theme.py: * Defines the dark theme stylesheet (DARK_THEME) used across the application for a consistent look and feel.

The ADAS class in main.py is the heart of the backend system.

``ADAS`` Class: * Manages the overall state of the ADAS (running, mode, configuration). * Initializes the FrameProcessor for perception tasks. * Handles communication with the GUI through PyQt signals (update_frame, status_update, play_alert). * Controls the main processing loop.
Processing Loop (``process_loop``): * Runs in a separate thread to prevent freezing the GUI. * Continuously reads frames from the selected video source. * Passes frames to the FrameProcessor. * Emits processed frames and alerts to the GUI. * If in CARLA mode, applies control commands (throttle, brake, steer) to the ego vehicle.
Video Source Initialization: * init_video_source(): Calls mode-specific initialization. * init_real_mode(): Initializes cv2.VideoCapture for left and right physical cameras based on paths/IDs from real_config.yaml. * init_carla_mode():
- Connects to the CARLA server using CarlaClient.
- Spawns an ego vehicle.
- Spawns additional vehicles for traffic simulation.
- Sets up stereo cameras on the ego vehicle based on carla_config.yaml.
- Listens to camera sensors to receive image data.
Frame Reading (``read_frame``, ``read_real_frame``, ``read_carla_frame``): * Abstracts frame acquisition based on the current mode.
CARLA Integration: * Manages the CarlaClient instance. * Spawns and destroys CARLA actors (vehicle, sensors). * Applies carla.VehicleControl based on user input or internal logic.

The FrameProcessor class in src/process_fame.py orchestrates the different perception modules.

``FrameProcessor`` Class: * Loads configuration from config/config.yaml which specifies model paths and parameters for perception modules. * Initializes:
- ObjectDetector
- UltrafastLaneDetector
- ObjectTracker
- (Commented out) CREStereo for disparity estimation.
- (Commented out) TrafficSignDetector.
- ``process_frame(left_img, right_img)`` Method: * Takes left and right input images. * Performs object detection on the left image. * Performs lane detection. * Estimates disparity CREStereo. * Updates the ObjectTracker with new detections. * Attaches 3D coordinates to tracks if disparity is available. * Generates a visualization of the detections/tracks on the output image. * Returns the processed image and any alerts.
Helper Functions: * detections_from_ultralytics(): Converts YOLO detection results into a format suitable for the ObjectTracker. * visualize_tracks(): Draws bounding boxes, track IDs, and (if available) 3D distances on an image. * attach_xyz_to_tracks(): Calculates and attaches 3D world coordinates (X, Y, Z) to tracks using a disparity map and the camera’s Q matrix.

These modules perform specific visual perception tasks.

src/perception/object_detection.py: * ObjectDetector class. * Uses a YOLO model for detecting objects. * detect(): Takes an image and returns detection results. * visualize(): Draws detections on an image.
src/perception/lane_detection.py: * UltrafastLaneDetector class. * Implements a lightweight lane detection model. * detect_lanes(): Gets lane points. * draw_lanes(): Visualizes detected lanes.
src/perception/disparity_estimation.py: * CREStereo class. * Designed for estimating disparity maps from stereo camera pairs. * estimate_disp(): Returns a disparity map.

Provides functionality to track objects over multiple frames. See src/tracker.py.

``Track`` Class: Represents a single tracked object.
``ObjectTracker`` Class: Implements a simple SORT-like algorithm. * update(detections): Associates new detections with existing tracks .

Facilitates interaction with the CARLA simulator.

carla_module/carla_client.py: * CarlaCameraConfig class: Holds configuration for CARLA cameras. * CarlaClient class: Manages connection and spawning of cameras.

Implements the Q&A functionality. See NLP/adas_chatbot.py.

``NLTKDownloader``, ``DataPreprocessor``, ``NeuralNet``, ``ModelTrainer``, ``Chatbot`` classes.
Uses NLP/data/moroccan_traffic_code.json for intent data.
Training is automatic if model files (NLP/models/processed_data_pytorch.pkl, NLP/models/qa_nn_model.pth) are missing.

Intended for stereo camera calibration. See src/calibration.py.

YAML files for managing configurations.