Key Concepts Explained

Stereo Vision and Disparity

  • Stereo Vision: Uses two cameras to capture slightly different views, mimicking human vision.

  • Disparity: The difference in horizontal position of a corresponding point in left/right images. Inversely proportional to depth.

  • Application: Calculating a disparity map allows depth estimation, crucial for 3D understanding. The CREStereo class is designed for this.

Object Detection (YOLO)

  • YOLO (You Only Look Once): Fast and accurate real-time object detection models.

  • Principle: Divides an image into a grid, predicts bounding boxes, confidence scores, and class probabilities in a single pass.

  • Application: Used in src/perception/object_detection.py to identify road users and objects.

Lane Detection

  • Purpose: Identifying lane markings on the road.

  • Importance: Essential for features like Lane Keeping Assist and Lane Departure Warning.

  • Method: The UltrafastLaneDetector (src/perception/lane_detection.py) uses a deep learning model.

Object Tracking (SORT-like)

  • Purpose: Maintain a consistent identity for detected objects across frames.

  • SORT (Simple Online and Realtime Tracking): * Relies on an external object detector. * Matches new detections to existing tracks using motion prediction, an affinity metric (e.g., IoU), and an assignment algorithm (e.g., Hungarian algorithm).

  • Application: src/tracker.py assigns stable IDs to detected objects.

NLP for Q&A (Intent Recognition)

  • NLP (Natural Language Processing): Enabling computers to understand human language.

  • Intent Recognition: Determining the underlying purpose of a user’s query.

  • Process in NLP/adas_chatbot.py: 1. Data: Predefined intents, patterns, and responses (moroccan_traffic_code.json). 2. Preprocessing: Tokenization, lemmatization. 3. Bag-of-Words (BoW): Input converted to a numerical vector. 4. Neural Network Classifier: Trained to map BoW vectors to intents. 5. Response Generation: Selects a response for the predicted intent.

CARLA Simulator

  • CARLA: Open-source simulator for autonomous driving research.

  • Features: Realistic 3D environments, vehicle physics, sensor models, API.

  • Role in this Project: Allows development and testing in a controlled virtual environment.