USING AI TO MAKE A SOPHISTICATED DECISION-MAKING SYSTEM THAT CAN ADVISE YOU IN REAL TIME WHETHER TO STOP OR KEEP GOING AT PEDESTRIAN CROSSINGS

ABSTRACT

This project is unusual because it blends AI with traffic safety laws for pedestrian crossings to develop a safety-critical intelligent decision-making framework. The proposed system addresses the decision-level challenge of determining in real time whether a vehicle should halt or proceed, whereas existing systems mostly concentrate on detecting pedestrians or forecasting their trajectories. A major part of this research is the use of Time-to-Collision (TTC) and Post-Encroachment Time (PET) together in an AI-based decision-making system. These safety indicators are used not only for post-analysis but also to make decisions because of a fail-safe monitoring system. This hybrid technique enables the system use both deep learning models, which are adaptable, and rule-based safety constraints, which are dependable. This makes it much less likely that you'll make harmful decisions when things are infrequent or uncertain. The suggested architecture also features a modular and extensible system design that meets rigorous latency constraints and smoothly mixes multi-sensor perception, feature fusion, and real-time inference. The findings of the trial reveal that this AI+ fail-safe method works better than both ordinary TTC-based rules and AI models that work on their own. This paper presents a practical and viable methodology for intelligent transportation systems, amalgamating data-driven intelligence with safety-critical traffic management at pedestrian crossings.

АННОТАЦИЯ

Этот проект необычен тем, что объединяет искусственный интеллект с правилами дорожного движения для пешеходных переходов с целью разработки критически важной интеллектуальной системы принятия решений. Предлагаемая система решает проблему принятия решений на уровне управления, определяя в режиме реального времени, следует ли транспортному средству остановиться или продолжить движение, тогда как существующие системы в основном сосредоточены на обнаружении пешеходов или прогнозировании их траекторий. Значительная часть этого исследования заключается в совместном использовании времени до столкновения (TTC) и времени после выезда на встречную полосу (PET) в системе принятия решений на основе искусственного интеллекта. Эти показатели безопасности используются не только для последующего анализа, но и для принятия решений благодаря отказоустойчивой системе мониторинга. Эта гибридная техника позволяет системе использовать как адаптивные модели глубокого обучения, так и надежные ограничения на основе правил. Это значительно снижает вероятность того, что вы примете вредные решения, когда что-то происходит редко или неопределенно. Предлагаемая архитектура также отличается модульным и расширяемым системным дизайном, который соответствует строгим ограничениям по задержке и плавно сочетает многосенсорное восприятие, слияние признаков и вывод в реальном времени. Результаты испытаний показывают, что этот метод AI+ с отказоустойчивостью работает лучше как обычных правил на основе TTC, так и автономных моделей ИИ. В данной статье представлена практичная и жизнеспособная методология для интеллектуальных транспортных систем, объединяющая интеллектуальный анализ данных с критически важным для безопасности управлением дорожным движением на пешеходных переходах.

Keywords: intelligent transportation systems (ITS); pedestrian crossing safety; safety-critical artificial intelligence; real-time decision-making; computer vision for traffic analysis; deep learning–based classification; autonomous and advanced driver assistance systems (ADAS); Time-to-Collision (TTC); Post-Encroachment Time (PET); a fail-safe decision framework, the synthesis of data from several sensors, and the management of traffic in smart cities1.

Ключевые слова: интеллектуальные транспортные системы (ИТС); безопасность пешеходных переходов; критически важный искусственный интеллект; принятие решений в реальном времени; компьютерное зрение для анализа дорожного движения; классификация на основе глубокого обучения; автономные и передовые системы помощи водителю (ADAS); время до столкновения (TTC); время после выезда на встречную полосу (PET); отказоустойчивая система принятия решений, синтез данных с нескольких датчиков и управление дорожным движением в умных городах. 1.

Introduction

Cities are less safe for driving now because of urbanization and the quick rise in the number of cars. This is especially true at crosswalks, when cars and people walk by each other in a lot of different ways. Statistics from all over the world show that many accidents involving pedestrians happened at or near crosswalks. Most of the time, this is because drivers don't pay enough attention to what's going on around them, don't react quickly enough, or don't think about how other people will act. Old-fashioned means to control traffic, including static traffic lights and road markers, don't easily adjust to changing traffic circumstances or people crossing the street in unexpected ways. Recent improvements in AI, especially in machine learning and computer vision, have made it possible to create intelligent transportation systems (ITS) that can see and understand their surroundings in real time. This makes it easier for people to make decisions automatically. Recent research has significantly advanced the recognition of pedestrians and the prediction of their trajectories; nonetheless, several methodologies remain concentrated on sensory-level tasks and fail to directly tackle the decision-making processes governing vehicular behavior at pedestrian crossings. When safety is very important, you need to be able to see things clearly and think about the situation, analyze the hazards, and stick to precise time limits to decide whether a car should stop or keep going. This paper presents an AI-driven intelligent decision-making system for real-time predictions regarding whether to halt or continue at pedestrian crossings, seeking to address these constraints. The proposed method combines multi-sensor perception, feature extraction utilizing safety-oriented metrics, and decision modeling based on deep learning into a single system. The system is ready for advanced driver assistance systems (ADAS) and self-driving automobiles since it has introduced Time-to-Collision (TTC) and Post-Encroachment Time (PET) to the decision pipeline and built in fail-safe supervisory logic. This will make the roadways safer for everyone, even people who are walking. [1]

Related work

You can look into pedestrian safety and smart traffic control in three main ways: by looking at how pedestrians see things, how cars behave, and how to make traffic management better. Deep feature extractors and convolutional neural networks (CNNs) are two types of computer vision that have done an excellent job at finding people and crosswalks in crowded city environments. Many new driver assistance systems are based on these ideas since they let you observe things as they happen. Several research have attempted to utilize probabilistic frameworks, trajectory forecasting models, and machine learning to predict the behavior of vehicles and people. [3] These plans try to guess where people or cars will go in the future so that accidents are less likely to happen. People often use these methods to make predictions, but they don't always help you figure out what to do with a car when you need to get there quickly. Some people have also argued that traffic lights should perform better and that infrastructure-based control technologies should be used to keep pedestrians safer. But these solutions often rely on fixed rules or central control and don't respond quickly to changes in traffic circumstances. These areas have come a long way, but not much has been done to make it clear in real time whether a car should stop or keep going at pedestrian crossings. AI models that can work on their own could make bad choices when they don't know what to do. On the other hand, rule-based solutions don't operate well in diverse areas. The current study addresses this deficiency by incorporating deep learning-based inference into a safety-critical decision framework that encompasses formal risk assessments and fail-safe supervisory logic, thereby facilitating robust and efficient real-time vehicle decision-making at pedestrian crossings.

Methodology

How the system is built up

Figure 1. The main idea behind the smart AI-based system that will aid individuals cross the street

The system is made up of different sections that work together to help people decide in real time whether to stop or keep going. Some of these parts are multi-sensor perception, feature fusion, artificial intelligence, and fail-safe control logic. These are the most critical parts of the system.

1. The Data Acquisition Module gathers real-time data from a multitude of sensors, like monocular cameras, radar or LiDAR units, and vehicle systems like GPS, IMU, and CAN bus. The information that was collected includes the weather, the speed of cars, the location of people walking, the brakes' status, steering information, and the distance to the pedestrian crossing. Module for Preprocessing and Syncing: The raw sensor data is synced in time and cleaned up by getting rid of noise, normalizing it, and extracting the region of interest (ROI), which is the area around the crossing. This step checks that the data from all the sensors is correct and consistent so that it may be used later.

2. The extraction of features and perception Module employs deep convolutional neural networks to find people and crosswalks in video sources. Then, algorithms for tracking many objects figure out how individuals move over time. We employ important spatial-temporal elements to show how people and cars interact. A supervised deep learning classifier considers vehicle speed, distance, Time-to-Collision (TTC), and Post-Encroachment Time (PET), and then puts them all into one state vector. Then it uses this vector to decide whether to stop or keep going. Safety Supervisor and Execution Module: When important TTC or PET thresholds are passed, a fail-safe supervisory system makes sure that safety regulations are obeyed by overriding AI decisions. The last choice is made by the vehicle control system or the human–machine interface. After that, it can issue ADAS alerts, help with the brakes, or do something on its own. We also save execution data so that we can look at it later and improve the model over time.

Mathematical Formulation.

In order to formally describe the proposed decision-making process, the system state at time ttt is represented as a feature vector that captures the dynamic interaction between the vehicle, pedestrians, and the surrounding environment:

={, , , }                                              (1)

where denotes the instantaneous vehicle speed, represents the longitudinal distance between the vehicle and the pedestrian crossing, corresponds to pedestrian-related features including position, velocity, and the number of pedestrians within the risk zone, and denotes environmental conditions such as lighting and weather. Using the recovered and combined characteristics, the vehicle's choice at time t is a binary classification problem. The decision function is defined as:

=(;)                                                  (2)

where {0,1}shows how the vehicle is acting. A value of 0 means "Continue" and a value of 1 means "Stop." It also shows the settings that the deep learning model has learned. You can use the function (.) to tell the system to stop based on how it is right now. After that, the rules for keeping people safe in Section checks this.

Teaching the Model.

The suggested decision-making model was trained using a labeled dataset that had both genuine and fake pedestrian crossing events, with varied quantities of traffic, pedestrian behaviors, and background factors. The right ground-truth vehicle action (stop or go) was added to each sample based on safety and traffic rules. We trained the model by lowering the cross-entropy loss function, which is explained below:

L=-log()+(1-)log(1-)]                           (3)

The real label is here, and it tells you how probable it is that the  -th sample will be in the stop class. It used stochastic gradient descent with adaptive learning rates to make sure that the training process converged steadily and was strong enough to handle class imbalance and changing traffic patterns. We used regularization methods and early stopping based on validation to stop overfitting and make things work better in real-time deployment. [2]

The Experiment's Results and Discussion.

We put the proposed intelligent decision-making system through a lot of different urban traffic conditions to see how well it functioned. The evaluation dataset included both real and fake pedestrian crossing situations, with different numbers of pedestrians, speeds of vehicles, and environmental elements like illumination and weather. We intentionally made our system different so we could test how well it works and how well it might operate in real-life situations when safety is critical. We used a number of evaluation parameters, like how often decisions were wrong, how long it took to make a choice, the false-stop rate, and the false-continue rate, to come up with a value that demonstrated how well the system functioned. The decision accuracy shows you how accurate the estimates are regarding whether to keep going or stop. The latency tells you how long it takes to decide after you acquire data from the sensors. The false-stop rate informs you how often cars stop when they don't need to, which can slow down traffic. The false-continue rate shows how often cars don't stop when they should, which is a huge safety issue. [3] The suggested AI-based method is quite good at making real-time forecasts. It always works better than regular decision rules that merely look at kinematic limits like speed or distance. Using Time-to-Collision (TTC) and Post-Encroachment Time (PET) together works very effectively to reduce down on false-continue events when two people are about to hit each other. This makes it safer for people to walk. The system also has a short decision latency, which is very important for advanced driver assistance systems that need to perform in real time. These results reveal that the proposed architecture is a solid balance between safety and efficiency, making it a good choice for smart cities and smart transportation systems.

Conclusion.

In summary, this research presented a safety-critical intelligent decision-making system that employs artificial intelligence to anticipate in real-time whether a vehicle should stop or continue at pedestrian crossings. The proposed system combines multi-sensor perception, feature fusion, deep learning-based inference, and formal traffic safety measures into a single structure that can be used in the real world. This is not the same as normal traffic control or systems that only use perception. By inserting Time-to-Collision (TTC) and Post-Encroachment Time (PET) directly into the decision-making process and using fail-safe supervisory logic, the system strikes a good compromise between being adaptable and dependable in changing urban traffic situations. [4] Extensive experimental evaluations across diverse pedestrian densities and traffic scenarios have demonstrated that the proposed methodology attains high accuracy in judgment while maintaining minimal latency, making it suitable for real-time applications. The reduction of false-continue incidents in safety-critical contexts illustrates the efficacy of amalgamating data-driven intelligence with rule-based safety constraints. These findings indicate that the proposed method could enhance pedestrian safety without compromising transit efficiency. [5] Future research will focus on broadening the proposed framework to include multi-agent interaction modeling, incorporating pedestrian intent prediction and collaborative vehicle-to-infrastructure communication. We will also run a lot of tests in the field and add the system to smart city traffic control platforms to make sure it works in real life and is easy for a lot of people to use. The WHO. A report on how safe it is to drive on roads all across the world.

References:

1. World Health Organization, Geneva, Switzerland, 2023.
2. Deep Learning by I. Goodfellow, Y. Bengio, and A. Courville. MIT Press in Cambridge, MA, USA, 2016.
3. A. Rasouli, I. Kotseruba, and J. K. Tsotsos, "Understanding pedestrian behavior in complex traffic scenes," IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 9, pp. 2376–2390, 2017. https://doi.org/10.1109/TITS.2016.26456344.
4. Behzod N. and Berdiev G. are working on a way to make it easier for banks to approve loans. The Journal of American Public Diplomacy and International Research (2993-2157) is about this. – 2023. – T. 12. – S. 21.5.
5. Shoniyozova YQ, Turdiyeva MA, and Norboyev BU TARMOQ HUJUMLARINING OLDINI OLISH TIZIMLARI // INNOVATSION MEXANIZMLAR VA STRATEGIK USTIBORIYATLAR ILMIY. - 2020. - 46-bet.