Reachability-Based Confidence-Aware Probabilistic Collision Detection in Highway Driving

Engineering ›› 2024, Vol. 33 ›› Issue (2) : 100 -118. DOI: 10.1016/j.eng.2023.10.010

Research

Author information +

History +

Received	Accepted	Published
2022-10-27	2023-10-11	2024-02-01
Issue Date	Revised Date
2024-06-12	2023-05-05

PDF (4220KB)

Abstract

Risk assessment is a crucial component of collision warning and avoidance systems for intelligent vehicles. Reachability-based formal approaches have been developed to ensure driving safety to accurately detect potential vehicle collisions. However, they suffer from over-conservatism, potentially resulting in false-positive risk events in complicated real-world applications. In this paper, we combine two reachability analysis techniques, a backward reachable set (BRS) and a stochastic forward reachable set (FRS), and propose an integrated probabilistic collision-detection framework for highway driving. Within this framework, we can first use a BRS to formally check whether a two-vehicle interaction is safe; otherwise, a prediction-based stochastic FRS is employed to estimate the collision probability at each future time step. Thus, the framework can not only identify non-risky events with guaranteed safety but also provide accurate collision risk estimation in safety-critical events. To construct the stochastic FRS, we develop a neural network-based acceleration model for surrounding vehicles and further incorporate a confidence-aware dynamic belief to improve the prediction accuracy. Extensive experiments were conducted to validate the performance of the acceleration prediction model based on naturalistic highway driving data. The efficiency and effectiveness of the framework with infused confidence beliefs were tested in both naturalistic and simulated highway scenarios. The proposed risk assessment framework is promising for real-world applications.

Graphical abstract

Keywords

Probabilistic collision detection / Confidence awareness / Probabilistic acceleration prediction / Reachability analysis / Risk assessment

Cite this article

Download citation ▾

Xinwei Wang, Zirui Li, Javier Alonso-Mora, Meng Wang. Reachability-Based Confidence-Aware Probabilistic Collision Detection in Highway Driving. Engineering, 2024, 33(2): 100-118 DOI:10.1016/j.eng.2023.10.010

登录浏览全文

4963

注册一个新账户忘记密码

1. Introduction

Autonomous vehicles (AVs) are expected to significantly benefit future mobility, and one of the prerequisites for enabling the public availability of AVs is to ensure autonomous driving safety [1]. Highways are structured environments designed for vehicles to drive at a consistently high speed for efficient road trips and are the first applications of Levels 1 and 2 automated vehicles. During the transition from human-driven and lower-level automated vehicles to high-level AVs, driving safety on highways must be addressed for both conventional vehicles and AVs. Extensive research has been conducted on risk assessment and collision detection has been conducted to identify driving risk and potential vehicle crashes [2], [3]. To accurately detect potential vehicle collisions, researchers have developed reachability-based formal approaches [4], because they can mathematically check whether the behavior of a system satisfies the given safety requirements.

Reachability analysis (RA) has been widely employed to verify driving safety [5], [6]. RA computes a complete set of states in which an agent (e.g., a vehicle) can reach an initial condition within a certain time interval [7]. Based on RA, safety verification can be performed by propagating all possible reachable spaces for the AVs and other traffic participants on the road. Thus, safety is ensured if such a forward reachable set (FRS) of the automated vehicle does not intersect that of other traffic participants during the propagation period. Corresponding with this definition, the FRS can formally verify safety between road users, but it easily results in over-conservative results because state propagation is feedforward-based and ignores traffic participant interactions (i.e., vehicles react to the surrounding environment and adjust the control output) [7].

Alternatively, RA can be conducted in a closed-loop manner [8]. Given a target set representing a set of undesirable states (e.g., collision states between two vehicles) and worst-case disturbances, we define the backward reachable set (BRS) as the set of states that can result in being in the target set during a certain time horizon. Specifically, a BRS is the state set in which a control strategy does not exist to prevent an AV from entering the target set under worst-case disturbances. Thus, an unsafe area can be directly identified by the BRS using the initial vehicle states. Note that the BRS can be computed offline in advance and the cached BRS used in real time. Although the BRS considers control reactions from the AV and is less conservative than the FRS, it still suffers from over-conservatism owing to the worst-disturbance closed-loop reactions.

We aim to use RA to evaluate driving risk and detect potential collisions. However, both of those RA approaches suffer from over-conservatism. The over-conservative of forward reachability is typically reduced by keeping the time horizon for an FRS small and frequently recomputing it. Although the BRS incorporates closed-loop feedback to consider the worst disturbance from the surrounding vehicle, general interactions between vehicles are not pursuit–evasion interactions [9]. It is reasonable to consider a more realistic situation: The interactions are not adversarial but may result in crashes.

1.1. Related studies

Driving risk assessments are crucial for identifying potential collisions and quantifying risk levels. Various surrogate measures of safety (SMoS) have been constructed to measure driving risk. Typically, SMoS can be calculated in a time series, including time-to-collision [10], [11], time-headway [12], and time-to-lane crossing [13]. However, these developed SMoS are mostly deterministic, which means that uncertainties in vehicle motion and environment are not considered. Although several probabilistic approaches [14], [15], [16], [17] have been integrated to improve the performance of SMoS, these methods can suffer from an additional computational load (particularly for long-term prediction) and cannot formally ensure driving risk.

Driving risk can also be assessed by estimating the current and future collision probabilities, given the surrounding road participants. Collision detection can generally be divided into three methodologies: neural network-based, probabilistic, and formal verification approaches. Neural networks can provide accurate vehicle collision detection by classifying safety–critical scenarios. For instance, a collision–detection model using a neural-network-based classifier was developed in Ref. [18]. The proposed model uses onboard sensor data, including acceleration, velocity, and separation distance as inputs into a neural network-based classifier and outputs whether alerts are activated for a possible collision. A specific animal-detection approach was proposed in Ref. [19], where a deep semantic segmentation convolutional neural network was trained to recognize and detect animals in dynamic environments. Although neural-network-based approaches are effective in identifying potential collisions, the trained classifier generally cannot include clear decision rules and is difficult to interpret.

Prediction-based approaches have also been widely adopted to address the uncertainties of surrounding road participants and better estimate collision probabilities. A conceptual framework for analyzing and interpreting dynamic traffic scenes for collision estimation was designed in Ref. [20]. Collision risks were estimated as stochastic variables and predicted by relying on driver behavior evaluation with hidden Markov models [20]. To calculate collision probability, Annell et al. [21] combined intention estimation and a long-term trajectory prediction module to construct a probability field for future vehicle positions. For a set of local path candidates, a collision risk assessment considering the lane-based probabilistic motion prediction of the surrounding vehicles was proposed in Ref. [22]. However, these methods typically require predefined position distribution parameters, which can affect the adaptability of probabilistic collision detection.

Formal verification approaches that can ensure system safety given a specific control input range and safety requirements have been employed to address collision detection [23], [24]. As a formal approach, RA computes a complete set of states in which an agent (e.g., a vehicle) can reach an initial condition within a certain time interval [7]. Based on RA, safety verification can be performed by propagating all possible reachable spaces of the AVs and other traffic participants forward in time and checking the overlaps. To reduce the over-conservative nature of forward reachability, Althoff et al. [25] developed a stochastic FRS discretizing the reachable space into grids with probability distributions. At each time step, the collision probability is provided by summing the probabilities of the states that vehicles intersect. However, this approach is based on Markov chains, which assume that the vehicle state and its control input evolve only according to the current state. In addition, it cannot explicitly address two-dimensional motion because lane-change maneuvers are not considered.

RA can also be conducted in a closed-loop manner with worst-case disturbances, namely the BRS [8]. Although the BRS can be constructed offline using advanced Hamilton–Jacobi–Isaacs partial differential equation (HJI PDE) solvers [26], it suffers from over-conservatism owing to worst-disturbance assumptions, which are not realistic. To fill this research gap, we aim to combine the BRS and stochastic FRS techniques into an integrated collision–detection framework, which can theoretically ensure safety in non-risky interactions and provide an accurate collision estimation in safety–critical scenarios.

1.2. Objectives and contribution

A stochastic FRS can accurately estimate collision when integrated with a prediction model. However, it cannot provide a safety guarantee, as the employed probabilistic distribution predictor can fail in corner cases, resulting in false negative/positive cases (e.g., in a safe cut-in scenario, the BRS can ensure safety, whereas the stochastic FRS estimates a biased collision probability, resulting in a false positive alarm). This motivated us to propose an integrated collision–detection framework that includes both the BRS and stochastic FRS to evaluate highway driving risk (Fig. 1). The BRS is first computed using the HJI PDE [8]. If the relative positions of vehicles are identified as unsafe by the BRS, a stochastic FRS considering the surrounding vehicle maneuvering modes is further established to calculate the collision probability at each future time step. The stochastic FRS shares the same reachable states as the FRS. In addition, each state of the stochastic FRS has an estimated probability. Ideally, a stochasti BRS should be used directly for collision detection, but the computation of a stochastic BRS is not readily viable owing to its closed-loop form.

Based on the stochastic FRS, the collision probability between two vehicles can be calculated by summing the state probabilities where the two vehicles spatially overlap. If the collision probability obtained is above a predefined threshold, the ego vehicle must execute an emergency brake or swerve to avoid collisions with other vehicles. The proposed framework benefits from both the BRS and FRS: Driving safety can be theoretically ensured when the relative vehicle positions are outside the unsafe area identified by the BRS; otherwise, the framework provides a collision probability based on a developed stochastic FRS.

To construct a stochastic FRS, we develop a long short-term memory (LSTM) model for multi-maneuver acceleration prediction on highways. The proposed model has two stages for maneuver prediction (i.e., lane-keeping, turning left/right on highways) and acceleration prediction, and the model input features are selected differently at each stage. We further incorporate a confidence-aware belief vector to generate a group of predicted acceleration distributions that can dynamically adjust the degree of confidence inferred from the current prediction accuracy [27]. The confidence-aware belief vector can result in a concentrated stochastic FRS when the LSTM model has a higher prediction accuracy and results in a more spread stochastic FRS when vehicles move unexpectedly.

The main contribution in this work is summarized as follows: ① We propose a multi-modal acceleration prediction model for surrounding vehicles and establish a stochastic FRS for each surrounding vehicle by leveraging the proposed acceleration predictor. Furthermore, we incorporate confidence awareness to generate a group of predicted acceleration distributions and dynamically update the degree of confidence, resulting in more accurate stochastic FRS and more agile collision detection results. ② An integrated probabilistic collision–detection framework that includes both the BRS and stochastic FRS is proposed to evaluate highway driving risk. Within the framework, an offline-computed and cached BRS is used online to check whether the car–car interaction safety can be theoretically ensured; if not, a stochastic FRS is computed online to provide an accurate collision probability at each future time step. We presented the results of stochastic FRS using the LSTM prediction model in our previous study [28]. In this paper, we significantly extend our previous study [28] by including an integrated collision–detection framework and infusing a confidence-aware belief vector for a more accurate stochastic FRS. Extensive and comprehensive experiments, which are different from those in Ref. [28] were conducted to validate the proposed framework. The remainder of the paper is organized as follows: Section 2 provides preliminaries on the BRS, FRS, and Markov-based stochastic FRS. In Section 3, a specific prediction-based confidence-aware stochastic FRS is developed. An integrated driving risk framework, including the BRS and stochastic FRS, is established in Section 4. Extensive experiments are presented in Section 5, and we conclude our paper in Section 6.

2. Preliminaries

2.1. BRS

Backward RA is considered to be an optimal control problem; thus, computing the reachable set is equivalent to solving an HJI PDE [8]. The system dynamics are defined by

\dot{x} = f (x, u, d)

, where

x \in R^{n_{1}}

and

u \in U \subset R^{n_{2}}

are the state and control input (

U

is the admissible control input set),

d \subset R^{n_{3}}

is the disturbance, and

n_{i} \in Z^{+} (i = 1, 2, 3)

are the dimensions. The system dynamics are assumed to be uniformly continuous and bounded.

In the context of two-vehicle interactions,

u

corresponds to the control of the ego vehicle, and

d

corresponds to the control of the surrounding vehicle, as its actions are treated as disturbance inputs. Specifically, let

(x_{e g o}, u_{e g o})

(x_{s}, u_{s})

represent the state and control of the ego/surrounding (s) vehicle, and

x_{r e l}

be the system states between vehicles, for instance, the relative two-dimensional distances

(y_{1, r e l}, y_{2, r e l})

and velocities

(v_{1, r e l}, v_{2, r e l})

. Therefore, the system dynamics are given by

{\dot{x}}_{r e l} = f (x_{r e f}, u_{e g o}, u_{s})

. The formal definition of the BRS, denoted by

BR (t)

, for the related system is,

(1)

\begin{matrix} BR (t) : = \{{\bar{x}}_{r e l} : \exists u_{s} (\cdot), \forall u_{e g o} (\cdot), \exists t^{⋆} \in [t, 0], \ x_{r e l} (t) = {\bar{x}}_{r e l} \land {\dot{x}}_{r e l} = f (x_{r e l}, u_{e g o}, u_{s}) \land x_{r e l} (t^{⋆}) \in T\} \end{matrix}

Here,

BR (t)

represents the set of unsafe states

{\bar{x}}_{r e l}

at time

t

, from which if the surrounding vehicle followed an adversarial policy

u_{s}

, any policy

u_{e g o}

would result in the state set

T

t^{⋆} \in [t, 0]

where two vehicles collide within a time horizon.

Assuming optimal (i.e., adversarial) surrounding vehicle actions,

BR (t)

can be computed by defining a value function

V (t, x_{r e l})

that obeys the HJI PDE, where the solution

V (t, x_{r e l})

provides the BRS as its zero-sublevel set:

(2)

\begin{matrix} BR (t) = \{x_{r e l} : V (t, x_{r e l}) \leq 0\} \end{matrix}

The HJI PDE is solved starting from the boundary condition

V (t, x_{r e l})

, which indicates whether the state

x_{r e l}

belongs to the collision set

T

. We cache the solution

V (t, x_{r e l})

for online use as a look-up table.

2.2. FRS and Markov-based stochastic FRS

An FRS is computed by considering all possible control inputs

u \in U

of a system

\dot{x} = f (x, u)

given an initial set of states

X_{0}

. The FRS of a system is formally defined as:

(3)

\begin{matrix} FR (t) : = \{\bar{x} : \exists u (\cdot), \exists t^{⋆} \in [0, t], x (0) \in X_{0} \land \dot{x} = f (x, u, d) \land x (t^{⋆}) = \bar{x}\} \end{matrix}

where

FR (t)

is an FRS of states

\bar{x}

at time

t^{⋆} \in [0, t]

from an initial state

x (0) \in X_{0}

at the current time

0

and subject to any input

u

belonging to the admissible control input set

U

Based on the definition of the FRS, we can mathematically formulate a stochastic version of the FRS as

S F R (t)

, where each state

\bar{x}

within the FRS is associated with the state probability

p (\bar{x})

(4)

\begin{matrix} S F R (t) : = {(\bar{x}, p (\bar{x})) : \exists u (\cdot), \exists t^{⋆} \in [0, t], x (0) \in X_{0} \land \dot{x} = f (x, u, d) \land x (t^{⋆}) = \bar{x}} \end{matrix}

One of the most frequently used techniques is the approximation of stochastic processes using Markov chains, which represent a stochastic dynamic system with discrete states [25]. The discretized time step series are denoted as

{0, 1, \dots, e}

, where

e

is the future final time step, and the duration of each time step is

d t

. Owing to stochastic characteristics, the system state at the predicted time step is not exactly known, and the probability

p_{i} (k)

is assigned to each state

i

at time step

k

(the discretized system state is denoted as

i, j

for simplicity). Subsequently, the probability vector

p (k + 1)

composed of probabilities

p_{i} (k)

over all states is updated as

(5)

\begin{matrix} p (k + 1) = Φ p (k) \end{matrix}

where

Φ

is the state transition matrix. Here,

Φ

is time-invariant as the model is assumed to be Markovian.

To implement a Markov chain model, we need to first discretize the system state if the original system is continuous. The vehicle dynamic system is represented as a tuple with four discretized elements, including the two-dimensional vehicle positions and velocities. Additionally, the control input is to be discretized. The detailed discretization parameters are described in Section 5.1.

Each element

Φ_{ji}

in the matrix,

Φ

represents the state transition probability from state

i

j

. Note that the transition probabilities depend on the discrete input

u

as well, for example, each discrete input

u

generates a conditional transition probability matrix

Φ^{u}

. Specifically, each element

Φ_{ji}^{u}

in the conditional matrix

Φ^{u}

is the possibility starting from the initial state

i

j

under control

u \in U

, where

u

represents the corresponding control input of

Φ_{ji}^{u}

. Therefore, the conditional probability

Φ_{ji}^{u}

is expressed as

(6)

\begin{matrix} Φ_{ji}^{u} = \{\begin{matrix} p_{i}^{u}, i f s t a t e i r e a c h e s s t a t e j w i t h i n p u t u \\ 0, o t h e r w i s e \end{matrix} \end{matrix}

where

p_{i}^{u}

is the control input probability for state

i

. The time index does not appear here because it is a Markov process. Thus, the overall state transition matrix is constructed as

(7)

\begin{matrix} Φ_{ji} = \sum_{u \in U} Φ_{ji}^{u} \end{matrix}

The probability distribution of the control input

p_{i}^{u}

is dynamically changed by another Markov chain with a transition matrix

Γ_{i}

, depending on the system state

i

. This enables more accurate modeling of driver behavior by considering the frequency and intensity of changes in the control input. Consequently, the transition matrices

Γ

are to be learned by observation or set by a combination of simulations and heuristics. By incorporating the two transition matrices

Φ

and

Γ

, a Markov-based stochastic FRS with probabilities

p (k)

over discretized states can be obtained at each predicted time step

k

In Althoff et al. [25], the control transition probability matrices

Γ

only depend on the current control input and the state at the current time. Although computational efficiency is ensured by using such a simplified Markovian setting, the future control input and trajectories of a vehicle can be influenced by historical information and interactions with the surrounding environment [29]. Therefore, in this paper, we do not assume that the vehicle system state is Markovian. Instead, to address historical information and vehicle interactions, we aim to use a vehicle control predictor with multi-maneuvering modes to generate and dynamically update the transition matrices at each time step

k

(8)

\begin{matrix} Φ_{ji} (k) = \sum_{u \in U} Φ_{ji}^{u} (k) \end{matrix}

3. Prediction-based confidence-aware stochastic FRS

In this section, to provide a more accurate prediction of the surrounding vehicles, we first introduce a two-stage multi-modal acceleration prediction model consisting of lane-change maneuver prediction and acceleration prediction modules. We then establish the stochastic FRS by incorporating the proposed acceleration prediction model and infusing a confidence belief vector. 3.1 Acceleration prediction of a surrounding vehicle, 3.2 Prediction-based stochastic FRS of a surrounding vehicle have been presented in Wang et al. [28] and are included here for completeness. Note that the prediction model is replaceable if the accelerations can be predicted using bivariate normal distributions.

3.1. Acceleration prediction of a surrounding vehicle

Typically, vehicle trajectories and accelerations are predicted using current and historical information [29]. Thus, the prediction accuracy can be improved compared with using only the current states as inputs. This motivated us to establish an LSTM-based network to probabilistically predict future vehicle accelerations using both current and historical vehicle information. An overview of the developed two-stage acceleration prediction model is shown in Fig. 2 [30].

3.1.1. Two-stage vehicle acceleration prediction

A two-stage multi-modal trajectory prediction model was developed in Ref. [30]. In this paper, we maintain the same lane-change maneuver prediction model in the first stage but develop a new acceleration prediction model in the second stage. This is necessary because the acceleration prediction is employed to enable the dynamic update of the conditional probability

Φ_{ji}^{u} (k)

in Eq. 8.

First, we briefly introduce the adopted lane-change maneuver prediction module from Wang et al. [30]. The input of the module is expressed as

(9)

\begin{matrix} X = [x^{(- h)}, \dots, x^{(- 1)}, x^{(0)}] \end{matrix}

where

X

represents all input features from the previous time step

- h

to the current time step

0

. At each historic time step, the collected input is composed of three parts:

x^{(\cdot)} = \{{x_{T}}^{(\cdot)}, b^{(\cdot)}, d^{(\cdot)}\}

, where

{x_{T}}^{(\cdot)}

is the position information for the vehicle being predicted and its surrounding vehicles,

b^{(\cdot)}

contains two binary values to check whether the predicted vehicle can turn left and right, and

d^{(\cdot)} \in [- 1, 1]

is the normalized deviation value from the current lane center.

As shown at the top of Fig. 2, LSTM models are used to encode and decode the lane-change maneuver prediction model, in which the encoding information is passed to fully connected layers before decoding. The output of the model is a probability distribution

P (m | X)

for each lane-change maneuver mode m (i.e., change to the left, change to the right, and maintain the same lane) in time steps 1 to

e

For the acceleration prediction in the second stage, the input includes the historic positions of the vehicle being predicted and surrounding vehicles in addition to the historic accelerations

{x_{A}}^{(- h : 0)}

of the vehicle being predicted:

(10)

\begin{matrix} X_{T} = [{x_{T}}^{(- h)}, \dots, {x_{T}}^{(- 1)}, {x_{T}}^{(0)}, {x_{A}}^{(- h : 0)}] \end{matrix}

Because we use additional acceleration information for the vehicle being predicted, we modify the input size of the LSTM encoder in Wang et al. [30] for the vehicle being predicted while maintaining the overall network structure unchanged. Detailed information on the second-stage model is provided in Refs. [29], [30].

Given the input

X_{T}

and corresponding maneuver mode probability distribution

P (m | X)

, the output

P (U | m, X_{T})

of the second-stage acceleration prediction model is a conditional acceleration distribution over

(11)

\begin{matrix} U = [u^{(1)}, \dots, u^{(e)}] \end{matrix}

where

u^{(\cdot)}

is the predicted vehicle acceleration at each time step within the prediction horizon. Note that the prediction horizon and time increment are the same as those for the reachable set computation.

Given the three defined maneuvers

m

, the probabilistic multi-modal distributions are calculated as

(12)

\begin{matrix} P (U | X) = \sum_{m} P_{Θ} (U | m, X_{T}) P (m | X) \end{matrix}

where outputs

Θ = [Θ^{(1)}, \dots, Θ^{(e)}]

are time-series bivariate normal distributions.

Θ^{(k)} = \cup_{m} {μ_{1 m}^{k}, μ_{2 m}^{k}, σ_{1 m}^{k}, σ_{2 m}^{k}, ρ_{m}^{k}}

corresponds to the predicted acceleration means and standard deviations along two dimensions and the correlation at future time step

k

under each maneuver mode

m

, respectively.

Under acceleration distributions

Θ

, the future vehicle trajectories are propagated as

(13)

\begin{matrix} \{\begin{matrix} \begin{matrix} v_{1 m}^{k + 1} = v_{1 m}^{k} + μ_{1 m}^{k + 1} d t \\ v_{2 m}^{k + 1} = v_{2 m}^{k} + μ_{2 m}^{k + 1} d t \end{matrix} \\ y_{1 m}^{k + 1} = y_{1 m}^{k} + (v_{1 m}^{k + 1} + v_{1 m}^{k}) d t / 2 \\ y_{2 m}^{k + 1} = y_{2 m}^{k} + (v_{2 m}^{k + 1} + v_{2 m}^{k}) d t / 2 \end{matrix} \end{matrix}

where

d t

is the time increment,

v_{1 m}^{k}, v_{2 m}^{k}, y_{1 m}^{k}, a n d y_{2 m}^{k}

are the propagated two-dimensional velocities and positions at future time step

k

for each maneuver mode

m

, respectively.

(v_{1 m}^{0}, v_{2 m}^{0}, y_{1 m}^{0}, y_{2 m}^{0})

denote the system states at time

t

To properly propagate probabilistic distributions from accelerations to trajectories, we assume that the trajectory variance and correlation propagation depend only on the acceleration at the current time rather than on the previous time steps. Otherwise, the trajectory variance/correlation at the future

k

step is determined by the acceleration variances/correlations from the current time step to the future

k

step, which complicates and makes the probabilistic distribution propagation intractable. Thus, the propagated trajectory standard deviations can be obtained as

\tilde{σ_{1 m}^{k}} = σ_{1 m}^{k} \cdot {(d t)}^{2} / 2

and

\tilde{σ_{2 m}^{k}} = σ_{2 m}^{k} {(d t)}^{2} / 2

, and the correlation as

ρ_{m}^{k} {(d t)}^{2} / 2

. Therefore, the propagated probabilistic distributions of the vehicle position are expressed as

\tilde{Θ^{(k)}} = {\{y_{1 m}^{k}, y_{2 m}^{k}, \tilde{σ_{1 m}^{k}}, \tilde{σ_{2 m}^{k}}, ρ_{m}^{k}\}}_{m = {1, 2, 3}}

3.1.2. Model training

Typically, a multi-modal prediction model is trained to minimize the negative log-likelihood (NLL) of its conditional distributions as

(14)

\begin{matrix} - l o g (\sum_{m} P_{Θ} (U | m, X_{T}) P (m | X)) \end{matrix}

For a more accurate collision probability estimation, we focus on the potential collision when two vehicles intersect along their trajectories. Therefore, we directly minimize the trajectory prediction errors propagated from the acceleration prediction in line with Zhou et al. [31] as

(15)

\begin{matrix} - l o g (\sum_{m} P_{\tilde{Θ}} (Y | m, X_{T}) P (m | X)) \end{matrix}

where

Y = [y^{(1)}, \dots, y^{(e)}]

is the propagated trajectories with distributions

\tilde{Θ}

, and

y^{(k)} = {y_{1 m}^{k}, y_{2 m}^{k}}

are the predicted positions of the vehicle at time step

k

under maneuver mode

m

To further improve the prediction performance, we separately train the lane-change maneuver and vehicle acceleration prediction models. This is because the proposed approach has a two-stage structure: the maneuver probabilities are first predicted and then predicted for the corresponding conditional vehicle acceleration distributions. For the maneuver prediction model, trained to minimize the NLL of the maneuver probabilities

- l o g (\sum_{m} P (m | X))

or the vehicle acceleration prediction, the adopted model is used to minimize

- l o g (\sum_{m} P_{\tilde{Θ}} (Y | m, X_{T}))

3.2. Prediction-based stochastic FRS of a surrounding vehicle

When predicting the future state of a surrounding vehicle, both the current state and historical information need to be considered [29]. In this paper, we use the acceleration prediction results from Section 3.1 to dynamically update the state transition probability matrix at each time step.

The system state

i

of the surrounding vehicle is represented as a tuple with four discretized elements, including two-dimensional vehicle positions and velocities. The system input is expressed as a two-dimensional acceleration

(a_{1}, a_{2})

. The probability of the current state is known in advance. Typically, we have an initial state

i

with probability

p_{i} (0) = 1

, or an initial probability distribution is provided to address the state uncertainties. In practice, from the current time, we need to calculate multiple stochastic FRS at multiple forwarded time steps and check the corresponding FRS at each future time step

k \in {1, 2, \dots, e}

At each predicted time step

k

, the acceleration prediction model provides a bivariate normal distribution function

f_{m}^{k} (a_{1}, a_{2})

for each maneuver mode

m

as follows:

(16)

\begin{matrix} f_{m}^{k} (a_{1}, a_{2}) = \frac{1}{2 π σ_{1} σ_{2 \sqrt{1 - ρ^{2}}}} e x p (- \frac{1}{2 (1 - ρ^{2})} [{(\frac{a_{1} - μ_{1}}{σ_{1}})}^{2} + {(\frac{a_{2} - μ_{2}}{σ_{2}})}^{2} - 2 ρ \frac{(a_{1} - μ_{1}) (a_{2} - μ_{2})}{σ_{1} σ_{2}}]) \end{matrix}

where

μ_{1}, μ_{2}, σ_{1}, a n d σ_{2},

provided by the prediction model, denote predicted means and standard deviations along two directions, and the correlation at future time instant

k

for each maneuver mode

m

, respectively. For brevity, the time and maneuver indices of the five parameters are omitted.

To propagate the system states, we calculate the conditional probability

p_{i}^{u} (k)

at time step

k

under state

i

and accelerations

u = (a_{1}^{u}, a_{2}^{u})

as follows:

(17)

\begin{matrix} p_{i}^{u} (k) = \frac{\tilde{p_{i}^{u}} (k)}{\sum_{u \in U} \tilde{p_{i}^{u}} (k)} \end{matrix}

(18)

\begin{matrix} \tilde{p_{i}^{u}} (k) = \sum_{m} λ_{m}^{k} \cdot \int_{{\underline{a}}_{2}^{u}}^{{\bar{a}}_{2}^{u}} \int_{{\underline{a}}_{1}^{u}}^{{\bar{a}}_{1}^{u}} f_{m}^{k} (a_{1}, a_{2}) d a_{1} d a_{2} \end{matrix}

where

λ_{k}^{m}

is the probability for maneuver mode m at time step

k

, and

{\underline{a}}_{1}^{u}, {\bar{a}}_{1}^{u}, {\underline{a}}_{2}^{u}, {\bar{a}}_{2}^{u}

are the integral boundaries of

u

The conditional state probability

p_{i}^{u} (k)

is implicitly relevant to the current and historical states. This is because current and historical information has been considered when providing the predicted acceleration results. This implies that the state transition matrix now has to be computed online.

Substituting Eqs. 17, 6 into Eq. 7, we can obtain the overall state transition matrix

Φ

. To distinguish the Markov-based approach which can compute the transition matrix offline, we denote the state transition matrix obtained with the prediction model at the predicted time step

k

Φ (k)

. Subsequently, at each predicted time step, the state probability vector is iteratively computed as

(19)

\begin{matrix} p (k + 1) = Φ (k) \times p (k) \end{matrix}

To measure driving risk, the collision probability at the current time

P_{c o l}

is expressed as the product of the collision probability at each predicted time step.

(20)

\begin{matrix} P_{c o l} = 1 - \prod_{k} (1 - \sum_{i \in H (k)} p_{i} (k)) \end{matrix}

where

H (k)

is the set of states occupied by the ego vehicle at time step

k

. The vehicle dimensions are considered when calculating the collision probability.

3.3. Infusing confidence belief

A prediction model applied to compute the stochastic FRS is not always accurate because vehicles can move unexpectedly. To address this problem, we adopt a confidence-aware belief vector to modify probabilistic motion predictions that exploit the modeled structure when the structure successfully explains the vehicle motion and degrades gracefully whenever the vehicle moves unexpectedly [27].

If prediction confidence is not considered, the probabilistic acceleration is expressed by Eq. 16. To address the prediction confidence level, similar to the study in Fridovich-Keil et al. [27], we infuse an additional coefficient

β

into the acceleration probability density function:

(21)

f_{m}^{k} (a_{1}, a_{2}, β) = \frac{1}{2 π (σ_{1} β) (σ_{2} β) \sqrt{1 - ρ^{2}}} exp (- \frac{1}{2 (1 - ρ^{2})} [{(\frac{a_{1} - μ_{1}}{σ_{1} β})}^{2} + {(\frac{a_{2} - μ_{2}}{σ_{2} β})}^{2} - 2 ρ \frac{(a_{1} - μ_{1}) (a_{2} - μ_{2})}{σ_{1} β σ_{2} β}])

where a positive coefficient

β

controls the confidence level of the prediction model. For instance, when setting

β

to infinity, the acceleration probability would be uniformly distributed and ignore the prediction model. When

β

is close to 0, the discretized input

(a_{1}, a_{2})

, which is closest to the predicted acceleration mean values

μ_{1}, μ_{2}

is assigned with probability one.

Consequently, the input conditional probability computation in Eq. 18 can be extended as

(22)

\begin{matrix} \tilde{p^{u}} (k, β) = \sum_{m} \sum_{β} λ_{m}^{k} b^{k} (β) \int_{{\underline{a}}_{2}^{u}}^{{\bar{a}}_{2}^{u}} \int_{{\underline{a}}_{1}^{u}}^{{\bar{a}}_{1}^{u}} f_{m}^{k} (a_{1}, a_{2}, β) d a_{1} d a_{2} \end{matrix}

where

b^{k} (β) \in (0, 1)

is the belief value for a specific

β

at time step

k

Therefore, the performance of the prediction model may change over time. For instance, the model may have a relatively worse prediction accuracy when the surrounding vehicle begins to make a lane change, as the lane-change maneuver may not be recognized promptly. To reflect the dynamic properties of the confidence level, we should update the belief of each confidence level frequently. Thus, a Bayesian belief vector regarding possible values of

β

is introduced.

Initially, each

β

is assigned with a uniform probability

b^{0} (β)

; subsequently, the belief vector evolves given posterior probabilities of the state and input under each

β

at time step

k

(23)

\begin{matrix} b^{k + 1} (β) = \frac{P (u^{(k - k^{'} + 1 : k)} | x^{(k - k^{'} + 1 : k)}; Θ^{(k - k^{'} + 1 : k)}, β) b^{k} (β)}{\sum_{\tilde{β}} P (u^{(k - k^{'} + 1 : k)} | x^{(k - k^{'} + 1 : k)}; Θ^{(k - k^{'} + 1 : k)}, \tilde{β}) b^{k} (\tilde{β})} \end{matrix}

where

(24)

\begin{matrix} P (u^{(s s : k)} | x^{(s s : k)}; Θ^{(s s : k)}, β) = \prod_{i = k - k^{'} + 1}^{k} \tilde{p^{u_{i}}} (k - i, β) \end{matrix}

is the posterior probability of the observed actual accelerations,

u^{(s s : k)}

with

t_{s s} = t_{m a x (k - k^{'} + 1, 1)}

from the previous

k^{'}

time steps.

In practice, the Bayesian belief vector with a relatively small set and small previous time steps, for instance, five discrete values of

β

and

k^{'} = 2

, can achieve significant improvement [27].

4. Integrated collision–detection framework

In this section, we propose an integrated collision–detection framework that combines the BRS and stochastic FRS described in the previous section. Hence, we also specify the system dynamics models for the BRS and stochastic FRS and prove the equivalence of different models in the BRS and stochastic FRS.

4.1. Framework

We propose an integrated collision–detection framework for highways by combining the BRS and stochastic FRS. Before employing this framework, a BRS is computed using the HJI PDE [32]. Although computing a BRS is time-consuming, this can be done offline, and the results of the BRS are cached in a look-up table for subsequent real-time risk assessment. Note that we are required to interpolate the BRS look-up table in practice as the cached states are discretized. A flowchart of the framework, including the three iterative steps, is shown in Fig. 3.

Step 1: We check whether the surrounding vehicles are inside the unsafe area identified by the BRS. If so, we proceed to the next step; otherwise, the safety of vehicle interactions is theoretically ensured at the current time.

Step 2: A stochastic FRS is generated online to calculate the collision probability at each predicted time step. Note that the stochastic FRS can be obtained using either heuristic rules [25] or the prediction-based approach introduced in Section 3.

Step 3: If the obtained collision probability is above a predefined threshold, the ego vehicle must replan its motion trajectories or the ego driver receives an alert to avoid potential crashes with the surrounding vehicle, which is not within the scope of this paper.

4.2. Vehicle dynamics and equivalent transformation between BRS and stochastic FRS

The computation of the BRS and (stochastic) FRS depends on the selection of the vehicle dynamic models. Ideally, we can use the same system dynamics for the ego to construct both the BRS and FRS, and the equivalent transformation, which is used to ensure a consistent control input range between different vehicle dynamics, is no longer necessary. However, in this paper, we use different system dynamics for BRS and FRS for two reasons. First, we construct the BRS using a bicycle model for the ego vehicle and a unicycle model for the surrounding vehicle following Li et al. [32] because these models are relatively more realistic than a point mass model. Second, we construct the FRS using a point mass model primarily to accommodate the control input predictors, which typically output two-dimensional acceleration values [31], [33]. The details of the vehicle dynamics employed in this paper are as follows:

To calculate the BRS, we define the vehicle dynamics of the ego and surrounding vehicles according to Li et al. [32]. A more complex ego vehicle model increases the dimensions of the relative dynamics model, resulting in an overly complicated calculation of the BRS. In the study of Leung et al. [8], computing the BRS using a seven dimensional relative system and discretization required approximately 70 h on a 3.0 GHz octa-core advanced micro devices (AMD) Ryzen 1700 central processing unit (CPU). In this paper, collision detection is guaranteed by a combination of BRS and stochastic FRS. The vehicle models used in the two modules (BRS and stochastic FRS) must be consistent. A high-dimensional vehicle model in the BRS significantly increases the computational load of the stochastic FRS, even with a parallel computation assumption. Therefore, as shown in Fig. 4, we apply a bicycle model for the ego vehicle, where the state includes longitudinal/lateral positions

y_{1, e g o} / y_{2, e g o}

, the heading angle

ψ_{e g o}

, and the ego vehicle velocity

v_{e g o}

. Its control input is the acceleration aego and steering angle

δ_{f}

. O is the center of vehicle rotation.

δ_{r} = 0

is the rear wheel steering angle and

β_{e g o} = \tan^{- 1} (\frac{l_{r}}{l_{f} + l_{r}} t a n δ_{f})

is the slip angle of the ego vehicle, where

l_{f} / l_{r}

is the distance from the front/rear to the vehicle reference point. For the surrounding vehicle, we model its state with two-dimensional positions

\frac{y_{1, s}}{y_{2, s}}

, the heading angle

ψ_{s}

, and the surrounding vehicle velocity

v_{s}

. The control input is acceleration

a_{s}

and angular velocity

ω_{s}

. Subsequently, the relative dynamics can be represented as

(25)

\{\begin{matrix} {\dot{y}}_{1, r e l} = \frac{v_{e g o}}{l_{r}} s i n β_{e g o} y_{2, r e l} + v_{s} c o s ψ_{r e l} - v_{e g o} c o s β_{e g o} \\ {\dot{y}}_{2, r e l} = - \frac{v_{e g o}}{l_{r}} s i n β_{e g o} y_{1, r e l} + v_{s} s i n ψ_{r e l} - v_{e g o} s i n β_{e g o} \\ {\dot{ψ}}_{r e l} = ω_{s} - \frac{v_{e g o}}{l_{r}} s i n β_{e g o} \\ {\dot{s}}_{e g o} = a_{e g o} \\ {\dot{v}}_{s} = a_{s} \end{matrix}

where

y_{1, e g o} / y_{2, e g o}

and

ψ_{r e l}

are relative two-dimensional coordinates and heading angles, respectively.

To calculate the stochastic FRS and accommodate acceleration prediction models that typically represent probabilistic accelerations with bivariate distributions, we employ a point-mass model to compute the stochastic FRS of the surrounding vehicle. The control input is simplified with two-dimensional accelerations, and future vehicle positions can then be directly propagated with the predicted accelerations. We assume that the planned trajectories of the ego vehicle are deterministic and are known in advance. Nevertheless, the motion uncertainties of an ego vehicle can be represented by extending its future position with additional adjacent states, which is left for future research.

A summary of the system dynamics used for BRS and FRS is provided in Table 1. Given the different vehicle dynamic models and control inputs employed for the BRS and stochastic FRS, it is essential to match the control input ranges among the three dynamic models. We term this process the equivalent vehicle dynamics transformation between the BRS and FRS. We provide a detailed equivalent transformation procedure in the Supplementary text in Appendix. A. The BRS and stochastic FRS can now be integrated into the same framework.

5. Experiments

In this section, we first introduce the naturalistic highway driving dataset highD employed for the prediction model training/testing as well as the experimental setup for the prediction model and BRS/FRS computation. Several experiments were designed and conducted with respect to three aspects: ① We tested the proposed acceleration prediction model by comparing it with three existing predictors, as the acceleration prediction model plays a vital role in the establishment of the stochastic FRS. ② We further tested the performance of the prediction-based confidence-aware FRS using naturalistic driving data from highD and simulated cut-in events. Risky cut-in events were simulated to test the collision estimation performance based on stochastic FRS. ③ To validate the integrated collision–detection framework, we first compared the identified unsafe areas between the BRS and FRS and then tested the framework for both risky and non-risky cut-in events.

5.1. Dataset and setup

The highD dataset [34], which contains bird–view naturalistic driving data on German highways, was used to train and test the acceleration prediction model. We randomly selected equal numbers of samples for the three lane-change maneuver modes, resulting in 135 531 (45 177 for each maneuver mode) and 19 482 (6494 for each mode) samples for training and testing, respectively. The original dataset sampling rate was 25 Hz, and we down sampled it by a factor of five to reduce the model complexity. We considered 2 s historic information as input and predicted within a 2 s horizon.

The prediction model was trained using Adam with a learning rate of 0.001, and the sizes of the encoder and decoder are 64 and 128, respectively. The size of the fully connected layeris 32. The convolutional social pooling layers consist of a 3 × 3 convolutional layer with 64 filters, a 3 × 1 convolutional layer with 16 filters, and a 2 × 1 max pooling layer, which reconsistent with the settings in Ref. [29].

Corresponding with the vehicle dynamics in Section 4.2, the BRS was computed offline in five dimensions: relative two-dimensional positions, heading angle, and velocities for the ego and surrounding vehicles, respectively. The relative longitudinal position was discretized from −10 to 40 m in 0.5 m increments, the relative lateral position from −4 to 4 m in 0.4 m increments, the heading angle from −45° to 45° in 9° increments, and ego/surrounding vehicle velocity from 20 to 40 m·s⁻¹ in 1 m·s⁻¹ increments, resulting in over five million states. The longitudinal position range setting was sufficient to identify an enclosed unsafe area for all simulations in this study, as shown in Fig. 14. The range of the control input is given in the Supplementary text to ensure an equivalent model transformation between BRS and FRS.

The FRS states were expressed in four dimensions, including two dimensions for position and velocities, respectively. The vehicle longitudinal position was discretized from −4 to 80 m in 2 m increments and the relative lateral position from −4 to 4 m in 1 m increments. The longitudinal velocity was discretized from 20 to 40 m·s⁻¹ in 0.4 m·s⁻¹ increments, and the lateral velocity from −2.5 to 2.5 m·s⁻¹ in 0.2 m·s⁻¹ increments, resulting in approximately half a million states. Here, the longitudinal position range of the FRS was from −4 to 80 m because we assumed that a vehicle could not move backward, and the marginal position to collide with a surrounding vehicle was −4 m, given the vehicle length at 4 m. It reached a maximum of 80 m with the highest longitudinal velocity (40 m·s⁻¹) in the prediction horizon (2 s). For the control input, we discretized the longitudinal (lateral) accelerations from −5 to 3 (−1.5 to 1.5) m·s⁻² in increments of 1 (0.5) m·s⁻², resulting in 63 acceleration combinations. We also added several constraints to limit acceleration selection, including maximal acceleration, strict forward motion, and maximal steering angle [17]. Ultimately, 37 million possible state transfers were generated. To alleviate the computational load, we assumed that an advanced graphics processing unit (GPU) [35] that enables 2048 × 28 parallel computations is available. The stochastic FRS with state probability distributions

p (k)

was calculated at each predicted future time step within 2 s in 0.4 s increments, for instance {0.4, 0.8, 1.2, 1.6, 2.0}.

5.2. Acceleration/trajectory prediction models

To validate the proposed two-stage acceleration prediction model (denoted as T-LSTMa), we compared T-LSTMa with three state-of-the-art probabilistic multi-modal predictors: T-LSTM [30], social convolutional trajectory predictor (S-LSTM) using convolutional neural networks to represent surrounding vehicles [29], and its variation S-LSTMa for fair comparisons, where the prediction output was modified to probabilistic accelerations. Utilizing the testing dataset from highD, we report the comparative results in Table 2, including five evaluation indicators, for instance, root mean square error (RMSE), average displacement error (ADE), final displacement error (FDE), NLL (the lower, the better), and average lane-change prediction F1 score LC-F1 (a metric calculated as the harmonic mean of the precision and recall [36]). We show the predictor performance on the overall testing dataset and compare the prediction results in terms of the three lane-change maneuver modes. Note that we did not directly evaluate the acceleration prediction accuracy. Instead, we compared the vehicle position prediction accuracy, which directly affects collision estimation.

Examining the overall comparison results, we observed that using acceleration prediction and then propagating the future vehicle position can significantly improve the prediction accuracy (see comparisons between S-/T-LSTMa and S-/T-LSTM). This could be owing to the additional physical information when using the predicted acceleration to propagate future positions. Meanwhile, S-/T-LSTMa and S-/T-LSTM had the same LC-F1 because they share the same lane-change prediction submodel. S-LSTMa exhibited the worst performance in terms of NLL and LC-F1. Specifically, the LC-F1 for lane-keeping trajectories was 0. This indicated that jointly predicting the acceleration and lane-change maneuver modes in one neural network results in undesirable results, and it is reasonable to consider decoupling the acceleration and lane-change maneuver mode predictions using two neural networks, which have been employed in T-LSTMa. In summary, our proposed acceleration predictor T-LSTMa achieved the best performance in terms of all indicators, with ADE < 0.15 m and FDE < 0.40 m.

We further tested the model’s prediction performance over different prediction horizons (one and three seconds). As shown in Table 3, the proposed predictor, T-LSTMa, achieved superior performance over all prediction horizons. In addition, both models obtained more accurate prediction results with shorter prediction horizons. For the remainder of this study, we used T-LSTMa with default settings to predict accelerations.

5.3. Confidence-aware position prediction and collision estimation

In this section, we compare different approaches for generating a stochastic FRS. Three different groups of coefficients β were selected: Only one distribution with β = [1]; three normal distributions with β = [1/2, 1, 2]; five normal distributions with β = [1/3, 1/2, 1, 2, 3]. This led to three different approaches to generating prediction-based stochastic FRS (denoted as PSRS, PSRS-3β, and PSRS-5β). A further increase in the number of distribution groups was not considered because we were required to ensure real-time computation of the stochastic FRS. The heuristic method in the study of Althoff et al. [25] was also adopted as a baseline to generate the stochastic FRS (denoted as HSRS).

To test the performance of the prediction-based confidence-aware FRS, we used naturalistic driving data from highD and simulated cut-in events. This is because highD does not contain safety–critical events; thus, we created simulated risky events to test the collision estimation performance based on the stochastic FRS.

5.3.1. highD trajectories

We assumed that the vehicle occupied the exact space state and the four adjacent states along the longitudinal and lateral directions. Naturalistic driving trajectories were randomly selected from highD, including lane-keeping and lane-change trajectories.

Examples of stochastic FRS predicted using different approaches are presented in Fig. 5, Fig. 6, respectively. For the lane-keeping trajectory, the position prediction accuracies at time t = 4.4 s for the four approaches were 44.33%, 45.56%, 46.84%, and 52.74%. For the lane-change trajectory, the position prediction accuracies at time t = 6.4 s for the four approaches were 24.54%, 29.38%, 32.58%, and 37.15%. Infusing confidence awareness can improve vehicle position prediction, particularly for lane-changing trajectories. The belief vector changes are shown in Fig. 7 for both lane-keeping and lane-change trajectories. For the lane-keeping trajectory, Fig. 7(a) and (b) show corresponding results from PSRS-3β and PSRS-5β; both had a stable acceleration prediction accuracy, as the surrounding vehicle moved as expected. Moreover, the belief value with the lowest β converged to one for both two approaches. This is because the confidence-infused prediction model provided a more accurate mean value of accelerations, resulting in a corresponding higher belief value with lower β in line with Eq. 24. For the lane-change trajectory, a similar convergence trend of the belief value changes can be observed in Fig. 7(c) and (d). However, the belief value of the lowest β (Fig. 7(d), PSRS-5β) required about 4 s to converge and less than 2 s in Fig. 7(c). This indicates that infusing a higher-dimensional belief vector can adjust more complicated prediction scenarios (e.g., lane-change trajectories) and provide more accurate prediction results.

We tested the four approaches using randomly selected 100 trajectories from highD for both lane-keeping and lane-change scenarios. The average position prediction performance is summarized in Table 4, which shows that when the prediction horizon is short (e.g., 0.4 s), all approaches could generate a stochastic FRS with higher prediction accuracy. However, when the prediction horizon increased, HSRS exhibited the worst performance, followed by PSRS. Again, infusing confidence belief effectively improves prediction accuracy; such an improvement is more significant in lane-change scenarios.

5.3.2. Simulated safe-critical cut-in trajectories

Safety-critical cut-in events were simulated to test the collision–detection performance of different stochastic FRS. In the simulated cut-in scenario, the ego vehicle in the middle lane and the surrounding vehicle in the right lane traveled with constant initial longitudinal speeds of

v_{e g o}

= 30 m·s⁻¹ and

v_{s}

= 25 m·s⁻¹, respectively. The surrounding vehicle was 15 m ahead of the ego vehicle at t = 1.0 s and began turning left with a lane-change duration of 7.5 s. The vehicle length and width were 4 and 2 m, respectively. The driving behaviors of the two vehicles were simulated using the classic intelligent driving model (IDM) for car-following and a lateral control model [37].

The IDM equations are expressed as

(26)

\{\begin{matrix} {d g}^{*} (t) = {d g}_{0} + m a x (0, v (t) T H + v (t) d v (t) / 2 \sqrt{a_{a} a_{b}}) \\ a (t) = m a x (a_{a} (1 - {(\frac{v (t)}{v_{0}})}^{4} - {(\frac{{d g}^{*} (t)}{d g (t)})}^{2}), a_{m i n}) \end{matrix}

where

{d g}^{*} (t)

and

d g (t)

are the desired and current longitudinal distance gaps, respecitvely;

v (t)

is the longitudinal speed;

d v (t)

is the longitudinal speed difference from the lead;

a (t)

the current accelerration. If there is no leading vehicle,

d v (t)

and

\frac{1}{d g (t)}

are set to zero. The IDM parameters are the longitudinal desired speed

{(v}_{0})

, time headway (TH), minimum gap

({d g}_{0})

, acceleration coefficients

{(a}_{a}), a n d d e c e l e r a t i o n

(

a_{b})

, respectively.

These IDM parameters were adopted from Kurtc [38] based on the car-following trajectories in highD. The ego vehicle had the desired speed as its initial longitudinal speed

v_{e g o}

, and the desired speed of the surroundings was set to 36.1 m·s⁻¹ (130 km·h⁻¹). The two vehicles shared the same values for the remaining IDM parameters: TH = 0.8 s,

{d g}_{0} = 6.0

a_{a}

= 1.0 m·s⁻²,

a_{ab}

= 1.0 m·s⁻². For the lateral cut-in behaviors, we adopted the polynomial curves according to Mullakkal-Babu et al. [37], which can provide lateral accelerations with smooth trajectories.

Given these cut-in scenario settings, a safety–critical cut-in event was created and a crash occurred at approximately t = 5.0 s. As shown in Fig. 8, the space state probabilities of the stochastic FRS were more concentrated using PSRS-3β and PSRS-5β. The future position prediction accuracies at this time were 31.81%, 33.4%, 36.73%, and 37.39% using HSRS, PSRS, PSRS-3β, and PSRS-5β, respectively. The belief dynamic updates for the simulated cut-in event are shown in Fig. 9. By employing PSRS-3β in Fig. 9(a), the belief value of the lowest β shortly converged to 1 in 2 s. As shown in Fig. 9(b), the acceleration prediction accuracy based on confidence levels β = 1/3 and β = 1/2 decreased to the same low value at t = 2.4 s. The difference in acceleration prediction accuracy became evident between β = 1/3 and β = 1/2, resulting in a belief value convergence for β = 1/3 later. Different confidence level settings and changes in the dynamic belief value eventually resulted in a better performance of PSRS-5β compared with PSRS-3β.

The collision detection results are shown in Fig. 10. In the beginning (from 0 to 1.2 s), the four FRS-based approaches all predicted a small collision probability because the surrounding vehicle does not start a lane change until t = 1 s. Subsequently, significant differences in collision estimation were observed for the four approaches. For instance, when the estimated collision probability using PSRS-5β was close to 0.20 at time t = 2.4 s, PSRS without infusing confidence awareness predicted the collision probability as 0.10, and HSRS had a predicted collision probability of approximately 0.05. This indicates that the stochastic FRS using confidence-aware prediction is agile and effective in identifying potential collisions in a risky cut-in event.

We also simulated a safety–critical cut-in event that did not result in a crash by changing the initial longitudinal speed of the surrounding vehicle to 27 m·s⁻¹. The increased longitudinal velocity of the cut-in vehicle resulted in a larger longitudinal gap when it entered the target lane, resulting in a safety–critical but collision-free cut-in event. During the cut-in process, the belief dynamic changes shown in Fig. 11 had a very similar trend to that in the cut-in crash event because the two events had the same parameter settings, except for the longitudinal speed difference. Fr the predicted collision probability shown in Fig. 12, the heuristic-based reachable set approach HSRS predicted the highest collision probability compared with the proposed prediction-based approach PSRS. Specifically, HSRS reached a maximum collision probability greater than 0.4 at t = 4 s, which corresponded to the time when the surrounding cut-in vehicle crossed the lane marker. This was because HSRS could not accurately predict the future positions of the surrounding vehicles and incorrectly estimated the high collision probability between two vehicles. The simplified heuristic approach, HSRS, also calculated a collision probability of 0.07 when the surrounding vehicle completed the cut-in process and became the leader of the ego vehicle in the target lane. Among the three PSRS approaches with different confidence awareness coefficients, we observed that the maximum collision probability was obtained at the critical time t = 4 s. Without the infusion of confidence awareness, PSRS calculated a collision probability of 0.16, whereas PSRS-3β/-5β obtained a smaller collision probability 0.07/0.03, owing to a more concentrated stochastic FRS. Given that the simulated cut-in event was safety–critical but collision-free, the PSRS approach accurately captures the potential collision risk with a smaller collision probability than the HSRS. In particular, the estimated collision probability by PSRS-5β was always below the predefined threshold of 0.05, indicating that the confidence-infused prediction approach PSRS-5β could not only accurately identify risky cut-in crash but also avoid false–positive results in a safety–critical but collision-free event.

Under the same simulated cut-in scenario settings, we varied the initial longitudinal speeds of the two vehicles from 20 to 35 m·s⁻¹ in increments of 1 m·s⁻¹, resulting in 16 × 16 = 256 cut-in events. Based on the employed longitudinal and lateral driving behavior models, 33 crashes were identified when 4 ≤

v_{e g o} - v_{s}

≤ 6 m·s⁻¹. To compare the collision detection performance, we statistically analyzed the simulated 21 cut-in events in which a crash occurred. The analysis results are shown in Fig. 13, where the timeliness value represents the average time required to crash when the collision probability reaches a threshold. Regardless of the selection of the collision probability threshold, our proposed approaches infusing confidence awareness (i.e., PSRS-5β/PSRS-3β) achieved a larger timeliness value, indicating a more adequate reaction time to potential crashes. The selection of a suitable collision probability threshold can vary in different scenarios, which we leave for future research.

5.4. Integrated collision–detection framework

We propose an integrated collision–detection framework based on BRS and stochastic FRS. First, we provide the comparative results between the two reachable set techniques: BRS and FRS. This explains the selection of the BRS rather than the FRS to formally check driving safety in the first step of the framework.

As shown in Fig. 14, we set the longitudinal speed to 30 and 28 m·s⁻¹ for the ego and surrounding vehicles, respectively, and illustrate comparative results using both the BRS and FRS as an example. The BRS was directly employed to identify unsafe areas when its cached state value function was less than zero. To obtain the unsafe area identified by the FRS, we first enumerated the initial relative surrounding vehicle positions and checked whether the two vehicles could collide using the FRS with a prediction horizon. We then enclosed all the relative positions that could result in a crash and identified the enclosed area in the blue line as unsafe. Note that we limited the unsafe area between −3.75 to 3.75 m in the lateral direction because we were required to address only potential risky interactions between adjacent lanes. Similarly, we did not check relative longitudinal positions behind the ego vehicle. As shown in Fig. 14, the FRS identified a larger unsafe area than the BRS. This was reasonable because the BRS further considered the ego reaction to the surrounding vehicle, resulting in a smaller unsafe area. Specifically, the identified unsafe areas were symmetric (Fig. 14(a)) because the lateral speed and relative heading angles of the initial vehicle states were both zero. In contrast, the surrounding vehicle had an initial lateral speed and a relative heading angle, as shown in Fig. 14(b). Subsequently, based on the FRS, we observed that the area with a lateral position between −3.00 to −3.75 m was now safe owing to the lateral speed of the surrounding vehicle. The BRS had an asymmetric shape owing to the lateral speed, and the area with a negative lateral relative position became safer. This example verifies that the BRS is less conservative than the FRS because the unsafe area identified by the BRS is smaller than that identified by the FRS, and the BRS-identified unsafe area is a subset of the FRS-identified area. This observation is consistent with the definitions of BRS and FRS.

We then provided two specific cut-in events to validate the proposed collision–detection framework. The cut-in events were selected from the simulated trajectories described in Section 5.3.2. During the first cut-in event, the initial longitudinal speeds of the ego and its surroundings were 30 and 28 m·s⁻¹, respectively. This led to non-risky and safe interactions between the two vehicles. As shown in Fig. 15, during a lane change, the integrated framework ensured safety because the relative positions (Fig. 15(a) and (b)) were not inside the unsafe area identified by the BRS. Thus, the establishment of the stochastic FRS was not activated, and the estimated collision probability always remained zero (Fig. 15(c)).

In the second cut-in event, the initial longitudinal speeds of the ego and its surroundings were 30 and 25 m·s⁻¹, respectively, resulting in a collision at approximately t = 5 s, as shown in Fig. 16. When we used the proposed collision–detection framework for this safety–critical event, safety could be ensured at the beginning (green squares in Fig. 16). This was because the surrounding vehicle began the lane-change maneuver at t = 1.0 s, and the driving risk can only be captured until the surrounding vehicle had a clear lane-change intention. Thus, at t = 2.0 s, the proposed framework could no longer ensure driving safety (Fig. 16(c), where the surrounding vehicle was inside the BRS unsafe area of the ego), and a stochastic FRS was established to accurately estimate the collision probability. However, the obtained collision probability was below a predefined threshold of 0.05 at t = 2.0 s, which is marked in blue. Thereafter, the framework provided an estimated collision probability above the threshold until a collision occurred at t = 5.0 s, where the estimated collision gradually increased to 1 (Fig. 16(f)). In summary, the proposed framework can reasonably ensure safety at the beginning and effectively identify potential crashes, owing to the established stochastic FRS.

In terms of computational efficiency, the proposed framework is promising for real-time risk assessment applications, and combining it with BRS can result in further time savings. This is because the BRS cached in a look-up table can quickly identify whether a vehicle is theoretically safe, requiring only a few milliseconds, whereas the stochastic FRS risk assessment can take between 30 and 50 ms depending on the experimental settings. Because most highway driving scenarios do not involve safety–critical vehicle interactions [34], the BRS can prevent unnecessary computation time for constructing a stochastic FRS if safety is ensured. Note that real-time capability is achieved based on an advanced GPU enabling 2048 × 28 parallel computations with a power consumption of up to 1.5 kW·h [39]. If the GPU operates continuously for stochastic FRS construction, it can consume 10% of the entire power consumption of an electric vehicle. However, naturalistic driving data indicate that vehicle interactions are not critical in most instances [34]. This implies that safety can be ensured by BRS in most scenarios. Consequently, the integrated risk assessment framework can benefit from a significant reduction in power consumption by not using GPU computation in most instances.

6. Conclusions

We developed a reachability-based framework for collision detection during highway driving. Inspired by two different RA approaches, a cached BRS is first employed to formally verify whether the current interaction safety can be theoretically ensured. Otherwise, a prediction-based confidence-aware stochastic FRS is calculated online at each predicted time step for collision probability estimation. If the estimated collision probability exceeds a predefined threshold, the ego vehicle can execute brakes or swerve to avoid potential crashes. Thus, the proposed framework can ensure risk-free interactions in non-risky events and provide accurate collision estimation for safety–critical events.

Using both naturalistic driving data and simulated safety–critical cut-in events, we conducted extensive experiments to validate the performance of the proposed acceleration prediction model, which effectively considered interactions between vehicles for constructing a stochastic FRS. Furthermore, we analyzed how infusing confidence beliefs can improve acceleration prediction accuracy. Simulation results showed that the proposed FRS-based approach PSRS-5β with five confidence levels can adjust complicated highway scenarios, resulting in more agile collision–detection results. The integrated framework was tested for both risky and non-risky events. We demonstrated that the proposed framework cannot only accurately identify potential collision events but also has the potential to avoid false–positive results in safety–critical but collision-free events.

The proposed approach assumes the use of an advanced GPU for efficient parallel computing in the stochastic FRS. Further efforts are required to implement hardware with lower power consumption for real-world testing. The developed acceleration predictor was trained on the highD dataset, which contains only highway trajectories. We can only apply our approach to diverse scenarios such as urban and country-road environments by training the predictor using a more diverse dataset. Future research can integrate risk assessment to enable safer and more efficient motion planning and employ the proposed risk assessment framework for an actual vehicle.

Acknowledgments

This research was supported by the proactive SAFEty systems and tools for a constantly UPgrading road environment (SAFE-UP) project. SAFE-UP has received funding from the European Union’s Horizon 2020 Research and Innovation Program (861570).

Compliance with ethics guidelines

Xinwei Wang, Zirui Li, Javier Alonso-Mora, and Meng Wang declare that they have no conflict of interest or financial conflicts to disclose.

References

Publishing order | Descend order by publishing year | Descend order by cited within

[1]	N. Kalra, S.M. Paddock. Driving to safety: how many miles of driving would it take to demonstrate autonomous vehicle reliability?. Transp Res Part A Policy Pract, 94 (2016), pp. 182-193

[2]	S. Lefèvre, D. Vasquez, C. Laugier. A survey on motion prediction and risk assessment for intelligent vehicles. ROBOMECH J, 1 (1) (2014), pp. 1-14

[3]	A. Mukhtar, L. Xia, T.B. Tang. Vehicle detection techniques for collision avoidance systems: a review. IEEE T Intell Transp, 16 (5) (2015), pp. 2318-2338

[4]	C. Pek, S. Manzinger, M. Koschi, M. Althoff. Using online verification to prevent autonomous vehicles from causing accidents. Nat Mach Intell, 2 (9) (2020), pp. 518-528

[5]	P. Falcone, M. Ali, J. Sjoberg. Predictive threat assessment via reachability analysis and set invariance theory. IEEE T Intell Transp, 12 (4) (2011), pp. 1352-1361

[6]	M. Althoff, J.M. Dolan. Online verification of automated road vehicles using reachability analysis. IEEE Trans Robot, 30 (4) (2014), pp. 903-918

[7]	M. Althoff, G. Frehse, A. Girard. Set propagation techniques for reachability analysis. Annu Rev Contr Robot, 4 (1) (2021), pp. 369-395

[8]	K. Leung, E. Schmerling, M. Zhang, M. Chen, J. Talbot, J.C. Gerdes, et al.. On infusing reachability-based safety assurance within planning frameworks for human-robot vehicle interactions. Int J Robot Res, 39 (10-11) (2020), pp. 1326-1345

[9]	M.P. Chapman, R. Bonalli, K.M. Smith, I. Yang, M. Pavone, C.J. Tomlin. Risk-sensitive safety analysis using conditional value-at-risk. IEEE T Automat Contr, 67 (12) (2021), pp. 6521-6536

[10]	D.N. Lee. A theory of visual control of braking based on information about time-to-collision. Perception, 5 (4) (1976), pp. 437-459

[11]	M.M. Minderhoud, P.H. Bovy. Extended time-to-collision measures for road traffic safety assessment. Accident Anal Prev, 33 (1) (2001), pp. 89-97

[12]	K. Vogel. A comparison of headway and time to collision as safety indicators. Accident Anal Prev, 35 (3) (2003), pp. 427-433

[13]	S. Mammar, S. Glaser, M. Netto. Time to line crossing for lane departure avoidance: a theoretical study and an experimental setting. IEEE T Intell Transp, 7 (2) (2006), pp. 226-241

[14]	N. Saunier, T. Sayed. Probabilistic framework for automated analysis of exposure to road collisions. Transp Res Rec, 2083 (1) (2008), pp. 96-104

[15]	G.A. Davis, J. Hourdos, H. Xiong, I. Chatterjee. Outline for a causal model of traffic conflicts and crashes. Accident Anal Prev, 43 (6) (2011), pp. 1907-1919

[16]	Y. Kuang, X. Qu, S. Wang. A tree-structured crash surrogate measure for freeways. Accident Anal Prev, 77 (2015), pp. 137-148

[17]	F.A. Mullakkal-Babu, M. Wang, X. He, B. van Arem, R. Happee. Probabilistic field approach for motorway driving risk assessment. Transport Res C Emer, 118 (2020), Article 102716

[18]	D. Katare, M. El-Sharkawy. Embedded system enabled vehicle collision detection: an ANN classifier. Proceeding of 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC); 2019 Jan 7-9; Las Vegas, USA, IEEE, Washington, DC (2019), pp. 0284-289

[19]

K. Saleh, M. Hossny, S. Nahavandi. Kangaroo vehicle collision detection using deep semantic segmentation convolutional neural network. Proceeding of 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA); 2016 Nov 30-Dec 2; Gold Coast, QLD, Australia, IEEE, Washington, DC (2016), pp. 1-7

[20]	C. Laugier, I.E. Paromtchik, M. Perrollaz, M.Y. Yong, J. Yoder, C. Tay, et al.. Probabilistic analysis of dynamic scenes and collision risks assessment to improve driving safety. IEEE Intel Transp SY, 3 (4) (2011), pp. 4-19

[21]	S. Annell, A. Gratner, L. Svensson. Probabilistic collision estimation system for autonomous vehicles. Proceeding of 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC); 2016 Nov; Rio de Janeiro, Brazil, IEEE, Washington, DC (2016), pp. 473-478

[22]	J. Kim, D. Kum. Collision risk assessment algorithm via lane-based probabilistic motion prediction of surrounding vehicles. IEEE T Intell Transp, 19 (9) (2018), pp. 2965-2976

[23]	C. Pek, P. Zahn, M. Althoff. Verifying the safety of lane change maneuvers of self-driving vehicles based on formalized traffic rules. Proceeding of 2017 IEEE Intelligent Vehicles Symposium (IV); 2017 Jun 11-14;Las Vegas, NV, USA, IEEE, Washington, DC (2017)

[24]	J. Törnblom, S. Nadjm-Tehrani. An abstraction-refinement approach to formal verification of tree ensembles. J. Guiochet, S. Tonetta, E. Schoitsch, M. Roy, F. Bitsch (Eds.), SAFECOMP 2019 Workshops, ASSURE, DECSoS, SASSUR, STRIVE, and WAISE; 2019 Sep 10; Turku, Finland, Springer, Berlin (2019)

[25]	M. Althoff, O. Stursberg, M. Buss. Model-based probabilistic collision detection in autonomous driving. IEEE T Intell Transp, 10 (2) (2009), pp. 299-310

[26]	S. Bansal, M. Chen, S. Herbert, C.J. Tomlin. Hamilton-Jacobi reachability: a brief overview and recent advances. 2017 IEEE 56th Annual Conference on Decision and Control (CDC); Proceeding of 2017 Dec 12-15; Melbourne, VIC, Australia, IEEE, Washington, DC (2017)

[27]	D. Fridovich-Keil, A. Bajcsy, J.F. Fisac, S.L. Herbert, S. Wang, A.D. Dragan, et al.. Confidence-aware motion prediction for real-time collision avoidance¹. Int J Robot Res, 39 (2-3) (2020), pp. 250-265

[28]	X. Wang, Z. Li, J. Alonso-Mora, M. Wang. Prediction-based reachability analysis for collision risk assessment on highways. Proceeding of 2022 IEEE Intelligent Vehicles Symposium (IV); 2022 Jun 5-9; Aachen, Germany, IEEE, Washington, DC (2022), pp. 504-510

[29]	N. Deo, M.M. Trivedi. Convolutional social pooling for vehicle trajectory prediction. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops; 2018 Jun 18-23; Salt Lake, UT, USA, IEEE, Washington, DC (2018)

[30]	X. Wang, J. Alonso-Mora, M. Wang. Probabilistic risk metric for highway driving leveraging multi-modal trajectory predictions. IEEE T Intell Transp, 23 (10) (2022), pp. 19399-19412

[31]	M. Zhou, X. Qu, X. Li. A recurrent neural network based microscopic car following model to predict traffic oscillation. Transport Res C Emer, 84 (2017), pp. 245-264

[32]	A. Li, L. Sun, W. Zhan, M. Tomizuka, M. Chen. Prediction-based reachability for collision avoidance in autonomous driving. Procceding of 2021 IEEE International Conference on Robotics and Automation (ICRA); 2021 May 30-Jun 5; Xi’an, China, IEEE, Washington, DC (2021)

[33]	J. Su, P.A. Beling, R. Guo, K. Han. Graph convolution networks for probabilistic modeling of driving acceleration. Proceeding of 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC); 2020 Sep 20-23; Rhodes, Greece, IEEE, Washington, DC (2020)

[34]

R. Krajewski, J. Bock, L. Kloeker, L. Eckstein. The highD dataset: a drone dataset of naturalistic vehicle trajectories on German highways for validation of highly automated driving systems. Proceeding of 2018 21st International Conference on Intelligent Transportation Systems (ITSC); 2022 Sep 18-Oct 12; Macao, China, IEEE, Washington, DC (2018)

[35]	D. Turner, D. Andresen, K. Hutson, A. Tygart. Application performance on the newest processors and GPUs. Proceedings of the Practice and Experience on Advanced Research Computing; 2018 Jul 22-26; Pittsburgh, PA, USA, Association for Computing Machinery, New York City (2018)

[36]	C. Goutte, E. Gaussier. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. Advances in Information Retrieval: 27th European Conference on IR Research, ECIR 2005; 2005 Mar 21-23; Santiago de Compostela, Spain, Springer, Berlin (2005)

[37]	F.A. Mullakkal-Babu, M. Wang, B. van Arem, B. Shyrokau, R. Happee. A hybrid submicroscopic-microscopic traffic flow simulation framework. IEEE T Intell Transp, 22 (6) (2020), pp. 3430-3443