Pedestrian recognition using automotive radar sensors

. The application of modern series production automotive radar sensors to pedestrian recognition is an important topic in research on future driver assistance systems. The aim of this paper is to understand the potential and limits of such sensors in pedestrian recognition. This knowledge could be used to develop next generation radar sensors with improved pedestrian recognition capabilities. A new raw radar data signal processing algorithm is proposed that allows deep insights into the object classiﬁcation process. The impact of raw radar data properties can be directly observed in every layer of the classiﬁcation system by avoiding machine learning and tracking. This gives information on the limiting factors of raw radar data in terms of classiﬁcation decision making. To accomplish the very challenging distinction between pedestrians and static objects, ﬁve signiﬁ-cant and stable object features from the spatial distribution and Doppler information are found. Experimental results with data from a 77 GHz automotive radar sensor show that over 95 % of pedestrians can be classiﬁed correctly under optimal conditions, which is compareable to modern machine learning systems. The impact of the pedestrian’s direction of movement, occlusion, antenna beam elevation angle, linear vehicle movement, and other factors are investigated and discussed. The results show that under real life conditions, radar only based pedestrian recognition is limited due to in-sufﬁcient Doppler frequency and spatial resolution as well as antenna side lobe effects.


Introduction
The use of radar sensors in automotive pedestrian recognition systems is of special interest since radar sensors are less influenced by environmental conditions (e.g.fog, rain, etc.) as other systems like video cameras (Wenger, 2007).Moreover, high resolution radar sensors are available in many modern vehicles as a part of Adaptive Cruise Control (ACC) systems.Abstract.The application of modern series production automotive radar sensors to pedestrian recognition is an important topic in research on future driver assistance systems.The aim of this paper is to understand the potential and limits of such sensors in pedestrian recognition.This knowledge could be used to develop next generation radar sensors with improved pedestrian recognition capabilities.A new raw radar data signal processing algorithm is proposed that allows deep insights into the object classification process.The impact of raw radar data properties can be directly observed 10 in every layer of the classification system by avoiding machine learning and tracking.This gives information on the limiting factors of raw radar data in terms of classification decision making.To accomplish the very challenging distinction between pedestrians and static objects, five signifi-15 cant and stable object features from the spatial distribution and Doppler information are found.Experimental results with data from a 77 GHz automotive radar sensor show that over 95 % of pedestrians can be classified correctly under optimal conditions, which is compareable to modern machine

Introduction
The use of radar sensors in automotive pedestrian recognition 30 systems is of special interest since radar sensors are less influenced by environmental conditions (e.g.fog, rain, etc.) as Correspondence to: A. Bartsch (armin.bartsch@bmw.de)Fig. 1.A common recognition system with no backtracking possibility is shown on the left.On the right, the new recognition method with possible backtracking of classification errors is shown.
other systems like video cameras (Wenger, 2007).Moreover, high resolution radar sensors are available in many modern vehicles as a part of Adaptive Cruise Control (ACC) systems.

35
If ACC systems and pedestrian recognition systems could use the same radar sensor, hardware costs for producing these systems are kept to a minimum.In this paper, the potential of a modern series production automotive radar sensor, designed for ACC systems, for pedestrian recognition is ex-40 plored.In particular, the limits of the radar sensor regarding decision making in pedestrian classification are investigated to see what future developments of automotive radar sensors are necessary to improve pedestrian recognition systems.
A radar based pedestrian recognition system consists of 45 two main components, a radar sensor and a signal processing unit, i.e. radar raw data preprocessing combined with a classification algorithm.In this paper, a radar signal processing unit is developed that allows the investigation of the potential Fig. 1.A common recognition system with no backtracking possibility is shown on the left.On the right, the new recognition method with possible backtracking of classification errors is shown.
If ACC systems and pedestrian recognition systems could use the same radar sensor, hardware costs for producing these systems are kept to a minimum.In this paper, the potential of a modern series production automotive radar sensor, designed for ACC systems, for pedestrian recognition is explored.In particular, the limits of the radar sensor regarding decision making in pedestrian classification are investigated to see what future developments of automotive radar sensors are necessary to improve pedestrian recognition systems.
A radar based pedestrian recognition system consists of two main components, a radar sensor and a signal processing unit, i.e. radar raw data preprocessing combined with a classification algorithm.In this paper, a radar signal processing unit is developed that allows the investigation of the potential and limits of the radar sensor in an online system.Thus, in case of a classification error, it can be examined if the sensor data was not sufficient to be able to choose the correct object Published by Copernicus Publications on behalf of the URSI Landesausschuss in der Bundesrepublik Deutschland e.V.
class or if the classification algorithm could not squeeze the necessary information out of the sensor's data because of a non-optimal classification algorithm.Misleading sensor raw data of an object is statistically not distinguishable from data of an object belonging to a different class.This could be data from a slowly moving pedestrian for example, that is almost identical to raw data from a small static object like a traffic sign because of quantization errors.
But classification decisions made by machine learning algorithms are non-transparent.Thus, it is not possible to examine why exactly the algorithm decided for the classification result in every case.The significance of the different object features, extracted from the sensor's data, was learned while processing a training data set.This curcial information is buried deep in the algorithm and cannot be easily obtained.Hence, machine learning does not allow for consistent backtracking from classification decisions to the critical information in the radar raw data that caused the decision (see Fig. 1).Human gait models (e.g., Nanzer, 2009, Kim, 2008, Ritter, 2007, Hornsteiner, 2008), which explain micro Doppler signatures, are limited to sensors with very high Doppler frequency resolution and need radar data over a longer, continous period of time instead of single frames.
That is why a different approach in pedestrian recognition without using machine learning and complex object features is needed.In this paper, a knowledge based pedestrian recognition system, consisting of simplified components, is shown that allows transparent classification decisions.The selection, significance testing, weighting of object features and classification algorithm details were found empirically.
A focus will be on boundary and surrounding conditions and their impact on radar raw data.The pedestrian's direction of movement, occlusion, radar elevation angle and other factors will be investigated to see how real life scenarios affect the recognition results in comparison to recognition under optimal conditions.

The radar sensor's raw data
In this paper, a 77 GHz band automotive scanning radar sensor is used.The sensor is designed for series production ACC systems.Table 1 shows some more detailed technical characteristics.Every 66 ms the sensor measures the received signal strength values and Doppler frequency shifts for every spatial resolution cell.A resolution cell is 1 • by 0.25 m.
The Doppler frequency shift is proportional to the radial component of the relative velocity between the sensor and the reflecting object and, therefore, the sensor can easily con- In every measurement cycle, one data frame consisting of two matrices is created.The so called "intensity image" contains the received signal strength values for all spatial resolution cells and the "frequency image" contains the radial relative velocities of reflection centers in every resolution cell.Figure 2 shows some example radar raw data of a pedestrian.These data matrices constitute the low-level radar raw data used for pedestrian recognition.

The pedestrian recognition algorithm
In this chapter, a novel method for pedestrian recognition avoiding machine learning and tracking is developed.The main concern in this step is to keep the signal processing from radar raw data to the classification result as simple and transparent as possible.This guarantees easy backtracking from classification decisions to the radar raw data.An even more powerful advantage of simple signal processing is that object features can be tested for their significance in classification by hand with low effort.
Tracking objects in radar raw data for better classification was discarded to benefit from single frame processing.For driver assistance systems, it is most essential to achieve minimum latency between appearance of a pedestrian in the raw radar data and the output of the correct classification decision.Tracking would introduce such latency and since radar based pedestrian recognition will be used together with other sensor systems in practice, tracking is more benefitial on a higher application level.based pedestrian recognition will be used together with other sensor systems in practice, tracking is more benefitial on a higher application level.Furthermore, this paper highlights the differentiation between static objects and pedestrians.This is challenging because walking pedestrians have very low speed compared to the sensor's velocity measurement resolution of 0.77 m/s.Other objects in typical traffic scenarios, e.g.cars, exhibit significant higher velocities and can be distinguished from pedestrians with less effort.

Raw data preprocessing
The signal processing chain starts with noise reduction and segmentation in the intensity image.Because simple lowpass filtering would blur the critical border between objects and surrounding, a combined bit-depth reduction and thresholding approach was used.The segmentation algorithm was developed in (Freund, 2007) and creates objects, i.e. sets of spatial neighboured sensor data that is likely from one physical object.This segmentation step has big influence on the classification result because it decides where the border is drawn between objects and the noisy surrounding.In the border area of objects, the frequency image contains artefacts that can extensively bias the classification algorithm.If the border between objects and the surrounding is drawn too close to the reflection center of objects, not all related resolution cells get attached to the object and important information about the object gets lost.The optimum in this trade-off is highly dependent on background noise in the intensity image.
In the frequency image, the ego-speed of the measurement vehicle v ego has to be compensated.If the vehicle moves 165 with v ego , a static object in front of the vehicle has a relative velocity of −v ego , which is measured by the sensor.Experiments showed that the vehicle's ego-speed signal, generated by the vehicle's dynamic stability control system, is too noisy for satisfactory compensation.To eliminate this error, an ob-170 ject which is known to be static is needed in the radar raw data, e.g. a tree or a road barrier that was identified by another sensor system like a camera.Using this information, the frequency image can be recompensated with sufficient accuracy for the following recognition process.In this pa-175 per, the information about which objects in the radar data are static was provided by hand to allow experiments with v ego = 0.

Feature extraction
In this section we propose a set of five object features that 180 change reliably and significantly when the object, represented by the segmented raw radar data, is a pedestrian or a static object.Since pedestrians are radar point targets (Yamada, 2005), their shape in the intensity image is the same as the one of any small reflective object.Yamada also shows 185 that the strength of a radar signal, reflected by a pedestrian, and thus the intensity in the intensity image, is highly fluctuative (about 20 dB).Hence, with information from the intensity image only, it is impossible to distinguish between pedestrians and small arbitrary objects.Nevertheless, in this 190 paper two object features from the intensity image, a size and a shape factor, are calculated to exclude big or elongate objects from potential pedestrians.
Figure 3 reveals that in a resolution cell the reflected power value from the intensity image and the radial relative veloc-195 ity value from the frequency image are statistically uncorrelated.The graphs depict the frequency distributions of resolution cells with certain reflection intensity and radial relative velocity for different types of objects.The statistical uncorrelation means that the mean and variance of a resolu-200 tion cell's radial velocity value are not correlated to its intensity value.Therefore, principal component analysis (PCA) or similar methods to transform possibly correlated data sets into uncorrelated data sets are obsolete.Another interesting point is that the relative velocity information is not correlated 205 with the distance from the object center since resolution cells with high intensity are located at the center of an object.
While the variance of the intensity values is almost equal for moving pedestrians and static objects, the variance of the radial velocity values varies significantly.The classifica-210 tion algorithm distinguishes pedestrians from static objects mainly using the higher variance of the relative radial velocity components in the frequency image data of pedestrians (see Sect. 3.2.3).Three object features from the frequency image data are calculated for every object, giving in combi-215 nation with two object features from the intensity image an Furthermore, this paper highlights the differentiation between static objects and pedestrians.This is challenging because walking pedestrians have very low speed compared to the sensor's velocity measurement resolution of 0.77 m s −1 .Other objects in typical traffic scenarios, e.g.cars, exhibit significant higher velocities and can be distinguished from pedestrians with less effort.

Raw data preprocessing
The signal processing chain starts with noise reduction and segmentation in the intensity image.Because simple lowpass filtering would blur the critical border between objects and surrounding, a combined bit-depth reduction and thresholding approach was used.The segmentation algorithm was developed in (Freund, 2007) and creates objects, i.e. sets of spatial neighboured sensor data that is likely from one physical object.This segmentation step has big influence on the classification result because it decides where the border is drawn between objects and the noisy surrounding.In the border area of objects, the frequency image contains artefacts that can extensively bias the classification algorithm.If the border between objects and the surrounding is drawn too close to the reflection center of objects, not all related resolution cells get attached to the object and important information about the object gets lost.The optimum in this trade-off is highly dependent on background noise in the intensity image.
In the frequency image, the ego-speed of the measurement vehicle v ego has to be compensated.If the vehicle moves with v ego , a static object in front of the vehicle has a relative veloc-ity of −v ego , which is measured by the sensor.Experiments showed that the vehicle's ego-speed signal, generated by the vehicle's dynamic stability control system, is too noisy for satisfactory compensation.To eliminate this error, an object which is known to be static is needed in the radar raw data, e.g. a tree or a road barrier that was identified by another sensor system like a camera.Using this information, the frequency image can be recompensated with sufficient accuracy for the following recognition process.In this paper, the information about which objects in the radar data are static was provided by hand to allow experiments with v ego = 0.

Feature extraction
In this section we propose a set of five object features that change reliably and significantly when the object, represented by the segmented raw radar data, is a pedestrian or a static object.Since pedestrians are radar point targets (Yamada, 2005), their shape in the intensity image is the same as the one of any small reflective object.Yamada also shows that the strength of a radar signal, reflected by a pedestrian, and thus the intensity in the intensity image, is highly fluctuative (about 20 dB).Hence, with information from the intensity image only, it is impossible to distinguish between pedestrians and small arbitrary objects.Nevertheless, in this paper two object features from the intensity image, a size and a shape factor, are calculated to exclude big or elongate objects from potential pedestrians.
Figure 3 reveals that in a resolution cell the reflected power value from the intensity image and the radial relative velocity value from the frequency image are statistically uncorrelated.The graphs depict the frequency distributions of resolution cells with certain reflection intensity and radial relative velocity for different types of objects.The statistical uncorrelation means that the mean and variance of a resolution cell's radial velocity value are not correlated to its intensity value.Therefore, principal component analysis (PCA) or similar methods to transform possibly correlated data sets into uncorrelated data sets are obsolete.Another interesting point is that the relative velocity information is not correlated with the distance from the object center since resolution cells with high intensity are located at the center of an object.
While the variance of the intensity values is almost equal for moving pedestrians and static objects, the variance of the radial velocity values varies significantly.The classification algorithm distinguishes pedestrians from static objects mainly using the higher variance of the relative radial velocity components in the frequency image data of pedestrians (see Sect. 3.2.3).Three object features from the frequency image data are calculated for every object, giving in combination with two object features from the intensity image an only five dimensional feature space which is sufficient for proper classification.only five dimensional feature space which is sufficient for proper classification.

The size of an object
The size of an object is a measure that is proportional to an object's mean radius and simply the square root of the distance-compensated number of resolution cells assigned to the object.Distance compensation is necessary because with increasing distance r from the sensor, the received power P D decreases with 1/r 4 and the width of the spatial resolution cells get larger since they are always 1 • in azimuth.With n as the distance-compensated number of resolution cells of the object, m 1 = √ n is the first object feature.

The shape of an object
The shape of a point target in the radar raw data is determined by the sensor's antenna pattern.In the intensity image, a small object has an elliptical shape with the longer axis in azimuthal direction and an axis ratio of about D/d = 1.8.
To obtain an object's shape feature, the difference of the object's shape compared to an ideal ellipse is calculated.An ideal ellipse of equal area and half-axes a, b is placed over the object's center (r c ,φ c ) (see Fig. 4).In this coordinate system, the ith resolution cell of an object with n cells has a local coordinate (x i , y i ).A distance measure for a cell to the ellipse's border is given by The size-normalized cumulative shape error gives the second Fig. 4.An illustration of the sensor coordinate system, an object with center at (r c , φ c ) and a local coordinate system for calculating the object's shape factor using (x , y ) coordinates.The maximum beam steering angle φ is the border of the azimuthal measurement range.

245
The Doppler spectrum (DS) is the histogram of an object's Doppler shift information, extracted from all resolution cells assigned to the object.It is a density function that tells how prominent different relative velocities are in the object.
The relative radial velocity axis v is divided into small in-250 tervals.Resolution cells with radial relative velocity v rr increase the counter for the corresponding intervals To get a densitiy function D Doppler , normalization is necessary such that D Doppler dv = 1 holds.

The size of an object
The size of an object is a measure that is proportional to an object's mean radius and simply the square root of the distance-compensated number of resolution cells assigned to the object.Distance compensation is necessary because with increasing distance r from the sensor, the received power P D decreases with 1/r 4 and the width of the spatial resolution cells get larger since they are always 1 • in azimuth.With n as the distance-compensated number of resolution cells of the object, m 1 = √ n is the first object feature.

The shape of an object
The shape of a point target in the radar raw data is determined by the sensor's antenna pattern.In the intensity image, a small object has an elliptical shape with the longer axis in azimuthal direction and an axis ratio of about D/d = 1.8.
To obtain an object's shape feature, the difference of the object's shape compared to an ideal ellipse is calculated.An ideal ellipse of equal area and half-axes a, b is placed over the object's center (r c ,φ c ) (see Fig. 4).In this coordinate system, the i-th resolution cell of an object with n cells has a local coordinate (x i , y i ).A distance measure for a cell to the ellipse's border is given by The size-normalized cumulative shape error gives the second object feature m 2 : only five dimensional feature space which is sufficient for proper classification.

The size of an object
The size of an object is a measure that is proportional to 220 an object's mean radius and simply the square root of the distance-compensated number of resolution cells assigned to the object.Distance compensation is necessary because with increasing distance r from the sensor, the received power P D decreases with 1/r 4 and the width of the spatial resolution 225 cells get larger since they are always 1 • in azimuth.With n as the distance-compensated number of resolution cells of the object, m 1 = √ n is the first object feature.

The shape of an object
The shape of a point target in the radar raw data is deter-230 mined by the sensor's antenna pattern.In the intensity image, a small object has an elliptical shape with the longer axis in azimuthal direction and an axis ratio of about D/d = 1.8.
To obtain an object's shape feature, the difference of the object's shape compared to an ideal ellipse is calculated.An 235 ideal ellipse of equal area and half-axes a, b is placed over the object's center (r c ,φ c ) (see Fig. 4).In this coordinate system, the ith resolution cell of an object with n cells has a local coordinate (x i , y i ).A distance measure for a cell to the ellipse's border is given by The size-normalized cumulative shape error gives the second object feature m 2 : Fig. 4.An illustration of the sensor coordinate system, an object with center at (rc, φc) and a local coordinate system for calculating the object's shape factor using (x , y ) coordinates.The maximum beam steering angle φ is the border of the azimuthal measurement range.An illustration of the sensor coordinate system, an object with center at (r c , φ c ) and a local coordinate system for calculating the object's shape factor using (x , y ) coordinates.The maximum beam steering angle φ is the border of the azimuthal measurement range.

Doppler spectrum
The Doppler spectrum (DS) is the histogram of an object's Doppler shift information, extracted from all resolution cells assigned to the object.It is a density function that tells how prominent different relative velocities are in the object.The relative radial velocity axis v is divided into small intervals.Resolution cells with radial relative velocity v rr increase the counter for the corresponding intervals structure are always located at the theoretical frequency bins (if interpolation would not be used), and the local minima right inbetween, independent from the object's speed.Thus, the wavy structure is an artefact caused by the interpolation algorithm in the Doppler frequency measurement inside the radar sensor.
It can be clearly seen in Fig. 5 that a radial moving pedestrian has a broad DS from 0 -2.5 m/s.In contrast, DS of static objects are very narrow since they only cover velocities from 0 -0.4 m/s.
To boil down the shape of a DS to key figures, three intervals The Doppler spectrum density function D Doppler is integrated over these intervals, giving three object features m 3 -m 5 , representing the proportional distribution of radial relative velocities in the object over the three intervals.
Radial velocities greater than 3 m/s are neglected since they are unlikely for pedestrians and thus not the matter in this paper.for the class "Static Object".Both valid classes, P for pedestrians and S for static objects, have a vector valued fuzzy membership function f C : R 5 → [0,1] 5 , C ∈ {P ; S} that maps the feature vector m of an object to a membership value vector A value of µ C,i close to one states a high probability of m i for the corresponding class C while a small value of µ C,i near zero indicates that the value of m i is unlikely for that 300 class.In this way, the membership values µ C,i of the five object features of every object are calculated for both classes P and S. Graphs of the membership functions f C,i are plotted in Fig. 6.The functions f S,1 and f S,2 are not defined because static objects can be of any shape and size.Hence, 305 all values of m 1 and m 2 have the same likelihood for the class S and thus their membership values for class S are suppressed in further calculations by giving them zero weight in the weighting step.
To decide whether class P or S is more appropriate for 310 an object, given fuzzy membership vectors u P and u S , the membership vectors are mapped to real membership values µ P , µ S ∈ [0,1].This is done by a weighted average of the membership vectors' components where w C = [w C,1 ,...,w C,5 ] T is the weighting vector for a class C ∈ {P ;S} and 1 p is a p-dimensional column vector of ones.
Experiments have shown that constant weighting vectors w C are not sufficient for proper classification.Instead, the 320 combination of the membership values µ C,i must be closer to a logical conjunction.This is achieved by allowing the weight vector w C (u C ) to be dependent on u C , meaning structure of the DS, especially of walking pedestrians, is not caused by movements of the object itself.The maxima of the wavy structure are always located at the theoretical frequency bins (if interpolation would not be used), and the local minima right inbetween, independent from the object's speed.Thus, the wavy structure is an artefact caused by the interpolation algorithm in the Doppler frequency measurement inside the radar sensor.
It can be clearly seen in Fig. 5 that a radial moving pedestrian has a broad DS from 0-2.5 m s −1 .In contrast, DS of static objects are very narrow since they only cover velocities from 0-0.4 m s −1 .
To boil down the shape of a DS to key figures, three intervals 2 for actual values of v 0 -v 2 ).The Doppler spectrum density function D Doppler is integrated over these intervals, giving three object features m 3 -m 5 , representing the proportional distribution of radial relative velocities in the object over the three intervals.
Radial velocities greater than 3 m s −1 are neglected since they are unlikely to be in pedestrians' DS and thus not the matter in this paper.

Classification
The challenge in the classification step is to decide whether an object is a pedestrian or a static object by using the corresponding feature vector m = [m 1 ,...,m 5 ] T only.The decision process is based on the knowledge that some values of these features are more probable if the object is of a certain class C. For example, a static object is very unlikely to have a significant fraction of Doppler information in the [v 2 ,v 3 ] interval.Hence, a value of m 3 close to one is very unlikely for the class "Static Object".Both valid classes, P for pedestrians and S for static objects, have a vector valued fuzzy membership function f C : R 5 → [0,1] 5 , C ∈ {P ; S} that maps the feature vector m of an object to a membership value vector u C = [µ C,1 ,...,µ C,5 ] T for the corresponding class.
A value of µ C,i close to one states a high probability of m i for the corresponding class C while a small value of µ C,i near zero indicates that the value of m i is unlikely for that class.In this way, the membership values µ C,i of the five object features of every object are calculated for both classes P and S. Graphs of the membership functions f C,i are plotted in Fig. 6.The functions f S,1 and f S,2 are not defined because static objects can be of any shape and size.Hence, all values of m 1 and m 2 have the same likelihood for the class S and thus their membership values for class S are suppressed in further calculations by giving them zero weight in the weighting step.
To decide whether class P or S is more appropriate for an object, given fuzzy membership vectors u P and u S , the membership vectors are mapped to real membership values µ P , µ S ∈ [0,1].This is done by a weighted average of the membership vectors' components .6. Plots of membership functions f P,i and f S,i that map values of object features m i to membership values µ P,i and µ S,i for the classes for pedestrians and S for static objects.t low membership values µ C,i are weighted stronger by reasing w C,i exponentionally: e exponential function for increasing the base weight w 0 C,i s chosen empirically.While γ and τ parametrize this lue dependent weighting, ξ C,i is used for compensating a rtain effect when a pedestrian has an unusual Doppler freency distribution.In Fig. 7, a DS of a walking pedestrian is w P,4 will rise exponentially, and the overall probability µ P for this object to be a pedestrian will be low.This conclusion would be wrong because the characteristic Doppler information for a pedestrian is just concentrated in the [v 2 ,v 3 ] inter-340 val rather than evenly distributed over the [v 1 ,v 3 ] interval.In this case, where m 4 has a very low and m 5 a high value, ξ P,4 is chosen small (ξ P,4 ≈ 1/γ) to reverse the value dependent weighting.Analogously, ξ P,5 is chosen if m 5 is very low in value while a significant share of Doppler information is in Since this cross-bonded value dependent weighting affects only w P,4 and w P,5 , all ξ C,i except ξ P,4 and ξ P,5 have a constant value of one.The decision whether an object is classified as a pedestrian or as a static object and, therefore, is assigned to the class 350 P or class S, is made by comparing the united membership values µ P and µ S .If µ P > µ S , the object is assigned to the class P and otherwise to class S. If both membership values µ P and µ S are small and hence the object is unlikely to be a pedestrian as well as a static object, it is declared as an 355 unknown object.This happens almost exclusively in case of segmentation errors.

Experimental results
Because a binary classifier is used and the number of unknown declared objects is negligible, the quality of the 360 pedestrian recognition system can be measured by only two key figures, r TP and r FP .The true positive rate r TP tells what fraction of real pedestrians is correctly classified as Fig. 6.Plots of membership functions f P ,i and f S,i that map values of object features m i to membership values µ P ,i and µ S,i for the classes P for pedestrians and S for static objects.Fig. 6.Plots of membership functions f P,i and f S,i that map values of object features m i to membership values µ P,i and µ S,i for the classes P for pedestrians and S for static objects.that low membership values µ C,i are weighted stronger by increasing w C,i exponentionally: The exponential function for increasing the base weight w 0 C,i was chosen empirically.While γ and τ parametrize this value dependent weighting, ξ C,i is used for compensating a certain effect when a pedestrian has an unusual Doppler fre-330 quency distribution.In Fig. 7, a DS of a walking pedestrian is depicted, obtained from a single data frame.In this case the object does not contain a resolution cell with a relative radial velocity in the [v 1 ,v 2 ] interval.Hence, the object feature m 4 will be zero and since there are some cells in the [v 2 ,v 3 ] in-335 terval, m 5 > 0 will hold.Because m 4 equals zero, its weight w P,4 will rise exponentially, and the overall probability µ P for this object to be a pedestrian will be low.This conclusion would be wrong because the characteristic Doppler information for a pedestrian is just concentrated in the [v 2 ,v 3 ] inter-340 val rather than evenly distributed over the [v 1 ,v 3 ] interval.In this case, where m 4 has a very low and m 5 a high value, ξ P,4 is chosen small (ξ P,4 ≈ 1/γ) to reverse the value dependent weighting.Analogously, ξ P,5 is chosen if m 5 is very low in value while a significant share of Doppler information is in 345 the [v 1 ,v 2 ] interval.Since this cross-bonded value dependent weighting affects only w P,4 and w P,5 , all ξ C,i except ξ P,4 and ξ P,5 have a constant value of one.The decision whether an object is classified as a pedestrian or as a static object and, therefore, is assigned to the class 350 P or class S, is made by comparing the united membership values µ P and µ S .If µ P > µ S , the object is assigned to the class P and otherwise to class S. If both membership values µ P and µ S are small and hence the object is unlikely to be a pedestrian as well as a static object, it is declared as an 355 unknown object.This happens almost exclusively in case of segmentation errors.

Experimental results
Because a binary classifier is used and the number of unknown declared objects is negligible, the quality of the  low membership values µ C,i are weighted stronger by increasing w C,i exponentionally: The exponential function for increasing the base weight w 0 C,i was chosen empirically.While γ and τ parametrize this value dependent weighting, ξ C,i is used for compensating a certain effect when a pedestrian has an unusual Doppler frequency distribution.In Fig. 7, a DS of a walking pedestrian is depicted, obtained from a single data frame.In this case the object does not contain a resolution cell with a relative radial velocity in the [v 1 ,v 2 ] interval.Hence, the object feature m 4 will be zero and since there are some cells in the [v 2 ,v 3 ] interval, m 5 > 0 will hold.Because m 4 equals zero, its weight w P ,4 will rise exponentially, and the overall probability µ P for this object to be a pedestrian will be low.This conclusion would be wrong because the characteristic Doppler information for a pedestrian is just concentrated in the [v 2 ,v 3 ] interval rather than evenly distributed over the [v 1 ,v 3 ] interval.In this case, where m 4 has a very low and m 5 a high value, ξ P ,4 is chosen small (ξ P ,4 ≈ 1/γ ) to reverse the value dependent weighting.Analogously, ξ P ,5 is chosen if m 5 is very low in value while a significant share of Doppler information is in the [v 1 ,v 2 ] interval.Since this cross-bonded value dependent weighting affects only w P ,4 and w P ,5 , all ξ C,i except ξ P ,4 and ξ P ,5 have a constant value of one.
The decision whether an object is classified as a pedestrian or as a static object and, therefore, is assigned to the class P or class S, is made by comparing the united membership values µ P and µ S .If µ P > µ S , the object is assigned to the class P and otherwise to class S. If both membership values µ P and µ S are small and hence the object is unlikely to be a pedestrian as well as a static object, it is declared as an unknown object.This happens almost exclusively in case of segmentation errors.

Experimental results
Because a binary classifier is used and the number of unknown declared objects is negligible, the quality of the pedestrian recognition system can be measured by only two key figures, r TP and r In the investigated range of walking paces from 0.58 m/s to 1.7 m/s, the DS is independent from the pedestrian's walking pace.
The test sample set consisted in every experiment of about 150 to 600 samples if not mentioned otherwise.

Influences on the Doppler spectrum
A walking pedestrian's DS turned out to be very stable in terms of outside influences compared to other tested object features from the frequency image.It is not affected by the height in which the radar beam hits the person, due to different angles of elevation of the antenna beam as long as enough power is reflected for proper segmentation in the intensity image.Furthermore, the DS is fairly independent from the walking pedestrian's pace (see Fig. 8).Walking paces from 0.58 m/s to 1.7 m/s were tested and did not affect the pedestrian's DS because many different velocities within the segmented object occured due to arm and leg swing.Nevertheless, the DS obviously is dependent on the pedestrian's angle of movement.A radar sensor can only measure the radial component v rr of an object's relative velocity v r .A good approximation is to assume the pedestrian centered in front of the sensor, i.e. φ = 0 (see Fig. 9).By using this approximation, the radial velocity component can be simplified to and it approaches zero for angles of movement near ϕ = 90 • .

390
Hence, lateral moving pedestrians have DS very similar to static objects.In Fig. 10, averaged DS of a walking pedestrian are plotted for different angles of movement ϕ.The closer ϕ approaches 90 • , the DS shows increasing similarity to the one of static objects (compare to Fig. 5).Hence, the true positive rate r TP after classification reaches 95.3 % for radially moving pedestrians (ϕ = 0 • ) and drops to 39.5 % for Fig. 9.A walking pedestrian with velocity v r relative to the sensor with angle of movement ϕ has a radial relative velocity component v rr that is measurable by the radar sensor.laterally moving pedestrians (ϕ = 90 • ). Figure 11 exhibits that the classification result is better than 88 % for |ϕ| ≤ 75 • .400 Backtracking from classification decisions to radar raw data, to see whether the sensor data or the classification algorithm is responsible for misclassified data frames, was extensively used in empirically optimizing the classifier.As a result, almost all misclassifications of pedestrians as static 405 objects can now be tracked down to sensor data that is indistinguishable, even by hand, from a static object's raw data.In Fig. 12, the averaged DS of correct and misclassified pedestrians are plotted as well as the averaged DS of static objects for comparison.The DS of the misclassified pedestrians is 410 almost the same as the one of static objects.Thus, in the case of misclassified pedestrians, the sensor does not deliver statistically different data from static objects and hence the Fig. 8. Averaged Doppler spectra of a walking pedestrian for different walking paces.The angle of movement was ϕ = 45 • in all cases.In the investigated range of walking paces from 0.58 m s −1 to 1.7 m s −1 , the DS is independent from the pedestrian's walking pace.
fraction of real pedestrians is correctly classified as such.On the other hand, the false positive rate r FP or false alarm rate tells what fraction of static objects are classified incorrectly as pedestrians.The true negative rate r TN = 1 − r FP and the false negative rate r FN = 1−r TP are implied under the abovementioned assuptions.
The test sample set consisted in every experiment of about 150 to 600 samples if not mentioned otherwise.

Influences on the Doppler spectrum
A walking pedestrian's DS turned out to be very stable in terms of outside influences compared to other tested object features from the frequency image.It is not affected by the height in which the radar beam hits the person, due to different angles of elevation of the antenna beam as long as enough power is reflected for proper segmentation in the intensity image.Furthermore, the DS is fairly independent from the walking pedestrian's pace (see Fig. 8).Walking paces from 0.58 m s −1 to 1.7 m s −1 were tested and did not affect the pedestrian's DS because many different velocities within the segmented object occured due to arm and leg swing.
Nevertheless, the DS obviously is dependent on the pedestrian's angle of movement.A radar sensor can only measure the radial component v rr of an object's relative velocity v r .A good approximation is to assume the pedestrian centered in front of the sensor, i.e. φ = 0 (see Fig. 9).By using this approximation, the radial velocity component can be simplified to In the investigated range of walking paces from 0.58 m/s to 1.7 m/s, the DS is independent from the pedestrian's walking pace.
The test sample set consisted in every experiment of about 150 to 600 samples if not mentioned otherwise.

Influences on the Doppler spectrum
A walking pedestrian's DS turned out to be very stable in terms of outside influences compared to other tested object features from the frequency image.It is not affected by the height in which the radar beam hits the person, due to differ-375 ent angles of elevation of the antenna beam as long as enough power is reflected for proper segmentation in the intensity image.Furthermore, the DS is fairly independent from the walking pedestrian's pace (see Fig. 8).Walking paces from 0.58 m/s to 1.7 m/s were tested and did not affect the pedes-380 trian's DS because many different velocities within the segmented object occured due to arm and leg swing.Nevertheless, the DS obviously is dependent on the pedestrian's angle of movement.A radar sensor can only measure the radial component v rr of an object's relative velocity v r .A good ap-385 proximation is to assume the pedestrian centered in front of the sensor, i.e. φ = 0 (see Fig. 9).By using this approximation, the radial velocity component can be simplified to and it approaches zero for angles of movement near ϕ = 90 • .

390
Hence, lateral moving pedestrians have DS very similar to static objects.In Fig. 10, averaged DS of a walking pedestrian are plotted for different angles of movement ϕ.The closer ϕ approaches 90 • , the DS shows increasing similarity to the one of static objects (compare to Fig. 5).Hence, the 395 true positive rate r TP after classification reaches 95.3 % for radially moving pedestrians (ϕ = 0 • ) and drops to 39.5 % for laterally moving pedestrians (ϕ = 90 • ). Figure 11 exhibits that the classification result is better than 88 % for |ϕ| ≤ 75 • .400 Backtracking from classification decisions to radar raw data, to see whether the sensor data or the classification algorithm is responsible for misclassified data frames, was extensively used in empirically optimizing the classifier.As a result, almost all misclassifications of pedestrians as static 405 objects can now be tracked down to sensor data that is indistinguishable, even by hand, from a static object's raw data.In Fig. 12, the averaged DS of correct and misclassified pedestrians are plotted as well as the averaged DS of static objects for comparison.The DS of the misclassified pedestrians is 410 almost the same as the one of static objects.Thus, in the case of misclassified pedestrians, the sensor does not deliver statistically different data from static objects and hence the In the investigated range of walking paces from 0.58 m/s to 1.7 m/s, the DS is independent from the pedestrian's walking pace.
The test sample set consisted in every experiment of about 150 to 600 samples if not mentioned otherwise.

Influences on the Doppler spectrum
A walking pedestrian's DS turned out to be very stable in terms of outside influences compared to other tested object features from the frequency image.It is not affected by the height in which the radar beam hits the person, due to different angles of elevation of the antenna beam as long as enough power is reflected for proper segmentation in the intensity image.Furthermore, the DS is fairly independent from the walking pedestrian's pace (see Fig. 8).Walking paces from 0.58 m/s to 1.7 m/s were tested and did not affect the pedestrian's DS because many different velocities within the segmented object occured due to arm and leg swing.Nevertheless, the DS obviously is dependent on the pedestrian's angle of movement.A radar sensor can only measure the radial component v rr of an object's relative velocity v r .A good approximation is to assume the pedestrian centered in front of the sensor, i.e. φ = 0 (see Fig. 9).By using this approximation, the radial velocity component can be simplified to and it approaches zero for angles of movement near ϕ = 90 • .
Hence, lateral moving pedestrians have DS very similar to static objects.In Fig. 10, averaged DS of a walking pedestrian are plotted for different angles of movement ϕ.The closer ϕ approaches 90 • , the DS shows increasing similarity to the one of static objects (compare to Fig. 5).Hence, the true positive rate r TP after classification reaches 95.3 % for radially moving pedestrians (ϕ = 0 • ) and drops to 39.5 % for laterally moving pedestrians (ϕ = 90 • ). Figure 11 exhibits that the classification result is better than 88 % for |ϕ| ≤ 75 • .400 Backtracking from classification decisions to radar raw data, to see whether the sensor data or the classification algorithm is responsible for misclassified data frames, was extensively used in empirically optimizing the classifier.As a result, almost all misclassifications of pedestrians as static 405 objects can now be tracked down to sensor data that is indistinguishable, even by hand, from a static object's raw data.In Fig. 12, the averaged DS of correct and misclassified pedestrians are plotted as well as the averaged DS of static objects for comparison.The DS of the misclassified pedestrians is 410 almost the same as the one of static objects.Thus, in the case of misclassified pedestrians, the sensor does not deliver statistically different data from static objects and hence the and it approaches zero for angles of movement near ϕ = 90 • .Hence, lateral moving pedestrians have DS very similar to static objects.In Fig. 10, averaged DS of a walking pedestrian are plotted for different angles of movement ϕ.The closer ϕ approaches 90 • , the DS shows increasing similarity to the one of static objects (compare to Fig. 5).Hence, the true positive rate r TP after classification reaches 95.3 % for radially moving pedestrians (ϕ = 0 • ) and drops to 39.5 % for laterally moving pedestrians (ϕ = 90 • ). Figure 11   developed classification algorithm is optimal within the restrictions of the chosen method for pedestrian recognition.

Occluded pedestrians
In this paper, two types of occlusions were investigated.In the first case, a pedestrian is hidden behind a single parked car and in the second case the pedestrian is in the gap be-420 tween two parked cars (see Fig. 13).In both cases the pedestrian moves laterally while being occluded such that the pedestrian is not in the line of sight to the sensor.The lateral movement is typical for pedestrians who try to cross the street without looking.

425
In the case where the pedestrian is in the shadowed area hidden behind one parked vehicle at the side of the road, the radar waves cannot hit the pedestrian directly.Therefore, the signal reflected by the pedestrian and received by the sensor is very weak.As a result, the pedestrian blurs with the 430 blocking car and often the segmentation algorithm is not able to seperate the two objects.In the classification step the true positive rate drops to r TP = 6.6%, which is only slightly better than the false alarm rate of r FP = 1.4%.
If the pedestrian is moving laterally in a gap between two 435 parked cars at the side of the road (see Fig. 13), the radar wave can propagate in this gap because of multiple reflections.Thus, the pedestrian reflects more power back to the radar sensor and can be segmented properly in the radar raw data if the gap is not smaller than 50 cm.Also, the recogni-440 tion result is much better, as the true poistive rate rises to values up to 29.4 %.This result is remarkable because now the recognition rate is only ten percent less than r TP = 39.5% in the case of a lateral moving pedestrian in line of sight (see Fig. 11).The recognition results for different sizes l of the 445 gap are shown in Tab. 3. Note that the false alarm rate r FP is significantly higher in case of narrow gaps up to 50 cm due to segmentation problems in the blurred occluded areas.

False alarms
False alarms, meaning that static objects are incorrectly clas-450 sified as pedestrians, are consequences of three root causes.Firstly, parasitic antenna side lobe effects can cause significant fractions in the DS of static objects that seem to have a higher velocity than static objects should have.Secondly, objects can now be tracked down to sensor data that is indistinguishable, even by hand, from a static object's raw data.In Fig. 12, the averaged DS of correct and misclassified pedestrians are plotted as well as the averaged DS of static objects for comparison.The DS of the misclassified pedestrians is almost the same as the one of static objects.Thus, in the case of misclassified pedestrians, the sensor does not deliver statistically different data from static objects and hence the developed classification algorithm is optimal within the restrictions of the chosen method for pedestrian recognition.

Occluded pedestrians
In this paper, two types of occlusions were investigated.In the first case, a pedestrian is hidden behind a single parked car and in the second case the pedestrian is in the gap between two parked cars (see Fig. 13).In both cases the pedestrian moves laterally while being occluded such that the pedestrian is not in the line of sight to the sensor.The lateral movement is typical for pedestrians who try to cross the street without looking.
In the case where the pedestrian is in the shadowed area hidden behind one parked vehicle at the side of the road, the radar waves cannot hit the pedestrian directly.Therefore, the signal reflected by the pedestrian and received by the sensor is very weak.As a result, the pedestrian blurs with the blocking car and often the segmentation algorithm is not able to seperate the two objects.In the classification step the true positive rate drops to r TP = 6.6%, which is only slightly better than the false alarm rate of r FP = 1.4%.
If the pedestrian is moving laterally in a gap between two parked cars at the side of the road (see Fig. 13), the radar wave can propagate in this gap because of multiple reflec- developed classification algorithm is optimal within the restrictions of the chosen method for pedestrian recognition.

Occluded pedestrians
In this paper, two types of occlusions were investigated.In the first case, a pedestrian is hidden behind a single parked car and in the second case the pedestrian is in the gap be-420 tween two parked cars (see Fig. 13).In both cases the pedestrian moves laterally while being occluded such that  the pedestrian is not in the line of sight to the sensor.The lateral movement is typical for pedestrians who try to cross the street without looking.

425
In the case where the pedestrian is in the shadowed area hidden behind one parked vehicle at the side of the road, the radar waves cannot hit the pedestrian directly.Therefore, the signal reflected by the pedestrian and received by the sensor is very weak.As a result, the pedestrian blurs with the 430 blocking car and often the segmentation algorithm is not able to seperate the two objects.In the classification step the true positive rate drops to r TP = 6.6%, which is only slightly better than the false alarm rate of r FP = 1.4%.
If the pedestrian is moving laterally in a gap between two 435 parked cars at the side of the road (see Fig. 13), the radar wave can propagate in this gap because of multiple reflec- tions.Thus, the pedestrian reflects more power back to the radar sensor and can be segmented properly in the radar raw data if the gap is not smaller than 50 cm.Also, the recognition result is much better, as the true poistive rate rises to values up to 29.4 %.This result is remarkable because now the recognition rate is only ten percent less than r TP = 39.5% in the case of a lateral moving pedestrian in line of sight (see Fig. 11).The recognition results for different sizes l of the gap are shown in Table 3.Note that the false alarm rate r FP  errors in the ego velocity compensation can offset the DS and finally, in case of errors in the segmentation step, resolution cells from the noisy surroundings of an object can get assigned to the object, resulting in DS fractions with higher velocity than possible for static objects.To see how good the specifity of the pedestrian recognition system is in practice, a real urban traffic scene was chosen (see Fig. 14).The vehicle drives along a road with lots of highly reflective static objects, such as cars, in the measurement range, which is the worst case scenario in terms of false alarms, since highly reflective objects cause high amplitudes of antenna side lobe 465 effects.As a result, in the described traffic scenario, a false alarm rate of r FP = 4.9% was obtained.

Driver assistance system scenario
A practical use of the introduced pedestrian recognition system would be collision avoidance by warning the driver of a 470 pedestrian which moves towards the road, occluded in a gap in a row of roadside parked cars.This scenario consists of parked cars as highly reflective static objects, lateral movement of the pedestrian, occlusion and linear movement of the measurement vehicle with about 8.5 m/s.With the combination of all those influences in one real scenario, a recognition rate of r TP = 19.3% was achieved.The false alarm rate in this particular measurement was r FP = 3.2%.

Discussion
The gathered results reveal three main development areas for pected rate of about r TP ≈ 4%, caused by false alarms, even if the pedestrian is central in front of the sensor (φ = 0) where Eq. ( 10) gives v rr = 0.This leads to the conclusion that micro Doppler effects (see (He, 2010)), e.g.swinging arms and legs, cause artefacts in the DS, which lead to these classifica-505 tion results.For a particular driver assistance system, a use case could be to require direct measurement (without the help of micro Doppler effects) of a lateral walking pedestrian's radial velocity component v rr in a range of |φ| ≥ 3 • , given a walking 510 pace of v r = 1.0m/s.This requirement gives a blind spot of 5.2 m width centered in front of the vehicle in 50 m distance.This blind spot could be observed by other sensor systems since it is usually directly visible.To fulfill these requirements, a radar sensor with v rr -measurement resolu-515 tion of at least 0.052 m/s or, respectively, Doppler frequency measurement resolution of 26.5 Hz is needed.The required measurement resolution is about fifteen times higher than the resolution of the radar sensor employed in this paper.
Antenna side lobe effects: Parasitic effects because of an-520 tenna side lobes affect the intensity image (ghost targets appear) as well as the frequency image (see Sect. 4.3), especially in the surrounding of highly reflective objects.The impact of antenna side lobes on the intensity image was modeled using a measured antenna pattern.Although in theory 525 compensation of this effect would be possible, further improvements of todays radar sensors concerning side lobes are needed.The knowledge of these effects is the key to optimize the segmentation algorithm.Spatial resolution: If a pedestrian is in line of sight and not 530 more than about 35 m distant from the sensor, the radar sen- is significantly higher in case of narrow gaps up to 50 cm due to segmentation problems in the blurred occluded areas.

False alarms
False alarms, meaning that static objects are incorrectly classified as pedestrians, are consequences of three root causes.Firstly, parasitic antenna side lobe effects can cause significant fractions in the DS of static objects that seem to have a higher velocity than static objects should have.Secondly, errors in the ego velocity compensation can offset the DS and finally, in case of errors in the segmentation step, resolution cells from the noisy surroundings of an object can get assigned to the object, resulting in DS fractions with higher velocity than possible for static objects.To see how good the specifity of the pedestrian recognition system is in practice, a real urban traffic scene was chosen (see Fig. 14).The vehicle drives along a road with lots of highly reflective static objects, such as cars, in the measurement range, which is the worst case scenario in terms of false alarms, since highly reflective objects cause high amplitudes of antenna side lobe effects.As a result, in the described traffic scenario, a false alarm rate of r FP = 4.9% was obtained.

Driver assistance system scenario
A practical use of the introduced pedestrian recognition system would be collision avoidance by warning the driver of a pedestrian which moves towards the road, occluded in a gap in a row of roadside parked cars.This scenario consists of parked cars as highly reflective static objects, lateral movement of the pedestrian, occlusion and linear movement of the measurement vehicle with about 8.5 m s −1 .With the combination of all those influences in one real scenario, a recognition rate of r TP = 19.3% was achieved.The false alarm rate in this particular measurement was r FP = 3.2%.

Discussion
The gathered results reveal three main development areas for future radar sensors used for pedestrian recognition: Doppler frequency resolution, antenna side lobes and spatial resolution.Doppler frequency resolution: An object at the position (r, φ) with relative velocity v r to the radar sensor (see Fig. 9) has a relative radial velocity component given by Equation 9 is the approximation of this equation assuming φ = 0.If the object moves on a straight lateral path (ϕ = 90 • ), it still causes a small Doppler shift given by v rr = v r • sinφ with absolute maxima at the azimuthal borders of the measurement range ± φ.The described experimental results for recognizing lateral walking pedestrians (see Sect. 4.1) are based on a walking pace of about v r = 1.2m s −1 .The theoretical maximum of the radial relative velocity component in this case is v rr = 0.17m s −1 at |φ| = φ, which is only about a fifth of the smallest measurable radial velocity of 0.77 m s −1 , given by the radar sensor's Doppler frequency measurement resolution.Nevertheless, in the experiment the recognition rate of r TP = 39.5% is much higher than the expected rate of about r TP ≈ 4%, caused by false alarms, even if the pedestrian is central in front of the sensor (φ = 0) where Eq. ( 10) gives v rr = 0.This leads to the conclusion that micro Doppler effects (see (He, 2010)), e.g.swinging arms and legs, cause artefacts in the DS, which lead to these classification results.For a particular driver assistance system, a use case could be to require direct measurement (without the help of micro Doppler effects) of a lateral walking pedestrian's radial velocity component v rr in a range of |φ| ≥ 3 • , given a walking pace of v r = 1.0m s −1 .This requirement gives a blind spot of 5.2 m width centered in front of the vehicle in 50 m distance.This blind spot could be observed by other sensor systems since it is usually directly visible.To fulfill these requirements, a radar sensor with v rr -measurement resolution of at least 0. measurement resolution is about fifteen times higher than the resolution of the radar sensor employed in this paper.
Antenna side lobe effects: Parasitic effects because of antenna side lobes affect the intensity image (ghost targets appear) as well as the frequency image (see Sect. 4.3), especially in the surrounding of highly reflective objects.The impact of antenna side lobes on the intensity image was modeled using a measured antenna pattern.Although in theory compensation of this effect would be possible, further improvements of todays radar sensors concerning side lobes are needed.The knowledge of these effects is the key to optimize the segmentation algorithm.
Spatial resolution: If a pedestrian is in line of sight and not more than about 35 m distant from the sensor, the radar sensor's spatial resolution is sufficient.However, if the pedestrian is occluded behind other objects, the corresponding radar data starts to blur with nearby objects.A finer spatial resolution might improve the separability of the objects in this case.If the pedestrian is further away from the sensor, the spatial resolution cells grow, since their width is a function of the azimuth angle.Hence, a distant pedestrian covers only few resolution cells, resulting in a DS of just few single peaks, which might not represent the DS well.A finer spatial resolution would improve the accuracy of the DS in this matter.

Summary and conclusions
Because pedestrians are point targets for radar sensors, object features from the intensity image cannot be used to distinguish between pedestrians and small static objects.Hence, information from the Doppler frequency image is needed.The DS, which is the histogram of the measured radial relative velocities of an object, turned out to be a stable object feature, holding the required information to reliably seperate pedestrians from static objects.A simple classification algorithm, based on five object features, was found with classification results compareable to modern machine learning algorithms.Radial moving pedestrians in line of sight can be recognized up to 95.3 % of the samples.The recognition rate drops to 39.5 % for lateral moving pedestrians and to 29.4 % for lateral moving pedestrians occluded in a gap between two parked cars.In case of misclassifications of pedestrians as static objects, the radar sensor does not deliver statistically different data as from static objects, making it impossible for classification systems (machine learning as well as the proposed solution) to decide for the correct class using only information from single radar raw data frames.Hence, pedestrian recognition using only single radar raw data frames shows poor classification results.
The radar sensor's main limitations in this use case turned out to be insufficient Doppler shift measurement resolution and antenna side lobe effects.Possibly, a finer spatial resolution would also improve the sensor's performance in rec-ognizing occluded pedestrians.Our findings indicate that pedestrian recognition using this low-level approach is limited.Only radar data with higher Doppler frequency measurement resolution (independent of the used technology, e.g.24 GHz, 77 GHz) would make major recognition improvements possible.This requirement of future automotive radar sensors is in line with claims in (Rasshofer, 2007).In state-of-the-art research on pedestrian recognition, today's radar sensors are mostly used to support camera-based systems (Gerónimo, 2010).
Manuscript prepared for Adv.Radio Sci. with version 4.2 of the L A T E X class copernicus.cls.Date: 9 May 2012 Pedestrian recognition using automotive radar sensors A. Bartsch, F. Fitzek, and R. H. Rasshofer BMW Group Research and Technology, Munich, Germany

Fig. 2 .
Fig. 2. Example of a pedestrian's radar raw data.The principle, how the sensor divides its spatial measurement range into resolution cells, is depicted on the left.The sensor measures the reflected power and Doppler shift in the reflected signal for every resolution cell, giving two data matrices, the intensity image and the frequency image (middle).The grey value of pixels in the intensity image refers to the reflected power while the grey value of pixels in the frequency image correspondes to the measured Doppler shift.A grey value of 128 indicates a Doppler shift of 0 Hz.

Fig. 2 .
Fig. 2. Example of a pedestrian's radar raw data.The principle, how the sensor divides its spatial measurement range into resolution cells, is depicted on the left.The sensor measures the reflected power and Doppler shift in the reflected signal for every resolution cell, giving two data matrices, the intensity image and the frequency image (middle).The grey value of pixels in the intensity image refers to the reflected power while the grey value of pixels in the frequency image correspondes to the measured Doppler shift.A grey value of 128 indicates a Doppler shift of 0 Hz.

Fig. 3 .
Fig.3.Relative frequency distributions of resolution cells with certain reflection intensity and relative radial velocity for different object types.The frequency distributions were retrieved from approximately 150 data frames giving about 2000 data points per graph.Since mean and variance of the relative radial velocity values are independent of the reflection intensity, information from the intensity image and from the frequency image is statistically uncorrelated.

Fig. 3 .
Fig. 3. Relative frequency distributions of resolution cells with certain reflection intensity and relative radial velocity for different object types.The frequency distributions were retrieved from approximately 150 data frames giving about 2000 data points per graph.Since mean and variance of the relative radial velocity values are independent of the reflection intensity, information from the intensity image and from the frequency image is statistically uncorrelated.

Fig. 3 .
Fig.3.Relative frequency distributions of resolution cells with certain reflection intensity and relative radial velocity for different object types.The frequency distributions were retrieved from approximately 150 data frames giving about 2000 data points per graph.Since mean and variance of the relative radial velocity values are independent of the reflection intensity, information from the intensity image and from the frequency image is statistically uncorrelated.

Fig
Fig. 4.An illustration of the sensor coordinate system, an object with center at (r c , φ c ) and a local coordinate system for calculating the object's shape factor using (x , y ) coordinates.The maximum beam steering angle φ is the border of the azimuthal measurement range.

Fig. 5 .
Fig.5.Averaged Doppler spectra of radial moving pedestrians and static objects.The Doppler spectrum function D Doppler is plotted over the relative radial velocity axis v. Ego-speed compensation is not necessary because the data was recorded with a non-moving radar sensor.

Fig. 5 .
Fig. 5. Averaged Doppler spectra of radial moving pedestrians and static objects.The Doppler spectrum function D Doppler is plotted over the relative radial velocity axis v. Ego-speed compensation is not necessary because the data was recorded with a non-moving radar sensor.
where w C = [w C,1 ,...,w C,5 ] T is the weighting vector for a class C ∈ {P ;S}.Experiments have shown that constant weighting vectors w C are not sufficient for proper classification.Instead, the combination of the membership values µ C,i must be closer to a logical conjunction.This is achieved by allowing the weight vector w C (u C ) to be dependent on u C , meaning that www.adv-radio-sci.net/10 Example of a single frame Doppler spectrum of a pedestrian.

Fig. 7 .
Fig. 7. Example of a single frame Doppler spectrum of a pedestrian.
360 pedestrian recognition system can be measured by only two key figures, r TP and r FP .The true positive rate r TP tells what fraction of real pedestrians is correctly classified as such.On the other hand, the false positive rate r FP or false alarm rate tells what fraction of static objects are clas-365 sified incorrectly as pedestrians.The true negative rate r TN = 1 − r FP and the false negative rate r FN = 1 − r TP are implied under the above-mentioned assuptions.

Fig. 7 .
Fig. 7. Example of a single frame Doppler spectrum of a pedestrian.

Fig. 8 .
Fig.8.Averaged Doppler spectra of a walking pedestrian for different walking paces.The angle of movement was ϕ = 45 • in all cases.In the investigated range of walking paces from 0.58 m/s to 1.7 m/s, the DS is independent from the pedestrian's walking pace.
Fig. 10.Averaged Doppler spectra of a walking pedestrian over various angles of movement ϕ.
Fig.8.Averaged Doppler spectra of a walking pedestrian for different walking paces.The angle of movement was ϕ = 45 • in all cases.In the investigated range of walking paces from 0.58 m/s to 1.7 m/s, the DS is independent from the pedestrian's walking pace. 370

Fig. 9 .Fig. 10 .
Fig.9.A walking pedestrian with velocity vr relative to the sensor with angle of movement ϕ has a radial relative velocity component vrr that is measurable by the radar sensor.

Fig. 9 .Fig. 8 .
Fig. 9.A walking pedestrian with velocity v r relative to the sensor with angle of movement ϕ has a radial relative velocity component v rr that is measurable by the radar sensor.

Fig. 9 .Fig. 10 .
Fig.9.A walking pedestrian with velocity vr relative to the sensor with angle of movement ϕ has a radial relative velocity component vrr that is measurable by the radar sensor.

Fig. 10 .
Fig. 10.Averaged Doppler spectra of a walking pedestrian over various angles of movement ϕ.

Fig. 11 .
Fig. 11.Classification result r TP for pedestrians walking with different angles of movement ϕ.

Fig. 12 .
Fig. 12. Averaged Doppler spectra of correct classified and incorrect classified pedestrians walking with various angles of movement.The similarity of Doppler spectra of incorrect classified pedestrians and static objects is a limitation of the radar sensor in pedestrian recognition, since the sensor does not deliver statistical different in this case. 415

Fig. 13 .
Fig. 13.Top: A lateral moving pedestrian is occluded behind a single parked car.Bottom: Now the lateral moving pedestrian is in a gap between two parked cars, where the radar wave can propagate due to multiple reflections.

Fig. 11 .
Fig. 11.Classification result r TP for pedestrians walking with different angles of movement ϕ.

Fig. 13 .
Fig. 13.Top: A lateral moving pedestrian is occluded behind a single parked car.Bottom: Now the lateral moving pedestrian is in a gap between two parked cars, where the radar wave can propagate due to multiple reflections.

Fig. 14 .
Fig. 14.A real urban traffic scene used for measuring the false alarm rate.On the left a photography of the scene is depicted and on the right the corresponding intensity image from the radar raw data.

480
future radar sensors used for pedestrian recognition: Doppler frequency resolution, antenna side lobes and spatial resolution.Doppler frequency resolution: An object at the position (r, φ) with relative velocity v r to the radar sensor (see Fig.9) 485 has a relative radial velocity component given byv rr = v r • cos(ϕ − φ) .(10) (Equation (9) is the approximation of this equation assuming φ = 0).If the object moves on a straight lateral path (ϕ = 90 • ), it still causes a small Doppler shift given by 490 v rr = v r •sinφ with absolute maxima at the azimuthal borders of the measurement range ± φ.The described experimental results for recognizing lateral walking pedestrians (see Sect. 4.1) are based on a walking pace of about v r = 1.2m/s.The theoretical maximum of the radial relative velocity com-495 ponent in this case is v rr = 0.17m/s at |φ| = φ, which is only about a fifth of the smallest measurable radial velocity of 0.77 m/s, given by the radar sensor's Doppler frequency measurement resolution.Nevertheless, in the experiment the recognition rate of r TP = 39.5% is much higher than the ex-500

Fig. 14 .
Fig. 14.A real urban traffic scene used for measuring the false alarm rate.On the left a photography of the scene is depicted and on the right the corresponding intensity image from the radar raw data.

Table 1 .
Technical details of the radar sensor used in this paper.
vert the Doppler shift information to radial velocity information.It is important to keep in mind that an object moving equidistant to the sensor does not cause any Doppler shift in the reflected signal.The sensor output for the Doppler information is not limited by frequency measurement resolution because it is internally interpolated to minimize quantization errors.
Doppler spectrum 250tervals.Resolution cells with radial relative velocity v rr increase the counter for the corresponding intervals[v i ,v j [ with v i ≤ |v rr | < v j .To get a densitiy function D Doppler , normalization is necessary such that D Doppler dv = 1 holds.Figure5depicts sample Doppler spectra, averaged over sev-255 eral hundred independent data frames.The wavy structure of the DS, especially of walking pedestrians, is not caused by movements of the object itself.The maxima of the wavy

Table 2 .
Relative radial velocity interval borders for Doppler spectrum feature extraction.The challenge in the classification step is to decide whether an object is a pedestrian or a static object by using the corresponding feature vector m = [m 1 ,...,m 5 ] T only.The deci-285 sion process is based on the knowledge that some values of these features are more probable if the object is of a certain class C. For example, a static object is very unlikely to have a significant fraction of Doppler information in the [v 2 ,v 3 ] interval.Hence, a value of m 3 close to one is very unlikely

Table 2 .
Relative radial velocity interval borders for Doppler spectrum feature extraction.

Table 3 .
Classification result for a lateral moving pedestrian in a gap between two parked cars.

Table 3 .
Classification result for a lateral moving pedestrian in a gap between two parked cars.% 29.4 % 28.5 % 20.1 % 25.2 %