基于图表示的智能车行人意图识别方法

吕超; 崔格格; 孟相浩; 陆军琰; 徐优志; 龚建伟

doi:10.15918/j.tbit1001-0645.2021.330

基于图表示的智能车行人意图识别方法

doi: 10.15918/j.tbit1001-0645.2021.330

1.
北京理工大学机械与车辆学院, 北京　100081
2.
上海汽车集团股份有限公司技术中心, 上海　201804

基金项目: 国家自然科学基金联合基金项目（U19A2083）；国家青年自然科学基金资助项目（61703041）；北京理工大学科技创新计划前沿交叉与学科创新专项计划；上海汽车工业科技发展基金公产学研项目

详细信息

作者简介:
吕超（1985—），男，博士，副教授，E-mail：chaolu@bit.edu.cn

中图分类号: U495
计量
- 文章访问数: 214
- HTML全文浏览量: 96
- PDF下载量: 0
- 被引次数: 0
出版历程
- 收稿日期: 2021-11-30
- 录用日期: 2022-03-22
- 刊出日期: 2022-08-17

Graph Representation Method for Pedestrian Intention Recognition of Intelligent Vehicle

1.
School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China
2.
SAIC Motor Technical Center, Shanghai 201804, China

摘要

摘要: 智能驾驶场景下的人车冲突问题与行人过街行为密切相关，为使高级驾驶辅助系统（advanced driving assistance system, ADAS）具备识别行人过街意图的功能，并对人车碰撞事件预警，提出一种基于图表示学习（graph representation learning, GRL）方法的行人过街意图识别框架。它采用开源工具对行人骨架信息进行识别，采用图方法，以行人在一段运动过程内每一帧的骨架关键点为节点，以骨架自然连接关系、相关关系和时域关系为边建立图模型，实现对行人动作序列的表征。以图结构数据为输入，基于支持向量机（support vector machine, SVM）训练行人过街意图识别模型。在自动驾驶数据集PIE上对所提出方法进行评估，结果显示，行人过街意图分类准确率可达90.29%，所提出方法能够有效识别行人过街意图，对提高智能车决策安全性具有重要意义。
- 行人意图识别 /
- 图表示学习 /
- 机器学习
Abstract: The problem of pedestrian-vehicle conflict in intelligent driving scenes is closely related to pedestrian crossing behavior. In order to enable advanced driving assistance system (ADAS) to have the function of identifying pedestrian crossing intentions and raising advanced warning of pedestrian-vehicle collision events, a pedestrian crossing intention recognition framework based on graph representation learning (GRL) method is proposed. It uses open source tools to generate pedestrian skeleton information. Then it establishes a graph model to represent the characteristics of pedestrian action sequence by taking the skeleton key points of each frame of pedestrian within a sequence as nodes, as well as taking the natural connections, the topological correlations and time-domain relationships between skeleton joints as edges. Taking the graph structure data as the input, the pedestrian crossing intention recognition model is trained based on support vector machine (SVM). The results show that the classification accuracy of pedestrian crossing intention can reach 90.29%. The proposed method can effectively identify the pedestrian crossing intention, which is of great significance to improve the safety of intelligent vehicle decision-making.
- pedestrian intention recognition /
- graph representation learning /
- machine learning

HTML全文

图 1 基于图表示的行人意图识别方法框架

Figure 1. Framework of pedestrian intention recognition method based on graph representation

下载: 全尺寸图片幻灯片

图 2 行人时空图模型构建示意图

Figure 2. Diagram of the construction of pedestrian spatio-temporal graph model

下载: 全尺寸图片幻灯片

图 3 图核方法内积投影示意图

Figure 3. Diagram of inner product projection of kernel method

下载: 全尺寸图片幻灯片

图 4 分类结果混淆矩阵分析图

Figure 4. Confusion matrix analysis of classification results

下载: 全尺寸图片幻灯片

图 5 行人过街意图判断结果示例

Figure 5. Examples of pedestrian crossing intention classification results

下载: 全尺寸图片幻灯片

表 1 行人骨架关键节点定义表

Table 1. Definition table of key nodes of pedestrian skeleton

节点编号	原始编号	节点名称	精度/%
0	1	鼻	34
1	6	左侧肩关节	79
2	7	右侧肩关节	77
3	12	左侧髋关节	81
4	13	右侧髋关节	81
5	14	左侧膝关节	78
6	15	右侧膝关节	77
7	16	左侧踝关节	73
8	17	右侧踝关节	70

下载: 导出CSV

表 2 基于图表示方法的行人意图识别模型销蚀实验

Table 2. Abation experiments of pedestrian intention recognition model based on graph representation method

特征			准确度/%			F₁分数/%			精确度/%
p	veh_s	d	GH	PK	ML	GH	PK	ML	GH	PK	ML
√			63.24	72.38	84.38	62.31	70.00	82.67	62.31	71.00	83.00
√	√		71.62	73.14	87.62	71.33	70.67	86.33	71.33	74.33	86.33
√	√	√	71.43	89.71	90.29	70.67	88.67	89.33	70.67	89.30	89.33

特征			回召率/%			平均值/%			时间/s
p	veh_s	d	GH	PK	ML	GH	PK	ML	GH	PK	ML
√			63.33	69.67	82.33	62.80	70.76	83.10	35957	100	25671
√	√		71.67	70.00	86.33	71.49	72.03	86.65	35340	98	25182
√	√	√	71.00	88.33	89.00	70.94	89.00	89.48	35134	121	25691

下载: 导出CSV

参考文献(9)

[1]	MUHAMMAD I P, WIWIK A, HANUGRA A S, et al. Early warning pedestrian crossing intention from its head gesture using head pose estimation[C]//International Seminar on Intelligent Technology and Its Applications (ISITIA). Surabaya, Indonesia. IEEE, 2021: 402 − 407.
[2]	RAÚL Q M, IGNACIO P A, DAVID F, et al. Pedestrian path, pose, and intention prediction through Gaussian process dynamical models and pedestrian activity recognition[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 20(5):1803 − 1814.
[3]	FANG Z, VAZQUEZ D, LOPEZ A M. On-board detection of pedestrian intentions[J]. SENSORS. 2017, 17(10): 2193
[4]	FRANCESCO P, RAJARATHNAM B, MARIA J P, et al. FuSSI-net: fusion of Spatio-temporal skeletons for intention prediction network[C]//54th Asilomar Conference on Signals, Systems, and Computers. Pacific Grove, CA, USA: IEEE, 2020: 68 − 72.
[5]	PABLO R G C, MING Y, QIAN Y Q, et al. Pedestrian Graph: Pedestrian crossing prediction based on 2D pose estimation and graph convolutional networks[C]//2019 IEEE Intelligent Transportation Systems Conference (ITSC). Auckland, New Zealand: IEEE, 2019: 2000 − 2005.
[6]	SVEN K, LORENZO B, ALEXANDRE A. PifPaf: composite fields for human pose estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA: IEEE, 2019: 11969 − 11978.
[7]	NEUMANN M, GARNETT R, BAUCKHAGE C, et al. Propagation kernels: efficient graph kernels from propagated information[J]. Machine Learning, 2016, 102(2): 209 − 245.
[8]	KONDOR R, PAN H. The multiscale laplacian graph kernel[C]//30th Conference on Neural Information Processing Systems (NIPS).Barcelona,Spain:Curran Associates,Inc., 2016: 2990 − 2998.
[9]	RASOULI A, KOTSERUBA I, KUNIC T, et al. PIE: a large-scale dataset and models for pedestrian intention estimation and trajectory prediction[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South): IEEE, 2019: 6261 − 6270.