All Issue

2024 Vol.20, Issue 2 Preview Page

Original Article

Comparison of Deep Learning Based Pose Detection Models to Detect Fall of Workers in Underground Utility Tunnels

딥러닝 자세 추정 모델을 이용한 지하공동구 다중 작업자 낙상 검출 모델 비교

김정수*

Jeongsoo Kim*

Research Specialist, Korea Institute of Civil Engineering and Building Technology, Goyang, Republic of Korea
30 June 2024. pp. 302-314
PDF Citation
Abstract
Purpose: This study proposes a fall detection model based on a top-down deep learning pose estimation model to automatically determine falls of multiple workers in an underground utility tunnel, and evaluates the performance of the proposed model. Method: A model is presented that combines fall discrimination rules with the results inferred from YOLOv8-pose, one of the top-down pose estimation models, and metrics of the model are evaluated for images of standing and falling two or fewer workers in the tunnel. The same process is also conducted for a bottom-up type of pose estimation model (OpenPose). In addition, due to dependency of the falling interference of the models on worker detection by YOLOv8-pose and OpenPose, metrics of the models for fall was not only investigated, but also for person. Result: For worker detection, both YOLOv8-pose and OpenPose models have F1-score of 0.88 and 0.71, respectively. However, for fall detection, the metrics were deteriorated to 0.71 and 0.23. The results of the OpenPose based model were due to partially detected worker body, and detected workers but fail to part them correctly. Conclusion: Use of top-down type of pose estimation models would be more effective way to detect fall of workers in the underground utility tunnel, with respect to joint recognition and partition between workers.
Keywords
Fall
Workers
Underground Utility Tunnel
Deep Learning
YOLOv8-pose
OpenPose
연구목적: 본 연구는 지하공동구 내 다수 작업자의 낙상을 자동으로 판별하기 위한 Top-down 방식의 딥러닝 자세 추정 모델 기반 낙상 검출 모델을 제안하고, 제안 모델의 성능을 평가한다. 연구방법: Top-down 방식의 자세 추정모델 중 하나인 YOLOv8-pose로부터 추론된 결과와 낙상 판별 규칙을 결합한 모델을 제시하고, 지하공동구 내 2인 이하 작업자가 출현한 기립 및 낙상 이미지에 대해 모델 성능지표를 평가하였다. 또한 동일한 방법으로 Bottom-up 방식 자세추정모델(OpenPose)을 적용한 결과를 함께 분석하였다. 두 모델의 낙상 검출 결과는 각 딥러닝 모델의 작업자 인식 성능에 의존적이므로, 작업자 쓰러짐과 함께 작업자 존재 여부에 대한 성능지표도 함께 조사하였다. 연구결과: YOLOv8-pose와 OpenPose의 모델의 작업자 인식 성능은 F1-score 기준으로 각각 0.88, 0.71로 두 모델이 유사한 수준이었으나, 낙상 규칙을 적용함에 따라 0.71, 0.23로 저하되었다. 작업자의 신체 일부만 검출되거나 작업자 간 구분을 실패하여, OpenPose 기반 낙상 추론 모델의 성능 저하를 야기한 것으로 분석된다. 결론: Top-down 방식의 딥러닝 자세 추정 모델을 사용하는 것이 신체 관절점 인식 및 개별 작업자 구분 측면에서 지하공동구 내 작업자 낙상 검출에 효과적이라 판단된다.
키워드
낙상
다중 작업자
지하공동구
딥러닝
YOLOv8-pose
OpenPose
References
  1. Beddiar, D.R., Oussalah, M., Nini, B. (2022). "Fall detection using body geometry and human pose estimation in video sequences." Journal of Visual Communication and Image Representation, Vol. 82, 103407. 10.1016/j.jvcir.2021.103407
  2. Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y. (2017). "Realtime multi-person 2d pose estimation using part affinity fields." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, US, pp. 7291-7299. 10.1109/CVPR.2017.143
  3. Chen, W., Jiang, Z.,Guo, H., Ni, X. (2020). "Fall detection based on key points of human-skeleton using OpenPose." Symmetry, Vol. 12, No. 5, 744. 10.3390/sym12050744
  4. Gao, M., Li, J., Zhou, D., Zhi, Y., Zhang, M., Li, B. (2022). "Fall detection based on OpenPose and MobileNetV2 network." IET Image Processing, Vol. 17, No. 3, pp. 722-732. 10.1049/ipr2.12667
  5. Hirschorn, O., Avidan, S. (2020). "Normalizing flows for human pose anomaly detection." In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seattle, US, pp. 13545-13554.
  6. Jocher, G. (2023). YOLOv8 by Ultralytics https://github.com/ultralytics/ultralytics
  7. Kim, G., Kim, J., Jung, W.-S. (2023a). "Deep learning-based object detection of panels door open in underground utility tunnel." Journal of The Korean Society of Disaster Information, Vol. 19, No. 3, pp. 665-672. (In Korean)
  8. Kim, J., Kim, B., Lee, H. (2024). "Fall recognition based on time-level decision fusion classification." Applied Sciences, Vol. 14, No. 2, 709. 10.3390/app14020709
  9. Kim, J., Lee, C.-W., Park, S.-H., Lee, J.-H., Hong, C.H. (2020a). "Development of fire detection model for underground utility facilities using deep learning: Training data supplement and bias optimization." Journal of Korea Academia-Industrial Cooperation Society, Vol. 21, No. 12, pp. 320-330. (In Korean)
  10. Kim, J., Park, S., Hong, C. (2023b). "A study on falling detection of workers in the underground utility tunnel using dual deep learning techniques." Journal of The Korean Society of Disaster Information, Vol. 19, No. 3, pp. 498-509. (In Korean)
  11. Kim, J., Park, S., Hong, C., Park, S.-H., Lee, J. (2022). "Development of AI detection model based on CCTV image for underground utility tunnel." Journal of The Korean Society of Disaster Information, Vol. 18, No. 2, pp. 364-373. (In Korean)
  12. Kim, J.-H., Choi, J.-H., Park, Y.-H., Nasridinov, A. (2020b). "Abnormal situation detection on surveillance video using object detection and action recognition." Journal of Korea Multimedia Society, Vol. 24, No. 2, pp. 186-198. (In Korean)
  13. Li, J., Gao, M., Li, B., Zhou, D., Zhi, Y., Zhang, Y. (2023). "KAMTFENet: A fall detection algorithm based on keypoint attention module and temporal feature extraction." International Journal of Machine Learning and Cybernetics, Vol. 14, pp. 1831-1844. 10.1007/s13042-022-01730-4
  14. Lim, J.-M., Yoo, S.-S., Shin, J.-S., Shin, S.-W. (2023). "A study on illumination analysis of an intelligent LED lighting fixture applied to underground common duct and visibility measurement by a low-light camera." Journal of the Korean Institute of Illuminating and Electrical Installation Engineers, Vol. 37, No. 5, pp. 7-14. (In Korean) 10.5207/JIEIE.2023.37.5.007
  15. Maji, D., Nagori, S., Mathew, M., Poddar, D. (2022). "Yolo-pose: Enhancing yolo for multi person pose estimation using object keypoint similarity loss." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, US, pp. 2637-2646. 10.1109/CVPRW56347.2022.00297
  16. Park, J.-T., Han, K.-P., Park, Y.-W. (2021). "A dangerous situation recognition system using human behavior analysis." Journal of Korea Multimedia Society, Vol. 24, No. 3, pp. 345-354. (In Korean)
  17. Park, S.M., Hong, C.-H., Park, S.-H., Lee, J., Kim, J.S. (2022). "Development of a deep learning-based fire extinguisher object detection model in underground utility tunnels." Journal of The Korean Society of Disaster Information, Vol. 18, No. 4, pp. 922-929. (In Korean)
  18. Si, J., Son, H., Kim, D., Kim, M., Jeong, J., Kim, G., Kim, Y., Kim, S. (2020). "Fall detection using skeletal coordinate vector and LSTM model." The Journal of Korean Institute of Information Technology, Vol. 18, No. 12, pp. 19-29. (In Korean) 10.14801/jkiit.2020.18.12.19
  19. Tan, M., Le, Q. (2019). "Efficientnet: Rethinking model scaling for convolutional neural networks." In Proceedings of the International Conference on Machine Learning, Long Beach, US, pp. 6105-6114.
  20. Yadav, S.K., Luthra, A., Tiwari, K., Pandey, H.M., Akbar, S.A. (2022). "ARFDNet: An efficient activity recognition & fall detection system using latent feature pooling." Knowledge-Based Systems, Vol. 239, 107948. 10.1016/j.knosys.2021.107948
  21. Yun, K., Park, J., Cho, J. (2020). "Robust human pose estimation for rotation via self-supervised learning." In IEEE Access, Vol. 8, pp. 32502-32517. 10.1109/ACCESS.2020.2973390
Information
  • Publisher :The Korean Society of Disaster Information
  • Publisher(Ko) :한국재난정보학회
  • Journal Title :Journal of the Society of Disaster Information
  • Journal Title(Ko) :한국재난정보학회논문집
  • Volume : 20
  • No :2
  • Pages :302-314
  • DOI :https://doi.org/10.15683/kosdi.2024.6.30.302
Journal Informaiton Journal of the Society of Disaster Information Journal of the Society of Disaster Information
  • NRF
  • KISTI Current Status
  • KISTI Cited-by
  • crosscheck
  • orcid
  • open access
  • ccl
Journal Informaiton Journal Informaiton - close