A Study of Real-Time Operations by Converting Human Skeleton Coordinates to Digital Avatars

Fei-lung Lin1, Jui-Hung Kao2, Yu-Yu Yen3, 4, Kuan-Wen Liao5 and Pu Huang6

  1. Institute of Technical and Vocational Education, National Taipei University of Technology, Taipei, Taiwan
    t110499005@ntut.edu.tw
  2. Department of Information Management, Shih Hsin University, Taipei, Taiwan
    kjhtw@mail.shu.edu.tw
  3. Center of General Education, Shih Hsin University, Taipei, Taiwan
    melyen@mail.shu.edu.tw
  4. Department of Biomedical Engineering, National Yang Ming Chiao Tung University, Taipei, Taiwan
    sheepkelly19.be11@nycu.edu.tw
  5. Department of Information Management, Shih Hsin University, Taipei, Taiwan
    m111660002@mail.shu.edu.tw
  6. School of political science and law, Shaoguan University, Shaoguan, China Taiwan
    20201047@sgu.edu.cn

Abstract

This study aims to develop a real-time motion recognition system that translates skeletal human movements into a virtual environment. This will be achieved through the use of advanced tech-niques for the accurate capture of human skeletons and coordinate conversion. This paper investi-gates the acquisition and processing of motion data for virtual characters using depth cameras to obtain depth information. This study identifies six specific actions: left kick, right kick, left punch, right punch, squatting, and sitting. The experimental process successfully integrated RGB+D cameras, Media Pipe, and OpenCV into Unreal Engine models to capture and display human skeletal and joint positions in real-time. The experimental results show that the system achieved a precision of 100% for all motion detections, with an accuracy of more than 94%. How-ever, the recall rate for specific actions was lower, reaching 88%.

Key words

Mixed Reality, Confusion Matrix, Motion Recognition

Digital Object Identifier (DOI)

https://doi.org/10.2298/CSIS241002067L

How to cite

Lin, F., Kao, J., Yen, Y., Liao, K., Huang, P.: A Study of Real-Time Operations by Converting Human Skeleton Coordinates to Digital Avatars. Computer Science and Information Systems, https://doi.org/10.2298/CSIS241002067L