1. General

Category

Please describe Other

Workplace safety

Category

Labor

2. Project Details

Company or Institution

Faculty of engineering, University of Kragujevac

Project

http://www.ai4workplacesafety.com/

General description of the AI solution

Prevention of workplace injuries implies an objective and timely detection of accident precursors, such are unsafe acts and unsafe conditions. The present manual safety reporting and monitoring have shown serious limitations, especially considering the size of halls and the number of people that need to be observed timely by safety management. Particularly, the reports indicate that a company of 1000 workers is exposed to the costs of ~$1.1M annually due to safety issues. This project aims to solve the described problem by developing computer vision algorithms for the detection of Nonuse of protective equipment (PPE) (OBJ1), and Unsafe acts during the pushing and pulling activities (OBJ2). By using the public and prospectively collected data, more reliable algorithms will be developed by accounting for the estimated posture, PPE position (OBJ1), acting forces, and cognitive state (OBJ2) of employees. The solution is designed as a series of successive modules. This enables one to develop and train some of its parts on the large/public data set and fine-tune them afterward on the data collected during the project (transfer learning). The project outcome will be the framework for automatic reporting of timepoints when safety irregularities occurred. With this feature, the effort and costs of safety management will be largely reduced – while the reliability of the safety reporting will be significantly increased. The considered problems (audit of equipment usage and ergonomics of repetitive tasks) are generic and related to many industries and injuries not reported as workplace accidents (i.e. backup pain and musculoskeletal disorders). Thus, besides the manufacturing (which employs most of the workforce in Serbia), the project outcomes could also find applications in ergonomics, sports, healthcare, etc.

Website

http://www.ai4workplacesafety.com/

Organisation

University of Kragujevac

3. Aspects

Excellence and Scientific Quality: Please detail the improvements made by the nominee or the nominees’ team or yourself if your applying for the award, and why they have been a success.

Detection and risk assessment of activities that do not cause direct injuries, such as unsafe pushing and pulling (P&P) and misuse of personal protective equipment (PPE), represents a complex and underestimated problem. To enhance this topic, our project will yield the first data sets related to these problems. The proposed AI solution emphasizes the fusion of anatomical information (by using AI for human pose estimation), corresponding values of the P&P forces (by using IoT modules), and cognitive state and fatigue level (by using wearable EEG). Compared to the state-of-the-art, our project will be the first to combine these aspects, which is excepted to end up with a more reliable and applicable safety recommendation system. To achieve this ambitious goal, we gathered leading Serbian experts in Computer vision, Computer science, Occupational health and safety, and Medicine. During the first three quartals of the project, we finalized proof-of-concept solutions (TRL3-4) for both project objectives: 1) PPE compliance study is submitted for publication, while 2) Recognition of unsafe P&P study is accepted for publication in a leading AI journal. Briefly, the first solution proposes an integrative approach that reduces the problem of PPE compliance to the binary classification while enabling compliance of arbitrary type and number of PPE that could be mounted on various body parts. To prove this hypothesis, we studied 18 different PPE types used across various industries for protecting 5 physiological body parts/functions. The second study proposed a novel approach for utilizing IoT module detects moments with increased P&P forces, while the assessment of pose ergonomics was performed from the employee pose reconstructed with the VIBE algorithm. Compared to previous studies, the proposed approach demonstrated improved performances with a unique ability to be easily adapted for performing compliance of various PPE or detection of various unsafe acts (besides P&P).

Scaling of impact to SDGs: Please detail how many citizens/communities and/or researchers/businesses this has had or can have a positive impact on, including particular groups where applicable and to what extent.

Potential users of the project outcomes are researchers (basic/applied), safety managers, healthcare professionals and regulatory bodies as well as companies interested for the commercialization of the project findings. The proposed AI solution for automated PPE compliance (OBJ1) could help to reduce the number of injuries and large costs (360B dollars annually to the US alone). Regarding the OBJ2, a typical consequence of the non-ergonomic execution of repetitive tasks is musculoskeletal disorders and backup pain. To illustrate the economic impact of investigating this problem, we highlight that solely in the US patients with musculoskeletal conditions incur costs of approximately $240 billion, of which $77 billion is related to musculoskeletal disorders. Although the ongoing digitalization eased the collection and centralization of data and safety reports, it opens the bigger problem: “How to efficiently extract meaningful information from such a large amount of incoming data streams?”. Due to the lack of software solutions for these purposes, a huge potential for improving workplace safety, and reducing the harms and cost for both employees, companies, and governments, remains unexploited. The proposed AI solutions will enable automatic detection and reporting of time points when safety irregularities occurred. In this way, the effort and time that safety professionals are spending on workplace observation will be largely reduced – while objectivity and reliability of safety reporting will be significantly increased. Besides the manufacturing industry (which employs most of the workforce worldwide), industries that could benefit from the project outcomes are also: ergonomics (both professionals and regulatory bodies), sports, and healthcare (rehabilitation, physical medicine) professionals. The mid and long-term impact of the project could be very high, since the considered problems (wearing equipment and ergonomic execution of manual tasks) are generic and common in many industries and workplaces.

Scaling of AI solution: Please detail what proof of concept or implementations can you show now in terms of its efficacy and how the solution can be scaled to provide a global impact ad how realistic that scaling is.

We emphasized that we have finalized proof-of-concept solutions for both project objectives: 1) Inspection of PPE use (submited for review in an international AI journal), and 2) Recognition of unsafe pushing and pulling tasks. Regarding the second task, our study “Assessment of the handcart pushing and pulling safety by using Deep Learning 3D pose estimation and IoT force sensors” is accepted for publication in the Expert systems with applications journal (which is leading journal for applied artificial intelligence, with the impact factor 5.452). Currently, we are working towards technology transfer and shaping technology to be used in industry practice. The companies behind the Serbian Association for Occupational Health and Safety (http://ubzrs.rs) showed interest to actively participate in further development and assessment phase, following the written support letter provided with the project proposal. As already mentioned, the problem of inspection of PPE usage and improper execution of pushing and pulling tasks are common in many fields of science and industry. Typical examples are sport (fitness, wrestling), healthcare (nurses, dentists, rehabilitation), transport and airplanes (stewardess), as well as workers in warehousing and manufacturing. Therefore, the expected project outcomes represent a good foundation for further studies in related scientific topics and fields. To sum-up, there is a high potential for the further development and spinning-off developed technology. Considering the present data/privacy/computational constraints, the technology is currently recommended as suited for the digitalization of PPE compliance at: 1) self-check points, and 2) safety-critical workplaces.

Ethical aspect: Please detail the way the solution addresses any of the main ethical aspects, including trustworthiness, bias, gender issues, etc.

In general, there is an increasing need and potential of applying Industry 4.0 technologies for improving workplace safety. For example, although previous studies demonstrated the applicability of AI-driven PPE compliance in construction engineering, the usage of surveillance technology to cover larger areas and multiple workers at once makes it challenging for such technologies to find a place in the industry practice. Furthermore, the current privacy regulations and costs/complexity of using AI for 24/7 surveillance of whole industry halls are also barriers for approaches that recommended real-time tracking of employees. Instead, we recommend the proposed technology as suited for the use in controlled conditions, such are: 1) self-check points (when users are asked to confirm their identity by using e.g. RFID card, while AI is used solely for the safety assessment (e.g. PPE compliance, pose ergonomy assessment) but not for the purpose of identification and tracking), and on 2) monitoring of particular workplaces/machines with high risk from injuries (so that AI could ensure timely detection and mitigation of occurred risks). According to our internal testings and experiments, the proposed technology does not suffer from fairness and gender equality issues, as underpinning methods for human pose reconstruction/classification and PPE detection/classification are not affected with gender/race variability (we do not deal with the operator identity). However, our recommendation/expectation is that the transfer learning of the trained models needs to be performed for a particular type of company or region where our AI needs to be applied/implemented in practice. For example, in the case of PPE compliance, this is achieved by linking the lists of considered PPE (classifiers) and corresponding body parts. By editing these lists, one could easily adapt and use the proposed procedure for an arbitrary type of PPE or industry (including manufacturing, healthcare, military/security, food, lumber, construction engineering, etc.).