Manual Assembly Augmented Reality Systems Implementation: A Systematic Literature Mapping
Resumo
Manual assembly operations can benefit from AR-based systems but the industry seems not to follow the idea. A gap between conceptual systems and actual ones on the shop floor in the industry might exist. Recommendations and guidelines for the development of AR-based manual assembly systems can be found, however, the prevalence of such applications in everyday industry remains relatively low. Due to the absence of reviews targeting systems’ application and usage, we performed a systematic literature mapping focusing on the implementation of AR-based manual assembly systems in real settings and the adaptations required. This paper reviewed the research on using AR to assist manual assembly from 2012 to 2022 and from 3330 papers a total of 77 were selected. The majority of the proposed systems were applied to a simulated environment and were evaluated by subjects that were not directly related to manual assemblies. It was also found that among the wide variety of tasks related to the presented manual assembly AR systems, the most common was Lego-based assembly. A preference was observed for manual assembly AR systems that display the actual assembly procedure, and provide guidance by presenting instructions within the visual field where the assembly is taking place. Regarding the main aspects of AR systems that have been evaluated to validate the results, we found: assembly time, number of errors, user experience, and cognitive load. Regarding implementing a manual assembly AR system in actual settings, it was found that the system should display only essential orientation to the task and be adaptative to the user’s knowledge. Our analysis indicates that current AR-based manual assembly systems are still in the process of identifying the necessary requirements for achieving an efficient and ergonomic system. Therefore, they have yet to reach widespread adoption as a common technology in the industrial setting.
Palavras-chave:
Manual Assembly, Augmented Reality, Implementation
Referências
Salvador S Agati, Rudieri D Bauer, Marcelo da S Hounsell, and Aleksander S Paterno. 2020. Augmented reality for manual assembly in industry 4.0: Gathering guidelines. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR). IEEE, 179–188.
Malek Alrashidi, Ahmed Alzahrani, Michael Gardner, and Vic Callaghan. 2016. A pedagogical virtual machine for assembling mobile robot using augmented reality. In Proceedings of the 7th Augmented Human International Conference 2016. 1–2.
Nikolaus Angel, Horst Orsolits, and Jose Garcia. 2022. Developing an AR Based Tool for Teaching Motion Analysis on Assembly Tasks. In 2022 8th International Conference of the Immersive Learning Research Network (iLRN). IEEE, 1–7.
JC Arbeláez, Roberto Viganò, and Gilberto Osorio-Gómez. 2019. Haptic augmented reality (HapticAR) for assembly guidance. International Journal on Interactive Design and Manufacturing (IJIDeM) 13 (2019), 673–687.
Doris Aschenbrenner, Michael Rojkov, Florian Leutert, Jouke Verlinden, Stephan Lukosch, Marc Erich Latoschik, and Klaus Schilling. 2018. Comparing different augmented reality support applications for cooperative repair of an industrial robot. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 69–74.
Faieza Abdul Aziz, AS Alsaeed, Shamsuddin Sulaiman, MK Ariffin, and Muhammad Faris Al-Hakim. 2020. Mixed reality improves education and training in assembly processes. Journal of Engineering and Technological Sciences 52, 4 (2020), 598–607.
Rudieri Dietrich Bauer, Thiago Luiz Watambak, Salvador Sergi Agati, Marcelo da Silva Hounsell, and Andre Tavares da Silva. 2021. Development and Evaluation of a PCB‘s Manual Assembly system using Augmented Reality and Total Quality. In Symposium on Virtual and Augmented Reality. 24–32.
Andreea Blaga, Cristian Militaru, Ady-Daniel Mezei, and Levente Tamas. 2021. Augmented reality integration into MES for connected workers. Robotics and Computer-Integrated Manufacturing 68 (2021), 102057.
Jonas Blattgerste, Patrick Renner, Benjamin Strenge, and Thies Pfeiffer. 2018. Insitu instructions exceed side-by-side instructions in augmented reality assisted assembly. In Proceedings of the 11th Pervasive technologies related to assistive environments conference. 133–140.
Jonas Blattgerste, Benjamin Strenge, Patrick Renner, Thies Pfeiffer, and Kai Essig. 2017. Comparing conventional and augmented reality instructions for manual assembly tasks. In Proceedings of the 10th international conference on pervasive technologies related to assistive environments. 75–82.
Francesco Bonavolontá, Dominique Dallet, Ernesto Erra, Andrea Grassi, Valentina Popolo, Annarita Tedesco, and Silvestro Vespoli. 2020. Measuring worker’s performance in augmented reality-assisted Industry 4.0 procedures. In 2020 IEEE International Instrumentation and Measurement Technology Conference (I2MTC). IEEE, 1–6.
Poonpong Boonbrahm and Charlee Kaewrat. 2014. Assembly of the virtual model with real hands using augmented reality technology. In Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments: 6th International Conference, VAMR 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22-27, 2014, Proceedings, Part I 6. Springer, 329–338.
Tim Bosch, Gu Van Rhijn, Frank Krause, Reinier Könemann, Ellen S Wilschut, and Michiel de Looze. 2020. Spatial augmented reality: a tool for operator guidance and training evaluated in five industrial case studies. In Proceedings of the 13th ACM International Conference on Pervasive technologies related to assistive environments. 1–7.
Sebastian Büttner, Michael Prilla, and Carsten Röcker. 2020. Augmented reality training for industrial assembly work-are projection-based AR assistive systems an appropriate tool for assembly training?. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–12.
Sebastian Büttner, Oliver Sand, and Carsten Röcker. 2015. Extending the design space in industrial manufacturing through mobile projection. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. 1130–1133.
Chih-Hsing Chu and Ching-Hung Ko. 2021. An experimental study on augmented reality assisted manual assembly with occluded components. Journal of Manufacturing Systems 61 (2021), 685–695.
Luís Fernando de Souza Cardoso, Flávia Cristina Martins Queiroz Mariano, and Ezequiel Roberto Zorzal. 2020. Mobile augmented reality to support fuselage assembly. Computers & Industrial Engineering 148 (2020), 106712.
Iñigo Fernández del Amo, John Ahmet Erkoyuncu, Rajkumar Roy, Riccardo Palmarini, and Demetrius Onoufriou. 2018. A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications. Computers in Industry 103 (2018), 47–71.
Abhiraj Deshpande and Inki Kim. 2018. The effects of augmented reality on improving spatial problem solving for object assembly. Advanced Engineering Informatics 38 (2018), 760–775.
Raquel Quesada Díaz and Anna Syberfeldt. 2022. Optimised Shop-Floor Operator Support on an Integrated AR and PLM Framework for Remanufacturing. In SPS2022: Proceedings of the 10th Swedish Production Symposium, Vol. 21. IOS Press, 425.
Wei Fang and Jianhao Hong. 2022. Bare-hand gesture occlusion-aware interactive augmented reality assembly. Journal of Manufacturing Systems 65 (2022), 169– 179.
Shuo Feng, Weiping He, Shaohua Zhang, and Mark Billinghurst. 2022. Seeing is believing: AR-assisted blind area assembly to support hand–eye coordination. The International Journal of Advanced Manufacturing Technology 119, 11-12 (2022), 8149–8158.
Francesca Ferrati, John Ahmet Erkoyuncu, and Samuel Court. 2019. Developing an augmented reality based training demonstrator for manufacturing cherry pickers. Procedia CIRP 81 (2019), 803–808.
Markus Funk and Albrecht Schmidt. 2015. Cognitive assistance in the workplace. IEEE Pervasive Computing 14, 3 (2015), 53–55.
Lorenzo Giunta, Jamie O’Hare, James Gopsill, Elies Dekoninck, et al. 2018. A review of augmented reality research for design practice: looking to the future. DS 91: Proceedings of NordDesign 2018, Linköping, Sweden, 14th-17th August 2018 (2018).
Mikell P Groover. 2011. Industrial automation and manufacturing systems. Pearson Brasil.
Jürgen Hahn, Bernd Ludwig, and Christian Wolff. 2015. Augmented reality-based training of the PCB assembly process. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia. 395–399.
Mario Heinz, Sebastian Büttner, Sascha Jenderny, and Carsten Röcker. 2021. Dynamic task allocation based on individual abilities-experiences from developing and operating an inclusive assembly line for workers with and without disabilities. Proceedings of the ACM on Human-Computer Interaction 5, EICS (2021), 1–19.
Bradley M Herbert, William Hoff, and Mark Billinghurst. 2020. Usability considerations of hand held augmented reality wiring tutors. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 277–282.
Hung-Yu Ho, An-Hsiang Wang, and Chia-Huang Wu. 2022. Effects of Training Methods on Performance of Scaffolding Task for Workers with Different Spatial Ability. Available at SSRN 4203648 (2022).
Jannike Illing, Philipp Klinke, Uwe Grünefeld, Max Pfingsthorn, and Wilko Heuten. 2020. Time is money! evaluating augmented reality instructions for time-critical assembly tasks. In Proceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia. 277–287.
Jannike Illing, Philipp Klinke, Max Pfingsthorn, and Wilko Heuten. 2021. Less is more! Support of parallel and time-critical assembly tasks with augmented reality. In Proceedings of Mensch und Computer 2021. 215–226.
Kazuyo IWAMOTO. 2019. Development of 4K Binocular Video See-through Display with Haptic Device for Task Training. In 2019 23rd International Computer Science and Engineering Conference (ICSEC). IEEE, 158–163.
Knoch S Alexandersson J. Jost M, A. 2022. PARTAS: A Personalizable Augmented Reality Based Task Adaption System for Workers with Cognitive Disabilities. InProceedings of the 15th International Conference on Pervasive Technologies Related to Assistive Environments 2022 Jun 29 (2022), 159–168.
Nawel Khenak, Jeanne Vézien, and Patrick Bourdot. 2020. Effectiveness of augmented reality guides for blind insertion tasks. Frontiers in virtual reality 1 (2020), 588217.
Bui Minh Khuong, Kiyoshi Kiyokawa, Andrew Miller, Joseph J La Viola, Tomohiro Mashita, and Haruo Takemura. 2014. The effectiveness of an AR-based contextaware assembly support system in object assembly. In 2014 IEEE Virtual Reality (VR). 57–62.
Sunwook Kim, Maury A Nussbaum, and Joseph L Gabbard. 2019. Influences of augmented reality head-worn display type and user interface design on performance and usability in simulated warehouse order picking. Applied ergonomics 74 (2019), 186–193.
Gyuwon Kwak, Ji Seong Min, and Wonil Hwang. 2020. Characteristics of usability problems from augmented reality-based user interface for assembly task. ICIC Express Letters 14, 4 (2020), 353–360.
Caroline Kwiatek, M Sharif, Shangkun Li, Carl Haas, and Scott Walbridge. 2019. Impact of augmented reality and spatial cognition on assembly in construction. Automation in construction 108 (2019), 102935.
Ze-Hao Lai, Wenjin Tao, Ming C Leu, and Zhaozheng Yin. 2020. Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing. Journal of Manufacturing Systems 55 (2020), 69–81.
Eva Lampen, Jonas Teuber, Felix Gaisbauer, Thomas Bär, Thies Pfeiffer, and Sven Wachsmuth. 2019. Combining simulation and augmented reality methods for enhanced worker assistance in manual assembly. Procedia CIRP 81 (2019), 588–593.
Traian Lavric, Emmanuel Bricard, Marius Preda, and Titus Zaharia. 2021. Exploring low-cost visual assets for conveying assembly instructions in AR. In 2021 International Conference on Innovations in Intelligent Systems and Applications (INISTA). IEEE, 1–6.
Traian Lavric, Emmanuel Bricard, Marius Preda, and Titus Zaharia. 2022. A low-cost AR training system for manual assembly operations. Computer Science and Information Systems 19 (2022), 1047–1073.
Federico Manuri and Andrea Sanna. 2016. A survey on applications of augmented reality. ACSIJ Advances in Computer Science: an International Journal 5, 1 (2016), 18–27.
Bernardo Marques, João Alves, Miguel Neves, Inês Justo, André Santos, Raquel Rainho, Rafael Maio, Dany Costa, Carlos Ferreira, Paulo Dias, et al. 2020. Interaction with virtual content using augmented reality: a user study in assembly procedures. Proceedings of the ACM on Human-Computer Interaction 4, ISS (2020), 1–17.
Jack Miller, Melynda Hoover, and Eliot Winer. 2019. Overcoming limitations of the HoloLens for use in product assembly. Electronic Imaging 31 (2019), 1–9.
Jack Miller, Melynda Hoover, and Eliot Winer. 2020. Mitigation of the Microsoft HoloLens’ hardware limitations for a controlled product assembly process. The International Journal of Advanced Manufacturing Technology 109 (2020), 1741– 1754.
Ruben Miranda and Esther Garcia-Carpintero. 2018. Overcitation and overrepresentation of review papers in the most cited papers. Journal of Informetrics 12, 4 (2018), 1015–1030.
Mohsen Moghaddam, Nicholas C Wilson, Alicia Sasser Modestino, Kemi Jona, and Stacy C Marsella. 2021. Exploring augmented reality for worker assistance versus training. Advanced Engineering Informatics 50 (2021), 101410.
Saeed Na’amnh, István Husti, and Miklós Daróczi. 2021. Implementing the Augmented Reality as an Industry 4.0 Application to Simplify the Busbar Bending Process during the Covid-19 Pandemic. Transactions of FAMENA 45, 3 (2021), 115–125.
G Nageswara Rao, K Aruna Kumari, D Ravi Shankar, and KG Kharade. 2021. A comparative study of augmented reality-based head-worn display devices. Materials Today: Proceedings (2021). https://doi.org/10.1016/j.matpr.2020.12.400
Elisa Yumi Nakagawa, Kátia Romero Felizardo Scannavino, Sandra Camargo Pinto Ferraz Fabbri, and Fabiano Cutigi Ferrari. 2017. Revisão sistemática da literatura em engenharia de software: teoria e prática. Elsevier: Rio de Janeiro, Brasil.
Alexander Neb, David Brandt, Ramez Awad, Silvana Heckelsmüller, and Thomas Bauernhansl. 2021. Usability study of a user-friendly AR assembly assistance. Procedia CIRP 104 (2021), 74–79.
Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology 64 (2015), 1–18.
Francesco Pilati, Maurizio Faccio, Mauro Gamberi, and Alberto Regattieri. 2020. Learning manual assembly through real-time motion capture for operator training with augmented reality. Procedia Manufacturing 45 (2020), 189–195.
Fernando R Pusda, Francisco F Valencia, Víctor H Andaluz, and Víctor D Zambrano. 2019. Training Assistant for Automotive Engineering Through Augmented Reality. In Augmented Reality, Virtual Reality, and Computer Graphics: 6th International Conference, AVR 2019, Santa Maria al Bagno, Italy, June 24–27, 2019, Proceedings, Part II 6. Springer, 146–160.
Moritz Quandt, A Ait Alla, Lars Meyer, and Michael Freitag. 2017. Success factors for the development of augmented reality-based assistance systems for maintenance services. Schmitt, RH, Schuh G (Hrsg) 7 (2017), 175–182.
Rafael Radkowski. 2015. Investigation of visual features for augmented reality assembly assistance. In Virtual, Augmented and Mixed Reality: 7th International Conference, VAMR 2015, Held as Part of HCI International 2015, Los Angeles, CA, USA, August 2-7, 2015, Proceedings 7. Springer, 488–498.
Anes Redžepagić, Christoffer Löffler, Tobias Feigl, and Christopher Mutschler. 2020. A sense of quality for augmented reality assisted process guidance. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 129–134.
Patrick Renner and Thies Pfeiffer. 2017. Evaluation of attention guiding techniques for augmented reality-based assistance in picking and assembly tasks. In Proceedings of the 22nd international conference on intelligent user interfaces companion. 89–92.
Mark Rice, Hong Huei Tay, Jamie Ng, Calvin Lim, Senthil Kumar Selvaraj, and Ellick Wu. 2015. [POSTER] Augmented Wire Routing Navigation for Wire Assembly. In 2015 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 88–91.
Mark Rice, Hong Huei Tay, Jamie Ng, Calvin Lim, Senthil Kumar Selvaraj, and Ellick Wu. 2016. Comparing three task guidance interfaces for wire harness assembly. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2279–2284.
E Fabián Rivera, Marco V Pilco, Paul S Espinoza, Edison E Morales, and Jessica S Ortiz. 2020. Training system for hybrid vehicles through augmented reality. In 2020 15th Iberian Conference on Information Systems and Technologies (CISTI). IEEE, 1–6.
Francisca S Rodriguez, Khadija Saleem, Jan Spilski, and Thomas Lachmann. 2021. Performance differences between instructions on paper vs digital glasses for a simple assembly task. Applied Ergonomics 94 (2021), 103423.
Marta Rossi, Alessandra Papetti, Michele Germani, and Marco Marconi. 2020. An Augmented Reality System for Operator Training in the Footwear Sector. Computer-Aided Design and Applications 18 (2020), 692–703.
Patrick RUPPRECHT, Hans KUEFFNER-MCCAULEY, Majesa TRIMMEL, Michael HORNACEK, and Sebastian SCHLUND. 2022. Advanced Adaptive Spatial Augmented Reality utilizing Dynamic in-situ Projection in Industrial Site Assembly. Procedia CIRP 107 (2022), 937–942.
Prasanth Sasikumar, Soumith Chittajallu, Navindd Raj, Huidong Bai, and Mark Billinghurst. 2021. Spatial perception enhancement in assembly training using augmented volumetric playback. Frontiers in Virtual Reality 2 (2021), 698523.
Florian Schuster, Uwe Sponholz, Bastian Engelmann, and Jan Schmitt. 2020. A user study on AR-assisted industrial assembly. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 135–140.
Mustafa Sirakaya and Ebru Kilic Cakmak. 2018. Effects of augmented reality on student achievement and self-efficacy in vocational education and training. International journal for research in vocational education and training 5, 1 (2018), 1–18.
Eleanor Smith, Gordon Semple, Dorothy Evans, Kenneth McRae, and Paul Blackwell. 2020. Augmented instructions: analysis of performance and efficiency of assembly tasks. In Virtual, Augmented and Mixed Reality. Industrial and Everyday Life Applications: 12th International Conference, VAMR 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22. Springer, 166–177.
Anmol Srivastava and Pradeep Yammiyavar. 2016. Design of multimodal instructional tutoring agents using augmented reality and smart learning objects. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 421–422.
Takaharu Suzuki, Hikaru Ikeda, Issei Takeuchi, Fumiya Matsunaga, Eri Sumitomo, and Yasushi Ikeda. 2020. Holonavi: A study on user interface for assembly guidance system with mixed reality in a timber craft of architecture. In 25th International Conference on Computer-Aided Architectural Design Research in Asia, CAADRIA 2020. The Association for Computer-Aided Architectural Design Research in Asia . . . , 691–700.
Ryuta Tanaka, Takamitsu Tanaka, and Kouichi Konno. 2021. An examination of displaying reassembly procedure by recording disassembly procedure with AR markers. In International Workshop on Advanced Imaging Technology (IWAIT) 2021, Vol. 11766. SPIE, 510–513.
Kätlin Värno, Kashif Mahmood, Tauno Otto, and Vladimir Kuts. 2019. Development of a smart workstation by using AR Technology. Annals of DAAAM & Proceedings 30 (2019).
Chao-Hung Wang, Ni-Hsin Tsai, Jun-Ming Lu, and Mao-Jiun J Wang. 2019. Usability evaluation of an instructional application based on Google Glass for mobile phone disassembly tasks. Applied Ergonomics 77 (2019), 58–69.
Peng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Sili Wei, Guangyao Xu, Weiping He, Xiangyu Zhang, and Jie Zhang. 2021. 3DGAM: using 3D gesture and CAD models for training on mixed reality remote collaboration. Multimedia Tools and Applications 80 (2021), 31059–31084.
Xuan Wang, SK Ong, and Andrew Yeh-Ching Nee. 2016. Multi-modal augmentedreality assembly guidance based on bare-hand interface. Advanced Engineering Informatics 30, 3 (2016), 406–421.
Xiangyu Wang, Soh K Ong, and Andrew YC Nee. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing 4 (2016), 1–22.
Sabine Webel, Uli Bockholt, Timo Engelke, Nirit Gavish, Manuel Olbrich, and Carsten Preusche. 2013. An augmented reality training platform for assembly and maintenance skills. Robotics and autonomous systems 61, 4 (2013), 398–403.
Stefan Werrlich, Elisabeth Eichstetter, Kai Nitsche, and Gunther Notni. 2017. An overview of evaluations using augmented reality for assembly training tasks. International Journal of Computer and Information Engineering 11, 10 (2017), 1068–74.
Stefan Werrlich, Phuc-Anh Nguyen, and Gunther Notni. 2018. Evaluating the training transfer of Head-Mounted Display based training for assembly tasks. In Proceedings of the 11th Pervasive Technologies Related to Assistive Environments Conference. 297–302.
Stefan Werrlich, Kai Nitsche, and Gunther Notni. 2017. Demand analysis for an augmented reality based assembly training. In Proceedings of the 10th international conference on pervasive technologies related to assistive environments. 416–422.
Ellen S Wilschut, Reinier Könemann, Molly S Murphy, Gu JW Van Rhijn, and Tim Bosch. 2019. Evaluating learning approaches for product assembly: using chunking of instructions, spatial augmented reality and display based work instructions. In Proceedings of the 12th ACM International Conference on Pervasive Technologies Related to Assistive Environments. 376–381.
Li-Chen Wu, I-Chen Lin, and Ming-Han Tsai. 2016. Augmented reality instruction for object assembly based on markerless tracking. In Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. 95–102.
Masahiro Yamaguchi, Shohei Mori, Peter Mohr, Markus Tatzgern, Ana Stanescu, Hideo Saito, and Denis Kalkofen. 2020. Video-annotated augmented reality assembly tutorials. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 1010–1022.
Wei Yan. 2019. Augmented reality applied to lego construction: Ar-based building instructions with high accuracy & precision and realistic object-hand occlusions. arXiv preprint arXiv:1907.12549 (2019).
Hailong Yang, Shuai Li, Xiaojun Zhang, and Qun Shen. 2021. Research on Satellite Cable Laying and Assembly Guidance Technology Based on Augmented Reality. In 2021 40th Chinese Control Conference (CCC). IEEE, 6550–6555.
Yumeng Yang, Joyce Karreman, and Menno De Jong. 2020. Comparing the effects of paper and mobile augmented reality instructions to guide assembly tasks. In 2020 IEEE International Professional Communication Conference (ProComm). IEEE, 96–104.
Jeroen Zegers, Vasilios Zogopoulos, and Dries Verhees. 2022. Recommender systems for Personalized Work Instructions. Procedia CIRP 109 (2022), 496–501.
Malek Alrashidi, Ahmed Alzahrani, Michael Gardner, and Vic Callaghan. 2016. A pedagogical virtual machine for assembling mobile robot using augmented reality. In Proceedings of the 7th Augmented Human International Conference 2016. 1–2.
Nikolaus Angel, Horst Orsolits, and Jose Garcia. 2022. Developing an AR Based Tool for Teaching Motion Analysis on Assembly Tasks. In 2022 8th International Conference of the Immersive Learning Research Network (iLRN). IEEE, 1–7.
JC Arbeláez, Roberto Viganò, and Gilberto Osorio-Gómez. 2019. Haptic augmented reality (HapticAR) for assembly guidance. International Journal on Interactive Design and Manufacturing (IJIDeM) 13 (2019), 673–687.
Doris Aschenbrenner, Michael Rojkov, Florian Leutert, Jouke Verlinden, Stephan Lukosch, Marc Erich Latoschik, and Klaus Schilling. 2018. Comparing different augmented reality support applications for cooperative repair of an industrial robot. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 69–74.
Faieza Abdul Aziz, AS Alsaeed, Shamsuddin Sulaiman, MK Ariffin, and Muhammad Faris Al-Hakim. 2020. Mixed reality improves education and training in assembly processes. Journal of Engineering and Technological Sciences 52, 4 (2020), 598–607.
Rudieri Dietrich Bauer, Thiago Luiz Watambak, Salvador Sergi Agati, Marcelo da Silva Hounsell, and Andre Tavares da Silva. 2021. Development and Evaluation of a PCB‘s Manual Assembly system using Augmented Reality and Total Quality. In Symposium on Virtual and Augmented Reality. 24–32.
Andreea Blaga, Cristian Militaru, Ady-Daniel Mezei, and Levente Tamas. 2021. Augmented reality integration into MES for connected workers. Robotics and Computer-Integrated Manufacturing 68 (2021), 102057.
Jonas Blattgerste, Patrick Renner, Benjamin Strenge, and Thies Pfeiffer. 2018. Insitu instructions exceed side-by-side instructions in augmented reality assisted assembly. In Proceedings of the 11th Pervasive technologies related to assistive environments conference. 133–140.
Jonas Blattgerste, Benjamin Strenge, Patrick Renner, Thies Pfeiffer, and Kai Essig. 2017. Comparing conventional and augmented reality instructions for manual assembly tasks. In Proceedings of the 10th international conference on pervasive technologies related to assistive environments. 75–82.
Francesco Bonavolontá, Dominique Dallet, Ernesto Erra, Andrea Grassi, Valentina Popolo, Annarita Tedesco, and Silvestro Vespoli. 2020. Measuring worker’s performance in augmented reality-assisted Industry 4.0 procedures. In 2020 IEEE International Instrumentation and Measurement Technology Conference (I2MTC). IEEE, 1–6.
Poonpong Boonbrahm and Charlee Kaewrat. 2014. Assembly of the virtual model with real hands using augmented reality technology. In Virtual, Augmented and Mixed Reality. Designing and Developing Virtual and Augmented Environments: 6th International Conference, VAMR 2014, Held as Part of HCI International 2014, Heraklion, Crete, Greece, June 22-27, 2014, Proceedings, Part I 6. Springer, 329–338.
Tim Bosch, Gu Van Rhijn, Frank Krause, Reinier Könemann, Ellen S Wilschut, and Michiel de Looze. 2020. Spatial augmented reality: a tool for operator guidance and training evaluated in five industrial case studies. In Proceedings of the 13th ACM International Conference on Pervasive technologies related to assistive environments. 1–7.
Sebastian Büttner, Michael Prilla, and Carsten Röcker. 2020. Augmented reality training for industrial assembly work-are projection-based AR assistive systems an appropriate tool for assembly training?. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–12.
Sebastian Büttner, Oliver Sand, and Carsten Röcker. 2015. Extending the design space in industrial manufacturing through mobile projection. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. 1130–1133.
Chih-Hsing Chu and Ching-Hung Ko. 2021. An experimental study on augmented reality assisted manual assembly with occluded components. Journal of Manufacturing Systems 61 (2021), 685–695.
Luís Fernando de Souza Cardoso, Flávia Cristina Martins Queiroz Mariano, and Ezequiel Roberto Zorzal. 2020. Mobile augmented reality to support fuselage assembly. Computers & Industrial Engineering 148 (2020), 106712.
Iñigo Fernández del Amo, John Ahmet Erkoyuncu, Rajkumar Roy, Riccardo Palmarini, and Demetrius Onoufriou. 2018. A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications. Computers in Industry 103 (2018), 47–71.
Abhiraj Deshpande and Inki Kim. 2018. The effects of augmented reality on improving spatial problem solving for object assembly. Advanced Engineering Informatics 38 (2018), 760–775.
Raquel Quesada Díaz and Anna Syberfeldt. 2022. Optimised Shop-Floor Operator Support on an Integrated AR and PLM Framework for Remanufacturing. In SPS2022: Proceedings of the 10th Swedish Production Symposium, Vol. 21. IOS Press, 425.
Wei Fang and Jianhao Hong. 2022. Bare-hand gesture occlusion-aware interactive augmented reality assembly. Journal of Manufacturing Systems 65 (2022), 169– 179.
Shuo Feng, Weiping He, Shaohua Zhang, and Mark Billinghurst. 2022. Seeing is believing: AR-assisted blind area assembly to support hand–eye coordination. The International Journal of Advanced Manufacturing Technology 119, 11-12 (2022), 8149–8158.
Francesca Ferrati, John Ahmet Erkoyuncu, and Samuel Court. 2019. Developing an augmented reality based training demonstrator for manufacturing cherry pickers. Procedia CIRP 81 (2019), 803–808.
Markus Funk and Albrecht Schmidt. 2015. Cognitive assistance in the workplace. IEEE Pervasive Computing 14, 3 (2015), 53–55.
Lorenzo Giunta, Jamie O’Hare, James Gopsill, Elies Dekoninck, et al. 2018. A review of augmented reality research for design practice: looking to the future. DS 91: Proceedings of NordDesign 2018, Linköping, Sweden, 14th-17th August 2018 (2018).
Mikell P Groover. 2011. Industrial automation and manufacturing systems. Pearson Brasil.
Jürgen Hahn, Bernd Ludwig, and Christian Wolff. 2015. Augmented reality-based training of the PCB assembly process. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia. 395–399.
Mario Heinz, Sebastian Büttner, Sascha Jenderny, and Carsten Röcker. 2021. Dynamic task allocation based on individual abilities-experiences from developing and operating an inclusive assembly line for workers with and without disabilities. Proceedings of the ACM on Human-Computer Interaction 5, EICS (2021), 1–19.
Bradley M Herbert, William Hoff, and Mark Billinghurst. 2020. Usability considerations of hand held augmented reality wiring tutors. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 277–282.
Hung-Yu Ho, An-Hsiang Wang, and Chia-Huang Wu. 2022. Effects of Training Methods on Performance of Scaffolding Task for Workers with Different Spatial Ability. Available at SSRN 4203648 (2022).
Jannike Illing, Philipp Klinke, Uwe Grünefeld, Max Pfingsthorn, and Wilko Heuten. 2020. Time is money! evaluating augmented reality instructions for time-critical assembly tasks. In Proceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia. 277–287.
Jannike Illing, Philipp Klinke, Max Pfingsthorn, and Wilko Heuten. 2021. Less is more! Support of parallel and time-critical assembly tasks with augmented reality. In Proceedings of Mensch und Computer 2021. 215–226.
Kazuyo IWAMOTO. 2019. Development of 4K Binocular Video See-through Display with Haptic Device for Task Training. In 2019 23rd International Computer Science and Engineering Conference (ICSEC). IEEE, 158–163.
Knoch S Alexandersson J. Jost M, A. 2022. PARTAS: A Personalizable Augmented Reality Based Task Adaption System for Workers with Cognitive Disabilities. InProceedings of the 15th International Conference on Pervasive Technologies Related to Assistive Environments 2022 Jun 29 (2022), 159–168.
Nawel Khenak, Jeanne Vézien, and Patrick Bourdot. 2020. Effectiveness of augmented reality guides for blind insertion tasks. Frontiers in virtual reality 1 (2020), 588217.
Bui Minh Khuong, Kiyoshi Kiyokawa, Andrew Miller, Joseph J La Viola, Tomohiro Mashita, and Haruo Takemura. 2014. The effectiveness of an AR-based contextaware assembly support system in object assembly. In 2014 IEEE Virtual Reality (VR). 57–62.
Sunwook Kim, Maury A Nussbaum, and Joseph L Gabbard. 2019. Influences of augmented reality head-worn display type and user interface design on performance and usability in simulated warehouse order picking. Applied ergonomics 74 (2019), 186–193.
Gyuwon Kwak, Ji Seong Min, and Wonil Hwang. 2020. Characteristics of usability problems from augmented reality-based user interface for assembly task. ICIC Express Letters 14, 4 (2020), 353–360.
Caroline Kwiatek, M Sharif, Shangkun Li, Carl Haas, and Scott Walbridge. 2019. Impact of augmented reality and spatial cognition on assembly in construction. Automation in construction 108 (2019), 102935.
Ze-Hao Lai, Wenjin Tao, Ming C Leu, and Zhaozheng Yin. 2020. Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing. Journal of Manufacturing Systems 55 (2020), 69–81.
Eva Lampen, Jonas Teuber, Felix Gaisbauer, Thomas Bär, Thies Pfeiffer, and Sven Wachsmuth. 2019. Combining simulation and augmented reality methods for enhanced worker assistance in manual assembly. Procedia CIRP 81 (2019), 588–593.
Traian Lavric, Emmanuel Bricard, Marius Preda, and Titus Zaharia. 2021. Exploring low-cost visual assets for conveying assembly instructions in AR. In 2021 International Conference on Innovations in Intelligent Systems and Applications (INISTA). IEEE, 1–6.
Traian Lavric, Emmanuel Bricard, Marius Preda, and Titus Zaharia. 2022. A low-cost AR training system for manual assembly operations. Computer Science and Information Systems 19 (2022), 1047–1073.
Federico Manuri and Andrea Sanna. 2016. A survey on applications of augmented reality. ACSIJ Advances in Computer Science: an International Journal 5, 1 (2016), 18–27.
Bernardo Marques, João Alves, Miguel Neves, Inês Justo, André Santos, Raquel Rainho, Rafael Maio, Dany Costa, Carlos Ferreira, Paulo Dias, et al. 2020. Interaction with virtual content using augmented reality: a user study in assembly procedures. Proceedings of the ACM on Human-Computer Interaction 4, ISS (2020), 1–17.
Jack Miller, Melynda Hoover, and Eliot Winer. 2019. Overcoming limitations of the HoloLens for use in product assembly. Electronic Imaging 31 (2019), 1–9.
Jack Miller, Melynda Hoover, and Eliot Winer. 2020. Mitigation of the Microsoft HoloLens’ hardware limitations for a controlled product assembly process. The International Journal of Advanced Manufacturing Technology 109 (2020), 1741– 1754.
Ruben Miranda and Esther Garcia-Carpintero. 2018. Overcitation and overrepresentation of review papers in the most cited papers. Journal of Informetrics 12, 4 (2018), 1015–1030.
Mohsen Moghaddam, Nicholas C Wilson, Alicia Sasser Modestino, Kemi Jona, and Stacy C Marsella. 2021. Exploring augmented reality for worker assistance versus training. Advanced Engineering Informatics 50 (2021), 101410.
Saeed Na’amnh, István Husti, and Miklós Daróczi. 2021. Implementing the Augmented Reality as an Industry 4.0 Application to Simplify the Busbar Bending Process during the Covid-19 Pandemic. Transactions of FAMENA 45, 3 (2021), 115–125.
G Nageswara Rao, K Aruna Kumari, D Ravi Shankar, and KG Kharade. 2021. A comparative study of augmented reality-based head-worn display devices. Materials Today: Proceedings (2021). https://doi.org/10.1016/j.matpr.2020.12.400
Elisa Yumi Nakagawa, Kátia Romero Felizardo Scannavino, Sandra Camargo Pinto Ferraz Fabbri, and Fabiano Cutigi Ferrari. 2017. Revisão sistemática da literatura em engenharia de software: teoria e prática. Elsevier: Rio de Janeiro, Brasil.
Alexander Neb, David Brandt, Ramez Awad, Silvana Heckelsmüller, and Thomas Bauernhansl. 2021. Usability study of a user-friendly AR assembly assistance. Procedia CIRP 104 (2021), 74–79.
Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology 64 (2015), 1–18.
Francesco Pilati, Maurizio Faccio, Mauro Gamberi, and Alberto Regattieri. 2020. Learning manual assembly through real-time motion capture for operator training with augmented reality. Procedia Manufacturing 45 (2020), 189–195.
Fernando R Pusda, Francisco F Valencia, Víctor H Andaluz, and Víctor D Zambrano. 2019. Training Assistant for Automotive Engineering Through Augmented Reality. In Augmented Reality, Virtual Reality, and Computer Graphics: 6th International Conference, AVR 2019, Santa Maria al Bagno, Italy, June 24–27, 2019, Proceedings, Part II 6. Springer, 146–160.
Moritz Quandt, A Ait Alla, Lars Meyer, and Michael Freitag. 2017. Success factors for the development of augmented reality-based assistance systems for maintenance services. Schmitt, RH, Schuh G (Hrsg) 7 (2017), 175–182.
Rafael Radkowski. 2015. Investigation of visual features for augmented reality assembly assistance. In Virtual, Augmented and Mixed Reality: 7th International Conference, VAMR 2015, Held as Part of HCI International 2015, Los Angeles, CA, USA, August 2-7, 2015, Proceedings 7. Springer, 488–498.
Anes Redžepagić, Christoffer Löffler, Tobias Feigl, and Christopher Mutschler. 2020. A sense of quality for augmented reality assisted process guidance. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 129–134.
Patrick Renner and Thies Pfeiffer. 2017. Evaluation of attention guiding techniques for augmented reality-based assistance in picking and assembly tasks. In Proceedings of the 22nd international conference on intelligent user interfaces companion. 89–92.
Mark Rice, Hong Huei Tay, Jamie Ng, Calvin Lim, Senthil Kumar Selvaraj, and Ellick Wu. 2015. [POSTER] Augmented Wire Routing Navigation for Wire Assembly. In 2015 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 88–91.
Mark Rice, Hong Huei Tay, Jamie Ng, Calvin Lim, Senthil Kumar Selvaraj, and Ellick Wu. 2016. Comparing three task guidance interfaces for wire harness assembly. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. 2279–2284.
E Fabián Rivera, Marco V Pilco, Paul S Espinoza, Edison E Morales, and Jessica S Ortiz. 2020. Training system for hybrid vehicles through augmented reality. In 2020 15th Iberian Conference on Information Systems and Technologies (CISTI). IEEE, 1–6.
Francisca S Rodriguez, Khadija Saleem, Jan Spilski, and Thomas Lachmann. 2021. Performance differences between instructions on paper vs digital glasses for a simple assembly task. Applied Ergonomics 94 (2021), 103423.
Marta Rossi, Alessandra Papetti, Michele Germani, and Marco Marconi. 2020. An Augmented Reality System for Operator Training in the Footwear Sector. Computer-Aided Design and Applications 18 (2020), 692–703.
Patrick RUPPRECHT, Hans KUEFFNER-MCCAULEY, Majesa TRIMMEL, Michael HORNACEK, and Sebastian SCHLUND. 2022. Advanced Adaptive Spatial Augmented Reality utilizing Dynamic in-situ Projection in Industrial Site Assembly. Procedia CIRP 107 (2022), 937–942.
Prasanth Sasikumar, Soumith Chittajallu, Navindd Raj, Huidong Bai, and Mark Billinghurst. 2021. Spatial perception enhancement in assembly training using augmented volumetric playback. Frontiers in Virtual Reality 2 (2021), 698523.
Florian Schuster, Uwe Sponholz, Bastian Engelmann, and Jan Schmitt. 2020. A user study on AR-assisted industrial assembly. In 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 135–140.
Mustafa Sirakaya and Ebru Kilic Cakmak. 2018. Effects of augmented reality on student achievement and self-efficacy in vocational education and training. International journal for research in vocational education and training 5, 1 (2018), 1–18.
Eleanor Smith, Gordon Semple, Dorothy Evans, Kenneth McRae, and Paul Blackwell. 2020. Augmented instructions: analysis of performance and efficiency of assembly tasks. In Virtual, Augmented and Mixed Reality. Industrial and Everyday Life Applications: 12th International Conference, VAMR 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22. Springer, 166–177.
Anmol Srivastava and Pradeep Yammiyavar. 2016. Design of multimodal instructional tutoring agents using augmented reality and smart learning objects. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 421–422.
Takaharu Suzuki, Hikaru Ikeda, Issei Takeuchi, Fumiya Matsunaga, Eri Sumitomo, and Yasushi Ikeda. 2020. Holonavi: A study on user interface for assembly guidance system with mixed reality in a timber craft of architecture. In 25th International Conference on Computer-Aided Architectural Design Research in Asia, CAADRIA 2020. The Association for Computer-Aided Architectural Design Research in Asia . . . , 691–700.
Ryuta Tanaka, Takamitsu Tanaka, and Kouichi Konno. 2021. An examination of displaying reassembly procedure by recording disassembly procedure with AR markers. In International Workshop on Advanced Imaging Technology (IWAIT) 2021, Vol. 11766. SPIE, 510–513.
Kätlin Värno, Kashif Mahmood, Tauno Otto, and Vladimir Kuts. 2019. Development of a smart workstation by using AR Technology. Annals of DAAAM & Proceedings 30 (2019).
Chao-Hung Wang, Ni-Hsin Tsai, Jun-Ming Lu, and Mao-Jiun J Wang. 2019. Usability evaluation of an instructional application based on Google Glass for mobile phone disassembly tasks. Applied Ergonomics 77 (2019), 58–69.
Peng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Sili Wei, Guangyao Xu, Weiping He, Xiangyu Zhang, and Jie Zhang. 2021. 3DGAM: using 3D gesture and CAD models for training on mixed reality remote collaboration. Multimedia Tools and Applications 80 (2021), 31059–31084.
Xuan Wang, SK Ong, and Andrew Yeh-Ching Nee. 2016. Multi-modal augmentedreality assembly guidance based on bare-hand interface. Advanced Engineering Informatics 30, 3 (2016), 406–421.
Xiangyu Wang, Soh K Ong, and Andrew YC Nee. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing 4 (2016), 1–22.
Sabine Webel, Uli Bockholt, Timo Engelke, Nirit Gavish, Manuel Olbrich, and Carsten Preusche. 2013. An augmented reality training platform for assembly and maintenance skills. Robotics and autonomous systems 61, 4 (2013), 398–403.
Stefan Werrlich, Elisabeth Eichstetter, Kai Nitsche, and Gunther Notni. 2017. An overview of evaluations using augmented reality for assembly training tasks. International Journal of Computer and Information Engineering 11, 10 (2017), 1068–74.
Stefan Werrlich, Phuc-Anh Nguyen, and Gunther Notni. 2018. Evaluating the training transfer of Head-Mounted Display based training for assembly tasks. In Proceedings of the 11th Pervasive Technologies Related to Assistive Environments Conference. 297–302.
Stefan Werrlich, Kai Nitsche, and Gunther Notni. 2017. Demand analysis for an augmented reality based assembly training. In Proceedings of the 10th international conference on pervasive technologies related to assistive environments. 416–422.
Ellen S Wilschut, Reinier Könemann, Molly S Murphy, Gu JW Van Rhijn, and Tim Bosch. 2019. Evaluating learning approaches for product assembly: using chunking of instructions, spatial augmented reality and display based work instructions. In Proceedings of the 12th ACM International Conference on Pervasive Technologies Related to Assistive Environments. 376–381.
Li-Chen Wu, I-Chen Lin, and Ming-Han Tsai. 2016. Augmented reality instruction for object assembly based on markerless tracking. In Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. 95–102.
Masahiro Yamaguchi, Shohei Mori, Peter Mohr, Markus Tatzgern, Ana Stanescu, Hideo Saito, and Denis Kalkofen. 2020. Video-annotated augmented reality assembly tutorials. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. 1010–1022.
Wei Yan. 2019. Augmented reality applied to lego construction: Ar-based building instructions with high accuracy & precision and realistic object-hand occlusions. arXiv preprint arXiv:1907.12549 (2019).
Hailong Yang, Shuai Li, Xiaojun Zhang, and Qun Shen. 2021. Research on Satellite Cable Laying and Assembly Guidance Technology Based on Augmented Reality. In 2021 40th Chinese Control Conference (CCC). IEEE, 6550–6555.
Yumeng Yang, Joyce Karreman, and Menno De Jong. 2020. Comparing the effects of paper and mobile augmented reality instructions to guide assembly tasks. In 2020 IEEE International Professional Communication Conference (ProComm). IEEE, 96–104.
Jeroen Zegers, Vasilios Zogopoulos, and Dries Verhees. 2022. Recommender systems for Personalized Work Instructions. Procedia CIRP 109 (2022), 496–501.
Publicado
06/11/2023
Como Citar
PEIXE, Lilian Cristina; AGATI, Salvador; HOUNSELL, Marcelo Da Silva.
Manual Assembly Augmented Reality Systems Implementation: A Systematic Literature Mapping. In: SIMPÓSIO DE REALIDADE VIRTUAL E AUMENTADA (SVR), 25. , 2023, Rio Grande/RS.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2023
.
p. 17–25.
