AI-supported knowledge transfer through audiovisual documentation of systems and processes

In a joint use case involving four companies, knowledge management was identified as a central field for innovation. In operational deployment environments, valuable experiential knowledge is generated directly in daily work, but it is often not systematically captured, is person-specific, or difficult to find. This creates risks of knowledge loss, for example, due to retirement or job changes. A practical, AI-supported solution is sought for capturing, structuring, and providing implicit field knowledge. The focus is on the audiovisual documentation of systems and processes, as well as easy use in daily work to support onboarding, problem documentation, and knowledge retention. Solution approaches already implemented: • Use of existing knowledge management solutions, e.g., documents, training materials, file storage, as well as office and collaboration systems. • Partial use of isolated knowledge platforms and multiple databases with limited networking. • Debriefing and interview formats for securing the knowledge of experienced employees (e.g., before retirement or offboarding). • Projects already underway or completed for the documentation of systems, the findings of which should be taken into account. • Previous approaches in the field of Augmented Reality (AR), whose experiences can be incorporated into solution development. • Low-code applications for documenting events with image material. Limitations of previous approaches: • Existing solutions are not sufficiently field-ready, particularly lacking hands-free use. • Uniform standards and a consistent structure for knowledge documentation are missing. • Knowledge is difficult to find and only limitedly usable, as it is distributed across various systems. • Implicit experiential knowledge has not been systematically captured and secured to date. • An intelligent, user-friendly solution for recording, structuring, and providing experiential knowledge has been lacking so far.

Possible solution approaches: - Use of a hands-free, AI-supported solution for knowledge acquisition in the field and ideally also in the office environment - Recording of work processes via video, photo, and audio with parallel voice commentary by employees - AI-supported transcription, segmentation, enrichment, tagging, and structuring of content - Provision of content via a central knowledge platform with AI-supported search - Examination of various recording formats, including audio recordings with supplementary images and video recordings of systems or processes - Examination of alternative technologies such as smart glasses, wearables, or other integrated technologies - Use for application areas such as offboarding, fieldwork, onboarding, independent knowledge retention, documentation of complex operational situations, and training Requirements for the solution - The solution must enable hands-free audio and video recording in the field, for example via helmet camera, smart glasses, wearables, or comparable hardware. - Operation must be possible without using hands, for example via voice triggers or buttons. - Employees must be able to comment on recordings in parallel by voice. - The solution must support video, photo, and audio recordings. - The solution must offer robust operational capability under field conditions, including outdoors, on construction sites, in plants, or on technical systems. - Sufficient battery life for operational use is required. - The solution must support the German language, including transcription and text enrichment. - The AI must be able to segment recordings into meaningful knowledge modules, sections, or chapters. - The solution must be able to enrich, tag, and structure content. - Use must be simple and intuitive for knowledge providers and knowledge recipients. - Documentation must be possible both on-site in systems and in the office. - Mobile, independent tools are needed; use on the company mobile phone is advantageous. - The control over the recording must lie with the knowledge providers. - Content may only be added to the knowledge pool after approval by the knowledge providers. - Co-determination, data protection, and personality rights must be considered technically and procedurally. - No permanent recording may occur. - The solution must be integrable into existing IT landscapes; an interface to existing office and collaboration systems is advantageous. - Digital sovereignty is a must for individual companies involved. - AI models used must be interchangeable and from Europe. Dependencies on nationally characterized models, especially from the USA or China, are to be avoided. - Experience with AI-based video analysis, speech-to-text, natural language processing, and knowledge management systems is expected. - Solutions for field-ready hardware or partnerships in this area are expected. - Experience in industrial or operational environments is advantageous. - Language and market understanding for the DACH region is advantageous. - Gamification elements to increase user acceptance and knowledge contributions are nice-to-have.