Innovative Technologies in Training and Education for Maintenance Team of NPPs

Many industries, such as nuclear power plants, chemical industry, oil and gas industry have dangerous working environments and hazardous conditions for employees. Maintenance, inspection and decommissioning activities in these safety-critical areas mean a serious risks, downtime is a significant financial loss.

The Virtual Reality Training Platform is reflecting on this shortcoming, by providing the possibility for maintenance workers to be trained and prepared for unexpected scenarios, and to learn complex maintenance protocols without being exposed to unnecessary danger, like high temperature, radiation, etc. Employees can have training for equipment maintenance, dismantling of facilities at closed NPP Units. One of the most significant and unique added value of the immersive virtual reality solution is that the operator can experience lifelike emergencies (detonation, havarias) under psychological pressure, while all of the physiology indicators can be monitored like eye-tracking. Users can work together anywhere in the world. A huge financial outage in industrial production is the preparation and maintenance downtime, which can be significantly reduced by the Virtual Training platform. This method can increase the accuracy, safety, reliability, and accountability of the maintenance and decommissioning procedures, while operational costs can be reduced as well.



In today’s industry, quick response and fast execution of well-learnt procedures is critical. Many people work in factories, where circumstances can be fatal in cases. For example, nuclear power plants have spots where people can only stay minutes due the harmful health effects of radiation. Dangerous places are not only present in power plants, there are also machines operating under water or in high altitude. People who are working in these environments can get injured easily if they are not attentive enough. However, maintenance of these machines has to be done, so maintenance workers must be very efficient, fast, precise and well-trained when they have to visit these places. Even when circumstances are not dangerous, there are several machines the faulty operation of which can cause huge risk or loss of money. These devices also have to be maintained regularly and efficiently.

Due to these facts, workers have to be trained several times and practice the movements very well before participating in real missions. Nowadays, most of the training is done on real copies of these machines, which are not currently operating and the only purpose of them is to help the training. Maintenance can be practiced in a very realistic way using these, however, there are also several drawbacks of the method. Ordering one more appliance can be very expensive to buy and maintain, while often requires much more space and occasionally operators as well. There is usually only one training appliance which is not flexible, so most employees can only get to use it few times if, because there are many people to train and travel costs may also be incurred.

The other problem is that while the appliance can be studied very closely, their environment cannot really be simulated even though this would be very important in many fields of application, especially when the real work has to be done in extreme circumstances. For example, firefighters can be trained how to operate water pumps and hoses efficiently but are not really able to feel the danger of situation when there are real people and real fire.

Interactive computer-based trainings are also available in many fields by now. It can be very cheap and flexible, but not close to trying a real machine, as using a keyboard or mouse cannot give the immersion needed to really memorise a procedure or series of movements.


VR training platform

In the Virtual Reality Training Platform developed by Bay Zoltán Nonprofit Ltd., the latest Virtual Reality technologies are used to help training of maintenance workers. It provides the possibility to practice complex working processes in advance, be prepared for unexpected situations and receive knowledge of the area safely, without any hazards. VR service can be applied for increasing the experience and knowledge of the personnel in the field of maintenance and operation in power plants, chemical industry, refinery plants and production companies. On the other hand, adequate operation training of high value machinery without imposing any risk on the real equipment state is also possible. The main purpose of the platform is decreasing human factor, assuring safer work and operation conditions and replacing expensive training centres with a safe and innovative education system with cost-effective periodic trainings.

Unlike in real appliance-based trainings, no special equipment is needed, so this solution can be cheaper and more flexible because a real machine does not have to be purchased. However, it still provides realism and precision unlike conventional computer programs and videos.

The other big advantage of computer support is that everything can be measured precisely during the training. For example the working time of maintenance or the hardest part of the procedure can be easily detected as all data can be recorded and analysed during the training without the need for any human staff, but operators can still help employees remotely during the training if necessary and the system can be used anywhere, even at the home of each employee, regardless of the distance from the original working place.


Structure of the system

The virtual training system contains key elements both on hardware and software side. Its most important part is a PC-connected VR headset – primarily Oculus Rift or HTC Vive – which is worn by the user during the training. The PC has to be powerful enough to maintain high-enough framerate (preferably 90Hz or more)) while rendering virtual reality content, or else users may feel motion sickness.

VR headsets are usually used with controller interaction, but this method is not immersive enough in most cases. The main drawback is the fact that controllers are designed to control computers and cannot represent everyday actions and movements naturally. In the real life, people do not push buttons or grab joysticks to assemble or disassemble machines and they will not be able to learn or practise the real movements of the procedures if they have to do so. Immersion is a critical point of virtual reality, which means that interaction methods also have to be as life-like and accurate as possible. For practising the assembly work, precise and latency-free (real-time) motion detection is essential. Many different devices are available on the market, however for our application, LEAP Motion provides the best solution, as its small sized, non-contact optical motion sensor can be fixed onto the VR headset itself and it does not disturb the free movement of the user. The sensor recognizes features of the human hand and is able to build up a skeleton using the position of the users’ real hand and fingers. The software side of the platform relies on Unity game engine, using which, this hand model gets transformed to the virtual space with the help of LEAP Motion’s SDK.

However, rendering the models of the user’s own hands in the virtual space and capturing its motion is not enough to fully replace controllers. If the aim is not to overlap virtual objects, but to be able to touch and grab them, an interaction engine is also necessary. In the early days, we used the default gesture based model provided by LEAP SDK for this purpose, the biggest disadvantage is of which is that it does not take physical qualities of the object into account.

The user can grab the nearest object whenever the “pinch” gesture is performed. Later, we began to develop an own, more precise way for interaction, which determines the fact of grabbing considering outlines, mass and size of touchable objects and the angle of the touching fingers.

Using this method, users can not only see their own real hand but are also able to work with it confidently in virtual reality without the distraction of any other devices.

Another issue in virtual reality is getting around large virtual spaces, which is also relevant in nuclear power plant maintenance. The platform has multiple solutions for this: on the one hand, workers can use a special “walker” called Cyberith, which uses optical flow sensors to determine direction and intensity of feet movement while users walk in place. On the other hand, the popular “teleport” mechanism can also be utilized. In this concept, users have to walk in the real area, but when a door or special barrier is reached, they get teleported to another spot, so there is no risk of outrunning the real space.

The advantage of the treadmill is that the operator can travel anywhere while they stay in the same position in the physical space. However, the Cyberith we use does not give full immersion in the field of simulating the principle of walking.

The step detection optical sensors do not sense the elevation of the foot, but rather a sliding motion, so this process is more like a controller: it has to be learned and accustomed to its special use. Depending on these artifacts, negative innervation may be developed which does not correspond to reality.

Another solution to implement motion into virtual reality is free movement. In this case, the operator walks in the physical space on their own legs like in reality and does not need to learn to walk again in virtual reality like on the treadmill. This method is much closer to real spatial motion. For maximizing freedom, we used a backpack computer because it is wireless – with 2 hours of battery time – and the operator is not limited by cables. For the motion tracking, we used the Stereolabs ZED stereo depth-sensing camera and inertial sensor that allows us to map our environment. By implementing SLAM (Simultaneous Localization and Mapping) algorithm for environment mapping and object and determine the actual position of the user, which is widely used in navigation and robotics besides VR and AR applications.

The disadvantage of the free movement solution used for the VR training platform is the limitation of the physical space. The boundaries of a platform set up in a room will be determined by the physical dimensions of the real environment. For this shortcoming, we implemented teleportation as a workaround.


Advantages in training

As stated earlier, the main purpose of the above-mentioned technologies is making training of maintenance workers more efficient and flexible. A simulation model is a great tool for training workforce because it can be done anywhere in the training room even before the production line is built. Software training with real data offers many benefits. If the control software is integrated into the simulation model, then the operator can acquire the same user interface as in real life, thereby gaining a holistic view of the production system. This allows them to study system parameters, weaknesses, operator reactions, and early problems in order to correct those.

Contrary to traditional procedure instructions and video trainings, the virtual training platform can effectively improve every moment of the practice, regardless of location and time. There is no need to build or rent expensive simulation halls, as virtually any environment can be easily built, and later, individual elements can be easily replaced and rearranged, making construction work cost-effective.

Another big advantage of the platform is flexibility. The system is designed to be very easily maintainable and extensible with many different modules. Training phases, tools and the whole environment can be very easily adjusted to very different situations if needed and can be used in a wide range of industrial practices. For instance, we successfully integrated a real-time radiation calculation and visualisation module, developed by IFE. This extension can display the actual level of radiation, position of shields and the radiation source as well. A heatmap also makes it easy to distinguish dangerous and safe spots and the dose of radiation an employee would take when working in such environment. The real-time data stream makes it possible to alert the user in case of a sudden radiation increase in the facility or segments of the plant and helps finding a way to leave the working zone avoiding dangerous spots. Using this extension, nuclear decommissioning can be made not only much easier and safer, but cheaper as well.


Other possible use-case scenarios

Aside from trainings, there are also some other efficient use-case scenarios of using VR, some of which we would like to introduce and discuss further.

In order for engineering teams to work in parallel phases, 3D visualization tools are needed to improve communication. The initial planning and design is always done in front of monitors, but once the base parameters of the facility and the list of objects to be placed are available, the imaginary concept can be constructed and tested in virtual reality. Rapid prototyping can be beneficial in any industry and this way, it can be way more efficient.

Using the VR platform can also be beneficial in product simulation if the concept is constructed in virtual reality before the real construction. In this field, we would like to determine and test how our preliminary plans, flow of materials would work, whether our control principles are appropriate, the size and location of the buffer are well estimated, and where the bottlenecks are. If the data that we are working on is based on real data and comes from a similar product family or from the same versions we can turn it to our [advantage in further applications. This is an iterative analysis where engineers have to examine the system from the most basic elements to determine what parameters require further analysis or changes, for example, to reduce cycle times. An important requirement is that the simulation should be able to validate our measurements and ideas, for which an easily parametric and flexible model is essential.


Dr. Szabolcs SZÁVAI, acting division director, Engineering Division, Department of Structural Integrity and Production Technologies
Address: H-3528 Miskolc, Iglói u. 2
Phone: +36-70/205-6455