Immersivity is referred today as one of the most important attributes of high quality Virtual Reality experiences. Indeed, much attention has been recently devoted to technologies that enable immersivity mainly as a consequence of the widespread adoption of Virtual Reality headsets, such as Google Cardboard, Oculus Rift, HTC Vive and Samsung Gear. Seldomly the term ?immersive? is used today to make reference to Augmented Reality experiences, as, notwithstanding the promises, they mainly run on smartphones or not yet fully developed Smartglasses. This happens mainly because of the limitation of current AR displays.
On Monday 10th October 2016, the city of Rome together with Zetema, one of the most important Italian cultural heritage management organizations, launched and opened to the public the “ARA as it was“, namely the first large scale augmented reality valorization project for the roman cultural heritage. Hosted in a beautiful building designed by the famous American architect Richard Meyer, Ara Pacis is the symbol of the power exerted by the Augustan lineage across the Roman Empire.
The Project
The project has been developed by ETT Solutions in collaboration with the Augmented Reality vendor, Inglobe Technologies, that provided the AR technology that made it possible. Basically, the Ara Pacis Augmented Reality system is based on immersive AR headsets that work in association with an Application that assists visitors through the discovery of the monument and its history.
We expect that this is going to be one of the most important benchmarks for Augmented Reality projects for Cultural Heritage as Zetema has already a business model around this project. Paying visitors are provided with Immersive Smartglasses, namely Samsung GearVR, powered by a Samsung S7 smartphone that runs the “Ara as it was” application.
The application is based on a unique feature of Inglobe’s AR technology that provides high performance 3D Tracking on a variety of devices, including Samsung Gear VR. This feature, together with the other features of the technology by Inglobe has made it possible to develop such an incredible and unique experience.
The Experience of an immersive Cultural Heritage site
Thanks to this approach, visitors can immerse themselves in the narrative path of the museum in first person. They can see the bas-reliefs and elements of the monument come to life in AR by providing the original shapes and colors together with information related to their history.
The experience is personalized and is based on a hybrid and multi-sensorial approach as it relies on sensorimotor, visual and auditory interaction, right there in the real world.
Tourist with VR AR headset visiting Ara Pacis in Rome
Talking about the solution, it allows to navigate through the different POIs of the monument. Each POI is a gateway to an immersive AR experience that leverages 3D object recognition and tracking (unique features of Inglobe’s AR technology) to provide visitors with information related to digital reconstructions both in terms of architecture and socio-historical meaning of the monument. Large FOV, precision of 3D Tracking, accuracy of Computer Graphics overlays and personalization of experience are just some of the takeaways of the solution.
When people think of improving efficiency in a business with digital technology, they often have an image of a robot on an Industry 4.0 assembly line. In practice, tasks involving people are also being performed more smoothly and rapidly with the help of technology. This case study describes a project that Inglobe has delivered for Huawei Technologies to provide its technicians Industrial Augmented Reality tools to carry out common tasks more efficiently.
Background of the project
Installation and maintenance of IT equipment can be complex. The outcome of a work order is affected by a number of issues. The technicians have too many procedures to remember and even with the best documentation, reading text and looking at images of a product can be misunderstood or time consuming. Extra time must be taken to avoid errors, which leads to inefficiencies in the performance of assigned tasks, security problems and, ultimately, higher costs.
Working with Huawei’s Technical Documentation Innovation team, Perey Research & Consulting invited Inglobe Technologies to provide an appropriate solution for developing AR-enhanced installation and inspection instructions through its HyperIndustry platform.
The HyperIndustry platform is the basis for delivering targeted applications to support field technicians during installation, assembly and maintenance of some enterprise-class equipment, improving their performance. This project focused on the SUN2000-KTL inverter, a device that converts DC power from photovoltaic power systems on commercial rooftops and in large power stations into AC power and feeds it into the power grid.
The goal of the project was to assess how HyperIndustry – Inglobe’s Augmented Reality platform for Industry 4.0 – could improve the installation and maintenance processes of Huawei’s SUN2000-KTL product.
The challenges of the Industrial Augmented Reality project
The project presented a series of technical and organizational challenges to the partners. First, the team members collaborating on the project were in different regions of the world (US, Europe and China) and all development had to be performed remotely. The Inglobe technical team never had direct access to the devices on which the project was to bring value. Nevertheless, all the tracking and testing operations were successfully completed on schedule.
The installation steps tested for AR-enhancement are complex. Huawei has published videos and images to support the technician, as well as specialized tools. Each step must be completed in order for the correct installation.
Further, Huawei required that the application rely entirely on natural feature tracking. Finally, the solution needs to work outdoors, in highly variable climatic and lighting conditions in which the SUN2000 inverters must be installed. Without use of any marker and relying solely on 3D models based on photographs, Inglobe Technologies’ 3D tracking component delivered excellent results on systems mounted on building rooftops.
How the AR experience is structured
AR-assisted procedures developed by Inglobe Technologies for Huawei focus on installation and maintenance instructions of SUN2000 inverters in photovoltaic systems installed outdoors.
The AR experience was created rapidly based on a detailed storyboard and using the HyperIndustry platform, an Experience Management System for Industry 4.0. The platform compiles all the necessary content and interactive modules into native mobile applications for use on Android and iOS smartphones and tablets.
First the technician can choose either English or Mandarin instructions. The Industrial Augmented Reality experience starts when the technician points the camera of the mobile device on which the application is running at the inverter. The target is automatically recognized, and a list of procedures available for the model appears on the screen. The procedures may be assigned by the maintenance manager to one or more technicians, or they can be initiated entirely by the engineer in the field.
Once the technician begins a procedure, a series of steps are shown, each accompanied by instructions that simplify the operation to be performed and by Augmented Reality elements which, depending on the need, may be highlights of the intervention points or procedural animations. HyperIndustry platform by Inglobe Technologies allows the developer to associate additional contextual information, such as technical documents, photos and videos previously prepared by the technical documentation team, with each step.
Field technicians independently perform technical procedures by executing all required steps. The procedures may be either followed linearly or accessed as required, as a result of the definition of branches and paths depending on the use case or the context. Checklists and steps in the application help the operator to perform inspections and audits. The platform records the technician’s performance, then sends all of the performance metrics and analytics to managers so they can assess the processes.
The technologies involved in the project
HyperIndustry is a modular platform based on a content management system that allows corporate asset management in the context of an Augmented Reality workflow for Industry 4.0. The project consists of 3D models, tracking data of the equipment, images, videos, animations, documents, and other information considered valuable for the design of procedures, and management of users and assignments.
The platform includes a series of profiles to be associated with business roles including technical manager, who can create and assign tasks with a user-friendly interface and the field technician who receives the requests on his device. An analytics control panel is included in order to analyze data and customizable indicators (KPIs).
The platform’s recognition of 3D objects uses an advanced 3D tracking technology designed to deliver AR experiences efficiently and reliably on mobile and wearable devices.
The Benefits for Huawei
Existing studies show that the use of Augmented Reality when performing complex procedures in the field can improve performance of technicians in the workplace. Augmented Reality–assisted instructions help the professional access all the relevant information where and when needed. By reducing the need for training and avoiding errors during the performance of tasks, AR can significantly reduce costs, compared to traditional methods of delivering documentation.
Compared with the traditional paper-based or digital instructions, one of the most obvious benefits related to the installation and maintenance instructions is the technician’s immediate understanding of each step in the operation. The use of Industrial Augmented Reality has, in fact, allowed Huawei to make understanding procedures faster and less ambiguous, reducing the cognitive load for the operator. Consequently, the average execution time of the procedures could decrease.
This ease of access to information is a key factor that not only speeds up the execution of procedures by new technicians, but can also help the experienced staff in solving occasional and complex problems, through a true “perceptual intelligence system”.
Rhonda Truitt, Director of US R&D Technical Communication Innovation & Best Practices at Huawei, referring to the project declared: “The results we achieved on this project through Inglobe’s HyperIndustry were nothing short of amazing. Object recognition on outdoor equipment was something other vendors told us could not be done. Our customers have responded overwhelmingly positive about this experience. I am confident that we will see a sharp decline in installation errors due to this AR experience for post installation checks. Inglobe has really impressed me.”
Plans for the future
Huawei plans to continue working on Augmented Reality workflows that integrate Technical Documentation, Product Lines and support for Customers and Technicians in the field.
Many vendors in the Augmented Reality space today provide platforms that promise to support technicians with AR enabled procedures for service and maintenance also in the automotive sector. But few do really offer the technical features that are required to deal with the complexities of the real world.
The majority of the platforms use planar images to trigger the AR contents. On the other hand, one of the key requirements of any enterprise class AR maintenance solution is the capability to deal with real systems and parts. Indeed, not always engines or mechanical parts can be identified or tracked by means of 2D images. Thanks to AR-media powerful tracking methods, the AR-media Dealer 2.0 solution provides the functionalities that are needed to identify and track 3D parts with the required flexibility, while at the same time enabling the access to step-by-step instructions and information relevant for diagnostics.
The solution is the outcome of the joint effort of Inglobe Technologies, a leading Augmented Reality Developer, and Lynx International. The platform that enables AR information for diagnostic and maintenance purposes is thought for Tablets and Smartglasses (Epson Moverio) and has the flexibility to extend the support to other devices as well. Thanks to a unique Mixed Reality feature, the platform can be used in association also with Virtual Reality glasses, like Samsung Gear, in low-cost blended AR/VR Training experiences.
The videos below show the AR-media Dealer solution in action:
Being able to provide contextual, high quality and spatially aligned digital information to real objects in Cultural Heritage sites is one of the biggest challenges for developers who try to provide a unique experience to their App users. This is what Augmented Reality is for.
In order to help developers deal with these challenges, at Inglobe Technologies we have worked hard to add new features to the AR-media Augmented Reality SDK. The new AR-media SDK v2.0 provides different tracking methods -including geolocation, planar images, device motion, 3D object tracking- alongside with 360° panoramas and interactive 3D contents on mobiles. The methods are packed together in such a way that they can be easily integrated into a single app. For more information take a look at this presentation:
And here goes a video that shows some of the features at work
If you are an interested reader, you can start to develop your own mobile Apps based on those features using the AR-media SDK today and make yourself familiar with the technology.
On December 14th, 2015 Inglobe Technologies took part to Renault Innovation Day in Rome. In the splendid setting of a memorable event, linked to the themes of innovation in Smart Cities, lifestyles and technologies – not only those found in the cars-, Inglobe Technologies presented the Dealer 2.0 Solution addressed to enhancing diagnostics and maintenance through Augmented Reality.
The solutions presented at the Renault Innovation Day have shown how it is possible, through our HyperIndustry platform, to interact in real-time with the vehicle and display data through smartphones and smartglasses, thus overlapping the data onto the vision of the vehicle. Thanks to the high degree of integration, the platform can significantly reduce the time needed to identify problems and quickly take action to resolve them. Thanks to the tight integration with the e-Dealer solution by Lynx International, the technology now fits into a broader platform that includes diagnostics, maintenance and e-check.
From “Experience Contextualization” to “Experience Personalization“. If we were asked to highlight the dominant theme of the conversations going on at the wonderful venue of Vinopolis, Central London, during The Technology Expo on 5-6 October, then there’d be no way for getting it wrong: the shift from Contextualization to Personalization would certainly be the winner. The shift well represents a trend that we are starting to experience within the new wave of technology coming from areas such as Augmented Reality, Virtual Reality, IoT, Wearables, 3D Printing, Bionics. And here is where a fundamental question emerges:
“What if data conveyed by intelligent sensors and equipment not only were provided to users when they are needed where they are needed, but were also presented by taking into account users’ personality, role, preferences, biases and emotions, in a word their “personal identity”?”
Based on the conversations we had chance to get involved in during the Tech Expo, this is exaclty the question that needs to be addressed as a whole new set of technologies is emerging thus allowing to shape the personal experience of the world according to one’s deepest desires, needs and expectations. As intelligent technologies progress, their disruptive potential is recognized by the larger community of users and observers, thus triggering the reflection and discussion not only around the involved technical challenges but also around the ethical implications. From AR/VR Systems, through personalized and embodied data presentation and management, to intelligent perceptual systems for autonomous navigation, a wide range of emerging technologies has been discussed during the Tech Expo.
As participants and supporters of The Tech Expo, we took part in many interesting discussions, we attended the best presentations, and we had chance to experiment with some of the latest solutions in the area. For this reason we can safely conclude that The Tech Expo is positioning itself as one of the most compact and stimulating European events focusing on the industrial, academic and societal implications of Emerging Technologies in general.
The challenge of communicating the essence of complex surgical procedures and products has always been a major concern for the medical sector. Inglobe Technologies and Tecres S.p.a. – an innovative company that offers solutions for orthopaedics, spine surgery and neurosurgery- have teamed up to create a set of tools that make the communication of advanced orthopaedic technology not only effective but also amusing.
Drawing on the X-ray visualization metaphor, Inglobe developed a Kinect-based Magic Mirror Experience and a mobile App that is helping Tecres engage customers with their solutions. The Tecres Magic Mirror is a huge high-definition interactive exhibit that the company has showcased, for medical visualization, for the first time on 25-27 March 2015 at the AAOS International Trade Fair in Las Vegas. The system recognizes humans as they approach the interaction area and scans the user before a metaphorical X-ray Mirror. The X-ray scheleton is controlled by the user’s gestures who can select and interact with the products thus focusing on the essence of the solutions offered by the company .
Medical Visualization on Mobile
Similarly, the Tecres Mobile App provides Healthcare Professionals with an immersive 3D experience focused on Tecres solutions for orthopaedics, spine surgery and neurosurgery. Through an innovative 3D and Augmented Reality interface, the App is currently employed as one of the company’s most valued enterprise tools to support the sales force present the company products. The App is available for smartphones and Tablets both on the App Store and Google Play.
On May 22 at the National Gallery of Umbria in Perugia, Italy, the INQUADRA project was presented to the public and to the press. INQUADRA is based on a new vision of how the unique pieces of art in the Museum can be experienced and communicated.
The trigger of the project was a new iconographic interpretation of room 18 of the gallery, which for years -thanks to its sumptuous frescos-has been intended to tell the story of the Perugia leader par excellence “Braccio da Montone”. After a careful re-interpretation of the whole cycle, a new attribution of roles and meanings has been proposed by researchers, especially for what concerns the role of the “Farnese” family and leaders. This new interpretation has inspired the creation of an innovative approach -based on a great practical use of Augmented Reality technology- that could better communicate the findings to the visitors. The INQUADRA project got born this way, thus allowing to apply AR to one of the jewels of the National Gallery of Umbria.
The project has disclosed a new horizon for the most important Museum in the region. INQUADRA allows the immediate interaction between the piece of art and the public: without human intermediaries or experts visitors can get an in-depth understanding of the meaning of the frescoes. The project also provides a unique tool in the national museum landscape by allowing, through the associated APP, to deepen the various scientific aspects of the room and by enabling a unique learning experience, which is thrilling and exciting at the same time.
With this operation, the Gallery plays one more card: by anticipating the use of Augmented Reality for the valorization of cultural heritage, the Gallery gets on top of what will be the future “Polo Museale Umbro” in which these innovations will be considered the milestones that make a place of culture and history a strategic point of contact between the past and the future. In this way, visitors will live exciting experiences while the desire for a deeper knowledge of history is stimulated.
The App “INQUADRA”, carried out as a collaboration between Promovideoand Inglobe Technologiesfor the National Gallery of Umbria, is now available as a free download for smartphones and tablets. This APP makes it possible to literally “enter” the frescoes in the Farnese Room and discover the history of the people represented in the works, as well as anecdotes and interesting information about the scenic and decorative elements. Even if some contents are accessible remotely, users are granted access to all the information in the App just inside the museum. One more reason to book a visit to this fantastic Museum.
Many of us are not aware of the level visualization technologies have reached in aircrafts today. It is commonly believed that advanced visualization technologies are available exclusively in the helmets of military pilots. That is not the case anymore.
Indeed, things have changed thanks to the fact that Head-up displays (HUDs) have become common equipment also in civil aircraft cockpits. One of the uses of HUDs is to provide a specific visual interface for pilots to help them perform their tasks more efficiently. Some of these interfaces take the form of what is called a “tunnel-in-the-sky“, a 3D geometry representing the navigation path that is displayed on a flat screen.
Despite its advantages, “Tunnel in the sky” systems are not as widespread as one may think, although they are available in some aircrafts as modern HUDs. The reasons why standard TS systems are not so widespread appear to reside on the following circumstances:
a) existing TS solutions are not as compact and cost-effective as they should be in order to be easily adopted for the general aviation;
b) according to recent studies the standard “tunnel-in-the-sky” system do not provide any crucial advantage in comparison with more traditional methods of presenting navigation information to pilots.
In order to overcome such difficulties, Inglobe Technologies, in collaboration with MSTUCA -Moscow State Technical University for Civil Aviation-, developed and evaluated experimentally a totally new “pocket-size” AR system that has capabilities that help pilots during operations. The results of this experimental study have been discussed in a paper titled “A Pocket-size Augmented Reality System for Flight Control“. The paper that has been presented at EuroVR 2014, 11th Conference and Exhibition of the European Association of Virtual and Augmented Reality 2014, held in Bremen (Germany) on December 10th has been awarded the “Best Paper Award” in the Conference.
The new AR system consists of low-cost items and does not suffer from the drawbacks tied with existing enhanced vision systems for pilots. The experiments made so far prove the effectiveness of the new AR solution. A series of flight tests have been also performed on small airplanes, one of which running on a Cessna 172 aircraft can be seen in the video below.
The benefits of the Flight Control System
The system has a number of advantages with respect to existing solutions:
1) it is low cost -dozens of times than traditional systems- and wearable –does not require any special installation-
2) it is accurate -because of the stereo mode- and natural -because of the extremely simple interface-
3) it helps solve the problem of flight safety.
When can the system be best employed? First, the system can be employed as an aid for commercial aircrafts during emergency situations when the standard systems fail (e.g. the crash of Air France A-330 in the Atlantic in 2009) or while landing in low visibility conditions (e.g. the crash of Polish president Tu-154 in 2010).
In the future the system may become the main way of presenting navigation information to pilots of general flying vehicles. The system itself could also be employed to make naval and terrain operations even safer.
Now, can you imagine what in the future the sky motorways will look like? We do have a bunch of ideas.
The Development Lab of Inglobe Technologies announced the release of the AR-media 3D SDK and platform on the first day of Augmented World Expo in Santa Clara, CA. The AR-media 3D SDK v.1.0.0 features advanced 3D object tracking, a new 3D mapping and tracking approach that allows to create first of a class augmented reality (AR) applications capable of recognizing and tracking objects independently of their size and geometry.
Inglobe Technologies first demonstrated the AR-media 3D Tracker prototype working on mobile devices in Santa Clara during Augmented World Expo 2013 catching the eye of many interested users. The new release will bring an enhanced version of the technology to the community of developers with extended support for multiple platforms and authoring environments, like for example Unity3D.
With the AR-media 3D SDK users will be able to create applications and systems that link the digital and physical world in ways never experienced before. The AR-media SDK enables a new generation of applications that redefine the boundary of indoor and outdoor augmented reality in domains like industrial maintenance, interactive gaming, architectural visualization and cultural heritage, just to mention some.
“While developing the AR-media 3D SDK we have tried to bridge the gap between what users and developers expect from augmented reality technology and what they actually have.” Said Graziano Terenzi, CEO at Inglobe Technologies. “Our 3D tracking technology adds a new form of intelligence to the environment and allows users to develop products and solutions never imagined before”.
The AR-media 3D SDK allows to create a new generation of gaming applications on site thus allowing developers to add digital interaction to real world objects around us and create funny and engaging experiences. By using the AR-media 3D SDK technology users can populate a historical site with virtual characters, help represent and reconstruct the past, like for example a battle or even take part to a battle as an active player and try to change the course of events.
The 3D object recognition and tracking technology of the AR-media 3D SDK does not just recognize planar images, but also complex 3D objects and is suitable for applications in the industry. This capability will address the growing need of different industries (like AEC, Aerospace, Automotive and Process Industry) to enhance their processes by means of the interaction of physical and digital information.
The white paper discusses a selection of Inglobe Technologies Augmented Reality (AR) solutions to enhance both training and operations in different industrial maintenance scenarios.
AR applications provide a technology that has the potential to support and speed up training, while at the same time increasing performance and effectiveness levels. Some statistics suggest performance improvements up to 30% with involved employees reporting higher levels of engagement. Applications of AR that support technicians in the field have the potential of reducing costs up to 25% throughout quicker maintenance or component substitution, identification and setup of new connections, solution of faults and misconfigurations, with less burden on backend personnel and system resources. Additionally, thanks to the information that operators collect on the field, the AR technology contributes to an improvement of the quality of enterprise data.
Development of AR applications in an industrial setting is a complex task, made even more challenging by the need to leverage the full power of native platforms (iOS, Android, Windows). This is certainly the case of modern, cutting edge AR mobile applications. Inglobe Technologies simplifies this complex task by providing software components, technology, expertise and customizable solutions that help managing complexity with reduced time from requirement analysis to deployment.
In order to present and educate on the software components needed to develop an effective AR solution for industrial maintenance, the document offers an overview of the challenges, enabling technologies and required solutions.
The document also clarifies how, according to the authors, the adoption of an AR strategy (i.e. a strategy for the introduction of AR for industrial maintenance), is not a consequence of an improvised choice, but rather the result of a path that requires the definition of a?Roadmap?for the development of Enterprise Class AR solutions.
The document addresses the challenge of making industrial maintenance more efficient under two main respects, i.e. the intelligent support of field force operations and training; section 1 introduces the dimensions from which the need for developing a Smart Maintenance solution derives; section 2 introduces the technologies that enable AR solutions for maintenance, including positioning technologies, pattern recognition, tracking and artificial vision as they allow to create effective and enhanced industrial solutions; section 3 offers an overview on the architecture of the proposed solution; a generic use case which is relevant in this context is discussed in section 4; section 5 discusses an approach to estimate the ROI (return on investment) by clarifying the reference parameters for evaluating quantitatively the benefits that the adoption of an AR strategy for maintenance provides; finally, section 6 offers an approach and Roadmap for the design and implementation of an enterprise class industrial Augmented Reality solution.
The development of new technological systems to improve the performance in the healthcare sector is one of the most challenging issues that we face today. There are several aspects of this problem though, including advanced medical visualization techniques, improved surgical procedures, improved safety in healthcare environments, improved training methods for healthcare staff.
The Augmented Reality community has recently started to provide solutions to problems emerging in this field. But in which way could augmented reality technology help healthcare staff improve their performance with respect to managing risks in a more effective way? The goal of the ongoing ANGELS Project, co-funded by European Commission, is to provide an answer to this question.
The video below shows some of the features of the current implementation of the system, including indoor location of risks, access to contextual information, management of risks, users and environments.
Basically, at this stage the system has the following main features:
it is capable of recognizing and identifying users profiles, thus providing them with contextual information about risk management in the environment
it helps users manage specific risk situations by means of AR guided procedures using mobile devices
it provides a real time update of data related with the risks in the environment
it allows to assess the understanding of risks and procedures by users and managers
it is a general purpose Risk Management tool: no restriction to specific environments in the Hospital
it allows to create structured, semi-structured and open training paths
More information about the project will be disclosed in the next few months as the pilot studies and large scale trials will be run in the partnering European Hospitals.
The consortium is formed by Policlinico Tor Vergata (Italy), Inglobe Technologies (Italy), Entropy Knowledge Network (Italy), University Jaume I (Spain), NCONZO (Chzech Republic), Hopital Broca (France).
