Let’s go to MARS! -Microsoft Hololens

Yes, Anyone can reach MARS now. Not only that, one can bring space station to his workbench.

Image Credit: NASA
Thanks to NASA’s JPL (jet Propulsion laboratory) and Microsoft Hololens. This blog is a primer on Microsoft Hololens.

Microsoft HoloLens is the first self-contained, holographic computer, enabling the engagement with digital content and interact with holograms in the real world. It has specialized components—like multiple sensors, advanced optics, and a custom holographic processing unit—enable us to go beyond the screen.
HoloLens helps to create holograms, virtual objects made of light and sound that appear in the world around you, just as if they are real objects. Holograms respond to gaze, gestures and voice commands, and can interact with real-world surfaces . With holograms, one can create digital objects that are part of real world.

Holograms makes it possible to create stunning immersive gaming field where game becomes more interactive and takes you “real” tour. Given below image shows shooting the virtual objects with hand gestures. For others, there is nothing to be seen, but for the one who is wearing hololens, room is full of objects.

The holograms that HoloLens renders appear in the holographic frame directly in front of the user’s eyes. Holograms add light to the virtual world, which means that one see both the light from the display and the light from your surroundings.
Holograms can have many different appearances and behaviors. Some are realistic and solid, and others are cartoonish and ethereal. Holograms can highlight features in the surroundings, and they can be elements in app’s user interface.
Holograms can also make sounds, which will appear to come from a specific place in surroundings. On HoloLens, sound comes from two speakers that are located directly above ears, without covering them. Similar to the displays, the speakers are additive, introducing new sounds without blocking the sounds from the existing environment.
Holograms aren’t just passive objects made of light and sound; they’re also an active part of world. Gaze at a hologram and gesture with your hand, and a hologram can start to follow you. Give a voice command to a hologram, and it can reply.
Holograms can also be occluded by real-world objects. For example, a holographic character might walk through a door and behind a wall, out of your sight.

More about holograms can be found here

What does HoloLens Contain:

Advanced optical lens
Intel processor with custom built HPU Holographic processing unit (2 GB CPU, GPU and 1 GB HPU)
Sensors (IMU, 1 Depth camera, 1 HD camera, 4 microphones, light, environmental sensor
Built in speaker
Wifi, battery, charger
Weighing more than half a KG (579 gram)

Building Hololens Application:

One can begin building apps for HoloLens with the Windows 10 SDK. There is no separate SDK for HoloLens; holographic app development uses Visual Studio 2015 Update 3 with the Windows 10 SDK (version 1511 or later) Developer is not supposed to have Hololens to build the app. There is Hololens emulator to build and test the code. However there are hardware requirements to make hololens emulator to work
The HoloLens emulator is based on Hyper-V and uses RemoteFx for hardware accelerated graphics.
Hardware requirements:

• 64-bit Windows 10 Pro, Enterprise, or Education (The Home edition does not support Hyper-V or the HoloLens emulator)
• 64-bit CPU
• CPU with 4 cores (or multiple CPU’s with a total of 4 cores)
• 8 GB of RAM or more
• In the BIOS, the following features must be enabled
  • Hardware-assisted virtualization
  • Second Level Address Translation (SLAT)
  • Hardware-based Data Execution Prevention (DEP)
• GPU (The emulator might work with an unsupported GPU, but will be significantly slower)
  • DirectX 11.0 or later
  • WDDM 1.2 driver or later

All holographic apps are Universal Windows apps, and all Universal Windows apps can be made to run on Microsoft HoloLens. With Windows 10 and familiarity with middleware tools like Unity, you can start building holographic experiences today.

Building blocks of Holographic development

• Gaze input
• Gesture input
• Voice input
• Spatial sound
• Spatial mapping

First three components (GGV) are used for interacting with user and hololens while the last three components are used to understand the surrounding environment.

Gaze is the first form of input and is used for targeting within holographic apps. Gaze tells you where the user is looking in the world and lets you determine their intent. In the real world, you’ll typically look at an object that you intend to interact with. This is the same with gaze.
HoloLens uses the position and orientation of your user’s head, not their eyes, to determine their gaze vector. You can think of this vector as a laser pointer straight ahead from directly between the user’s eyes

Gestures are one of the three primary forms of input on HoloLens. Once you’ve targeted a hologram with gaze, gestures allow you interact with the hologram. Gesture input lets you interact with your holograms naturally using your hands. Below are the identified gestures.

Press and release: The primary way in which you’ll select or activate holograms in your world is the air-tap gesture, which consists of a press followed by a release.
Bloom is a special system gesture that is used to go back to the Start Menu from within a holographic application. It is similar to pressing the Windows key on a keyboard or the Xbox button on an Xbox controller.
Hold: Holding a press beyond the system’s Hold threshold.
Manipulation: A press, followed by absolute movement of your hand through 3-dimensional world.
Navigation: A press, followed by relative movement of your hand or the controller within a 3-dimensional unit cube, potentially on axis-aligned rails. More on this below.

Voice input can be a natural way to communicate your intent. Voice is especially good at traversing complex interfaces because it lets users cut through nested menus with one command.

Tutorial for each can be found here

Also one needs game engines like Unity or Vuforia (or if some one wants to build his own middleware gaming engine, he can use Windows Holographic APIs with DirectX 11)

Possible fields where Hololens can play a vital role: Edcuation, medical

With the mixed world capabilities of Hololens, many usecases are possible.

Case studies

1) Learning enhancement (For example, to understand about Jet craft, one can have virtual jet and play around with the nuts and bolts of it- Japan airlines built usecase around this)

2) Exploration and simulation in Medical field ( For example, operations can be simulated and given inputs to explore the possibilities)
3) Enhanced shopping experience at retail stores