Magic Leap Finally Announced; Remains Mysterious

Yesterday Magic Leap announced that it will ship developer edition hardware in 2018. The company is best known for raising a lot of money. That’s only partially a joke, since the teased hardware has remained very mysterious and never been revealed, yet they have managed to raise nearly $2 billion through four rounds of funding (three of them raising more than $500 million each).

The announcement launched Magic Leap One — subtitled the Creator Edition — with a mailing list sign up for “designers, developers and creatives”. The gist is that the first round of hardware will be offered for sale to people who will write applications and create uses for the Magic Leap One.

We’ve gathered some info about the hardware, but we’ll certainly begin the guessing game on the specifics below. The one mystery that has been solved is how this technology is delivered: as a pair of goggles attaching to a dedicated processing unit. How does it stack up to current offerings?

What is Magic Leap?

Keeping the technology a mystery for years was a pretty good move. It let everyone imagine something much more advanced than could be possible right now. You likely remember the marketing image of an elephant in the palms of a user’s hands that broke the initial announcement back in 2014. The Wired article summarized Rony Abovitz’s boast as a new technology:

Magic Leap founder and CEO Rony Abovitz insists what he’s building isn’t mobile computing, Oculus-style virtual reality, or even augmented reality.

With the cat out of the bag we think those claims overstep. This is augmented reality as far as the broader term is concerned, but goes beyond into Mixed Reality (MR). MR is a type of AR that includes awareness of surroundings. So the Magic Leap promises to understand where tables, chairs, windows, doors, and hopefully cats and dogs are located when adding virtual items to your surroundings.

Magic Leap v. Hololens v. CastAR

These wearable goggles have the most in common with Microsoft’s Hololens and CastAR because all three feature see-through lenses that do not block out all of the world around you in the way that HTC Vive or Oculus Rift (both VR technologies) do. What’s interesting is the key differences between the three.

Unfortunately, CastAR closed their doors earlier this year but we still loved the technology. It was built on a set of wearable glasses that included two 720p projectors, one above each eye. The augmented reality experience depended on a retro-reflective material to use as a projection surface. That surface reflected the projected images back at the exact same angle they were received so several people can use the same surface at the same time without interfering with each other. One of the keys to success here is that what you’re seeing is actually coming from the physical location the user expects to see it.

Microsoft’s Hololens is a curved visor which you wear like a facemask, spanning from temple to temple and from the tip of your nose to above your brow. That visor is the projection surface so the user sees the real world, but the effect of projecting on the visor overlays images on what the user sees. The benefit of this is that you don’t need to have the retro-reflective material or any other gear separate from the head-mounted unit itself to make the experience happen.

Magic Leap appears to use lenses as the projection surface, like the Hololens, but uses two separate lenses instead of a single large visor. We’re keen to try this out because it may solve a problem we noticed when testing the Hololens: field of view (FOV).

The Hololens has a limited area where it can project that is smaller than the total viewable part of the visor itself. If your eye catches the “edge” of where the virtual items can be projected it breaks the experience. Looking at the smaller lenses of the Magic Leap it would be fantastic if this synchronizes the FOV for both the real and the virtual world, leaving no way to interrupt the perceived experience. Alas, hearing from reporters who received demos of the current prototypes, Magic Leap does have a rectangular projection area inside of the round lenses. We hope the production hardware can improve on this.

I was fortunate enough to try out the CastAR while still in development and it had a great FOV. But that was limited to the size of the retro-reflective material so you can say there was a similar experience-breaking issue.

The Secret Sauce… or Not

So, it can track head movement, it has a wearable transparent screen in front of each eye, and it has cameras and sensors to pick up the surroundings. How do you make this stand out from existing offerings?

The most verbose source of information we’ve found is the Rolling Stone article that ran yesterday. There is a lengthy section on light field and how a breakthrough in handling it is the revolutionary technology Magic Leap is built upon. The assertion is that your eye only needs specific important parts of the light field and your brain builds the rest. If you feed in those magic parts, your brain perceives something real is in front of your eyeballs.

The secret sauce is a chip — basically a custom sensor/DSP that knows how to take in a virtual image, reconcile it with the lightfield sensed by the cameras, and spit out an artificial light field sophisticated enough to fool your brain into filling in the gaps. The article makes it sound like they built a fab in the basement floor of their Florida-based operations. That’s rather incredible; the stuff you expect from Tony Stark (or some evil villian’s lair). It’s more likely they’re getting wafers fabbed and shipped bare for processing during in-house assembly of prototypes.

These light field breakthroughs are a mighty claim and we hope it’s true, because such an advancement would indeed be revolutionary. It seems we’ll have to wait and see. We want to read the white paper!

An interesting tidbit the Rolling Stone article does show is an image of the first proof of concept prototype. If you look close you can see two bits of glass held at 45 degrees to a chin rest like at the eye doctor. Despite hints of more, the system has been fundamentally glasses-based since the beginning.

Guess the Hardware

Time for everyone’s favorite game! How did they do it?

There’s almost no hardware information available in this reveal. But guessing is more fun anyway. Here’s what we can gather from the photos. The headmounted goggles have two wires coming off of them. They merge into one which snakes around to the Discman shaped, belt-mounted computing device that drives it. The handheld controller is wireless, and there is no apparent power supply as the disc-shaped unit incorporates a battery.

To be honest, the controller and CPU aren’t all that interesting. These are solved problems as far as hardware goes. The magic of Magic Leap is in the headset itself.

Position tracking is perhaps the most important part of this entire system. Sure, you could argue that optics make or break it and we’ll get to that in a minute. But if your perfect graphics are out of sync with the head movement of the user, motion sickness will ensure that the headgear will not be used.

Having seen the Intel Euclid demoed at Maker Faire last May, it’s obvious that camera tech for making sense of movement and the environment has come of age. The small Euclid has an entire computer and battery inside, which the Magic Leap moves to your belt. Fitting the rest into these goggles is not beyond belief. The guessing game becomes: what are the obvious lenses and sensors shown above used for?

There is a peripheral vision lens on either “corner” of the Magic Leap, two front-mounted cameras also on the corners, and two windows on the bridge above the nose that look like they might be for IR. The green circle shown on the left is a mystery (power button and indicator?) and the corresponding camera lens on the right is as well. Motion tracking likely uses at least two depth cameras — blasting an IR laser projector to draw a grid that can be interpreted by an IR camera. This is how Intel’s Realsense does it and Hololens takes a similar approach. The other cameras are likely used for object recognition and lighting condition processing.

The small holes at the bottom of the lenses and on the side show locations for three of the four microphones used to sense sound.

What Do You Think of Magic Leap?

We want to hear from you in the comments below. Did you spot any hints from what has been shown off of the hardware? Put on your reverse engineering hat and tell us what you can glean from this very limited info? Is this revolutionary or merely another incremental development in the VR/AR/MR scene?

Of course there are a lot of other questions left hanging out there. What will the Magic Leap One hardware cost? How do you interface with the controller, and what frameworks can be used for getting your code onto the machine? We’re looking forward to hearing what you think.