The Daily Roundup is our comprehensive coverage of the VR industry wrapped up into one daily email, delivered directly to your inbox.
The Daily Roundup is our comprehensive coverage of the VR industry wrapped up into one daily email, delivered directly to your inbox.
The Daily Roundup is our comprehensive coverage of the VR industry wrapped up into one daily email, delivered directly to your inbox.
The Daily Roundup is our comprehensive coverage of the VR industry wrapped up into one daily email, delivered directly to your inbox.
The Daily Roundup is our comprehensive coverage of the VR industry wrapped up into one daily email, delivered directly to your inbox.
Keep your finger on the pulse of the XR industry with the Daily Roundup, the most important news in one daily email.
The Daily Roundup is our comprehensive coverage of the VR industry wrapped up into one daily email, delivered directly to your inbox.
Speaking at the IEDM conference late last year, Meta Reality Labs’ Chief Scientist Michael Abrash laid out the company’s analysis of how contemporary compute architectures will need to evolve to make possible the AR glasses of our sci-fi conceptualizations.
While there’s some AR ‘glasses’ on the market today, none of them are truly the size of a normal pair of glasses (even a bulky pair). The best AR headsets available today—the likes of HoloLens 2 and Magic Leap 2—are still closer to goggles than glasses and are too heavy to be worn all day (not to mention the looks you’d get from the crowd).
If we’re going to build AR glasses that are truly glasses-sized, with all-day battery life and the features needed for compelling AR experiences, it’s going to take require a “range of radical improvements—and in some cases paradigm shifts—in both hardware […] and software,” says Michael Abrash, Chief Scientist at Reality Labs, Meta’s XR organization.
That is to say: Meta doesn’t believe that its current technology—or anyone’s for that matter—is capable of delivering those sci-fi glasses that every AR concept video envisions.
But, the company thinks it knows where things need to head in order for that to happen.
Abrash, speaking at the IEDM 2021 conference late last year, laid out the case for a new compute architecture that could meet the needs of truly glasses-sized AR devices.
The core reason to rethink how computing should be handled on these devices comes from a need to drastically reduce power consumption to meet battery life and heat requirements.
“How can we improve the power efficiency [of mobile computing devices] radically by a factor of 100 or even 1,000?” he asks. “That will require a deep system-level rethinking of the full stack, with end-to-end co-design of hardware and software. And the place to start that rethinking is by looking at where power is going today.”
To that end, Abrash laid out a graph comparing the power consumption of low-level computing operations.
As the chart highlights, the most energy intensive computing operations are in data transfer. And that doesn’t mean just wireless data transfer, but even transferring data from one chip inside the device to another. What’s more, the chart uses a logarithmic scale; according to the chart, transferring data to RAM uses 12,000 times the power of the base unit (which in this case is adding two numbers together).
Bringing it all together, the circular graphs on the right show that techniques essential to AR—SLAM and hand-tracking—use most of their power simply moving data to and from RAM.
“Clearly, for low power applications [such as in lightweight AR glasses], it is critical to reduce the amount of data transfer as much as possible,” says Abrash.
To make that happen, he says a new compute architecture will be required which—rather than shuffling large quantities of data between centralized computing hubs—more broadly distributes the computing operations across the system in order to minimize wasteful data transfer.
A starting point for a distributed computing architecture, Abrash says, could begin with the many cameras that AR glasses need for sensing the world around the user. This would involve doing some preliminary computation on the camera sensor itself before sending only the most vital data across power hungry data transfer lanes.
To make that possible Abrash says it’ll take co-designed hardware and software, such that the hardware is designed with a specific algorithm in mind that is essentially hardwired into the camera sensor itself—allowing some operations to be taken care of before any data even leaves the sensor.
“The combination of requirements for lowest power, best requirements, and smallest possible form-factor, make XR sensors the new frontier in the image sensor industry,” Abrash says.
This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.
Very fascinating in theory, not always applicable in practice. Sensor data fusion is made for a purpose… if you could avoid it and do most of computations directly on the sensors, we would do it already today. So I guess that this approach may help, but it is not the only thing to consider
I get paid $98 per hour to do a regular job at home.~kk138~I never thought this was possible, after all one of my closest friends made $25k in just 3 weeks doing this side job He made me join.~kk138~See this page for more information. —->>>> http://glo.wf/jOd3F
As he described, in a modern computer chip, most energy consumption is not the computation itself but carrying the data between each transistors. I think that a stacked memory directly on top of the computing transistor and new architectures are the way to go with this. Easier said than done though. If that is achieved, we could see a new era of power efficiency, now that reducing the node size is reaching atom limits soon. Happy to see that this problem starts being tackled.
All I know is I have faith in this guy and his team. They’ve impressed me at every turn with pretty much everything they’ve done in the VR space, both inside and outside the headsets.
Working online from home and earning more than $15,000 simply doing easy work. (MPB893) last month I made and received $17915 from this job by doing easy work part-time. Go to this web and follow the instructions.
– – – – >>>> https://rq.fyi/OCd35q *******************************************************
Facebook blocked me form watching this video because I was seeking through it too fast. Like wtf?! Why is that even a thing.
I had a similar issue while referencing it for the report. Very weird.
Mad scientist has got a captive-audience and no peer reviewed citations. Have faith when in his company!
Yeah, like Tesla but with a quarter trillion dollars for the next couple decades. He could be the president of America by then!
Michael, if you are reading this, where are the 140 degrees FOV and 30 ppd you predicted in 5 year time in the Oculus Connect 3 (Oct-2016)??? And the 4K per eye??
Cambria will be released end of this 2022 with just half 4K (ie 2160×2160 per eye) and miserable FOV.
Please, meet your predictions before talking again about future.
hmm.. has he sais that ALL this features will be seen at once in one headset?
we have 140degreees FOV we have 4k per eye we have 37ppd
but not all in one headset so far
Yeah, he was getting us hot in 2016 predicting that ALL that will be in the future headsets of Oculus. Unfortunately most of the Oculus founders left when Facebook didn´t like PC-VR
He wasn’t wrong about the pace of technology. Those things are here. It’s not really his fault the market or corporations have different ideas. Such a headset could come out tomorrow but it would be expensive as hell and be for “enterprise.”
Big corporations can be very wrong as well. One as to be brave enough to state their ideas even if they are differnt from the CEO. A succesor of Rift like Half-dome with 4K per eye and 140FOV (without the varifocal thing) and DP cable will be sold like candies at the gate of a school.
all that running on xr2 is tricky ….that and price and production is the limiting factors
looks like Cambria might be as high as $1200 and was delayed by a year and still doesn’t have everything …and that is still subsidized
Hardwired algorithms can not be updated. With the current progress in computer vision and pattern recognition hardware will become obsolete very fast.
It’s a cool idea, but I am not sure our current tech level can build something at a consumer price point that would be profitable. Batteries are heavy, even tiny ones. That weight just increases with the ability to store more charge. That’s not even considering the projection system to place the AR images on the lens, the logic board to code it, memory storage, etc. I want to see research continue, but I think we are gonna need incredible advances before a consumer model becomes reality.