82 // Spatial computing with Yiliu Shen-Burke

In a discussion on Metamuse, Adam Wiggins, Mark McCranigan, and guest Elio Schoenberg explore the transformative potential of spatial computing and its impact on user experience. Elio, founder of Soft Space, highlights the challenges developers face in creating immersive software that interacts with the user's central nervous system, emphasizing the importance of comfort and trust. They discuss the evolution of virtual and augmented reality technologies, noting Apple's Vision Pro as a significant advancement. Elio shares his vision for Soft Space, a 3D augmented reality app that leverages spatial memory for organizing ideas, anticipating a future where spatial computing enhances productivity and creativity.

• Spatial computing requires developers to handle user trust more carefully than traditional software.
• Immersive software can have a visceral effect on the user's central nervous system, unlike standard apps on a phone.

"With spatial computing, there's a level of trust that the user is placing in you as a developer that most software developers have not had to handle on a phone."

• The quote highlights the unique responsibility developers have when creating spatial computing applications, emphasizing the potential physical and psychological impacts on users.

• The podcast, Metamuse, focuses on the team and ideas behind the Muse product, not the product itself.
• Participants include Adam Wiggins, Mark McCranigan, and guest Elio Schoenberg from Soft Space.

"Hello and welcome to Metamuse. Muse is a tool for deep work on iPad and Mac, but this podcast isn't about Muse, the product. It's about the small team and the big ideas behind it."

• This quote sets the stage for the podcast's focus on broader themes and innovative ideas rather than specific product details.

• Elio Schoenberg shares his experience with Wim Hof breathing techniques.
• The technique involves intense breathing cycles and breath-holding, which enhances focus and cognitive performance.

"I started trying some Wim Hof breathing before some of my, like, cafe work sessions, which is equal parts actually very invigorating and effective."

• The quote illustrates how breathwork can be a practical tool for improving focus and productivity in everyday settings.

• Soft Space is a 3D augmented reality app that organizes ideas and information spatially.
• The app leverages human spatial memory and reasoning, which are underutilized in traditional computing.

"Soft Space shows you the true shape of your ideas."

• This statement emphasizes the unique approach of Soft Space in visualizing and organizing information in a way that aligns with natural human cognitive abilities.

• The discussion touches on the evolution of virtual reality and augmented reality technologies.
• The potential for these technologies to transform how we interact with information and environments is significant.

"I think we're still a little ways out from really mainstream adoption of computers that you wear over your eyes."

• This quote reflects the current state of VR and AR, acknowledging both the excitement and the challenges associated with mainstream adoption.

• The physical environment plays a crucial role in the creative process.
• Virtual environments can replicate some aspects of physical spaces, democratizing access to creative spaces.

"This physical space was in and of itself a framework on which the creative process hung."

• The quote highlights the importance of environment in creativity and how virtual spaces can offer similar benefits.

• Elio's background in architecture influenced his approach to designing virtual environments.
• The transition to a focus on information architecture and user interface design reflects the evolution of Soft Space.

"I was in architecture school at the time. I ended up dropping out to keep running this idea."

• This quote underscores the shift from traditional architecture to digital innovation, driven by a vision for new ways of thinking and organizing information.

• Virtual reality is fully immersive, while augmented reality combines digital and physical elements.
• Mixed reality encompasses a range of interactions between virtual and physical spaces.

"Virtual reality, you're referring to something that is 100% immersive. You have no awareness of your surroundings."

• The quote provides a clear distinction between VR and AR, setting the stage for understanding their applications and implications.

• Spatial computing involves systems that engage with human spatial awareness.
• The concept emphasizes designing with three-dimensional space, offering new opportunities and challenges.

"Spatial computing... as a computer or computing system that engages with the fact that you are a human being in space very directly."

• This quote encapsulates the essence of spatial computing, focusing on its human-centered approach and potential to transform interactions with technology.

• The podcast explores the future possibilities of spatial computing and its impact on human capabilities.
• There is a focus on the potential for spatial computing to unlock new ways of learning and interacting with information.

"What will this unlock? What does this add to our lives and to our work that we would be poorer for if we didn't have it?"

• This quote challenges listeners to consider the transformative potential of spatial computing beyond its novelty and technical aspects.

• VR systems are increasingly replacing traditional methods in fields like surgery and architecture.
• The potential lies in using VR to work with 2D information in a 3D environment.
• Apple's marketing emphasizes 2D information in a 3D context, hinting at a broader application beyond obvious 3D use cases.

"What I am personally motivated by in building soft space is the belief that there's tremendous value to working with 2D information in a 3D environment."

• The speaker is motivated by the potential of VR to enhance the interaction with 2D information in a spatial context.

"I would like to think that the design minds at Apple are pursuing a very similar thesis that there is tremendous value in letting people work with 2D information."

• Suggests that Apple's approach to spatial computing aligns with the speaker's vision of integrating 2D data within a 3D framework.

• New technologies often reveal unexpected use cases after their initial development.
• VR/AR has found applications in gaming and immersive experiences but lacks deep integration into daily life.
• Productivity software in VR/AR remains underexplored, focusing primarily on 2D interfaces.

"It's often the case for any new technology that you figure out something new and impressive you can do with computing or some other technology first, and then you sort of figure out how that can be used."

• Highlights the process of discovering practical applications for new technologies post-development.

"It doesn't become a thing that's deeply integrated to your life."

• Indicates the current lack of VR/AR integration into everyday life beyond entertainment and niche applications.

• Differentiates between 2D UI paradigms and inherently 2D information.
• The concept of 'blocks' in apps like Roam Research and Notion inspired a shift towards block-based spatial computing.
• Emphasizes the potential of spatial computing to display relationships between data blocks more effectively than traditional 2D displays.

"I essentially shifted my own development model toward working with blocks, because blocks to me map so much better to the underlying material of thought and of creativity than a Word Doc or an Excel spreadsheet do."

• The speaker shifted focus to block-based models for better representation of thought and creativity.

"In an infinite spatial canvas or an infinite 3D spatial canvas, you have many more options."

• Spatial computing offers more flexibility and options for representing complex information relationships.

• AI and large language models have significantly impacted knowledge work and productivity tools.
• Tools for Thought community supports the development of innovative software for knowledge management.
• AI could potentially reduce the manual effort involved in maintaining complex knowledge systems.

"AI has upended everything about everything, but it's definitely upended our working assumptions about what knowledge work was."

• AI is transforming the landscape of knowledge work and productivity tools.

"Maybe if some of those friction points and some of those barriers could be lowered, then we could have tools that you could on their own be making a lot of the connections."

• AI could automate tedious aspects of knowledge management, enhancing productivity tools.

• VR fitness apps demonstrate the potential for spatial computing to achieve personal goals.
• The future of technology should balance augmentation and unmediated human interactions.
• Spatial navigation in the brain may correlate with conceptual understanding, supporting spatial computing models.

"I really like this model for how spatial computing can fit into our lives and work, or actually any technology for that matter can fit into our lives and work."

• Advocates for technology that enhances personal growth and can be seamlessly integrated into daily life.

"There has been some research recently that suggests our brains use or creatively misuse spatial navigation neural circuitry to keep track of concepts and memories."

• Indicates a possible neurological basis for the effectiveness of spatial computing in conceptual understanding.

• The development of VR/AR technology dates back to the 60s and 70s.
• Advances in hardware miniaturization have fueled recent developments in VR/AR.
• Despite impressive technological achievements, many platforms have not succeeded commercially.

"Going back to even the 60s and 70s, people strapping these ridiculous contraptions to their head and trying to figure out head tracking and all this kind of stuff."

• Highlights the long history and challenges of developing VR/AR technology.

"Lots of big companies and lots of investment dollars went into many platforms, most of which have not panned out but nevertheless have produced some very impressive things."

• Acknowledges the significant investment and mixed success in the VR/AR industry.

• The hardware and operating system platforms for spatial computing have only recently reached a level that is "good enough" for general-purpose computing, but this level is still very expensive.
• The challenge in spatial computing is that the minimum viable product requires a very high bar in terms of visual acuity, pixel density, and motion response.
• The human nervous system's complexity and the high expectations set by other consumer technologies are significant hurdles for spatial computing hardware.

"It's only been about a year or just over a year, that there has existed a hardware and operating system platform that just barely got over the line of good enough for a general-purpose computing tool like ours."

• This quote highlights the recent progress in spatial computing hardware, suggesting it has only just become viable for general computing tasks.

"The challenge with spatial computing, in my humble opinion, has been that the minimum viable product is actually not minimum at all. It's a very, very high bar."

• This statement underscores the high standards required for spatial computing hardware to be considered viable, emphasizing the difficulty in achieving these standards.

• The human nervous system's resistance to being tricked by digital environments is a major barrier to spatial computing technology.
• The competition from other advanced consumer technologies like smartphones increases the challenge for spatial computing to gain traction.
• Market expectations for success have increased, requiring a large user base for a product to be considered successful.

"The greatest is that you're dealing with the human nervous system. It's almost like, thank goodness our nervous system is actually laggy enough this hard to trick."

• This quote explains the difficulty in creating convincing digital environments due to the human brain's ability to detect inconsistencies.

"Everything else in the consumer tech space has gotten so good... the competition for spatial computing is therefore that much greater."

• This highlights the challenge spatial computing faces in competing with highly advanced existing technologies.

• Meta has focused on achieving a balance between cost and quality in their VR devices, notably with the Quest 2.
• The Quest 2 is seen as barely sufficient for productivity applications, with ongoing improvements in price and quality.
• Meta has opened up new possibilities for augmented reality by repurposing VR cameras for pass-through capabilities.

"What Meta has done really, really well is getting the price right for this technology and getting the sort of like absolutely minimum acceptable quality at that price."

• This quote emphasizes Meta's strategy of balancing affordability with basic quality standards in their VR devices.

"Even a really muddy and terrible view of what's going on around you physically is infinitely better than none."

• This statement reflects the significance of even low-quality augmented reality in improving user comfort and experience.

• Apple's Vision Pro is seen as a significant development in the spatial computing space, indicating Apple's commitment to this technology.
• Apple's approach focuses on high-quality visual experiences and the use of eye-tracked input systems, avoiding traditional hand controllers.
• The Vision Pro represents a continuation of existing ideas, but with Apple's emphasis on quality and premium pricing.

"This is only good news. The fact that Apple has entered in this way that feels like it's very central and very core to their plans for the future of Apple."

• This quote expresses optimism about Apple's entry into spatial computing, seeing it as a positive development for the industry.

"It kind of felt like if Apple had released instead of the iPhone... a BlackBerry, but it had a Retina display."

• This analogy suggests that while Apple's Vision Pro is technologically advanced, it doesn't introduce a fundamentally new paradigm.

• Apple's strategy involves focusing on 2D content within a 3D environment, leveraging existing user familiarity with 2D interfaces.
• Apple aims to create a controlled framework for app development, ensuring consistency and quality across applications.
• Collaboration with Unity allows Apple to support immersive 3D experiences, while maintaining a structured approach for developers.

"They're really doubling down on 2D content, I mean, as an initial set of use cases... like a floating 2D window."

• This quote highlights Apple's focus on integrating traditional 2D applications within their spatial computing framework.

"We have figured out the exact perfect framework for what Spatial software is, and it's very specific."

• This statement reflects Apple's intention to guide developers with a specific framework for spatial app development, ensuring quality and consistency.

• Apple's approach to building platforms focuses on creating a stable and comfortable ecosystem, even at the expense of experimental ideas.
• The company prioritizes user experience and market success over adopting new computing paradigms.
• Apple's conservative approach allows for cross-platform learning and adaptation, particularly from more experimental platforms like Meta.

"Apple has done a great job. You know, it might be frustrating in certain moments for developers or for users, but overall it works."

• Apple prioritizes a functioning, reliable platform over innovation, which can frustrate developers but ensures stability.

"You can very easily take the things that you learn from those experiments on the meta platform and bring them over to Apple."

• Apple's ecosystem can benefit from innovations proven on other platforms, allowing for integration without initial risk.

• Platforms represent significant economic opportunities, influencing how they are developed and controlled.
• The economic implications of controlling a platform are as critical as the software and technological aspects.
• Historical lessons from the open Internet and desktop eras highlight the accidental nature of their economic models.

"It's a key lesson from the mobile era, that this is as much an economic system as a software system."

• Economic considerations are integral to platform development, influencing how systems are structured and controlled.

"We kind of got the desktop and open Internet stuff kind of by accident before people really figured out what was going on."

• The open nature of early Internet platforms was unintentional, shaped before the full economic potential was understood.

• AR/VR systems demand low latency to ensure a seamless user experience, impacting both hardware and software design.
• Vision OS is a real-time operating system developed by Apple to address latency and responsiveness.
• Apple's resources allow it to create new operating system paradigms, setting it apart from competitors like Meta.

"The AR VR systems are a vindication of that interest because as you alluded to earlier, they really have to be very low latency."

• Low latency is crucial for AR/VR systems to function effectively, validating the focus on performance.

"Apple has put so much, because Apple can do this, they have the resources and the technical expertise to essentially write up, you know, a whole new operating system paradigm from scratch in order to address latency and responsiveness."

• Apple's unique position allows it to develop new systems to tackle latency challenges, highlighting its technical prowess.

• High-performance software development requires either a system-level enforcement or a dedicated focus from leadership.
• The natural tendency of software development is towards slowness unless countered by strict performance criteria.
• The balance between hardware capabilities and software efficiency is crucial for maintaining high performance.

"It's basically impossible to have really fast software unless you have one of two conditions. One is you have basically a dictator who's absolutely fanatical about it."

• Achieving high-performance software often requires either strong leadership or systemic enforcement.

"There's this equilibrium process where that's the natural resting place and you just get slowness that accumulates."

• Software naturally tends toward inefficiency unless actively managed, necessitating performance-focused strategies.

• Social norms and user comfort play significant roles in the adoption and acceptance of VR technology.
• Wearing headsets in public or social settings can be perceived as isolating, impacting social interactions.
• The future of VR and smart glasses depends on understanding and respecting social signals and discomfort.

"People feel often uncomfortable wearing headsets. Not because of, I don't know, it's heavy or there's wires or something, but because they know they're in an immersive environment that's cutting them off a little bit from their surroundings."

• The immersive nature of VR can create social discomfort due to perceived isolation from the environment.

"I think there could very well be, you know, an important signal in people's discomfort with head mounted displays that we should respect and understand."

• Understanding and respecting social discomfort with head-mounted displays is crucial for future VR development.

• The future of spatial computing involves balancing immersive technology with unmediated life experiences.
• Smart glasses should be used in specific contexts and not worn constantly to maintain a healthy balance.
• The technology industry needs to develop products that encourage appropriate usage time and context.

"I think that the future I would want to live in and to have future generations live in, where smart glasses are a thing would be one where we don't wear them all the time."

• The ideal future involves smart glasses used selectively, not continuously, to balance technology with real life.

"So far the technology industry has not been great at making tech that encourage you to then stop using it after the appropriate amount of time has passed or the task has been completed."

• The tech industry must focus on creating products that promote balanced usage and discourage overuse.