Abstract: 

In an era where the digital realm continually broadens its horizons, AI's sophisticated applications are becoming pivotal in sculpting the landscape of games, simulations, and the emerging Metaverse. This comprehensive tutorial unravels the groundbreaking contributions of generative AI, neural methodologies, and autonomous AI within immersive 3D spaces.

Witness how these state-of-the-art technologies dismantle conventional barriers, facilitating the effortless creation and digitization of virtual assets and scenes marked by unparalleled accuracy and fidelity—milestones that champion the rise of the creator economy. When autonomous AI synchronizes with generative AI, the result is a game-changer: automated processes and independent agents that will redefine the very foundation of gaming, simulations, and the Metaverse.

Tailored for AI researchers, game developers, simulation specialists, and Metaverse enthusiasts, this session unfurls a rich tapestry of knowledge and state-of-the-art techniques in applying AI to 3D environments and experiences. Merging profound theoretical insights with hands-on techniques, from core algorithms to their tangible application, attendees will be equipped with an arsenal of knowledge. Experience instructive lectures, live tutorials, and dynamic discussions. Dive deep into this immersive tutorial and emerge inspired to apply artificial intelligence to 3D virtual environments, to build virtual worlds and games, train robots and usher in the era of the Metaverse and empower the creator economy.

Session Outline:

• Module 1: Foundations of Generative and Autonomous AI (Presented by Matt/Chen-Hsuan)

o Learning Objectives: Acquire a holistic understanding of Generative and Autonomous AI's roles and potential within virtual ecosystems.

o Engagement Activity: A compelling presentation encompassing the realms of generative and autonomous AI and neural methods and their use cases, methods and applications.

• Module 2: Crafting 3D Magic with Neural Radiance Fields (NeRFs) and Diffusion Models (Presented by Chen-Hsuan)

o Learning Objectives: Grasp the mechanics and methodologies underpinning 3D asset generation via diffusion models and NeRFs.

o Engagement Activity: Experience a demo and technical details involved in generating detailed 3D digital assets, be it furniture, intricate characters, or evocative scenes using NeRFs.

• Module 3: Neuralangelo – High-Fidelity Neural Surface Reconstruction (Presented by Chen-Hsuan)

o Learning Objectives: Discover the magic of neural surface reconstruction and its prowess in replicating real-world scenes in 3D.

o Engagement Activity: See Neuralangelo in action firsthand through an interactive Colab notebook exploration which you can try on your own.

• Module 4: Breathing Life into Games with Autonomous Agents as NPCs (Presented by Matt)

o Learning Objectives: Master the art of embedding intelligent, self-governing agents within games, focusing on NPCs, agent training, and hyper-automation.

o Engagement Activity: An engaging presentation with a deep dive into the future of autonomous multi-modal agents and state-of-the-art learning methods. A look at real-world applications for embodied agents like non-player characters and robotics training as well as a look at disembodied agents, swarms and their applications in hyper-automation and their future applications.

- Gain a foundational understanding of Generative and Autonomous AI.
- Learn the technical aspects required for implementation.
- Acquire hands-on experience through project work.
- Understand the ethical implications and future prospects of using AI in virtual worlds.

Background Knowledge:

- Basic understanding of AI and machine learning concepts.
- Some coding experience, preferably in Python or C++.
- Familiarity with game development or simulation engineering is advantageous but not a strict requirement.

Bio: 

Chen-Hsuan Lin is a senior research scientist at NVIDIA Research. He received his Ph.D. in Robotics from Carnegie Mellon University, where he was advised by Prof. Simon Lucey. His research interests are computer vision and machine learning, with a focus on 3D reconstruction, view synthesis, and 3D content creation. Chen-Hsuan is a recipient of the NVIDIA Graduate Fellowship (2019) and spent internships at Facebook AI Research (FAIR) and Adobe Research. He received his M.S. in Robotics from CMU and B.S. in Electrical Engineering from National Taiwan University.