Zaid Zaim
Coding an Immersive Copilot using Unity / .NET and Azure OpenAI!
#1about 3 minutes
Introducing the concept of an immersive AI copilot
The talk introduces building an interactive AI character inspired by Disney's WALL-E using immersive technologies and generative AI.
#2about 3 minutes
Using Microsoft Mesh for collaborative XR experiences
Microsoft Mesh integrates into Microsoft Teams to provide a platform for immersive meetings, trainings, and social events.
#3about 1 minute
The core technologies for building an immersive copilot
The project is built using three main components: the Unity game engine, the Mesh Toolkit for deployment, and Azure OpenAI for intelligence.
#4about 5 minutes
Integrating Azure OpenAI service with Unity via REST API
A C# script in Unity calls the Azure OpenAI REST API to handle communication, parse the JSON response, and manage text-to-speech output.
#5about 2 minutes
Using the Mesh Toolkit for cross-platform deployment
The Mesh Toolkit provides shaders, visual scripting, and an optimizer to prepare and deploy the Unity environment to the Mesh platform.
#6about 6 minutes
Live demo of the AI copilot on a HoloLens device
A live demonstration shows interaction with the AI character "Anna" using voice commands on a Microsoft HoloLens.
#7about 6 minutes
Live demo in a shared Microsoft Mesh environment
The AI copilot is demonstrated within a shared, multi-user Microsoft Mesh space, showing collaborative interaction with the character.
#8about 6 minutes
Integrating immersive spaces directly into Microsoft Teams
The final demo showcases how users can join a Microsoft Mesh immersive space directly from a standard Microsoft Teams meeting.
Related jobs
Jobs that call for the skills explored in this talk.
Wilken GmbH
Ulm, Germany
Senior
Amazon Web Services (AWS)
Kubernetes
+1
Matching moments
03:03 MIN
Live demo of the immersive AI assistant Anna
How Mixed Reality, Azure AI and Drones turned me into a Magician?
03:32 MIN
Fusing mixed reality, AI, and drones for magical experiences
How Mixed Reality, Azure AI and Drones turned me into a Magician?
01:51 MIN
Exploring the core technology stack for the project
How Mixed Reality, Azure AI and Drones turned me into a Magician?
04:24 MIN
A practical walkthrough of the Azure AI Foundry playground
How Mixed Reality, Azure AI and Drones turned me into a Magician?
02:43 MIN
Building enterprise-grade AI agentic solutions
AI for Enterprise Developers - Dr. Damir Dobric
02:06 MIN
Creating an intelligent cockpit experience with AI and AR
Software defines the vehicle: Why customers and developers will love cars even more
01:27 MIN
Getting started with the Azure AI Foundry platform
How Mixed Reality, Azure AI and Drones turned me into a Magician?
02:41 MIN
Exploring the tools for drone and MR development
Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
Featured Partners
Related Videos
35:26How Mixed Reality, Azure AI and Drones turned me into a Magician?
Zaid Zaim
38:28Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!
Zaid Zaim
31:39Livecoding with AI
Rainer Stropek
23:33Azure AI Foundry for Developers: Open Tools, Scalable Agents, Real Impact
Oliver Will
1:12:04WeAreDevelopers LIVE - the weekly developer show with Chris Heilmann and Daniel Cranney
1:09:49Inside the AI Revolution: How Microsoft is Empowering the World to Achieve More
Simi Olabisi
24:24Best practices: Building Enterprise Applications that leverage GenAI
Damir
26:35Agentic AI - From Theory to Practice: Developing Multi-Agent AI Systems on Azure
Ricardo
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
Yadimen Consulting Limited
Leeds, United Kingdom
API
Azure
DevOps
Continuous Integration
autonomous-teaming
München, Germany
Remote
API
React
Python
TypeScript
Microsoft
Barcelona, Spain
C
API
C++
Java
Azure
+5
Missionat Humanoid
Remote
£80-90K
Intermediate
Python
PyTorch
Machine Learning
Avanade
Charing Cross, United Kingdom
Junior
Azure
Flask
Django
Python
Docker
+2