On October 6, 4pm to 5:30 pm (CEST, Berlin), Mohamed Kari will present his coauthors' and his work on TransforMR: Pose-Aware Object Substitution for Composing Alternate Mixed Realities at this year's International Symposium on Mixed and Augmented Reality (ISMAR '21). Join the publicly streamed paper session on YouTube.

While continuous advances in geometric scene understanding have contributed to the physical coherence of virtual objects in mixed reality scenes, semantic scene understanding and functional— rather than physical—reasoning remain hard problems. Through pose-aware object substitution, TransforMR presents a way of composing alternate mixed realities that feature semantically interpretable scenes. The ML-based computer vision pipeline comprises real-time instance segmentation, video inpainting, 3D object detection for humans, and 3D object detection for vehicles. It runs as a cloud-assisted system at approximately 15 frames per second where perception is offloaded to a GPU-accelerated host in cloud.

Tune-in to the live paper session on YouTube on October 6, 4pm to 5:30 (CEST) or check-out the video and paper.