XR / Virtual production

Q and A about functionality and how to solve a special task for your application.

Moderator: Support

Post Reply
VincentPix
Posts: 44
Joined: 06 Nov 2014, 15:07

XR / Virtual production

Post by VincentPix » 09 Oct 2020, 17:18

Hi everyone,

I'm working on a XR config (just for fun at this moment). I imagine you saw the "Mandalorian" technic to have the content of the screen following the tracking of the camera.

For now i code in C# a view and projection doing the stuff quite well => Input : position and size of the screen + camera position => return view matrix + proj matrix.
(If you want this piece of code, I'm ok to share. Just ask)

It's all right until I merged my content scene. My view matrix completely disturb the light calculation it seems.
In the scene of the attached snapshots there's just a skybox (irradiance only) and Directional Light with shadow.

If someone has a hint...
Attachments
Snap2.jpg
Snap1.jpg

chriss0212
Posts: 665
Joined: 18 Jan 2012, 20:56
Location: wuppertal
Contact:

Re: XR / Virtual production

Post by chriss0212 » 12 Oct 2020, 08:34

Hi Vincent

maybe the Design Eyepoint setting in the camera will fix you problem:
Design Eyepoint
At times you might need to calculate lighting independently from the camera's actual position. Especially projection mapping setups need to be able to have different locations for the rendering point and for the viewer's point. This is because you need to render a scene from the position of the projector in order to map it correctly onto the projection's screen. But still the scene's specular points and reflections need to be calculated from the point of view, for example an audience in front of the screen. This becomes even more visible when using more than one projector, each creating different specular reflections.

The Design Eyepoint can be attached to an anchor in the scene that is located at the position of the viewer. Now regardless of the position of the active camera, all shaders will use that position as the camera position for lighting purposes.
Greetings

Christian

VincentPix
Posts: 44
Joined: 06 Nov 2014, 15:07

Re: XR / Virtual production

Post by VincentPix » 12 Oct 2020, 12:52

My previous answer doesn't appear in the forum. ?
If it's lost : the design eyepoint was what I was looking for but didn't work. :(

I made a "demo" scene to show the problem in the attached vza. You'll also find in it the hints I'm using but for now not the complete solution.
Attachments
XR01.vza
(284.06 KiB) Downloaded 25 times

chriss0212
Posts: 665
Joined: 18 Jan 2012, 20:56
Location: wuppertal
Contact:

Re: XR / Virtual production

Post by chriss0212 » 13 Oct 2020, 08:32

Dear Vincent

Your problem seems to be related to your script.

If i use a "normal camera" and animate it, it works as expected.

Sorry, but i am far away to be a scripting guru... so i can not help there ;)

Greetings

Christian
XR01_CK.vzs
(58.11 KiB) Downloaded 15 times

VincentPix
Posts: 44
Joined: 06 Nov 2014, 15:07

Re: XR / Virtual production

Post by VincentPix » 13 Oct 2020, 14:04

Yes exactly and more precisely the view matrix. If you change the scipted view matrix by a simple "look at" while keeping the scripted projection matrix, the rendering is ok.
This is more a 3D matrix problem than a scripting problem I think. And how the view matrix intervene in lighting, irradiance, texturing, SSAO...

So, does anyone know if there is another way to have this "window effect" without the view matrix / light calculation problem ?

Thanks

VincentPix
Posts: 44
Joined: 06 Nov 2014, 15:07

Re: XR / Virtual production

Post by VincentPix » 20 Oct 2020, 08:00

Problem solved by myself.
If interested on this subject. Feel free to send me a massage.

Post Reply