Mixed Reality: the "Recorder" feature in MARUI v1.4.4

I received some feedback about the last videos. (Thanks again!)
It seems that the "Artist's POV" makes it a bit hard to understand for people who are new to VR what is going on.
Instead, I was told, the first shot should illustrate the work environment by showing the Maya 3D scene in the real world.
I didn't want to "cheat" however, and just produce a fancy concept video. I only wanted to use what MARUI really can do.

Well, the solution was to just make MARUI do more.
So I implemented a "VR Camcorder" - a small module that allows recording videos from a different point of view than what the user sees, which can then just be inserted in a video recorded from that position.

The first attempt was still a bit crappy, but came in handy as release video for the new MARUI version 1.4.4.

The Recorder can be activated from the recording menu in the Maya shelf.
Use either the HMD or one of the controllers to set the Recorder to the position of the camera in reality.

As soon as you open the Recorder, it will start saving the video images as a PNG sequence.


To make good MixedReality videos, you'll need a bit of knowledge about your camera:
the so called "intrinsics" focal length and centroid.

The notation of the camera intrinsics is a bit different in MARUI:
they are relative to the width and height of the image.

So let's say your camera has 2/3" sensor (8.8mm x 6.6mm) and you record it with 15mm focal length, then fx would be 15mm / 8.8mm = 1.7 and fy = 15mm / 6.6mm = 2.27.

However, in most cases (unless you have quite professional gear) the effective focal length will probably be hard to calculate.

Rather you would use some camera calibration toolkit like MRPT, which will require you to take a series of pictures of a chessboard pattern and then calculate the focal length and centroid.

The output may look like this:

resolution=[1920 1080]
cx=970.40973
cy=512.52167
fx=1845.70793
fy=1861.81341

The numbers here are all in pixels, meaning the focal length is about 1845 pixels. It is also different horizontally (x) and vertically (y).

So for MARUI the resulting values are (approximately):

cx=970 / 1920 = 0.50
cy=512 / 1080 = 0.47
fx=1845 / 1920 = 0.96
fy=1861 / 1080 = 1.72

Now you can see why MARUI wants these parameters in such a strange format:

(1) MARUI started as an academic research project, where people are used to this calibration procedure.

(2) If you want to record at a higher or lower resolution, you would usually have to update the parameters (because the size of "a pixel" just changed). But since MARUI uses image-size relative values, you can just switch to 1280x720 or 960x540 without changing the values.


I'll try to make that easier to use in the future.

In the meantime: check out MARUI - Maya VR PlugIn.

More Character Animation in MARUI v1.4.3

Since the last video was not a very good illustration of how you can do 3D Character Animation in VR with MARUI, I tried to make a bit more elaborate animation.

(It actually was a lot of fun to do, but it reminded me that I didn't do any animation in years, so my skills have gotten from bad to worse...)

This also shows some of the changes in the new v1.4.3.
The Animation Menu and Time Slider are now combined, meaning you won't have to hold the "Shift" key anymore to set a keyframe.
It also shows the way of the motion to change the current frame, which should be very helpful for new users.

Making this also made me realize that it's important to know the current frame, so I quickly added that to the menu.
It's not in the v1.4.3 MARUI because it's a hack I just put in after the release.
It'll definitely be in the next release, but if you want early access, just send an email to contact@marui-plugin.com .

Character Animation in VR

One of the really cool things about MARUI is that you can take your existing character rig and start animating it in VR.

In order to properly test that, I went out and bought a rig... and it worked right out of the box in MARUI:

Video of my first attempt to animate this little rascal.
My animation skills suck, but you get the idea.

You can see how I use the left controller to flip through the frames,
and the animation menu (SHIFT + TIME buttons) to set keyframes and turn ghosting on or off.

I have yet to make the use of the animation menu a bit easier.

The real cool thing gets a bit lost in the video at the end:
because you now have two hands, you can still move objects while flipping through the frames. So if you have Auto-Keyframe enabled, you can make animations superfast: push the time with the left hand and push the object with the right. Done!

One thing I have to admit though: even though it is a low-poly rig, it dropped the framerate on my 2-year-old notebook to about 30. It's still usable, but I saw visible juddering in the Vive. Guess I have to optimize the code a bit... or finally buy "VR ready" hardware.

Give it a try on your own rig and let me know if you're having any troubles:
www.marui-plugin.com