When I talk about the full body virtual reality system ImmotionRoom that we’re developing at Immotionar, the first question that people asks me is always “Why Kinect? Kinect is dead”. I’ve already answered to this question, explaining that yes, it is slowly dying, but Kinect is still the best off-the-shelf infrared body-tracking sensor that we can find on the market (due to the mega-training samples fed to its algorithm). So, while we hope that other competitors like Orbbec and VicoVR will reach its quality level, we have still to use it… and this is the same reason why lots of research projects still employ the kinect.
But then there is another important question people make me: “I’m a Vive user… and Vive and Kinect interfere a lot. I like your videos about full body virtual reality, but I will surely not be able to use it…”
Well, let’s be honest: this is true. Strangely, infrared-based tracking of Oculus Rift seems not disturbed by Kinects, while laser-based Vive Lighthouse is really disturbed a lot. Someone says that radiations with wavelengths around 900nm are the issue. This is a real problem if someone wants to use our system or is doing some research projects using virtual reality and Kinect (I know a guy working on this for rehabilitation purposes), so I want to give you some advices on how to handle this problem:
- First of all, we noticed that things are now slightly better. Don’t know if it happened by chance, but it seems that compared to 6 months ago (when we shot the above video), Vive tracking is disturbed less by the Kinects. Maybe they’ve made the algo more stable and able to recover better from various problems. Six months ago, Vive was quite unusable with Kinects, while now I find it only annoying: sometimes the HMD screen gets grayed out and you can’t see anything, but I managed to use our system for some minutes with only few graying;
- The real problem happens when the HMD (or the controller) directly faces the Kinect, so it is completely hit by lots of full-power infrared-rays. So to solve the issue, it’s better to put the Kinect in a position so that it never faces directly your headset. In our user manual, our advice is to put it in a lower position, like on a table, so that if you don’t duck, you have no problems, since your face is really above the sensors. Look the images below to have an idea of what I mean with “low position”. We’ve seen that also putting the Kinect high in height can help, because rays arrive more skewed to the headset;
- Another thing that works is putting Kinect farther: the more your headset is close to the sensor, the higher are the chances of getting interferences. If you use your Vive with a Kinect that is far away, you’ll experience less problems. Remember that Kinect v2 can see your skeleton up to a distance of 4.5m; and that can detect raw depth up to 9m;
- Connecting Lighthouse towers in high position and with the sync cable helps a bit;
- Having a Kinect with the same orientation of a Lighthouse (e.g putting a Kinect and a Lighthouse station close to each other, facing the same direction) makes the problem even worse;
- In case of interference, putting the object in direct view with Lighthouse stations, re-set the object in sync: when a controller is getting lost, putting it in air, re-enables it;
- Solve the problem using filters on Kinect to filter-away the wavelength causing the interference: I’ve never tried this solution by myself, but someone told me that it works, since you remove the interfering rays. You put this filters directly on your Kinect, so to block all rays with the wavelengths causing the problem.
These hints will not solve your issues completely, but will let you use Vive with Kinect in a satisfying way.
We’ve recently discovered that these issues hold only for Kinect v2. Kinect v1, the little brother of Kinect v2, which has worse performance, shows no problems in working with Vive. We just tested it: I tried to put my Vive headset in front of the sensor and cause an interference in any way, but it never happened. I can imagine that sometimes it may even happen, but it’s a very rare case. This made us happy, since this means that our recent Kinect v1 integration will let even Vive people to try our system happily! So, if you’re in doubt and you don’t need great accuracy, use a Kinect v1 with Vive for your research projects.
About the other sensors (Orbbec, VicoVR) we still don’t know about interferences but we’ll investigate in the future.
As always, if you have further advices, opinions, etc… I’d be glad to hear from you in the comments section! And of course I’d be also glad if you like and share this article!
24 January 2017 at 12:08
How do you actually set this up? Would like to try it thanks
24 January 2017 at 17:44
Well, if you talk about using Vive + Kinect, just use any kind of Kinect (v1 or v2) with a Vive. With Kinect v1, you can put it where you prefer, while with kinect v2 you have to put the sensor so that it does not face the headset directly, as I’ve explained in the article.
If you talk about making Vive + Kinect to make full body VR, you can accomplish it with our ImmotionRoom SDK, that already does all the job if you use Unity3D. This is what I do every day 🙂 Go to http://www.immotionar.com and download the runtime for free to see potentiality of full body in virtual reality. There is a manual that explains you how to set up the room for this.
LikeLiked by 1 person
5 February 2017 at 12:38
Do you have any further information on point 7, the filters? This sounds like the only real solution.
Unfortunately, both Vive and Kinect v2 operate at 850nm infrared. How would such a filter not completely block out the Kinect for example?
5 February 2017 at 15:53
It’s something a friend told us… I’ve never used them myself. Seems that Kinect operates at multiple frequencies, so you can block some and it works quite the same. I can ask and return with an answer if I get a satisfactory one
5 February 2017 at 16:26
Another option would be to use polarizing filters, but that would block out a lot of the infrared light and you would probably need a filter for each of the sensors on HMD and controllers.
6 February 2017 at 15:22
The friend has answered me. He talked about two ways:
– First one is using polarizing filters, like the one used for the 3D cinema glasses. Some of them works with IR light, too
– Second one are filters for professional photography, but they’re hard to find since they were made for analogical photography, with films that were infrared-sensitive. Or polarizing filters for infrared cameras.
Don’t know how much this can help you