Virtual Desktop
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Virtual Desktop
I've been thinking about a true virtual desktop for a while but I haven't had time to pursue it. After finding the cool little utility Infinite Screen http://ynea.futureware.at/cgi-bin/index.pl I had enough pieces that I could at least demo the concept. I threw this together in a couple of minutes with a Vuzix mouse emulator I wrote and the Infinite Screen app. I don't think it's usable, but it gives at least an idea of how a VR desktop should work. I have been claiming on a bunch of threads that HD resolutions and high FOV are not necessary for desktop replacement, but it's hard to put into words - so here's my little video. Maybe one of these days when I finish the 15 other projects I'm working on, I would like to work on an OS window manager that supports VR properly.
[youtube]http://www.youtube.com/watch?v=iXlFOi-7qmA[/youtube]
edit: before I get blasted - I mis-stated the resolution in the vid. It's a Vuzix 920VR so that's like 640x480, right?
[youtube]http://www.youtube.com/watch?v=iXlFOi-7qmA[/youtube]
edit: before I get blasted - I mis-stated the resolution in the vid. It's a Vuzix 920VR so that's like 640x480, right?
Last edited by brantlew on Sun Aug 05, 2012 3:26 pm, edited 1 time in total.
-
- Sharp Eyed Eagle!
- Posts: 381
- Joined: Mon Aug 09, 2010 6:08 pm
- Location: Houston
Re: Virtual Desktop
Good! I'll use this with my ST1080. Can the mouse and head move independently now? I found Infinite Screen through this thread. What didn;t you like about it? I haven't tried it yet. The thread also mentions two utilities for Nvidia GPUs, RivaTuner and nTune, which both have some kind of virtual screen resolution tool. I think Windows has its own tool built-in, doesn't it? I keep looking for more programs. There's SphereXP, which I've mentioned previously. I found an HP TouchSmart feature that creates a continuous circling desktop, left and right, no up and down. It's probably only available on TouchSmart computers, though. There are lots of those cube desktop programs, but those aren't as useful for HMDs as free-panning.
Here's a bunch of desktop enhancement tools, but they're probably all just cube programs. Here are two cool examples of how a traditional desktop can be reskinned and customized, using programs such as ObjectDesktop, Rainmeter, and Samurize. The second one has a nice spaceship window overlooking a planet in the background, which would be amazing for VR, especially in 3D. A reskinned, scrolling desktop could enable someone with an HMD to inhabit an immersive sci-fi desktop environment fulltime, potentially even in stereo 3D.
SphereXP offers the best example of my ideal - being in a virtual environment, like out on the ocean, or a scenic field, or indoors, and being surrounded by a sphere of windows superimposed onto the background. It would be like having monitors all around you. But SphereXP isn't customizable like ObjectDesktop is. I don't know if it's good for serious use. I lasted used it in 2007. I don't know how much it's been improved. But that kind of desktop environment is the future, having a scenic three-dimensional landscape with superimposed windows positioned in a sphere.
Here's a bunch of desktop enhancement tools, but they're probably all just cube programs. Here are two cool examples of how a traditional desktop can be reskinned and customized, using programs such as ObjectDesktop, Rainmeter, and Samurize. The second one has a nice spaceship window overlooking a planet in the background, which would be amazing for VR, especially in 3D. A reskinned, scrolling desktop could enable someone with an HMD to inhabit an immersive sci-fi desktop environment fulltime, potentially even in stereo 3D.
SphereXP offers the best example of my ideal - being in a virtual environment, like out on the ocean, or a scenic field, or indoors, and being surrounded by a sphere of windows superimposed onto the background. It would be like having monitors all around you. But SphereXP isn't customizable like ObjectDesktop is. I don't know if it's good for serious use. I lasted used it in 2007. I don't know how much it's been improved. But that kind of desktop environment is the future, having a scenic three-dimensional landscape with superimposed windows positioned in a sphere.
-
- Certif-Eyed!
- Posts: 559
- Joined: Mon Dec 05, 2011 3:02 am
- Location: Geekenhausen
Re: Virtual Desktop
Very cool stuff brantlew. I think it's very usable for someone who uses a HMD with the FOV beyond focus away from a monitor, so no expanded/cloned desktop over multiple devices is needed for simple things. Would it work in a game menu as well?
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Re: Virtual Desktop
@Synexious: Thanks for the link. I haven't tested any of those other desktop managers that you mentioned but I've seen a few of them. Most of them are designed with an outside perspective and not necessarily what we are looking for. The ones with an "inside" perspective are more applicable but I get the feeling that the interfaces to them are a bit complicated and we want something that just responds to simple head movement. The Infinite Screen was so small, simple in concept, and just took a mouse interface so it was easy to create the mock up with.
The thing that doesn't work well is the fact that is uses the mouse for desktop motion - so in this mockup I don't have independent control of the mouse and view. What we need is an alternative interface (not the keyboard) that could be used to reposition the view. Plus it needs a toggle switch to turn the tracking feature on permanently instead of having to press Ctrl-Shift to enable it. If I had independent head and mouse and could type normally then it would be usable.
Some other problems were Windows-centric. Windows does not allow you to resize a window beyond the size of the active desktop. So all those little windows in my demo are no bigger than 800x600. Unfortunate, because what I wanted to demo was a Word document expanded from the floor to the ceiling. Not possible it seems in Windows 7. Other nitpicks are the fact that the taskbar and icons don't move. That means you would still have problems with large FOV units.
Anyway, I sent an email to the author of the software. If we can get him to supply another interface beyond mouse input then Infinite Screen + head tracker + GlovePIE would make a very serviceable desktop.
@STRZ: Unfortunately it would not work within a game. It's actually a very simple piece of software that just moves all the windows on your desktop around. Games don't operate within the window manager, so they would not respond to this technique.
The thing that doesn't work well is the fact that is uses the mouse for desktop motion - so in this mockup I don't have independent control of the mouse and view. What we need is an alternative interface (not the keyboard) that could be used to reposition the view. Plus it needs a toggle switch to turn the tracking feature on permanently instead of having to press Ctrl-Shift to enable it. If I had independent head and mouse and could type normally then it would be usable.
Some other problems were Windows-centric. Windows does not allow you to resize a window beyond the size of the active desktop. So all those little windows in my demo are no bigger than 800x600. Unfortunate, because what I wanted to demo was a Word document expanded from the floor to the ceiling. Not possible it seems in Windows 7. Other nitpicks are the fact that the taskbar and icons don't move. That means you would still have problems with large FOV units.
Anyway, I sent an email to the author of the software. If we can get him to supply another interface beyond mouse input then Infinite Screen + head tracker + GlovePIE would make a very serviceable desktop.
@STRZ: Unfortunately it would not work within a game. It's actually a very simple piece of software that just moves all the windows on your desktop around. Games don't operate within the window manager, so they would not respond to this technique.
-
- Golden Eyed Wiseman! (or woman!)
- Posts: 1498
- Joined: Fri Jul 08, 2011 11:47 pm
Re: Virtual Desktop
I just remembered that my Z800 came with one of these programs - never used it however.
Coupled with its award-winning Z800 3DVisor,
eMagin Corporation’s XtendDesk™ software
lets you work on your computer as you’ve
never worked before. Using the Z800
3DVisor’s headtracking capability, XtendDesk
enables you to access multiple windows, simply
by turning your head, creating an almost
unlimited virtual desktop.
- cybereality
- 3D Angel Eyes (Moderator)
- Posts: 11407
- Joined: Sat Apr 12, 2008 8:18 pm
Re: Virtual Desktop
This sounds great, in theory, but I feel like you'd get a major headache trying to program or even just read the forum with headtracking like that. I mean, ultimately I'd want something like that, but more if it were a large curved projection screen (like the computer the kid had in "Gamer").
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Re: Virtual Desktop
@Cyber: Possibly. It's not really useable in this demo setup so I didn't even try to do anything productive with it. But I think it could work if the implementation were done well.
One thing you gotta realize is that my display is set to 800x600, so the text and graphics are really large. That helps alleviate the problems with focus and distortion that the HMD naturally has. It's still a bit "fuzzy" around the edges but you don't have to strain to focus on text like you do at 1024x768.
Another thing that I believe would help is optical. I'm not normally a fan of optical head tracking because it limits your movement somewhat but it's a perfect fit for this particular application and optical tracking can be much more accurate and responsive which is what you would need for precision focusing. I think if you track the head motions near perfectly then it would be a very realistic experience. Someone out there using FreeTrack and GlovePie should try this out.
I could very well be wrong but even my quick test convinced me it was possible.
One thing you gotta realize is that my display is set to 800x600, so the text and graphics are really large. That helps alleviate the problems with focus and distortion that the HMD naturally has. It's still a bit "fuzzy" around the edges but you don't have to strain to focus on text like you do at 1024x768.
Another thing that I believe would help is optical. I'm not normally a fan of optical head tracking because it limits your movement somewhat but it's a perfect fit for this particular application and optical tracking can be much more accurate and responsive which is what you would need for precision focusing. I think if you track the head motions near perfectly then it would be a very realistic experience. Someone out there using FreeTrack and GlovePie should try this out.
I could very well be wrong but even my quick test convinced me it was possible.
-
- Golden Eyed Wiseman! (or woman!)
- Posts: 1498
- Joined: Fri Jul 08, 2011 11:47 pm
Re: Virtual Desktop
Optical is easy and convenient, but seriously the Hydra (magnetic tracking) has it beat every way. As long as you are careful with your emitter placement to avoid interference, that is.
The problem with optical is that when you turn around 45 degrees, it starts to lose tracking. While you can just not turn that far, its still a PITA even in flight sims etc. The Hydra gives you absolute tracking (just like optical) but with the advantage of being able to look ANYWHERE.
I still use optical currently, due to me not having a Hydra -> TrackIR interface written yet, but I dont really see the value of my TrackIR anymore compared to it. Razer should release a TrackIR competitor and give Naturalpoint a heart attack.
The problem with optical is that when you turn around 45 degrees, it starts to lose tracking. While you can just not turn that far, its still a PITA even in flight sims etc. The Hydra gives you absolute tracking (just like optical) but with the advantage of being able to look ANYWHERE.
I still use optical currently, due to me not having a Hydra -> TrackIR interface written yet, but I dont really see the value of my TrackIR anymore compared to it. Razer should release a TrackIR competitor and give Naturalpoint a heart attack.
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Re: Virtual Desktop
A couple thoughts on optical.
- The failure of optical to track beyond 45 degrees is not the fault of the technique but of the implementation. Instead of a light triangle on a hat, if you were to wear a headband with lights around your head you could (with the proper software) track well beyond 45 degrees.
- As a practical solution, even a 45 degree tracker could have huge benefits. Just thinking out loud...imagine you are flying on a plane. You put your open laptop on your lap, look straight ahead, and point the integrated webcam upwards towards your face. Now you put on your (inoffensive) glasses-style HMD (Vuzix or ST1080) with a few light beacons attached to it. And now you can work on a 40" virtual monitor (or larger) directly in front of you. (You can't see it all at once but you have that much desktop available) That could be huge in terms of usefulness and marketability. And it doesn't rely on far flung technologies. The hardware is basically all there now (especially if the ST1080 comes through). It just needs the proper software.
- The failure of optical to track beyond 45 degrees is not the fault of the technique but of the implementation. Instead of a light triangle on a hat, if you were to wear a headband with lights around your head you could (with the proper software) track well beyond 45 degrees.
- As a practical solution, even a 45 degree tracker could have huge benefits. Just thinking out loud...imagine you are flying on a plane. You put your open laptop on your lap, look straight ahead, and point the integrated webcam upwards towards your face. Now you put on your (inoffensive) glasses-style HMD (Vuzix or ST1080) with a few light beacons attached to it. And now you can work on a 40" virtual monitor (or larger) directly in front of you. (You can't see it all at once but you have that much desktop available) That could be huge in terms of usefulness and marketability. And it doesn't rely on far flung technologies. The hardware is basically all there now (especially if the ST1080 comes through). It just needs the proper software.
Last edited by brantlew on Sun Jan 22, 2012 12:40 am, edited 2 times in total.
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Re: Virtual Desktop
Good news. I got in touch with the creator of Infinite Screen. Here is our last correspondence:
hi
this is a good idea, give me time to think and do ...
greetings Robert
##################################
On Jan 21, 2012 10:32 PM, Brant Lewis wrote:
Hi, and thanks for responding.
Instead of working with the limitations of the mouse, I think a good solution is to create another interface. The mouse and keyboard interface work well with monitors. For the VR glasses, it would be better to have a different way to control the view instead of the mouse. You could then have independent mouse motion and view motion. For example - if you provide a IP socket interface then the VR glasses can send messages directly to Infinite Screen to "move left" or "move right" instead of sending mouse input. Is this something that would be possible for you to implement?
Thanks again,
Brant
-
- Golden Eyed Wiseman! (or woman!)
- Posts: 1498
- Joined: Fri Jul 08, 2011 11:47 pm
Re: Virtual Desktop
Indeed, but if you have a whole lot of multiple identical lights, you can lose the ability to do absolute tracking unless you can differentiate some of them. Otherwise, its possible for unexpected events (framerate loss for example) to cause it to lose the track. However, things such as TrackIR / Freetrack could be improved by supporting more IR's in the tracker.
If you are only wanting a virtual desktop it wil lbe fine probably, but you'll still find it annoying when you accidentally look just a little too far and find it losing track until you move your head back.
If you are only wanting a virtual desktop it wil lbe fine probably, but you'll still find it annoying when you accidentally look just a little too far and find it losing track until you move your head back.
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Re: Virtual Desktop
Hey, I just noticed that Infinite Screen 1.4 was released and now has an independent IP socket interface. That means that I can implement my test app with independent mouse and head motion which should make it truly usable as a virtual desktop. Super busy, but I'll try to find some time this week to test this.
Edit: I'll try to implement this as a FreePIE plugin so that a variety of head-trackers can be used.
Edit: I'll try to implement this as a FreePIE plugin so that a variety of head-trackers can be used.
-
- Sharp Eyed Eagle!
- Posts: 377
- Joined: Sat Apr 12, 2008 9:45 pm
Re: Virtual Desktop
Great. Been looking for a working virtual desktop.
-
- Petrif-Eyed
- Posts: 2166
- Joined: Mon Jun 22, 2009 8:36 am
- Location: Stockholm, Sweden
Re: Virtual Desktop
Nice, how about a custom plugin, not part of the Core? If there are evey duty frameworks around it maybe that is better?
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Re: Virtual Desktop
Well I'm writing the Vuzix module as part of the core. How would I assemble the Infinite Screen module as an independent plugin?CyberVillain wrote:Nice, how about a custom plugin, not part of the Core? If there are evey duty frameworks around it maybe that is better?
-
- Petrif-Eyed
- Posts: 2166
- Joined: Mon Jun 22, 2009 8:36 am
- Location: Stockholm, Sweden
Re: Virtual Desktop
Just create a classlibrary project and reference FreePIE.Contracts, copy the output to the plugins folder of FreePIE guibrantlew wrote:Well I'm writing the Vuzix module as part of the core. How would I assemble the Infinite Screen module as an independent plugin?CyberVillain wrote:Nice, how about a custom plugin, not part of the Core? If there are evey duty frameworks around it maybe that is better?
edit: Yeah, Vuzix is a given Core plugin
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Re: Virtual Desktop
Oh jeez. Don't these marketers even read these threads before they post?
-
- Petrif-Eyed
- Posts: 2166
- Joined: Mon Jun 22, 2009 8:36 am
- Location: Stockholm, Sweden
Re: Virtual Desktop
Probably a botbrantlew wrote:Oh jeez. Don't these marketers even read these threads before they post?
- cybereality
- 3D Angel Eyes (Moderator)
- Posts: 11407
- Joined: Sat Apr 12, 2008 8:18 pm
Re: Virtual Desktop
Don't bother replying to the spammers, just hit the report button and me or another mod will delete their account (and all their posts).
-
- One Eyed Hopeful
- Posts: 2
- Joined: Sun Aug 05, 2012 1:24 pm
Re: Virtual Desktop
Hi, I'm a total noob in here, but Rift looks too sexy not try it out.
One of the ideas I thought about was the virtual desktop as well, so this thread is a great read. The difference is I plan to do it on linux and I hope to use the compiz plugin interface with hillcrest as a input device to control it.
I don't have the hardware, so probably I'll start by mocking libfreespace and use some caught mouse events to control it for now.
My current worry is the resolution of the panels. I actually wanted to use this setup to work and reading the recent postings on this forum, I'm not sure it is viable option at all. Do you think Rift will be good enough?
One of the ideas I thought about was the virtual desktop as well, so this thread is a great read. The difference is I plan to do it on linux and I hope to use the compiz plugin interface with hillcrest as a input device to control it.
I don't have the hardware, so probably I'll start by mocking libfreespace and use some caught mouse events to control it for now.
My current worry is the resolution of the panels. I actually wanted to use this setup to work and reading the recent postings on this forum, I'm not sure it is viable option at all. Do you think Rift will be good enough?
- cybereality
- 3D Angel Eyes (Moderator)
- Posts: 11407
- Joined: Sat Apr 12, 2008 8:18 pm
Re: Virtual Desktop
I honestly doubt the Rift will be useful for work, for a few reasons.
1) The resolution is not that high at 640x800 per eye. This is also a strange aspect ratio.
2) The optics will warp the image, and make text look strange along the edges especially.
So I think it will be mostly for playing games, but maybe it could work.
1) The resolution is not that high at 640x800 per eye. This is also a strange aspect ratio.
2) The optics will warp the image, and make text look strange along the edges especially.
So I think it will be mostly for playing games, but maybe it could work.
- brantlew
- Petrif-Eyed
- Posts: 2221
- Joined: Sat Sep 17, 2011 9:23 pm
- Location: Menlo Park, CA
Re: Virtual Desktop
@fanhaf: I agree with Cyber. The Rift prototype is not going to be the proper display for virtual desktops because of the low resolution. And it wouldn't even work with the way I am doing it here because the taskbar would be out of your field of view. You need a complete virtual desktop where all elements could be within the field of view.
You should check out this thread instead. These guys are on the right track and it's a Linux solution also.
http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15088
You should check out this thread instead. These guys are on the right track and it's a Linux solution also.
http://www.mtbs3d.com/phpBB/viewtopic.php?f=138&t=15088
-
- One Eyed Hopeful
- Posts: 2
- Joined: Sun Aug 05, 2012 1:24 pm
Re: Virtual Desktop
Thanks, this is what I was looking for
-
- One Eyed Hopeful
- Posts: 40
- Joined: Fri Jan 21, 2022 12:10 am
Re: Virtual Desktop
Wow, shameless plug there buddy.