ARMARJournal

From DigitalBlacksmith

Jump to: navigation, search

ARMAR Development Journal

Contents

[edit] Oct 2008

[edit] 18 Oct

Made a set of doxy documentation:

here (login)

Other things to try

Attachments

CViewSetup -- move origin before VGUI paint

VGui_Paint

OK -- getting somewhere with this..maybe 

 
Msg("}}}}}}}}}}}}}}}}  tools ipanel z pos = %d\n",vgui::ipanel()->GetZPos(root));
 

This shows the VGUI panel at 25 and the tools at 28

Going to try and vary it and watch it in stereo


OK I am abandoning attempts to tweak the rendering of the existing HUD

I already have code for a player slaved HUD...its hud_vguiscreen.cpp

But it lags so I am going to try attaching a vgui screen like described in this tutorial.

I did the rpg one..works well and best of all NO LAG

So now I am going to try and attach a panel to a player:

Added this to player.cpp:

Lines: 431 

 
Msg("--------Spawinging player control panels....\n");
vm->SpawnControlPanels();
 

This indicated that it was trying to find a model....

Need to try to get some attachment points.

What's the model?

Found it in c_baseplayer.cpp and a few other players (search on player.mdl)

OK..this is worthy of its own page...here.

[edit] 17 Oct

Finished the 3d HUD using the VGUI screen class.

Trouble is, I am getting the client-server lag.

[edit] Ideas

Try to disable physics and prediction

Use teleport

Check out this on the client side:

cl_hud_vguiscreen.cpp: 

 
//-----------------------------------------------------------------------------
//  Computes the panel center to world transform
//-----------------------------------------------------------------------------
void hud_VGuiScreen::ComputePanelToWorld()
{
    // The origin is at the upper-left corner of the screen
    Vector vecOrigin, vecUR, vecLL;
    ComputeEdges( &vecOrigin, &vecUR, &vecLL );
    m_PanelToWorld.SetupMatrixOrgAngles( vecOrigin, GetAbsAngles() );
}
 

Maybe alter vecOrigin manually? From in_main.cpp?

It looks like the client can use:

GetAbsOrigin()

Interesting..


[edit] 16 Oct

Finally getting some time to get back at research..been trying to get ahead in OS.

OK. Immediate task at hand is to build a 3D HUD class.

The issue: The NVIDIA Stereo driver clobbers the HUD elements for Valve. Basically, the NVIDIA Stero driver makes assumptions about how "deep" in the scene the HUD is, and sperates the HUD accordingly in the left and right eye...unfortunetly, this seperation is wrong.

I did a lot of investigation. It is def something in the NVIDIA driver. Supposedly, it's possible to set a filter in the driver, where objects inside the filter aren't shown in stereo. Not sure how to do this.

I did find an "alternative driver" by KindDragon (aka. "Russian Driver"). This is a very nice stereo driver. However, it's not usable for us, because it is already using the Windows Detours API, which interferes with the Valve Source video hook I'm using for video see through stereo...

[edit] Sep 2008

[edit] 24 Sep

[edit] Wednesday Meeting

[edit] What I am doing

  • Demo: 3D VGUI
  • Kevin needs a fast machine
  • Candidacy exam

[edit] Others

  • Nitek working on Klein code
  • Order HDMI cable
  • Test projector

[edit] NVIS

  • Do Stereo test

[edit] 2 Sep

Today I am working on tweaking the NVIDIA Stereo driver to try and optimize stero in the HeadPlay.

Involves a ton of registry hacks. 

HKEY_LOCAL_MACHINE\SOFTWARE\NVIDIA Corporation\Global\Stereo3D

Going to post the experience to my [Page].

[edit] August 2008

[edit] 26 Aug

Working on demo demo demo

I have a twitter account: http://twitter.com/stevehenderson

[edit] 20 Aug

Here's the code to load a config file: 

Msg( "Executing dedicated server config file\n" );
Q_snprintf( szCommand,sizeof(szCommand), "exec %s\n", cfgfile );
engine->ServerCommand( szCommand );


[edit] 15-19 Aug

Worked on submitting the final version of the VRST08 paper.

[edit] 14 Aug

rsync -azv -e ssh --exclude 'backup' /raid root@magritte:/mnt/usbdrive/goya_bu/

[edit] 10 Aug

Here's the config calib I've been using with OT: 

armar_coord_frame_x "-12"
armar_coord_frame_y "-40"
armar_coord_frame_z "25"
armar_tracker_head_pitch "0"
armar_tracker_head_roll "0"
armar_tracker_head_yaw "90"
armar_tracker_head_z "0.5"
armar_tracker_head_x "1.25"
armar_tracker_head_y "-1.0"

Found some interesting methods for drawing debug text:

Search for: 

AddTextOverlay

The debuboverlay_shared.cpp file has a bunch of methods.

Also the "point_message" hammer entity will allow you to add it to the map directly.

[edit] 7 Aug

In order not to break the AC code - I'm not going to touch any of the DART maintenance controllers.

The server code for the Albany demo is going here: 

dll/AR/maintenance_controller.cpp

Will need to add a header file..

Starting a new Demo page...ARMAR Albany Demo

Thinking about using config files for each state..basically use console commands to toggle visibility, etc.

Something like this to load a config file: 

char szCommand[256];
				
Msg( "Executing dedicated server config file\n" );
Q_snprintf( szCommand,sizeof(szCommand), "exec %s\n", cfgfile );
engine->ServerCommand( szCommand );

This file has some interesting looking calls to save to a config file: 

C:\ARMAR\ARMARSP\src\public\vgui\ISystem.h

 
// user config
virtual KeyValues *GetUserConfigFileData(const char *dialogName, int dialogID) = 0;
// sets the name of the config file to save/restore from.  Settings are loaded immediately.
virtual void SetUserConfigFile(const char *fileName, const char *pathName) = 0;
// saves all the current settings to the user config file
virtual void SaveUserConfigFile() = 0;
 

I can't find any uses in the API for these methods...

[edit] 6 Aug

Hooked up 10 x V100 Optitrack Cameras using the new cage -- the tracking is much better, and more robust. I think this will be a workable solution providing we can get a good camera layout inside the turret.


[edit] 5 Aug

[edit] Modeling

Worked on a decent model we could use for developing the demo...involved a lot of changes to the smdExport script to handle nested groups..believe I have it working now.

[edit] Camera cage

The camera cage for rigid and flexible V100 positioning
The camera cage for rigid and flexible V100 positioning

Also setup a "camera cage" that would allow for rigid mounting of the cameras...I built it using components we purchased from McMaster Carr -- basicallu 6ft steel angles and 2" triangle brackets.. You can see a list of the items at the bottom of the McMaster-Carr catalog Page.

I used about 20 1.5 x 1.5 inc 6 foot sections (Item 4664T54) and a ton of corner plates (Item 4664T22)

The cage is pretty sturdy..though I order some more, thicker members (Item 4664T67 ) for reinforcement.

Lessons Learned

  • Use thicker gauge angles (Item 4664T67 ).
  • Use extra corner triangles -- use on all possible sides of a junction
  • Goes up pretty fast


[edit] Script

Spent some time thinking about how best to author the scipt for the demo..Going to try using console commands as the primary interface that manipulates the objects in the scene at each point in the script..So nothing has to be recompiled.

[edit] 4 Aug

Trying to get a barebones model done of the turret that can be used to register content on the real turret and also could be used to provide missing LRUs.

The ExportSMD script is choking on nested groups...it works well for one level froups but not others

Need to implement a recursive crawler. Should be easy!

--Steve 12:44, 4 August 2008 (EDT)

I got the crawler working..after much agnst..have tested it a bit and seems to work...had trouble with empty transforms that are used as subgroups but don't have polys. I write about the solution here.

--Steve 22:03, 4 August 2008 (EDT)

Dr. Feiner and I were discussing a MagicWand idea -- a dynamic tool that would allow one to use a tracked Optitrack rigid body to position things inside Valve..

Installed Stereo on the Dell XPS 1410 w/ GeForce 7850..Here

[edit] 2-3 Aug

  • Finished a decent lower half turret
  • Tweaked and tested the export script - had to alter the Shape node detection. Tested with the CDA, Ammo can, and lower frame. Works well. Need to watch this as a saw a bunch of ignored shape nodes in the cda export -- check the output.
  • I also discovered the power of Maya references -- they are awesome. Instead of importing a file, and risk version control issues, just make a reference to it. If you change the reference (e.g. shaders, etc) then these changes will persist to the referencing file. But some changes will kill any transformations in you referencing file so be careful
  • I also updated the batch file creator to add in a -noxbox flag
  • In Hammer/vmf's us the following to keep things in place (ignore physics impacts) 
"spawnflags" "1294"

This will keep them stable except for hand grenades and possible other weapons

The technique of using Maya as a central registration tool, then dropping everything in hammer at the origin seems to work. The downside is you have to export models when they change location, and this takes time.

It would be better if you simply compiled the models in model space -- then dropped them in hammer at the angles and location (the world transform). However, this could get dicey when you think about model orgins, and manually reading the transform information from May.

A compromise might involve reading just the transforms from Maya and outputting prop_physics_overide objects into hammer with the requisite transform. This may be tough.

--Steve 02:14, 3 August 2008 (EDT)

[edit] 1 Aug

  • Did a demo for some CU folks looking to do some fire fighting / first responder application
  • Tried like heck to optimize head tracking with the Optitrack
    • Need to pay attention to reflections -- moved a bunch of stuff arounf on the HWD to mititgate
    • Get a lot of jitter in YAW -- need to explore Inertia Cube for orientation
  • Ordered bunch of stuff to build a cage

[edit] July 2008

[edit] 31 July

Did a demo for PBS today

Fixed the memory leak

Internal 1394b connection is toast on the DellXPS...luckily, I found a PCMCIA-express card..works well..still requires external power

[edit] 30 July

I am hooking up a demo that uses the IS900 for ehad tracking but features an OT tracked rigid body

So we have mixed coordinate systems

The IS900 (In ARMAR) is showing the origin at (45, -1, 21)

I'm going to adjust this using the rigid body offset

OK here is the rotation offset I am using: 

45
209
-21

I confirmed this with an axis aligned static prop version of the CDA....then tweaked position offsets on the tracked version until I could use the tracked version as a guide.


I am going to attempt to register position by hand (dream) and then use ARTag as a fallback

Noticed a memory leak (bad one) in ARMARMessageServer...FIX THIS

OK I have it tracking a nice CDA: 

FOV=35
rigid_body_pos_offset.x=45.4;
rigid_body_pos_offset.y=-0.5;
rigid_body_pos_offset.z=18.2;    

rigid_body_ang_offset.x=180;
rigid_body_ang_offset.y=0;
rigid_body_ang_offset.z=0;

[edit] Flicker Problem

I did have a flicker problem -- PGR's are picking up IR flicker from OT camera illuminators....I tuned this out by setting the Exposure on the cameras to ~ 30.00 ms

PGR Slider Tweak note -- manipulate the slider with mouse...then use the left and right arrow keys.

(off, b b m, labz)

[edit] 29 July

Ok today I worked on setting up the OC demo for PBS..removed the OptiTrack markers from the HWD and reinstalled the IS900 sensor. I also reconfigured the DellXPS as a CGUI machine and ran it across the network..the ARMARMessageServer is talking to the GLUI tool but haven't tested with the engine yet.

I found out that recalibrating the OT at 5m seemd to work pretty well

Also, when registering rigid bodies, its important that you don't have any echoes in the FOV..that is if you see multiple markers, and use the "Create from Visible Markers" option, the rigid body will register the echoes..so be careful to get a "clean" rigid body..recalibrate!

I tested out the revised export MEL script and ran into some issues...modified the code a bit. Changes documented [MayaValveExport_DB|here]].

When I exported the CDA, I had 2 groups cda_base and cda_lights. Added these groups to an emtpy layer titled export Carefully deleted history and froze all the transforms. After export, I copies all the cda_lights*.smd files into the cda_base directory. I then executed this qc (living in the modelsrc/cda_base folder -- run cda.bat): 


$modelname "cda\cda.mdl"
$model "body" "cda_base.smd"
$cdmaterials "models\cda\"
$scale 1
//$hboxset "default"
//$hbox 0 "polymsh" -6.164  -0.003  -2.859  6.166  8.204  2.858
$surfaceprop "item"
$keyvalues  { prop_data { "base" "Item.Small"  } } 
$illumposition -0.001 0.001 4.100
$sequence idle "cda_base_idle" loop fps 5.00

$bodygroup studio
{
    studio "cda_lights.smd"
    
} 

$collisionmodel "cda_base_phy.smd" {

    //Mass in kilograms
    $concave
    $mass 2.0    
}

It works pretty well.

I also did some research on our APC battery problem...here.

(r=lusk, e=ggg)

[edit] 28 July

CDT June Yi is aboard..set up portal here.

  • Brought the OT stream down to Sleep value of 20
  • Enabled DSP -- wors well
  • Took out the "no double packet code" This was a swag I thought might have been causing the mysterious zeros in the ARMAR Message Server stream..turns out it was a threaded issue


I nixed the following defaults, which will be needed if you want to use the IS900 w/ the OC map and others: 

ConVar armar_coord_frame_x( "armar_coord_frame_x", "-284", 0);
ConVar armar_coord_frame_y( "armar_coord_frame_y", "-139.3", 0);
ConVar armar_coord_frame_z( "armar_coord_frame_z", "-23.5", 0);

[edit] 25-28 July

Weekend in Boston (OFF)

[edit] 24 July

Happy Birthday Dad

I've been working on the OT tracker..spent 36 hours struggling with pipes. Here's some random thoughts:

Had a lot of problems with named pipes..need to pay attention to OVERLAPPED settings as well as sharing on both the client and the server..need to check all the other named pipe using objects

Also, I ended up placing OptitrackClient.cpp in its own console app..this seemed to really help


I do get one error from the tracker..it places all zeroes in a few positions...I think this is coming from the tracker 9conflict in cameras -- rivalry) or perhaps there is a bug in OptiTrackClient..


  • Need to work through the orient mapping..believe I have something switched in the streamed text
  • FPS is pretty high on the OptitrackClient..maybe bring it down some


[edit] 23 July

Kicking some ideas around for the CAVE


[edit] 19-22 July

[edit] Clobbered Pipes

I am having an issue when part of the pipe is getting shifted...Basically its dropping characters from the buffer which causes all kinds of issues...quanternions roll into position etc...

The problem goes away when I reduce the traffic on the pipe (lower tracking HZ)

Ideas:

  • May be time to enforce overlapped pipes
  • Do some type of avergaing and check for outliers (hack)

--Steve 23:42, 22 July 2008 (EDT)

Still working on the Optitrack..I have it tracking objects and the player in Valve ARMAR..

It's a little jittery.

Called Optitrack tech Support ... nice folks out in Oregon.

Here's what they said:

  • Ensure you have the latest SDK and Calibration tools -- current version 1.3.035 (I do now)
  • For the tracked object, it may help to override default residuals -- increasing it
  • You want 150 data points for calibration (Medium or higher)
  • Don't worry about getting all Great results for the cameras

This was helpful but I am still getting some jitter. Going to bring in a Double Exponential Smoothing algorithm for prediction.

Here's a nice paper by LaViola on DESP and Kalman Filter approaches:

Joseph J. LaViola - Double exponential smoothing: an alternative to Kalman filter-based predictive tracking
EGVE '03: Proceedings of the workshop on Virtual environments 2003 pp. 199--206, New York, NY, USA, 2003
UrlBibtex
Auteur : Joseph J. LaViola
Titre : Double exponential smoothing: an alternative to Kalman filter-based predictive tracking
Dans : EGVE '03: Proceedings of the workshop on Virtual environments 2003 -
Adresse : New York, NY, USA
Date : 2003

[edit] 18 July

Lately I've been hooking up the NaturalPoint OptiTrack tracker...more to follow on this..

I've also been thinking a lot about the modeling pipeline -- how to handle the growing number of model, how to register them, etc

I've concluded that Hammer is not the best place to integrate multiple models..too hard to verfiy positions, coordinate systems, etc. The biggest problem is that the reference mesh is way to big to put into Hammer ( > 200K polys). I will be using the reference mesh a great deal to verify things now that the turret is gone. I'd like to have confidence that the exported Valve model matches the reference mesh as closely as possible.

So, ideally we would be able to set up most of the scene in Maya..place the CDA, PDA and all of the other models over the reference mesh. Then export the entire kit such that the transformations in Maya were preserved.

So for the past 24h or so, I've been working on a modification to Prall's export script that would allow for me to export and compile multiple models in one step.

The new script allows me to subdivide the exported models into groups...the script then rips each group and creates a model (complete suite of smd, qc, bat) for each group/model. The transformations are preserved. I then drop these all at origin in Hammer (can do this with manual edits to the vmf). Then add in the non Maya stuff (sprites, artag_models, optitrack_models, etc)

If I need to tweak a position, I tweak maya and rerun the uber export tool.

The script is complete but I need to exercise and test the entire pipeline...

[edit] 8 July

Ok finally got the stereo hook working again....sort of..could not get the video flip to work..it was hooking the video for the left image but would not pick up the stereo pair properly.

Of course I tried a million things, so its hard to say which worked out.

The hook works but locks up the machine (no blue screen just freezes video)

Here are my NVIDIA settings

Not sure what is happening here...maybe driver?

I new lesson learned was recompiling the Valve Mod source (game code)..once I did that it worked. Maybe it was a DirectX mismatch..

In remembrance of this saga I am starting a new blog:

NVIDIA Stereo Hook

Got it I ended up getting the NVIDIA 162.50 pair working..Read about the configuration here.


[edit] Moving On...

As you may know, we had to send back the turret...I made a couple of last minute turret measurements prior to shipping in order to verify the model.

Front Mains: center to center -- 117.5cm, diameter - 7.5 cm

Ammo Can Lid: 24cm wide

Ammo Can: 23cm wide

Baseboard to Ammo Lid Top: 45.75 cm

[edit] 7 July

Installed the Micorsoft Platform SDk as well as the Direct X SDK (June 2008) and it's breathing again! Praise the Lord!

--Steve 11:15, 7 July 2008 (EDT)


[edit] Intersense DLL Reinstall

Moving these files into the Windows/System32 dir: 

ARMDLL32.DLL
isense.dll

Intersense IS900 tracking restored..

--Steve 11:40, 7 July 2008 (EDT)

[edit] Stereo

Going to move on to stereo testing.

Tried the 91.31 Forceware/Stereo combo...blue screens I might have screwed it up by first installing the Dell drivers and not downgrading with NVClean.

Here are the various combinations of driver and 3d driver I am trying.

I'm going to roll back (uninstall) all the drivers I currently have and use Nasty File Remover to delete the old files before trying the new (162.50) drivers

[edit] 4 July

Trouble in Paradise this 4th of July

Reinstalled windows and the ARMAR code on warhol

Having trouble with the hook

Initially, the hook would crash with a "Could not hook" message

I discovered, by trying to hook firefox.exe, that I was missing glut32.dll

Now the hook will inject, but crashes when hooking the DirectX interface. Not sure why yet

I did hook HL2DM once, but can't anymore...ug

I was also able to hook some of the examples from the DirectX 9 (2006) library

Ideas/possibilities:

  • DirectX library mismatch -- perhaps time to try a more recent version of the DirectX SDK
  • Visual Studio libraries jacked, Platform SDK, or other tool configuration


--Steve 05:21, 4 July 2008 (EDT)

Trying to install the Microsoft Platform SDK (version Windows SDK for Windows Server 2008 and .NET Framework 3.5)

[edit] May-June 2008

Worked exclusively on the Indigenous Controls code on the secure site.

[edit] April 2008

[edit] 20 Apr

I spent the day working on the Uber-panel for the AC demo.

You can check it out here (login req).

A couple of things for general consumption.

The vgui::HTML control is awesome. It allows you to render html directly into the VGUI, ala Java swing html parsing. You have to set it up correctly, the key points being:

You need to tell the html loader to not copy offline: 

 
//The true flag here is vital
infoHTML->OpenURL( localURL , true);
 

You also lose transparency control, as the html portion is a bitmap that is scrapped from the browser. Still, this will save a ton of time when you consider the ability to render internet content, images, tables, etc.

[edit] 19 Apr

I revisted the ARTag integration, as it got messed up during my calibration exercise.

The code is now working (rotates and translates accordingly). Checked in and updated on both warhol and eva.

Check it out here.

[edit] 18 Apr

I jumped back on the AC demo...follow along on the secure side.

[edit] 17 Apr

I finished up the screenshots...this was a nice forcing function to get a lot done. You can read the full details Follow along here.

In summary:

  • MyHudPanel has been revamped
  • Integrated in_main and MyHudPanel and CCmaintenanceController
  • Added an armar_set_model_alpha (or is it armar_set_model_transparency) command to set the transparency of any entity by supplying the name of the entity and an alpha value

ToDO:

Based on feedback from Dr. Feiner we need to address:

  • Registration of the manual turret traverse
  • Close cropping of text on left hand side

[edit] 15-16 Apr

Dr. Feiner wants to generate some screenshots for our stakeholders.

Revisiting the demo code. Follow along here

[edit] 14 Apr

A lot going on..currently deployed to the lab. Came down to work on the AC project.

We are moving away from the turret to the engine as not to conflict with any NDA issues

Going to do an electrical harness check.

The xenerac display interferes with the IS900...ug.


[edit] 13 Apr

Today I am Tuning the Connector AC


[edit] 12 Apr

I've spent the past 24 hours integrating the AC code into ARMAR.

I ended up doing a major rework of the image capture pipeline...this was necessary to support the synchronization of the rendured frame with ARTag and AC (debug overlays in particular).

Took me an entire day to do the rework, but it's now tracking..need to test stereo.

Things to note/Long range TODO's

  • There are five flags and four indices in a ImageBuffers object that control the synch of Grabbing, Surface Loading, ARTag, AC, and rendering. This ImageObject also has two buffers per left and right camera. The indices and flags are used to point the Grabber, Surface Loader, ARTag processor and AC processor at the appropriate image
  • Right now, only the PGR grabber points directly at the images in the ImageBuffer. I'm trying to avoid the number of times I move pixels....Can probably do the same in the DSVL code (USB).
  • Right now the four flags kind of move in lock step..need to look at sneaking the surface loader ahead once the grabber is done
  • Drawing AC overlays is a huge hit on frame rate..
  • Need to thread AC...

TODOS:

  • Get PGR running with the AC GUI
  • Calibrate PGR 6mm lens
  • Work the segmentation tuning
  • Finish the demo

[edit] 11 Apr

I spent today finalizing the VGUI menu control that is used in the AC demo. This involved adding a few classes to the client and the server:

[edit] Client changes

[edit] CClientMaintenanceController (cl_dll/AR/ClientMaintenanceController.cpp)

This is a centralized maintenance controller that is meant to offload the demo control logic out of in_main.cpp into a single location. Highlights:

  • It is a HUDElement which means it can use Valve's client/server communication mechanism
  • I added an new global message type called ARMARMsg which looks like this: 
int   float String

Where int is the global gesture code, float is an optional value, and String is a description code ("GESTURE", "ERROR", "STATE", etc) Only GESTURE is used now.

  • It has a state machine, that is used to guide through a particular demo/procedure. In the future we need to figure our how to model and store this in a database -- lots of work so I am hand coding it now. Probably best to either use an interface, with different instantiations for different procedures...
  • The ClientMaintenanceController is added in vgui_int.cpp

[edit] CConnectorPanel (cl_dll/AR/ConnectorPanel.cpp)

This is the code for the VGUI Panel (Frame actually) that lives to support an AC demo. Its a small frame that currently has a scroll list...will later add other elements for things like resitance, etc.

A HUGE thing to remember is that the parent of a VGUI element makes a big difference when it gets painted..

[edit] Server changes

[edit] CGestureController (dll/AR/gesture_controller.cpp)

This LogicalEntity talks to the ARMARMessage server about gestures. It then shoots them via the HUD Element protocol to the ClientMaintenanceController. Its in the fgd so ensure you add it to any maps where you need AC.

[edit] 10 Apr

Today was VGUI Day. Spent all day and evening working out the details on:

The learning curve was steep, but I needed to climb it.

The key lessons learned was the bindings between the VGUI classes, the Build Mode, and the resource file. Also the stuff on schemes. This should save a lot of time in the future.

[edit] 7-9 Apr

OK..for the past few days I've been working hard on the AC code. Going to keep this short until it gets done.

Notes:

  • Need to optimize..possible choke points:
    • Need to reuse inverse matrix operations..do them once per frame. Everything needs to be in the AC.updateTransorms() method
    • Compiler optimization -- build a release version and check it out
    • Use better cameras
    • Keep the segmentation window small
    • Stats are your friend....break off loops when it will be worse
  • Smoothing and State Transitions
    • These are a little shaky, particularly on the uptick of the connectorAC.
  • Made some fundamental definitions that govern overlapping regions, what defines a control, etc

Admin:

I also started a doxygen trail in doc..Installed doxygen and also GraphViz

Order some parts and components to build another HeadPlay -- this one with FireFly cameras and zero metal components. This includes another HeadPlay, plastic hardware and building materials (from McMaster-Carr which ROCKS), some 15ft firewire cables with 90deg ends, etc, etc


I stood up a secure wiki here.

[edit] 6 Apr

Today I took a break from the software side and worked on the displays.

I mounted a PGR firefly to a Xenarc 700TSv 800x400 LCD. It works pretty well and will be a nice replacement for the 15 in I'm using now.


I also worked on the new HWD..replacing the USB cameras with Two Point Grey Dragon FLy 2 with Remote Heads. I mounted the camera control units in a sandwich configuration...they are two inches thick and are heavier than I hoped. I think it will weigh in less than the InnnerOptic Videsee (but that puppy is parallax free).

Lessons learned:

  • Use large holes
  • Order more 2x64 and 80 washers

UPDATE: Going to build another display with just FireFly MV and microlens, as this rig is HUGE

[edit] 5 Apr

Tweaked Ali's code...ignored black px. Runs faster. Checked in 04 1320 APR.

I have the tracking locked. Checked in.

Need to work the KJ piece. Trying a short cut now.

[edit] 4 Apr

Spent the entire day and evening working on the segmentation code.

Ported the Java AC code (ImpageProcessor and Component Handler) over to C++...did some testing. Stack depth was a serious issue, and without Java's garbage collection it turned into a nightmare..

Gutted the code and went with an example from Ali Rahimi. (Why not OpenCV or CVBlobs? I don't think they are fast enough...didn't try them fully but decided to avoid the temptation to reach for the 5000lb hammer)

Works pretty well, though I have to do the statistical piece and figure out how to filter out black pixels. They are labeled as well. Here are some notes:

  • Ali's code takes a raw buffer of arbitrary dimensions. I tried to mod his code for sub image parameterization. No luck.
  • Decided to modify my own ComponentHandler class to work with arbitrary dimensions. This means:
    • I removed all OpenCV IIMage types from CH
    • When you instantiate the class, you must set the maximum expected dimensions for memory allocation -- this should be easy to figure out give the AC window size and the expected location envelop
    • Any post image processing (e.g. viewing in a window, dumping to disk) will to to be handled outside the class
    • I do said post processing and it works well
  • The classes crank right along so no worries.


Started to get to the point where I needed to do a lot of viewing of the segmentation. The ARTag example I was using was (a) getting messy and (b) didn't have a GUI So.........

[edit] Building a Generic AC GUI Tool

I decided to embark on a generic C++/Visual Studio GUI tool that I could use to test AC concepts and segmentation. Ideally this tool would:

  • Allow one to play with crop settings, filters
  • Be written in C++ such that portion of the code could be later added to ValveHook
  • Include socket commo in case we need to go with the IPC rout or use with another application
  • Be easy to install and run from the command line


The idea is to use it to test an AC before using in ARMAR..once everything is tested, then we can launch the same AC in ARMAR

I downloaded an installed the GLUI library. Had to get a third party distribution to get it working.. Meshed it with the ARTag code I was using and got it working.

The entire glui lib and my VS2003 capable application (/exmaples/AugmentedControlsTool) is in the svn repos (glui).

Now I need to play with the smaller control window...(not shown above)

--Steve 02:55, 4 April 2008 (EDT)

I also installed Wubi on my office machine and runs pretty well...nice to have as an option.

TODO:

  • Finish GUI Tool
  • Use it to finalize component bins
  • Do some stats -- just the min I need now
  • Work projection piece (use simple trig -- See DirectX book)

[edit] 3 Apr

I ported the Augmented Controls from Java to C++.

Its running very slowly, but working. I'm not cropping the segmentation area at all.

Now I need to optimize it by:

  • Setting the MVM
  • Getting the 2d projection transform of the augmented control (e.g. a circle)
  • Use this projected transform and some function for the AC as the seed for the segmentation

I spent about 12 hours trying to work the cvBlobsLib. But I had some serious mashing of MFC versus non-MFC code, so I decided to re-use the code I did in Java...I don't think performance is going to be an issue considering the optimization that will occur in the AC.

I also received the special right angle fire wire cables to use with my LCD.

Some other random thoughts..

  • Use a second camera. Coordinate view transform between the camera corresponding to the user, and the fixed camera on the AC. Show a vgui_screen with the second camera's image and cue the user to "look in" this window..when they do, boom, they are attached to the fixed camera. What are the UI implications on seeing through someone else's eyes? Is there a metaphor for this?
  • Zoom camera stabilization

[edit] 2 Apr

Today I started to work on the code needed for Augmented Controls using ARTag.

I think its going to be possible to modify the Kjeldsen segmentation approach such that the transformation matrix from camera coordinates to interface coordinates occurs dynamically.

Here's the run down:

  1. Define some interface
  2. Get the MVM from ARTAG
  3. Find the interface coordinates in 3D using the MVM
  4. Follow this approach the generate the 2D camera coorinates of the interface corners. (BTW ez = fx I think)
  5. Crank the A-HAT matrix
  6. Use A-HAT to take camera coordinates into interface coordinates to parse the gesture

Kevin also stopped by and we talked about MonoSlam for markerless tracking.

[edit] 1 Apr

I continue to try and find the transform that I need to change the Model View Matrix from ARTag to Valve.

I've tried what I think should be the rotations -- but it doesn't work out.

It should be a matter of rotating the world coordinate system so that the basis axis align. Then, there will need to be some local rotations.

I have discovered a lot, and covered a lot of ground.

You can follow along here.

I suspect I am not chaining the transforms correctly, or perhaps something more fundamental like not understanding what the Model View Matrix is..

I also discovered that rotating the MVM introduces some errors -- numerical instabilities, very minor. I'd like to get the clean tracking that you get in OpenGL using the ARTag examples.

Have decided to put this on the back burner for now, and go with the code I have that works.

This was all done to support Calibrating Intersense Tracking with ARTag. SO I don't really need it now for the paper..

[edit] March 2008

[edit] 31 Mar

I'm going to run some OpenGL only checks to see if I can't confirm for sure that the MVM I am sending on is correct.

I want to solidify what is happening in this MVM so I can translate it to a Valve Friendly one.

As you can imagine, in OpenGL space, Y is up, negative Z if the look axis. Of course in Valve Source, Z is up and +X is the look axis. So there should be some pre translation I can do before giving it to artag_model.cpp.

What I currently have works, but is a hack and I'm not 100% sure that the transformations are correct even thought they look decent in the application.

In summary the strategy is:

  • Confirm the rotation and translation of the "pure" MVM from ARTag. Identify the axis orientation.
  • Once this is known, translate the model back to the origin in OpenGL/ARTag space, do the requisite rotations,

swap axes ( + Zvalve = + Yartag, etc).


[edit] 28-30 Mar

I ended up trying to examine the MVM Matrix that is coming in from ARTag, and comparing it the the MVM I am sending now to the automated calibration routine.

I noticed a couple of things:

  • I am rotating the MVM in the AxisAngleQuarternion calls. I thought these just added a dynamic offset, but upon

further inspection I notices I was also injesting a 90 degree rotation in one axis and a 180 degree in another.

  • I also noticed that in the ARTag code, in TangibleObject.h, I was trying the use some GLUT code to rotate.

I don't remember this having any effect,but I need to check.


[edit] 27 Mar

2241:

I have it very close...problem getting the rotation transforms on the same sheet.

Line 306 has a AxisAngleQuaternion call..set to 0 now. I set it to 180 and got usefule results when I checked Wb * A-1 against the player pos (a good test BTW)

--Steve 22:42, 27 March 2008 (EDT)


OK..I tired implementing this in code.

Getting some weird results when I pull the inverses...singular matrices and other abnormalities.

Ideas:

  • Valve VMatrix -- Am I using it correctly? Documentation from vmatrix.h: 
// VMatrix always postmultiply vectors as in Ax = b.
// Given a set of basis vectors ((F)orward, (L)eft, (U)p), and a (T)ranslation, 
// a matrix to transform a vector into that space looks like this:
// Fx Lx Ux Tx
// Fy Ly Uy Ty
// Fz Lz Uz Tz
// 0   0  0  1

// Note that concatenating matrices needs to multiply them in reverse order.
// ie: if I want to apply matrix A, B, then C, the equation needs to look like this:
// C * B * A * v
// ie:
// v = A * v;
// v = B * v;
// v = C * v;

[edit] 14-26 Mar

Ok -- first entry after the demo. This will be a catch up entry.


[edit] Demo

Demo: The demo went well, but there is much room for improvement:

  • 3D model normals: some of the exported 3D models (manual gun traverse, others) have some inverted normals
  • Jumping "noclip" start: the demo always started with clipping enabled, which you had to shut off quickly before the player died. **FIXED**
  • Default VGUI: The default task for CGUI is bad. It doesn't start on the first task..clips text, etc. Need to relook the entire MyHUDPanel code. This is not as tightly coupled as I'd like.
  • Label color: Dr. Feiner suggested black with white border
  • Registration: The registration is not as good as I'd like. trying to fix it here.

[edit] APG

I went to APG and did some tracker testing and took some more photos.

Built another doghouse which fits nicely.

TODOs:

  • Secure doghoue with anchors; remount lighting
  • Get the IS1200 working. I had issues with the IS1200. Have since sent it back to InterSense.
  • Use the new textures to redo the models

[edit] Markerless Tracking

It's become more and more apparent that we will require some markerless tracking. I spent some time exploring this option. ImageProcessing#Markerless_Tracking check it out here.

I think the way ahead on this is to use BazAR to get the features. Then use the model based approach outline by Bleser and others.

I made a sandbox in DirectX (See C:\DX9SDK\examples\luna) and started playing with correspondence point/3D model correlation.

[edit] 13 Mar

Day 4 in the lab

To do:

  • Investigate slow fps in the USB camera. Dr. Feiner said way to slow. I agree
  • Keep it simple..
  • Replace vectored box with spot light
  • Get a nonHMD (or use another HMD) version going WITH PGR cameras and WITH cropped subimages and see how the fps stack up

--Steve 18:21, 13 March 2008 (EDT)

[edit] 12 Mar

Day 3..

  • Created an armar_maintenance_controller class on the server side. This will track state variables, and turn things on and off. There is a FindEntityByName() method that will be used a lot to find things in the scene
  • Implemented the armar_tracker_head_pitch, ..roll, ..yaw methods -- make a difference
  • Went back to Maya and moved the CDA and the elevation mount to reflect the setup in the lab. This will be an obvious change in a real LAV
  • Trying to get better ARTag tracking going. Its hard on the CDA as the markers are small. Doing some semantic zooming..showing a course grain icon and pointer at long range, then a box at close range.

--Steve 18:21, 13 March 2008 (EDT)

[edit] 11 Mar

This is the end of Day 2 of my deployment to the lab to prep for the demos next week.

Here's a random stream..

[edit] Intersense Tracking

I finally got serious about getting the Intersense tracking going. This involved many tasks:

Hooking up the tracker
I continue to use the client-server mixed approach that Ohan and Mike Sorvillo did. I decided to start by using the IS900. I zeroed out the coordinate space such that 0,0,0 was the floor of the turret centered on the fwd left upright pipe.
Scaling and registering the model
I next registered the turret model i have going in Maya..I learned that obj exports from GeoMagic come out in decimeters, so I had to scale everything I did before by 0.1...I next registered it to a particular configuration I have set up by my desk..
Tracker to player translation
Because the tracker is sitting fwd and down from the player's head, I had to rework some transformations..and got it! This was a big win, as its something I've been struggling with for sometime. I posted the guts of the process here. I need to finish the orientation offsets, as its very likely the tracker won't be coplanar with the user's head (glasses are angled down a bit anyways..).
Testing
I did some preliminary testing..the registration is better than its ever been -- but not good enough. There appears to be some lag. The frame rate has dropped way down..not sure why. Ohan had added some tick threshold in in_main.cpp so maybe its there.
Elevation Crank Modeling
I next revisited the LAV25 elevation pump model...I scaled, retextured it, then trimmed it down so its just the box and handle (now known as elevationbox.mdl). Did some preliminary registration as both a prop-static_overload and an artag_model..Put it in the mod. Textures are way bright..need to edit the .vmt.

Here are the commands I now use to start the map: 

map demo
noclip
sv_gravity 0
fov 30
armar-intersense_enable 1
armar_tracker_head_z -2
armar_tracker_head_x -3
armar_tracker_head_y 2.5

Also, here are the no kidding IS900 coordinates of the two uprights I'm using for registration:

  • fwd-left : (-0.45, 0.03, 38.6)
  • fwd-right: (29.3, 36.5, 38.55)

TODO:

  • Investigate low fps -- SOLVED: Run it in full screen mode --Steve 15:07, 12 March 2008 (EDT)
  • Better material lighting for elevationbox.mdl -- DONE: Added some textured scratches. Need to revisit --Steve 15:07, 12 March 2008 (EDT)
  • Orientation offsets in inmain.cpp
  • ARTag-ize eleavtionbox.mdl -- right now its transposed I think. --- Done --Steve 15:07, 12 March 2008 (EDT)

[edit] The Friendly Source SDK Community

There is a very strong and friendly social network of coders out there...Not your typical fraternal-knowledge is power-crowd..The more I learn about these fellow comrades and trailblazers the more I am impressed. Very knowledgeable and helpful to boot.

Today, I made contact with some other folks out there in the Steam community who are doing similar work. This came after participating in the Source SDK Coder's Mailing list. Adam, aka carbon_adam, is doing some awesome serious game/artistic work. Check it out here.

This prompted me to look into leveraging and participating more in the Steam online community... As a result, I am now officially a part of it. This is going to be a good way to get help and share ideas!

--Steve 04:07, 12 March 2008 (EDT)

[edit] 10 Mar

Ok...today I got a lot done.

I added some more MOD side client code to tweak the sub images for the video see through display. Check it out here.

Confirmed something I learned a few days ago..tweaking the left sub image negates the shared focal point with ARTag..read more here.

Re-enabled and studied Cheng-Chih's HUD class. Need to add a SetVisible() con command in the client side MyHudPanel.cpp code

Updated subversion..getting ready to deploy to the lab

Road ahead:

  • Build a scene for the demo..and a work flow
  • Feed these into the HUD class
  • Figure out what to do with focal point issue...either translate MVM or use symmetric cropping.
  • Use the ARTag model to revist Intersense registration
  • Calibrate the ARTag intrinsic parameters

--Steve 04:26, 10 March 2008 (EDT)

[edit] 7-9 Mar

Reworked the ARMARMessage Server to use overlapped pipes

Added a DisplayController to adjust resizing of subimages from within Valve Source MOD. Had originally used a seperate machine running a Java app but it works better when you are single pilot from within the MOD.

[edit] The Friendly Source SDK Community

There is a very strong and friendly social network of coders out there...Not your typical fraternal-knowledge is power-crowd..The more I learn about these fellow comrades and trailblazers the more I am impressed. Very knowledgeable and helpful to boot.

Today, I made contact with some other folks out there in the Steam community who are doing similar work. This came after participating in the Source SDK Coder's Mailing list. carbon_adam is doing some awesome serious game/artistic work. Check it out here.

This prompted me to look into leveraging and participating more in the Steam online community... As a result, I am now officially a part of it. This is going to be a good way to get help and share ideas!

[edit] 10 Mar

Ok...today I got a lot done.

I added some more MOD side client code to tweak the sub images for the video see through display. Check it out here.

Confirmed something I learned a few days ago..tweaking the left sub image negates the shared focal point with ARTag..read more here.

Re-enabled and studied Cheng-Chih's HUD class. Need to add a SetVisible() con command in the client side MyHudPanel.cpp code

Updated subversion..getting ready to deploy to the lab

Road ahead:

  • Build a scene for the demo..and a work flow
  • Feed these into the HUD class
  • Figure out what to do with focal point issue...either translate MVM or use symmetric cropping.
  • Use the ARTag model to revist Intersense registration
  • Calibrate the ARTag intrinsic parameters

--Steve 04:26, 10 March 2008 (EDT)

[edit] 7-9 Mar

Reworked the ARMARMessage Server to use overlapped pipes

Added a DisplayController to adjust resizing of subimages from within Valve Source MOD. Had originally used a seperate machine running a Java app but it works better when you are single pilot from within the MOD.

Retrieved from "http://www.digitalblacksmith.com/wiki/index.php/ARMARJournal"
Personal tools