ARTagInARMAR
From DigitalBlacksmith
Contents |
[edit] Valve Source ARTag Model
In order to bring ARTag objects into Valve, I decided to author a specific class called the artag_model.
The artag_model class links an arbitrary model file to and artag coordinate frame. The artag_model can be specified in Hammer, which allows the modeler to link a model file with a particular ARTag coordinate system.
At runtime, the artag_model implementing class (which is a Source SDK entity) communicates with an external server that send position information for ARTag objects. If the artag_model receives position information for its coordinate frame, it will move its model to the appropriate position and orientation specified by a model view matrix that is passed by the server.
I did a lot of work to get the transformation from ARTag to ARMAR..even made some YouTube videos:
Verifying ARTag Transformations
[edit] Using ARTag with any camera source
Getting ARTag running with DirectShow
[edit] USB Webcam
This part of the thread is dedicated to integrating video from 2xLifeCam 6000 N cameras into ARMAR.
[edit] DSLV - DirectShow Video Processing Library
This is a nice wrapper for direct show that abstracts away much of the heavy coding.
Comes with a few examples.
Has the ability to pass XML strings to the SDK to configure the camera:
e.g.
<?xml version="1.0" encoding="UTF-8"?> <!--Sample XML file generated by XMLSpy v2005 rel. 3 U (http://www.altova.com)--> <dsvl_input xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\Thomas\My Documents\projects\ARToolKit & DSVideoLib\ARVideoLib\DsVideoLib\DsVideoLib.xsd"> <camera show_format_dialog="false" frame_width="800" frame_height="600"> <pixel_format> <RGB32/> </pixel_format> </camera> </dsvl_input>
Notice, here I set the camera resolution using the frame_width and frame_height attributes in the camera tag.
I have an example working with a single camera..now moving on to multiple cameras.. --Admin 23:31, 19 January 2008 (EST)
[edit] Multiple Cameras and DSVL
Need to figure out whatthe directX device names are for the cameras.
This link says we need to run the DirectX SDK graphedit.exe:
C:\DX90SDK\Utilities\graphedt.exe
C:\Program Files\Microsoft Platform SDK for Windows Server 2003 R2\Bin
When you run this, go to Graph menu, Insert filters, then explore in the Video Capture Filters..
The DSVL XML Schema says we can specify a camera name like this:
@device:*:{860BB310-5D01-11d0-BD3B-00A0C911CE86}
My first camera I think is this:
@device:*:{65E8773d-8F56-11D0-A3B9-00A0C9223196}
Note: The code is HEX so zeros are zeros, digits need to be [0-9A-F]..not sure if caps will be an issue..
Going to cram this into a test xml and use the DSVL precompiled binaries (C:\webcam\dslv\bin\glutSample.exe) to test...
<?xml version="1.0" encoding="UTF-8"?>
<!--Sample XML file generated by XMLSpy v2005 rel. 3 U (http://www.altova.com)-->
<dsvl_input xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\Thomas\My Documents\projects\ARToolKit & DSVideoLib\ARVideoLib\DsVideoLib\DsVideoLib.xsd">
<camera show_format_dialog="false" frame_width="800" frame_height="600" device_name="@device:*:{860BB310-5D01-11d0-BD3B-00A0C911CE86}">
<pixel_format>
<RGB32/>
</pixel_format>
</camera>
</dsvl_input>
OK - that worked...note placement of the device_name as an attribute in the camera node..
Now try camera #2...--Admin 00:01, 20 January 2008 (EST)
Here's the device handle for the second DX-6000N on my USB bus..
@device:*:{65E8773d-8f56-11D0-A3B9-00A0C9223196}
Wait..this is the same as the other...so we need to expand left of the * in the handle to pick up the unique handle..ugh
I'm thinking WDM strings - so caleld friendly strings - maybe easier..going to try that route using the Java Media Freamework Studio detect capture device tool to interrogate the WDM strings for the camera..
I get this:
vfw:Microsoft WDM Image Capture (Win32):0
The zero on the end is the device count in series so hopefully this will work...here goes..
That didn't work.
Found this post.
GOT IT
I had to hack on the dsvl source and examine what the dsvl_helpers.cpp file was getting from the USB query versus what my requested device was. This string worked:
<?xml version="1.0" encoding="UTF-8"?>
<!--Sample XML file generated by XMLSpy v2005 rel. 3 U (http://www.altova.com)-->
<dsvl_input xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\Thomas\My Documents\projects\ARToolKit & DSVideoLib\ARVideoLib\DsVideoLib\DsVideoLib.xsd">
<camera show_format_dialog="false" device_name="@device:pnp:\\?\usb#vid_045e&pid_00f8&mi_00#6&95ef5cc&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global"> <pixel_format>
<RGB32/>
</pixel_format>
</camera>
</dsvl_input>
Note, the device name is inside the camera tag..it didn't work the other way. Also & have to be html friendly as mentioned by others..
Here's my other camera xml:
<?xml version="1.0" encoding="UTF-8"?>
<!--Sample XML file generated by XMLSpy v2005 rel. 3 U (http://www.altova.com)-->
<dsvl_input xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\Thomas\My Documents\projects\ARToolKit & DSVideoLib\ARVideoLib\DsVideoLib\DsVideoLib.xsd">
<camera show_format_dialog="false" device_name="@device:pnp:\\?\usb#vid_045e&pid_00f8&mi_00#6&1c18a671&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global">
<pixel_format>
<RGB32/>
</pixel_format>
</camera>
</dsvl_input>
RIGHT EYE CAMERA (flips H and V because its mounted upside down:
<?xml version="1.0" encoding="UTF-8"?>
<!--Sample XML file generated by XMLSpy v2005 rel. 3 U (http://www.altova.com)-->
<dsvl_input xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\Thomas\My Documents\projects\ARToolKit & DSVideoLib\ARVideoLib\DsVideoLib\DsVideoLib.xsd">
<camera show_format_dialog="true" device_name="@device:pnp:\\?\usb#vid_045e&pid_00f8&mi_00#6&95ef5cc&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global"> <pixel_format>
<RGB32 flip_h="true" flip_v="true" />
</pixel_format>
</camera>
</dsvl_input>
LEFT EYE CAMERA:
<?xml version="1.0" encoding="UTF-8"?>
<!--Sample XML file generated by XMLSpy v2005 rel. 3 U (http://www.altova.com)-->
<dsvl_input xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\Thomas\My Documents\projects\ARToolKit & DSVideoLib\ARVideoLib\DsVideoLib\DsVideoLib.xsd">
<camera show_format_dialog="false" device_name="@device:pnp:\\?\usb#vid_045e&pid_00f8&mi_00#6&1c18a671&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global">
<pixel_format>
<RGB32/>
</pixel_format>
</camera>
</dsvl_input>
USB Bandwidth Issues
When I try to run two copies of glutSample.exe, one of the channels starts pixelating and becomes unusable...more trouble in paradise. Probably need to use separate USB bus for each..ug
--Admin 02:43, 20 January 2008 (EST)
I did this..I picked up a PCI USB Controller (PCI Card with USB). This added another enhanced controller and now noth run great.
--Admin 20:51, 21 January 2008 (EST)
[edit] DSVL XML
Here are some links on the DSVL XML:
http://www.hitlabnz.org/forum/showthread.php?t=518
Direct Show Video Library XSD (with my comments)
[edit] Getting the USB Code working in Valve
Updated handles for the cameras in EVA
@device:pnp:\\?\usb#vid_045e&pid_00f8&mi_00#6&1d5ad4f4&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global
and
@device:pnp:\\?\usb#vid_045e&pid_00f8&mi_00#7&1f05d4d6&0&0000#{65e8773d-8f56-11d0-a3b9-00a0c9223196}\global
--Admin 17:34, 8 February 2008 (EST)
I wrote a USBCamera class instantiating the Valve Hook VideoCapture interface. I have the USB ARTag example running on my home machine. However, the machine in the lab is not working. The hook causes the machine to lockup and reboot. So I am going to note the differences in configurations. The goal is to create ONE SINGLE config that I can use at home or in the lab.
Here is the configuration of ANNA. Need to compare it to RUBENS.
- DSLV.dll - C:\WINDOWS\system32
- run_mod.bat:
THIS IS A HUGE DIFFERENCE:
"c:\steam\steam.exe" -applaunch 215 -dev -dxlevel 90 -width 800 –height 600 -game "c:\steam\steamapps\SourceMods\Armar" -allowdebug %1 %2 %3 %4 %5 %6 %7 %8 %9
Output from ARTAg in ttnlog.txt:
version: <ARTag Rev2k trial/demo version: Windows OpenGL, expires June 30/2008, Mark Fiala 2006>
DSLV header file, line 38, C:\ARTag\include\camera_dsvl.v:
sprintf(xmlstring,"<?xml version=\"1.0\" encoding=\"UTF-8\"?><dsvl_input xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"> <camera show_format_dialog=\"false\" frame_width=\"%d\" frame_height=\"%d\"><pixel_format><RGB24/> </pixel_format> </camera></dsvl_input>",desired_width,desired_height);
Other possibly related issues:
- DirectX SDKs (ANNA Summer 2003, Rubens DEC 2006)!!
- Half2 MP vs. HL2
Do we need to go with Orange Box?
Note:
- Both machines are running DirectX version 9.0c (4.09.0000.0904)
[edit] USBCameraCode in Irrlicht
Click here.
[edit] AMCap
The AMCap project that comes bundled with the DirectX SDK is a pretty good place to get started. However, this is a HUGE project, and a lot more than what I need. Going to look around for some other stuff..
[edit] Links
A single frame grab from a stream - NON-MFC
ESCAPI - Extremely Simple Camera API (Wrapper for Direct Show)
- This looks promising but doesn't come with source code...Author will send it on request
[edit] Sub Image Journal
OK those settings look pretty good, and maintain the MVM matrix from ARTag. But now there is a lot of divergence in the virtual content.
--Steve 10:49, 10 March 2008 (EDT)
I tried this for the left image:
(137,107, 137, 107)
This looked a little better:
(132,107,132,107)
This equates to a 376 width..376*0.75=282...(480-282)/2= 99
So try:
(132,99,132,99)
Going to prime the right image with this as well..
--Steve 10:42, 10 March 2008 (EDT)
Based on the latest good measurement, It looks like the preferred subimage dimensions are around 365x265. Thats close to a 4:3 aspect ratio
So I'm going to force feed a 366x276 symmetric resolution on the left image and see what happens.
--Steve 09:58, 10 March 2008 (EDT)
New issue:
I have some pretty good control going for the sub cropping of images sent to the stereo video texture in the backbuffer of Valve Source. We now have independent control of the placement and size of the left and right sub images, as well as the ability to "toe in" and "toe out" to change the vergence. (If you remeber the UNC work, a sub image is extracted to correspond to the appx 28deg fov of the display, and this sub image is then resized to the resolution of the display).
However, there is a new translation issue that needs addressing.
Because the ARTag thread uses the *entire* left image (in the interest of maximum resolution), it isn't cropped and resized like the one that is sent to the left eye.
So the nice video see through characteristic of placing virtual content on top of the same pixels where the marker is detected doesn't directly apply. Tthe model video matrix coming from ARTag will need translation to account for the actual video that is viewed by the user. In short, the camera notion in ARTag space doesn't match the camera notion in user space.
Keeping the center of the subimages anchored to the camera's center (in effect cropping symeterically from the sides) would be one solution, but then we can't dynamically change the vergence point. A second (very bad option) would be to let ARTag chew on the subimage, but that sacrfice perfromance and require dynamic alteration of ARTag's camera (which isn't possible without restarting ARTag).
I will keep playing around with it and thinking and we can keep tuning. I think it will be as simple as adding a global x and y delta that gets added to any ARTag MVM coming into valve. Of course, this will have to be dynamic to allow us the keep everything in synch if we do the "UNC vergence trick"
In the short run, to prep the demo, I will prob postpone dynamic vergence, and just position the left sub image to share the same focal point as the ARTag image. Then adjust the right image..something like that.
--Admin 03:21, 10 March 2008 (EDT)
These are some pretty nice settings (image only):
Left: (159, 153, 116, 52)
Right: (149, 171, 136, 43)
--Admin 03:21, 10 March 2008 (EDT)
After modifying the USB mount on the headplay display these are my sub crop dimensions. All units are in pixels from the closest margin.
--Admin 00:43, 6 March 2008 (EST)
- Right camera
- 114 ULX
- 135 ULY
- 155 LRX
- 65 LRY
--Admin 17:46, 7 March 2008 (EST)
I added some capability into the MOD to send image adjustments to the Hook Display Controller.
The display controller is used to manipulate either the LEFT or the RIGHT image. It defaults to the left image.
To swap control between the images, use the "]" key.
To control image cropping:
- Upper Left X decrease: 9
- Upper Left X increase: 0
- Upper Left Y decrease: o
- Upper Left Y increase: l
- Lower Right X increase: -
- Lower Right X decrease: +
- Lower Right Y increase: [
- Lower Right Y decrease: '
[edit] Added Keybindings
Powerpoint of the key map (also in SVN in hook land)
(inside <content>/Config/config.cfg)
bind "9" "+dc_ux_dec" bind "0" "+dc_ux_inc" bind "-" "+dc_lx_inc" bind "=" "+dc_lx_dec" bind "o" "+dc_uy_dec" bind "l" "+dc_uy_inc" bind "[" "+dc_ly_inc" bind "'" "+dc_ly_dec" bind "]" "+dc_swap_image" bind "y" "+dc_up" bind "h" "+dc_down" bind "j" "+dc_left" bind "k" "+dc_right" bind "," "+dc_toein" bind "." "+dc_toeout"
[edit] Added Classes
- Client Side:
- Added classes DisplayController.cpp and DisplayController.h
- Static invocation added to in_main.cpp. There is a compiler directive to disable
--Admin 17:48, 7 March 2008 (EST)
Using this new tool, I have tuned the cropping of the USB cameras as:
#define DEFAULT_LEFT_ULX 140 #define DEFAULT_LEFT_ULY 136 #define DEFAULT_LEFT_LRX 135 #define DEFAULT_LEFT_LRY 69 #define DEFAULT_RIGHT_ULX 149 #define DEFAULT_RIGHT_ULY 171 #define DEFAULT_RIGHT_LRX 136 #define DEFAULT_RIGHT_LRY 43
[edit] Toshiba Cameras in InnerOptic Display
This part of the thread describes integrating the Toshiba cameras that are mounted in the InnerOptic Display.
[edit] Demo Params
Until I crack the code on the camera calibration, I'm using the following adjusment:
fov 28 set_artag_pos_offset 15 -5 -4
I need to look into the camera FY and CY values..
I suspect the issue lies in there.
--Steve 01:36, 11 September 2007 (EDT)
[edit] Milestone - ARTag Integration and Camera Calibration
--Steve 01:06, 11 September 2007 (EDT)
I cracked the code on the coordinate tranlsation.
Am now working on sizing the model.
Also I did some camera calibration for the InnerOptic cameras:
<results>
<ImageCount>33</ImageCount>
<focus_lenX>746.62619824105536</focus_lenX>
<focus_lenY>747.51968727388157</focus_lenY>
<PrincipalX>302.61227445979057</PrincipalX>
<PrincipalY>205.58944515364613</PrincipalY>
<Dist1>-0.17369199823116283</Dist1>
<Dist2>0.099304878998261847</Dist2>
<Dist3>-0.00086214293205871058</Dist3>
<Dist4>0.0036681983879416442</Dist4>
<focus_lenX_er>2.4785711171719775</focus_lenX_er>
<focus_lenY_er>2.1104484288647765</focus_lenY_er>
<PrincipalX_er>2.1228189458549442</PrincipalX_er>
<PrincipalY_er>2.9233624254290436</PrincipalY_er>
<Dist1_er>0.0066815151344468164</Dist1_er>
<Dist2_er>0.042778842052153217</Dist2_er>
<Dist3_er>0.00068329385181974372</Dist3_er>
<Dist4_er>0.00056811043703621628</Dist4_er>
<dc_AllImage_errX>0.18724894930596306</dc_AllImage_errX>
<dc_AllImage_errY>0.2169954154272315</dc_AllImage_errY>
<Calib_Date>9/11/2007 12:10:44 AM</Calib_Date>
<Calib_Type>1</Calib_Type>
</results>
This entailed writing an image capture routine for the Matrox. (located in ARMAR root)
However, I'm getting weird results, prob because we are using a deinterlaced video in ARTag. I tried to calibrate the deinterlaced video using the GTK camera capture utility I got terrible results from the calib tool - unusable. I suspect it can't handle the non standard aspect ratio (?).
So...I halved the horizontal focul length, then brought it down further to 300. Will continue to play with it. May be worth the time to use the MatLab toolbox.. tool
[edit] Transformation Issues
Am now having some trouble translating the ARTag ModelView Matrix into Valve.
Read More Here.
[edit] Status Report - Deinterlaced Working
--Steve 14:26, 6 September 2007 (EDT) Cool beans!! I wrote my own deinterlace algorithm, so can now avoid using 3 digitizers.
The ARTag thread updates the ARTag by asking the MatroxCapture class for the ARTag image. The MatrocCapture class returns gets a copy of the full image from the left matrox digitizer. It then executes the deinterlace and sends the result to the ARTagThread and eventually it gets to ARTag.
Here's the nuts and bolt's of the deinterlace routine:
//Global buffer in the class constructor
int totalSize = 3*CAMERA_IMAGE_WIDTH*CAMERA_IMAGE_HEIGHT;
buffer = (unsigned char*) malloc(sizeof(unsigned char)*totalSize);
/******************************************************************************
*
* Utility to get a pixel from the supplied x,y coordinate in the
* supplied unsigned char block.
*
*******************************************************************************/
void MatroxCapture::paintPixel(unsigned char* block, int pitch, int x, int y, int r, int g, int b) {
block[x*3+2+y*pitch*3]=r;
block[x*3+1+y*pitch*3]=g;
block[x*3+0+y*pitch*3]=b;
}
/**************************************************************
* Return a deinterlaced Image
***************************************************************/
unsigned char* MatroxCapture::deinterlacedImage(unsigned char* source, int pitch, int width, int height) {
int totalSize = pitch*height;
int offsety =0;
for(int y=0; y < height; y=y+2) {
for(int x=0; x < width; x=x+1) {
if(3==3) {
int r,g,b;
int BytesPerPixel = 3;
unsigned char* pPixelAddress = ( (unsigned char*)source ) + x * BytesPerPixel + y * pitch;
paintPixel(buffer,640,x,offsety,pPixelAddress[2],pPixelAddress[1],pPixelAddress[0]);
}
}
offsety++;
}
return buffer;
}
/**************************************************************
*
* Gets the AR Tag image.
***************************************************************/
unsigned char* MatroxCapture::getARTagImage()
{
return(deinterlacedImage(getLeftImage(),getLeftPitch(),640,480));
//return SrcImageDataPtr;
//return getLeftImage();
}
[edit] Multiple Digitizer Test REDUX
OK..when I ported the hello world example that worked to the hook I have issues... matrox complains about conflict with digitizer....also slow fps..
Going to try and write my own deinterlace algo to avoid the third digitizer.
--Steve 21:45, 4 September 2007 (EDT)
[edit] All About Interlaced Video
[edit] Multiple Digitizer Test
OK..tried adding a third digitizer for the ARTag..works! KEY: You cannot have more than 2 digitizers in ContinuousGrab Therefore, have to use grab each time you want an image.
Code:
avail c:\bak\rubens_bu_26JUL\artag\artag_rev2k_sdk_windows_1207\Sample_projects\visual_studio_6\matrox_opengl
//© 2006, National Research Council Canada
//BASIC_ARTAG_OPENGL.CPP - demonstration of ARTag Rev2 OpenGL support- last update Nov 2006
//Oct 4/2005 by Mark Fiala, National Research Council of Canada - IIT/CVG group
// email: mark.fiala@nrc-cnrc.gc.ca
//
//-with this demo app, the modelview matrix of OpenGL is set by an ARTag function
//call. This program demonstrates initializing ARTag, loading an array file,
//associating an individual array from this file (an array file can contain many
//arrays, starting and ending with the "coordframe" and "\coordframe" keywords),
//calling the main artag_find_objects() function, calling artag_is_object_found() to
//see if an object is loaded, and finally the artag_set_object_opengl_matrix()
//function to set the OpenGL modelview matrix to allow rendering relative to the array.
//-OpenGL lines, untextured triangles, untextured quads and textured quads are used
//ismar_logo.ppm image is used to texture a quad).
//
//- link in artag_rev2_vs6.lib
//- I created a Win32 Console App in Microsoft Visual Studio 6 for this demo
//
//-this demonstrates use of either OpenCV's "cvcam" for camera capture for use with
//USB,USB2 webcams, or "pgrflycapture" for use with Point Grey's Dragonfly firewire
//camera (www.ptgrey.com). For each, uncomment the appropriate include line and
//link in the correct library. Refer to license agreements for both Intel's OpenCV
//and Point Grey's Pgrflycapture software, and use one of these two camera inputs
//if you agree with their license terms.
//
//-USING OpenCV's "cvcam"
//-#include my wrapper "camera_dragonfly_640_480_windows.c"
//-link in artag_rev2_vs6.lib (from \lib directory) and cv.lib & highgui.lib (from OpenCV 4
//from Intel or from the copy I included in \opencv directory)
//
//-USING Thomas Pintaric's DSVL (DirectShow Video Processing Library) -for faster camera input than cvcam
//-#include wrapper "camera_dsvl.c" -follow directions in file to download it from Sourceforge
//-link in DSVL.lib (from DSVL Sourceforge download)
//
//-USING DRAGONFLY
//-#include my wrapper "camera_cvcam_windows.c"
//-link in artag_rev2_vs6.lib (from lib directory) & pgrflycapture.lib (from www.ptgrey.com)
//
//-to link in files in Visual Studio, do project tab ->Settings->Link,
//add them to "Object/Library modules" list
#include <windows.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <mil.h>
#define M_PI 3.1415927
#define PI 3.14159265358979
#include "glut.h"
//#include "camera_cvcam_windows.c" //link cv.lib cvcam.lib
//#include "camera_dsvl.c" //(faster than cvcam) link DSVL.lib
//#include "camera_dragonfly_1_4r27_windows.c"
//#include "camera_dragonfly_1_5_3_0015_windows.c"
//#include "camera_image_single_frame.c"
#include "TangibleObject.h"
#include "NamedPipeServer.h"
#include "artag_rev2.h"
#define SCREEN_WIDTH 800
#define SCREEN_HEIGHT 600
#define DEFAULT_FX 677 //focal length approx 850-1150 for 640x480 Dragonflies
#define DEFAULT_FY 704 //about 400 for 320x240 webcam
#define CAM_NUM 0 //which camera input
#define ARRAY_FILENAME "panel_set.cf"
unsigned char *cam_image,*cam_tex_img; //cam_image is cam_width*cam_height from camera, cam_tex_img is 1024*1024
char rgb_greybar; //1=RGB, 0=greyscale
int cam_width, cam_height;
unsigned char *screen_image;
char mirror_on=0; //set to 1 to flip image horizontally for "magic mirror" effect
int artag_object_id; //artag object # for array
//camera params ("K" matrix) - perfect (no radial ,etc distortion) pinhole model
float camera_cx,camera_cy; //camera params
//variables for OpenGL for camera image
GLuint camera_texID; //handle to camera image texture
double camera_opengl_dRight,camera_opengl_dLeft,camera_opengl_dTop,camera_opengl_dBottom;
GLuint pgm_texID;
unsigned char *texpgm;
#define FIDUCIAL_COUNT 1
#define TEST_OBJECT 0
GLfloat fiducialArray[FIDUCIAL_COUNT][16];
double fiducialMatrix[FIDUCIAL_COUNT][3][4];
double fiducialVisible[FIDUCIAL_COUNT];
unsigned char* demo_read_ppm(char *file_name, int *width, int *height);
NamedPipeServer *namedPipeServer;
////////////////
//MMMMMMMMMMMM
///////////////
//BEGIN MATROX DEFINITIONS//
/* MIL function specifications. */
//Number of parameters in Slave Method
#define FUNCTION_NB_PARAM 1
#define FUNCTION_OPCODE_ADD_CONSTANT 1
#define FUNCTION_PARAMETER_ERROR_CODE 1
#define FUNCTION_SUPPORTED_IMAGE_DEPTH 8
//END MATROX DEFINITIONS//
/////////////////////////////////////////////////
//PRIMITIVES
////////////////////////////////////////////////
const static int CAMERA_IMAGE_HEIGHT = 480;
const static int CAMERA_IMAGE_WIDTH = 640;
const static int COMPOSITE_IMAGE_HEIGHT = 480;
const static int COMPOSITE_IMAGE_WIDTH = 1280;
const static int COMPOSITE_TARGET_IMAGE_HEIGHT = 480;
const static int COMPOSITE_TARGET_IMAGE_WIDTH = 1280;
/////////////////////////////////////////////////
//ATTRIBUTES
////////////////////////////////////////////////
MIL_ID MilApplication;
MIL_ID MilSystem ;
MIL_ID MilDigitizerARTag ;
MIL_ID MilDigitizerLeft ;
MIL_ID MilDigitizerRight ;
MIL_ID MilCompositeImage;
MIL_ID MilCompositeTargetImage;
MIL_ID MilARTagImage;
MIL_ID MilLeftSubImage;
MIL_ID MilRightSubImage;
long SizeBand, SizeX, SizeY, Type;
TangibleObject *testObject;
////////////////
//MMMMMMMMMMMM
///////////////
void initMatrox(void)
{
/* Setup Matrox Capture Stuff */
MappAlloc(M_DEFAULT,&MilApplication);
MsysAlloc(M_SYSTEM_MORPHIS,M_DEV0,M_DEFAULT,&MilSystem);
//Note M_DEV0 is right based on the InnerOptics configuration
//Note M_DEV1 is left based on the InnerOptics configuration
MdigAlloc(MilSystem, M_DEV0, "M_NTSC", M_DEFAULT , &MilDigitizerRight);
MdigAlloc(MilSystem, M_DEV1, "M_NTSC", M_DEFAULT , &MilDigitizerLeft);
MdigAlloc(MilSystem, M_DEV1, "M_NTSC", M_DEFAULT , &MilDigitizerARTag);
SizeBand = MdigInquire(MilDigitizerLeft, M_SIZE_BAND, M_NULL);
SizeX = MdigInquire(MilDigitizerLeft, M_SIZE_X, M_NULL);
SizeY = MdigInquire(MilDigitizerLeft, M_SIZE_Y, M_NULL);
//Allocate color buffer for composite image
MbufAllocColor(MilSystem,
//SizeBand,
3,
COMPOSITE_IMAGE_WIDTH,
COMPOSITE_IMAGE_HEIGHT,
8 + M_UNSIGNED,
M_IMAGE + M_BGR24 + M_PACKED + M_GRAB, &MilCompositeImage);
//M_IMAGE + M_BGR32 + M_PACKED + M_GRAB, &MilCompositeImage);
//Allocate color buffer for composite image
MbufAllocColor(MilSystem,
//SizeBand,
3,
COMPOSITE_TARGET_IMAGE_WIDTH,
COMPOSITE_TARGET_IMAGE_HEIGHT,
8 + M_UNSIGNED,
M_IMAGE + M_BGR24 + M_GRAB, &MilCompositeTargetImage);
//Allocate color buffer for AR Tag image
MbufAllocColor(MilSystem,
//SizeBand,
3,
CAMERA_IMAGE_WIDTH,
CAMERA_IMAGE_HEIGHT,
8 + M_UNSIGNED,
M_IMAGE + M_BGR24 + M_PACKED + M_GRAB, &MilARTagImage);
//Allocate Left and Right Child Buffers
//Create the left image starting at 0,0 in composite (upper left) and spanning one camera image wide
MbufChildColor2d(MilCompositeImage, M_ALL_BANDS, 0, 0, CAMERA_IMAGE_WIDTH, CAMERA_IMAGE_HEIGHT, &MilRightSubImage);
//Create the right image starting at CAMERA_IMAGE_WIDTH,0 in composite (upper midpoint) and spanning one camera image wide
MbufChildColor2d(MilCompositeImage, M_ALL_BANDS, CAMERA_IMAGE_WIDTH, 0, CAMERA_IMAGE_WIDTH, CAMERA_IMAGE_HEIGHT, &MilLeftSubImage);
//Create a B&W copy of the left image starting at 0,0 in composite (upper left) and spanning one camera image wide
//MbufChild2d(MilCompositeImage, 0, 0, CAMERA_IMAGE_WIDTH, CAMERA_IMAGE_HEIGHT, &MilBWSubImage);
//Clear the composite image
MbufClear(MilCompositeImage, 0L);
//Set up the digitizers (frame grabbers)
//Note M_CH0 is right based on the InnerOptics configuration
//Note M_CH1 is left based on the InnerOptics configuration
//WONT LET YOU GO > 1.0
MdigControl(MilDigitizerLeft,M_GRAB_SCALE_X , 1.0);
MdigControl(MilDigitizerRight,M_GRAB_SCALE_Y , 1.0);
MdigControl(MilDigitizerARTag,M_GRAB_FIELD_NUM , 1);
//MdigControl(MilDigitizerLeft,M_GRAB_FIELD_NUM , 1);
//MdigControl(MilDigitizerRight,M_GRAB_FIELD_NUM, 1);
MdigControl(MilDigitizerLeft,M_CAMERA_LOCK, M_ENABLE + M_FAST);
MdigControl(MilDigitizerARTag,M_CAMERA_LOCK, M_ENABLE + M_FAST);
MdigControl(MilDigitizerRight,M_CAMERA_LOCK, M_ENABLE + M_FAST);
MdigChannel(MilDigitizerRight, M_CH0);
MdigChannel(MilDigitizerLeft, M_CH1);
MdigChannel(MilDigitizerARTag, M_CH1);
/* Grab continuously on the display. */
//==>
//MdigGrabContinuous(MilDigitizerLeft, MilARTagImage);
//MdigGrabContinuous(MilDigitizerLeft, MilLeftSubImage);
//MdigGrabContinuous(MilDigitizerRight, MilRightSubImage);
/* Display the image. */
//MdispSelect(MilDisplay, MilImage);
}
/**************************************************************
*
* Gets the Left images.
***************************************************************/
unsigned char* getLeftImage()
{
unsigned char *SrcImageDataPtr;
long SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
MdigGrab(MilDigitizerLeft, MilLeftSubImage);
/* Read image information. */
MbufInquire(MilLeftSubImage, M_HOST_ADDRESS, &SrcImageDataPtr);
MbufInquire(MilLeftSubImage, M_SIZE_X, &SrcImageSizeX);
MbufInquire(MilLeftSubImage, M_SIZE_Y, &SrcImageSizeY);
//M_SIZE_BIT = which returns the depth per band, in bits.
MbufInquire(MilLeftSubImage, M_SIZE_BIT, &SrcImageSizeBit);
MbufInquire(MilLeftSubImage, M_PITCH_BYTE, &SrcImagePitchByte);
//printf("got paired Image..size x=%d, size y=%d\n", SrcImageSizeX, SrcImageSizeY);
//MimResize(MilCompositeImage, MilCompositeTargetImage, M_FILL_DESTINATION, M_FILL_DESTINATION, M_DEFAULT);
//MbufInquire(MilCompositeTargetImage, M_HOST_ADDRESS, &SrcImageDataPtr);
return SrcImageDataPtr;
}
/**************************************************************
*
* Gets the AR Tag image.
***************************************************************/
unsigned char* getARTagImage()
{
//Copy from composite
//void MbufCopyClip(MIL_ID SrcBufId, MIL_ID DestBufId, long DestOffX, long DestOffY)
///==>
//MbufCopyClip(MilCompositeImage, MilARTagImage, 0,0);
MdigGrab(MilDigitizerARTag, MilARTagImage);
unsigned char *SrcImageDataPtr;
long SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
/* Read image information. */
MbufInquire(MilARTagImage, M_HOST_ADDRESS, &SrcImageDataPtr);
MbufInquire(MilARTagImage, M_SIZE_X, &SrcImageSizeX);
MbufInquire(MilARTagImage, M_SIZE_Y, &SrcImageSizeY);
//M_SIZE_BIT = which returns the depth per band, in bits.
MbufInquire(MilARTagImage, M_SIZE_BIT, &SrcImageSizeBit);
MbufInquire(MilARTagImage, M_PITCH_BYTE, &SrcImagePitchByte);
printf("got Image..size x=%d, size y=%d\n", SrcImageSizeX, SrcImageSizeY);
//MimResize(MilCompositeImage, MilCompositeTargetImage, M_FILL_DESTINATION, M_FILL_DESTINATION, M_DEFAULT);
//MbufInquire(MilCompositeTargetImage, M_HOST_ADDRESS, &SrcImageDataPtr);
return SrcImageDataPtr;
}
/**************************************************************
*
* Gets the pitch of the left image.
***************************************************************/
int getARTagPitch()
{
long SrcImagePitchByte;
MbufInquire(MilARTagImage, M_PITCH_BYTE, &SrcImagePitchByte);
return (SrcImagePitchByte);
}
/**************************************************************
*
* Gets the pitch of the left image.
***************************************************************/
int getLeftPitch()
{
long SrcImagePitchByte;
MbufInquire(MilLeftSubImage, M_PITCH_BYTE, &SrcImagePitchByte);
return (SrcImagePitchByte);
}
/**************************************************************
*
* Gets a composite image of both Left and Right images.
***************************************************************/
unsigned char* getPairedImage()
{
unsigned char *SrcImageDataPtr;
long SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
/* Read image information. */
MbufInquire(MilCompositeImage, M_HOST_ADDRESS, &SrcImageDataPtr);
MbufInquire(MilCompositeImage, M_SIZE_X, &SrcImageSizeX);
MbufInquire(MilCompositeImage, M_SIZE_Y, &SrcImageSizeY);
//M_SIZE_BIT = which returns the depth per band, in bits.
MbufInquire(MilCompositeImage, M_SIZE_BIT, &SrcImageSizeBit);
MbufInquire(MilCompositeImage, M_PITCH_BYTE, &SrcImagePitchByte);
//printf("got paired Image..size x=%d, size y=%d\n", SrcImageSizeX, SrcImageSizeY);
//MimResize(MilCompositeImage, MilCompositeTargetImage, M_FILL_DESTINATION, M_FILL_DESTINATION, M_DEFAULT);
//MbufInquire(MilCompositeTargetImage, M_HOST_ADDRESS, &SrcImageDataPtr);
return SrcImageDataPtr;
}
/**************************************************************
*
* Gets the pitch of the composite image.
***************************************************************/
int getPitch()
{
long SrcImagePitchByte;
MbufInquire(MilCompositeImage, M_PITCH_BYTE, &SrcImagePitchByte);
return (SrcImagePitchByte);
}
/**************************************************************
*
* Gets the Right images.
***************************************************************/
unsigned char* getRightImage()
{
unsigned char *SrcImageDataPtr;
long SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
/* Read image information. */
MbufInquire(MilRightSubImage, M_HOST_ADDRESS, &SrcImageDataPtr);
MbufInquire(MilRightSubImage, M_SIZE_X, &SrcImageSizeX);
MbufInquire(MilRightSubImage, M_SIZE_Y, &SrcImageSizeY);
//M_SIZE_BIT = which returns the depth per band, in bits.
MbufInquire(MilRightSubImage, M_SIZE_BIT, &SrcImageSizeBit);
MbufInquire(MilRightSubImage, M_PITCH_BYTE, &SrcImagePitchByte);
//printf("got paired Image..size x=%d, size y=%d\n", SrcImageSizeX, SrcImageSizeY);
//MimResize(MilCompositeImage, MilCompositeTargetImage, M_FILL_DESTINATION, M_FILL_DESTINATION, M_DEFAULT);
//MbufInquire(MilCompositeTargetImage, M_HOST_ADDRESS, &SrcImageDataPtr);
return SrcImageDataPtr;
}
/**************************************************************
*
* Gets the pitch of the right image.
***************************************************************/
int getRightPitch()
{
long SrcImagePitchByte;
MbufInquire(MilRightSubImage, M_PITCH_BYTE, &SrcImagePitchByte);
return (SrcImagePitchByte);
}
int count=0;
void convertToAR3x4Matrix(float srcMatrix[16], double dstMatrix[3][4]) {
// Convert OpenGL style 16-element array (representing modelview matrix) to 3x4 matrix (representing transformation matrix)
int i,j;
for (i=0; i<3; i++) {
for (j=0; j<4; j++) {
dstMatrix[i][j] = *(srcMatrix+4*j+i);
}
}
}
//////////
void opengl_tick(void)
{
if(5==5) {
int j=0;
int source_offset,dest_offset;
//grab camera image
//camera_grab_bgr_blocking(CAM_NUM,cam_image,cam_width,cam_height);
//USE MATROX
cam_image = getARTagImage();
//printf("ARTag image pitch = %d\n", getARTagPitch());
//printf("composite pitch = %d\n", getPitch());
//printf("cam width, height=%d, %d\n", cam_width, cam_height);
//main ARTag function - process image for markers and 'coordframe' marker arrays
artag_find_objects(cam_image,1);
testObject->updateState();
if (testObject->IsFound())
{
fiducialVisible[TEST_OBJECT] = 1;
testObject->getPoseArray(fiducialArray[TEST_OBJECT]);
convertToAR3x4Matrix(fiducialArray[TEST_OBJECT], fiducialMatrix[TEST_OBJECT]);
} else {
fiducialVisible[TEST_OBJECT] = 0;
}
//copy camera image into 1024x1024 texture image
source_offset=0; dest_offset=0;//200*3+550*1024*3;//0;
for(j=0;j<cam_height;j++)
{
memcpy(cam_tex_img+dest_offset,cam_image+source_offset,cam_width*3);
source_offset+=cam_width*3; dest_offset+=1024*3;
}
//copy image into OpenGL buffer
glBindTexture(GL_TEXTURE_2D, camera_texID);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 1024, 1024, 0, GL_BGR_EXT, GL_UNSIGNED_BYTE,cam_tex_img);
glutPostRedisplay();
count=0;
}
namedPipeServer->update();
}
void opengl_draw(void)
{
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
// draw a quad for the background video
glLoadIdentity();
glDisable(GL_LIGHTING);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, camera_texID);
glBegin(GL_QUADS);
glColor3f(1.0f, 1.0f, 1.0f);
//draw camera texture, set with offset to aim only at cam_width x cam_height upper left bit
//normal operation (magic mirror off)
if(mirror_on==0)
{
glTexCoord2f(0.0f, (float)cam_height/1024.0);
glVertex3f(camera_opengl_dLeft, camera_opengl_dBottom, -1024);
glTexCoord2f(0.0f, 0.0f);
glVertex3f(camera_opengl_dLeft, camera_opengl_dTop, -1024);
glTexCoord2f((float)cam_width/1024.0, 0.0f);
glVertex3f(camera_opengl_dRight, camera_opengl_dTop, -1024);
glTexCoord2f((float)cam_width/1024.0, (float)cam_height/1024.0);
glVertex3f(camera_opengl_dRight, camera_opengl_dBottom, -1024);
glEnd();
}
//flip background image for "magic mirror" effect
else
{
glTexCoord2f(0.0f, (float)cam_height/1024.0);
glVertex3f(camera_opengl_dRight, camera_opengl_dBottom, -1024);
glTexCoord2f(0.0f, 0.0f);
glVertex3f(camera_opengl_dRight, camera_opengl_dTop, -1024);
glTexCoord2f((float)cam_width/1024.0, 0.0f);
glVertex3f(camera_opengl_dLeft, camera_opengl_dTop, -1024);
glTexCoord2f((float)cam_width/1024.0, (float)cam_height/1024.0);
glVertex3f(camera_opengl_dLeft, camera_opengl_dBottom, -1024);
glEnd();
}
//draw simple augmentations
if(testObject->IsFound())
{
//printf("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Found it!\n");
//artag_set_object_opengl_matrix(artag_object_id,mirror_on);
testObject->loadOpenGLMatrix();
glDisable(GL_TEXTURE_2D);
glBegin(GL_TRIANGLES);
glColor3f(1, 0, 0);
glVertex3f(0,0,0);
glColor3f(0, 1, 0);
glVertex3f(10,0,0);
glColor3f(0, 0, 1);
glVertex3f(10,0,10);
//
glColor3f(1, 0, 0);
glVertex3f(0,0,0);
glColor3f(0, 1, 0);
glVertex3f(10,0,0);
glColor3f(0, 0, 1);
glVertex3f(10,0,10);
glEnd();
glBegin(GL_QUADS);
glColor3f(0, 1, 0);
glVertex3f(-5,25,0);
glVertex3f(-5,35,0);
glVertex3f( 5,35,0);
glVertex3f( 5,25,0);
glEnd();
glBegin(GL_LINES);
glColor3f(1, 0, 0);
glVertex3f(0,0,0);
glColor3f(0, 0, 1);
glVertex3f(00,0,100);
glEnd();
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, pgm_texID);
glBegin(GL_QUADS);
glColor3f(1.0f, 1.0f, 1.0f);
glTexCoord2f(0.0f, 1.0f); glVertex3f(-50,50,0);
glTexCoord2f(0.0f, 0.0f); glVertex3f(-50,150,0);
glTexCoord2f(1.0f, 0.0f); glVertex3f( 50,150,0);
glTexCoord2f(1.0f, 1.0f); glVertex3f( 50,50,0);
glEnd();
} else {
//printf("no marker found..\n");
}
glutSwapBuffers();
}
void opengl_reshape(int w, int h)
{
//Setup a new viewport.
glViewport (0, 0, w, h);
}
//empty keyboard and mouse move,click functions
void opengl_mouse_moved(int x, int y)
{
int w=glutGet(GLUT_WINDOW_WIDTH),h=glutGet(GLUT_WINDOW_HEIGHT);
}
void opengl_mouse_clicked(int button, int state, int x, int y)
{
}
void opengl_key_down(unsigned char k, int x, int y)
{
if(k==27) exit(0); //escape key exits program
else if(k==' ') mirror_on=mirror_on?0:1; //space bar toggles "magic mirror" mode (hor. mirroring)
}
void terminateMe(void)
{
close_artag();
//close_camera(CAM_NUM);
}
int main(int argc, char** argv)
{
int width,height;
char version_string[1024];
//int argc = 0;
//char** argv = (char**)malloc(sizeof(char*) * 10);
//argc = 0;
//argv = (char**)malloc(sizeof(char*) * 10);
// initialize glut
glutInit(&argc, argv);
// initialize glut
//glutInit(&argc, argv);
glutInitDisplayMode(GLUT_DOUBLE | GLUT_DEPTH | GLUT_RGB | GLUT_STENCIL);
// create the window
glutInitWindowSize(SCREEN_WIDTH,SCREEN_HEIGHT);
glutInitWindowPosition(100, 100);
glutCreateWindow("OpenGL with ARTag");
glutSetWindowTitle("OpenGL with ARTag");
glClearColor(0, 0, 0, 0.0);
// setup callbacks
atexit(terminateMe);
glutDisplayFunc(opengl_draw);
glutIdleFunc(opengl_tick);
glutReshapeFunc(opengl_reshape);
glutKeyboardFunc(opengl_key_down);
glutPassiveMotionFunc(opengl_mouse_moved);
glutMouseFunc(opengl_mouse_clicked);
// set up gl parameters
glEnable(GL_DEPTH_TEST);
glBlendFunc(GL_SRC_ALPHA,GL_ONE);
glShadeModel(GL_SMOOTH);
glEnable(GL_TEXTURE_2D);
glCullFace(GL_BACK);
glFrontFace(GL_CW);
//initialize camera
//rgb_greybar=1;
//if(init_camera(CAM_NUM,640,480,rgb_greybar,&cam_width,&cam_height)) {printf("Can't start camera\n");exit(1);}
//printf("Camera started with resolution=%dx%d\n",cam_width,cam_height);
//USE MATROX
initMatrox();
cam_width = 640;
cam_height = 320;
//display version (optional)
artag_get_version(version_string);
printf("version: <%s>\n",version_string);
// ARTag initialization
if(init_artag(cam_width,cam_height,3)) {printf("ERROR: Can't start ARTag\n");exit(1);}
//set camera params
artag_set_camera_params(DEFAULT_FX,DEFAULT_FY,cam_width/2.0,cam_height/2.0);
//load coordframe file
if(load_array_file(ARRAY_FILENAME)) printf("ERROR loading <%s>\n",ARRAY_FILENAME);
else printf("Loaded array file <%s>\n",ARRAY_FILENAME);
//associate object with array named "base0"
artag_object_id=artag_associate_array("toolbar1");
printf("object id=%d", artag_object_id);
int g = artag_associate_array("base0");
printf("object id=%d", g);
//Make a tangible object where we can apply smoothing
testObject = new TangibleObject(artag_associate_array("toolbar1"), mirror_on);
testObject->setFiltering(true);
////////////////////////////NAMED PIPE/////////////////////////////
namedPipeServer = new NamedPipeServer();
//set viewing frustrum to match camera FOV
glMatrixMode (GL_PROJECTION);
glLoadIdentity ();
camera_opengl_dRight = (double)cam_width / (double)(2.0 * DEFAULT_FX);
camera_opengl_dLeft = -camera_opengl_dRight;
camera_opengl_dTop = (double)cam_height / (double)(2.0 * DEFAULT_FY);
camera_opengl_dBottom = -camera_opengl_dTop;
glFrustum(camera_opengl_dLeft, camera_opengl_dRight, camera_opengl_dBottom, camera_opengl_dTop, 1.0, 102500.0);
camera_opengl_dLeft *= 1024;
camera_opengl_dRight *= 1024;
camera_opengl_dBottom *= 1024;
camera_opengl_dTop *= 1024;
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
//rgb image
cam_image=(unsigned char*)malloc(cam_width*cam_height*4+100);
if(cam_image==NULL) {printf("failed malloc\n");exit(1);}
//camera texture
cam_tex_img=(unsigned char*)malloc(1024*1024*4+100);
if(cam_tex_img==NULL) {printf("failed malloc\n");exit(1);}
// make a texture for the video
glGenTextures(1, &camera_texID);
glBindTexture(GL_TEXTURE_2D, camera_texID);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
//load example PPM image for textured quad
glGenTextures(1, &pgm_texID);
glBindTexture(GL_TEXTURE_2D, pgm_texID);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
texpgm=demo_read_ppm("test_texture.ppm",&width,&height);
if (texpgm!=NULL)
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE,texpgm);
else {printf("Couldn't load <test_texture.ppm>\n");exit(1);}
// show the window and start the loop
glutShowWindow();
glutMainLoop();
return 0;
}
//------------------------------------------------------------------------------------------------
//PPM files encoded byte order: red green blue
unsigned char* demo_read_ppm(char *file_name, int *width, int *height)
{
int i,size;
int max_grey;
FILE *in_file;
char c,comment[256];
unsigned char uc,*image;
in_file=fopen(file_name,"rb");
if(in_file==NULL) return NULL;
c=fgetc(in_file); if(c!='P') {printf("%s Not a PPM file\n",file_name);exit(1);}
c=fgetc(in_file); if(c!='6') {printf("%s Not a PPM file\n",file_name);exit(1);}
c=fgetc(in_file); if(c!='\n') {printf("%s Not a PPM file\n",file_name);exit(1);}
while(1)
{
fgets(comment,256,in_file);
if((comment[0]!='#')&&(comment[0]!=0x0a)&&(comment[0]!=0x0d)) break;
}
sscanf(comment,"%d %d",width,height);
fgets(comment,256,in_file);
sscanf(comment,"%d",&max_grey);
size=(*width)*(*height);
image=(unsigned char*)malloc(size*3+10);
if(image==NULL) {printf("READ_PPM() error: Couldn't malloc image\n");exit(1);}
for(i=0;i<size*3;i++)
{
uc=fgetc(in_file);
*(image+i)=uc;
}
fclose(in_file);
return image;
}
[edit] Thread Outcome Status report 18:00, 4 September 2007 (EDT)
OK..The thread is working, and yielding a 21 fps speed.
I wrote a Thread wrapper for ARTag object in the hook project (rather than rewrite the ARTag class).
* Issues:
o The ARTag object is being sent the entire 640x480 image (with both fields) from the left camera...NEXT UP: Try to instantiate an independent MILDigitizer with a single field from the left camera..
o When the hook calls to update the image in the thread, I pass a pointer...should this be a memcopy instead..are we blocking the digitizer's next read?
Here's the code on the wrapper, and the current ARTag class..note special care of creating thread inside a class (TODO: this works, but is this legit....or a hack?)
Header:
#include <windows.h>
#include "artag.h"
#pragma once
class ARTagThread
{
public:
ARTagThread(void);
~ARTagThread(void);
void update(unsigned char*, int);
unsigned char* activeImage;
HANDLE theThread;
DWORD threadId;
bool needUpdate;
};
Class:
#include ".\ARTagThread.h"
#include "main.h"
static DWORD instanceThread(void *param)
{
ARTag* artagController = new ARTag();
ARTagThread *obj=(ARTagThread *)param;
while(1) {
if(obj->needUpdate) {
add_log("\nThread: I needed update, so that's what I did!!");
//Call ARTag update
artagController->updateImage(obj->activeImage);
//clear update
obj->needUpdate = false;
//sleep(10);
}
}
return 1;
};
ARTagThread::ARTagThread(void)
{
add_log("Startiing the ARTag Thread!");
//Make the controller
needUpdate = false;
DWORD dwThreadId;
//Start the thread
theThread = CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)instanceThread, this,0,&dwThreadId);
if(theThread == NULL) {
add_log("ERROR Starting Thread");
} else {
add_log("Thread running with proc id = " + dwThreadId);
}
}
ARTagThread::~ARTagThread(void)
{
}
void ARTagThread::update(unsigned char* srcImage, int pitch)
{
add_log("\nsomeone called the thread update accessor with pitch %d", pitch);
int blockSize = sizeof(unsigned char)*pitch*240;
//mem copy the image data
this->activeImage = srcImage;
//this->activeImage = (unsigned char*)malloc(blockSize);
//memmove(activeImage, srcImage, blockSize);
//set need update
this->needUpdate = true;
}
Current ARTag header and class:
/******************************************************************************
*
* The ARTag class is used to track ARTag fiducials in an
* image.
*
* @author Steve Henderson (henderso@cs.columbia.edu)
*
******************************************************************************/
#pragma once
/////////////////////////////////////////////////
//DEFINITIONS
////////////////////////////////////////////////
class ARTag
{
public:
/////////////////////////////////////////////////
//PRIMITIVES
////////////////////////////////////////////////
/////////////////////////////////////////////////
//ATTRIBUTES
////////////////////////////////////////////////
HANDLE hPipe;
int updateCount;
/////////////////////////////////////////////////
//METHODS
////////////////////////////////////////////////
/**************************************************************
*
* Update the class
*
***************************************************************/
void updateImage(unsigned char* new_image);
/////////////////////////////////////////////////
//CONSTRUCTORS/DESTRUCTORS
////////////////////////////////////////////////
ARTag(void);
~ARTag(void);
};
/******************************************************************************
*
* The ARTag class is used to track ARTag fiducials in an
* image.
*
* @author Steve Henderson (henderso@cs.columbia.edu)
*
******************************************************************************/
#include <windows.h>
#include "main.h"
#include ".\ARTag.h"
#include <stdio.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include "glut.h"
#include "artag_rev2.h"
#include "TangibleObject.h"
////////////////////////////////////////////////////////////////
// PRIMITIVES
////////////////////////////////////////////////////////////////
#define SCREEN_WIDTH 800
#define SCREEN_HEIGHT 600
#define CAMERA_WIDTH 640
#define CAMERA_HEIGHT 480
#define DEFAULT_FX 1100 //focal length approx 850-1150 for 640x480 Dragonflies
#define DEFAULT_FY 1100 //about 400 for 320x240 webcam
#define CAM_NUM 0 //which camera input
#define ARRAY_FILENAME "C:\\ARMAR\\ValveHook\\D3D_starterkit_v3.0b\\D3D9\\panel_set.cf"
////////////////////////////////////////////////////////////////
// ATTRIBUTES
////////////////////////////////////////////////////////////////
unsigned char *cam_image,*cam_tex_img; //cam_image is cam_width*cam_height from camera, cam_tex_img is 1024*1024
char rgb_greybar; //1=RGB, 0=greyscale
int cam_width, cam_height;
unsigned char *screen_image;
char mirror_on=0; //set to 1 to flip image horizontally for "magic mirror" effect
int artag_object_id; //artag object # for array
//camera params ("K" matrix) - perfect (no radial ,etc distortion) pinhole model
float camera_cx,camera_cy; //camera params
//variables for OpenGL for camera image
GLuint camera_texID; //handle to camera image texture
double camera_opengl_dRight,camera_opengl_dLeft,camera_opengl_dTop,camera_opengl_dBottom;
GLuint pgm_texID;
unsigned char *texpgm;
TangibleObject *testObject;
///////////////////////////
// PIPE STUFF
///////////////////////////
//Name given to the pipe
#define g_szPipeName "\\\\.\\pipe\\mynamedpipe"
//Pipe name format - \\servername\pipe\pipename
//This pipe is for server on the same computer,
//however, pipes can be used to
//connect to a remote server
#define BUFFER_SIZE 256 //1k
#define ACK_MESG_RECV "Message received successfully"
float f=0;
//THE SINGLE MODEL VIEW MATRIX
float mm[16];
/**************************************************************
*
* Constructor
***************************************************************/
ARTag::ARTag(void)
{
add_log("******************INITIALIZING ARTag CONTROLLER*********************");
updateCount=0;
int width,height;
char version_string[1024];
int argc=0;
char** argv = (char**)(malloc(sizeof(char*) * 10));
// initialize glut
glutInit(&argc, argv);
//glutInitDisplayMode(GLUT_DOUBLE | GLUT_DEPTH | GLUT_RGB | GLUT_STENCIL);
glutInitDisplayMode(GLUT_DOUBLE | GLUT_DEPTH | GLUT_RGB | GLUT_STENCIL);
// create the window
glutInitWindowSize(SCREEN_WIDTH,SCREEN_HEIGHT);
glutInitWindowPosition(10000, 10000);
glutCreateWindow("OpenGL with ARTag");
glutSetWindowTitle("OpenGL with ARTag");
glClearColor(0, 0, 0, 0.0);
// setup callbacks
/*
atexit(terminate);
glutDisplayFunc(opengl_draw);
glutIdleFunc(opengl_tick);
glutReshapeFunc(opengl_reshape);
glutKeyboardFunc(opengl_key_down);
glutPassiveMotionFunc(opengl_mouse_moved);
glutMouseFunc(opengl_mouse_clicked);
*/
/*
// set up gl parameters
glEnable(GL_DEPTH_TEST);
glBlendFunc(GL_SRC_ALPHA,GL_ONE);
glShadeModel(GL_SMOOTH);
glEnable(GL_TEXTURE_2D);
glCullFace(GL_BACK);
glFrontFace(GL_CW);
//initialize camera
/*
rgb_greybar=1;
if(init_camera(CAM_NUM,640,480,rgb_greybar,&cam_width,&cam_height)) {printf("Can't start camera\n");exit(1);}
printf("Camera started with resolution=%dx%d\n",cam_width,cam_height);
*/
cam_width = CAMERA_WIDTH;
cam_height = CAMERA_HEIGHT;
//display version (optional)
artag_get_version(version_string);
add_log("version: <%s>\n",version_string);
// ARTag initialization
if(init_artag(cam_width,cam_height,3)) {add_log("ERROR: Can't start ARTag\n");}
//set camera params
artag_set_camera_params(DEFAULT_FX,DEFAULT_FY,cam_width/2.0,cam_height/2.0);
//load coordframe file
if(load_array_file(ARRAY_FILENAME)) add_log("ERROR loading <%s>\n",ARRAY_FILENAME);
else add_log("Loaded array file <%s>\n",ARRAY_FILENAME);
//associate object with array named "base0"
artag_object_id=artag_associate_array("toolbar1");
//set viewing frustrum to match camera FOV
glMatrixMode (GL_PROJECTION);
glLoadIdentity ();
camera_opengl_dRight = (double)cam_width / (double)(2.0 * DEFAULT_FX);
camera_opengl_dLeft = -camera_opengl_dRight;
camera_opengl_dTop = (double)cam_height / (double)(2.0 * DEFAULT_FY);
camera_opengl_dBottom = -camera_opengl_dTop;
glFrustum(camera_opengl_dLeft, camera_opengl_dRight, camera_opengl_dBottom, camera_opengl_dTop, 1.0, 102500.0);
camera_opengl_dLeft *= 1024;
camera_opengl_dRight *= 1024;
camera_opengl_dBottom *= 1024;
camera_opengl_dTop *= 1024;
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
//rgb image
cam_image=(unsigned char*)malloc(cam_width*cam_height*4+100);
if(cam_image==NULL) {printf("failed malloc\n");exit(1);}
//camera texture
cam_tex_img=(unsigned char*)malloc(1024*1024*4+100);
if(cam_tex_img==NULL) {printf("failed malloc\n");exit(1);}
// make a texture for the video
glGenTextures(1, &camera_texID);
glBindTexture(GL_TEXTURE_2D, camera_texID);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
//load example PPM image for textured quad
glGenTextures(1, &pgm_texID);
glBindTexture(GL_TEXTURE_2D, pgm_texID);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
//Make a tangible object where we can apply smoothing
testObject = new TangibleObject(artag_associate_array("toolbar1"), mirror_on);
testObject->setFiltering(true);
//SET UP PIPE//
//Connect to the server pipe using CreateFile()
this->hPipe = CreateFile(
g_szPipeName, // pipe name
GENERIC_READ | // read and write access
GENERIC_WRITE,
0, // no sharing
NULL, // default security attributes
OPEN_EXISTING, // opens existing pipe
0, // default attributes
NULL); // no template file
if (INVALID_HANDLE_VALUE == this->hPipe)
{
add_log("\nError occurred while connecting"
" to the server: %d", GetLastError());
//One might want to check whether the server pipe is busy
//This sample will error out if the server pipe is busy
//Read on ERROR_PIPE_BUSY and WaitNamedPipe() for that
}
else
{
add_log("\nCreateFile() was successful.");
}
}
/**************************************************************
*
* Update the ARTag Processor.
***************************************************************/
void ARTag::updateImage(unsigned char* new_image)
{
if(updateCount==5) {
updateCount=0;
add_log("looking for ARTag objects...");
cam_image = new_image;
//main ARTag function - process image for markers and 'coordframe' marker arrays
artag_find_objects(cam_image,1);
testObject->updateState();
if (testObject->IsFound())
{
add_log("\nFound ARTAG object!!!!");
testObject->getPoseArray(mm);
//convertToAR3x4Matrix(fiducialArray[TEST_OBJECT], fiducialMatrix[TEST_OBJECT]);
add_log("\n------------------trying to reach pipe------------------------");
//TRY TO REACH THE PIPE
char szBuffer[BUFFER_SIZE];
sprintf(szBuffer, "UPDATE toolbar1 %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f %f" , mm[0], mm[1], mm[2], mm[3], mm[4], mm[5], mm[6], mm[7], mm[8], mm[9], mm[10], mm[11], mm[12], mm[13], mm[14], mm[15]);
//strcpy(szBuffer, "GET HELLO");
DWORD cbBytes;
bool goodRead, goodWrite = false;
//Send the message to server
BOOL bResult = WriteFile(
this->hPipe, // handle to pipe
szBuffer, // buffer to write from
strlen(szBuffer)+1, // number of bytes to write, include the NULL
&cbBytes, // number of bytes written
NULL); // not overlapped I/O
if ( (!bResult) || (strlen(szBuffer)+1 != cbBytes))
{
add_log("\nError occurred while writing"
" to the server: %d", GetLastError());
goodWrite= false;
}
else
{
add_log("\nWriteFile() was successful.");
goodWrite = true;
}
if(goodWrite) {
//Read server response
bResult = ReadFile(
this->hPipe, // handle to pipe
szBuffer, // buffer to receive data
sizeof(szBuffer), // size of buffer
&cbBytes, // number of bytes read
NULL); // not overlapped I/O
if ( (!bResult) || (0 == cbBytes))
{
add_log("\nError occurred while reading"
" from the server: %d", GetLastError());
//CloseHandle(hPipe);
goodRead= false;
}
else
{
add_log("\nReadFile() was successful.");
goodRead= true;
}
if(goodRead) add_log("\nServer sent the following message: %s", szBuffer);
}
} else {
}
} else {
updateCount++;
}
}
/**************************************************************
*
* Destructor
***************************************************************/
ARTag::~ARTag(void)
{
}
[edit] Status report 17:49, 4 September 2007 (EDT)
I have integrated ARTag into Valve via the hook and a sep named pipe server. Currently, an ARTag controller class is called every frame and sent the left image from the stereo image pair that forms the video see through backbuffer. However, I am running into a frame rate issue. Without ARTag the hook generates a 25-28 fps throughput on the video see through display. With ARTag, goes to 12 fps. Going to try threading the ARTag controller so the render loop inside the hook is not waiting around for ARTag. --Steve 17:49, 4 September 2007 (EDT)
[edit] Background/Starting Out
One of the issues I ran into early on in getting ARTag to work with the cameras in the InnerOptic is the interlaced video from the cameras. The NTSC interlacing causes tearing, which interferes with ARTag recognition. So, I experimented with only grabbing one field (see code below). This greatly improved tracking, but halves the vertical resolution. --Steve 17:56, 4 September 2007 (EDT)
* Issues:
o Can we use a seperate digitizer in the Matrox API to do this single field grab without interfereing with the normal 2 grabs required for the vide see through background?
ARTag code for InnerOptic cameras grabbed by Matrox Mil (on Matrox Morphis Board):
//© 2006, National Research Council Canada
//BASIC_ARTAG_OPENGL.CPP - demonstration of ARTag Rev2 OpenGL support- last update Nov 2006
//Oct 4/2005 by Mark Fiala, National Research Council of Canada - IIT/CVG group
// email: mark.fiala@nrc-cnrc.gc.ca
//
//-with this demo app, the modelview matrix of OpenGL is set by an ARTag function
//call. This program demonstrates initializing ARTag, loading an array file,
//associating an individual array from this file (an array file can contain many
//arrays, starting and ending with the "coordframe" and "\coordframe" keywords),
//calling the main artag_find_objects() function, calling artag_is_object_found() to
//see if an object is loaded, and finally the artag_set_object_opengl_matrix()
//function to set the OpenGL modelview matrix to allow rendering relative to the array.
//-OpenGL lines, untextured triangles, untextured quads and textured quads are used
//ismar_logo.ppm image is used to texture a quad).
//
//- link in artag_rev2_vs6.lib
//- I created a Win32 Console App in Microsoft Visual Studio 6 for this demo
//
//-this demonstrates use of either OpenCV's "cvcam" for camera capture for use with
//USB,USB2 webcams, or "pgrflycapture" for use with Point Grey's Dragonfly firewire
//camera (www.ptgrey.com). For each, uncomment the appropriate include line and
//link in the correct library. Refer to license agreements for both Intel's OpenCV
//and Point Grey's Pgrflycapture software, and use one of these two camera inputs
//if you agree with their license terms.
//
//-USING OpenCV's "cvcam"
//-#include my wrapper "camera_dragonfly_640_480_windows.c"
//-link in artag_rev2_vs6.lib (from \lib directory) and cv.lib & highgui.lib (from OpenCV 4
//from Intel or from the copy I included in \opencv directory)
//
//-USING Thomas Pintaric's DSVL (DirectShow Video Processing Library) -for faster camera input than cvcam
//-#include wrapper "camera_dsvl.c" -follow directions in file to download it from Sourceforge
//-link in DSVL.lib (from DSVL Sourceforge download)
//
//-USING DRAGONFLY
//-#include my wrapper "camera_cvcam_windows.c"
//-link in artag_rev2_vs6.lib (from lib directory) & pgrflycapture.lib (from www.ptgrey.com)
//
//-to link in files in Visual Studio, do project tab ->Settings->Link,
//add them to "Object/Library modules" list
#include <windows.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <mil.h>
#define M_PI 3.1415927
#define PI 3.14159265358979
#include "glut.h"
//#include "camera_cvcam_windows.c" //link cv.lib cvcam.lib
//#include "camera_dsvl.c" //(faster than cvcam) link DSVL.lib
//#include "camera_dragonfly_1_4r27_windows.c"
//#include "camera_dragonfly_1_5_3_0015_windows.c"
//#include "camera_image_single_frame.c"
#include "TangibleObject.h"
#include "NamedPipeServer.h"
#include "artag_rev2.h"
#define SCREEN_WIDTH 800
#define SCREEN_HEIGHT 600
#define DEFAULT_FX 677 //focal length approx 850-1150 for 640x480 Dragonflies
#define DEFAULT_FY 704 //about 400 for 320x240 webcam
#define CAM_NUM 0 //which camera input
#define ARRAY_FILENAME "panel_set.cf"
unsigned char *cam_image,*cam_tex_img; //cam_image is cam_width*cam_height from camera, cam_tex_img is 1024*1024
char rgb_greybar; //1=RGB, 0=greyscale
int cam_width, cam_height;
unsigned char *screen_image;
char mirror_on=0; //set to 1 to flip image horizontally for "magic mirror" effect
int artag_object_id; //artag object # for array
//camera params ("K" matrix) - perfect (no radial ,etc distortion) pinhole model
float camera_cx,camera_cy; //camera params
//variables for OpenGL for camera image
GLuint camera_texID; //handle to camera image texture
double camera_opengl_dRight,camera_opengl_dLeft,camera_opengl_dTop,camera_opengl_dBottom;
GLuint pgm_texID;
unsigned char *texpgm;
#define FIDUCIAL_COUNT 1
#define TEST_OBJECT 0
GLfloat fiducialArray[FIDUCIAL_COUNT][16];
double fiducialMatrix[FIDUCIAL_COUNT][3][4];
double fiducialVisible[FIDUCIAL_COUNT];
unsigned char* demo_read_ppm(char *file_name, int *width, int *height);
NamedPipeServer *namedPipeServer;
////////////////
//MMMMMMMMMMMM
///////////////
//BEGIN MATROX DEFINITIONS//
/* MIL function specifications. */
//Number of parameters in Slave Method
#define FUNCTION_NB_PARAM 1
#define FUNCTION_OPCODE_ADD_CONSTANT 1
#define FUNCTION_PARAMETER_ERROR_CODE 1
#define FUNCTION_SUPPORTED_IMAGE_DEPTH 8
//END MATROX DEFINITIONS//
/////////////////////////////////////////////////
//PRIMITIVES
////////////////////////////////////////////////
const static int CAMERA_IMAGE_HEIGHT = 480;
const static int CAMERA_IMAGE_WIDTH = 640;
const static int COMPOSITE_IMAGE_HEIGHT = 480;
const static int COMPOSITE_IMAGE_WIDTH = 1280;
const static int COMPOSITE_TARGET_IMAGE_HEIGHT = 480;
const static int COMPOSITE_TARGET_IMAGE_WIDTH = 1280;
/////////////////////////////////////////////////
//ATTRIBUTES
////////////////////////////////////////////////
MIL_ID MilApplication;
MIL_ID MilSystem ;
MIL_ID MilDigitizerLeft ;
MIL_ID MilDigitizerRight ;
MIL_ID MilCompositeImage;
MIL_ID MilCompositeTargetImage;
MIL_ID MilARTagImage;
MIL_ID MilLeftSubImage;
MIL_ID MilRightSubImage;
long SizeBand, SizeX, SizeY, Type;
TangibleObject *testObject;
////////////////
//MMMMMMMMMMMM
///////////////
void initMatrox(void)
{
/* Setup Matrox Capture Stuff */
MappAlloc(M_DEFAULT,&MilApplication);
MsysAlloc(M_SYSTEM_MORPHIS,M_DEV0,M_DEFAULT,&MilSystem);
//Note M_DEV0 is right based on the InnerOptics configuration
//Note M_DEV1 is left based on the InnerOptics configuration
MdigAlloc(MilSystem, M_DEV0, "M_NTSC", M_DEFAULT , &MilDigitizerRight);
MdigAlloc(MilSystem, M_DEV1, "M_NTSC", M_DEFAULT , &MilDigitizerLeft);
SizeBand = MdigInquire(MilDigitizerLeft, M_SIZE_BAND, M_NULL);
SizeX = MdigInquire(MilDigitizerLeft, M_SIZE_X, M_NULL);
SizeY = MdigInquire(MilDigitizerLeft, M_SIZE_Y, M_NULL);
//Allocate color buffer for composite image
MbufAllocColor(MilSystem,
//SizeBand,
3,
COMPOSITE_IMAGE_WIDTH,
COMPOSITE_IMAGE_HEIGHT,
8 + M_UNSIGNED,
M_IMAGE + M_BGR24 + M_PACKED + M_GRAB, &MilCompositeImage);
//M_IMAGE + M_BGR32 + M_PACKED + M_GRAB, &MilCompositeImage);
//Allocate color buffer for composite image
MbufAllocColor(MilSystem,
//SizeBand,
3,
COMPOSITE_TARGET_IMAGE_WIDTH,
COMPOSITE_TARGET_IMAGE_HEIGHT,
8 + M_UNSIGNED,
M_IMAGE + M_BGR24 + M_GRAB, &MilCompositeTargetImage);
//Allocate color buffer for AR Tag image
MbufAllocColor(MilSystem,
//SizeBand,
3,
CAMERA_IMAGE_WIDTH,
CAMERA_IMAGE_HEIGHT,
8 + M_UNSIGNED,
M_IMAGE + M_BGR24 + M_PACKED + M_GRAB, &MilARTagImage);
//Allocate Left and Right Child Buffers
//Create the left image starting at 0,0 in composite (upper left) and spanning one camera image wide
MbufChildColor2d(MilCompositeImage, M_ALL_BANDS, 0, 0, CAMERA_IMAGE_WIDTH, CAMERA_IMAGE_HEIGHT, &MilRightSubImage);
//Create the right image starting at CAMERA_IMAGE_WIDTH,0 in composite (upper midpoint) and spanning one camera image wide
MbufChildColor2d(MilCompositeImage, M_ALL_BANDS, CAMERA_IMAGE_WIDTH, 0, CAMERA_IMAGE_WIDTH, CAMERA_IMAGE_HEIGHT, &MilLeftSubImage);
//Create a B&W copy of the left image starting at 0,0 in composite (upper left) and spanning one camera image wide
//MbufChild2d(MilCompositeImage, 0, 0, CAMERA_IMAGE_WIDTH, CAMERA_IMAGE_HEIGHT, &MilBWSubImage);
//Clear the composite image
MbufClear(MilCompositeImage, 0L);
//Set up the digitizers (frame grabbers)
//Note M_CH0 is right based on the InnerOptics configuration
//Note M_CH1 is left based on the InnerOptics configuration
//WONT LET YOU GO > 1.0
MdigControl(MilDigitizerLeft,M_GRAB_SCALE_X , 1.0);
MdigControl(MilDigitizerRight,M_GRAB_SCALE_Y , 1.0);
MdigControl(MilDigitizerLeft,M_GRAB_FIELD_NUM , 1);
MdigControl(MilDigitizerRight,M_GRAB_FIELD_NUM, 1);
MdigControl(MilDigitizerLeft,M_CAMERA_LOCK, M_ENABLE + M_FAST);
MdigControl(MilDigitizerRight,M_CAMERA_LOCK, M_ENABLE + M_FAST);
MdigChannel(MilDigitizerRight, M_CH0);
MdigChannel(MilDigitizerLeft, M_CH1);
/* Grab continuously on the display. */
MdigGrabContinuous(MilDigitizerLeft, MilLeftSubImage);
MdigGrabContinuous(MilDigitizerRight, MilRightSubImage);
/* Display the image. */
//MdispSelect(MilDisplay, MilImage);
}
/**************************************************************
*
* Gets the Left images.
***************************************************************/
unsigned char* getLeftImage()
{
unsigned char *SrcImageDataPtr;
long SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
/* Read image information. */
MbufInquire(MilLeftSubImage, M_HOST_ADDRESS, &SrcImageDataPtr);
MbufInquire(MilLeftSubImage, M_SIZE_X, &SrcImageSizeX);
MbufInquire(MilLeftSubImage, M_SIZE_Y, &SrcImageSizeY);
//M_SIZE_BIT = which returns the depth per band, in bits.
MbufInquire(MilLeftSubImage, M_SIZE_BIT, &SrcImageSizeBit);
MbufInquire(MilLeftSubImage, M_PITCH_BYTE, &SrcImagePitchByte);
//printf("got paired Image..size x=%d, size y=%d\n", SrcImageSizeX, SrcImageSizeY);
//MimResize(MilCompositeImage, MilCompositeTargetImage, M_FILL_DESTINATION, M_FILL_DESTINATION, M_DEFAULT);
//MbufInquire(MilCompositeTargetImage, M_HOST_ADDRESS, &SrcImageDataPtr);
return SrcImageDataPtr;
}
/**************************************************************
*
* Gets the AR Tag image.
***************************************************************/
unsigned char* getARTagImage()
{
//Copy from composite
//void MbufCopyClip(MIL_ID SrcBufId, MIL_ID DestBufId, long DestOffX, long DestOffY)
MbufCopyClip(MilCompositeImage, MilARTagImage, 0,0);
unsigned char *SrcImageDataPtr;
long SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
/* Read image information. */
MbufInquire(MilARTagImage, M_HOST_ADDRESS, &SrcImageDataPtr);
MbufInquire(MilARTagImage, M_SIZE_X, &SrcImageSizeX);
MbufInquire(MilARTagImage, M_SIZE_Y, &SrcImageSizeY);
//M_SIZE_BIT = which returns the depth per band, in bits.
MbufInquire(MilARTagImage, M_SIZE_BIT, &SrcImageSizeBit);
MbufInquire(MilARTagImage, M_PITCH_BYTE, &SrcImagePitchByte);
//printf("got paired Image..size x=%d, size y=%d\n", SrcImageSizeX, SrcImageSizeY);
//MimResize(MilCompositeImage, MilCompositeTargetImage, M_FILL_DESTINATION, M_FILL_DESTINATION, M_DEFAULT);
//MbufInquire(MilCompositeTargetImage, M_HOST_ADDRESS, &SrcImageDataPtr);
return SrcImageDataPtr;
}
/**************************************************************
*
* Gets the pitch of the left image.
***************************************************************/
int getARTagPitch()
{
long SrcImagePitchByte;
MbufInquire(MilARTagImage, M_PITCH_BYTE, &SrcImagePitchByte);
return (SrcImagePitchByte);
}
/**************************************************************
*
* Gets the pitch of the left image.
***************************************************************/
int getLeftPitch()
{
long SrcImagePitchByte;
MbufInquire(MilLeftSubImage, M_PITCH_BYTE, &SrcImagePitchByte);
return (SrcImagePitchByte);
}
/**************************************************************
*
* Gets a composite image of both Left and Right images.
***************************************************************/
unsigned char* getPairedImage()
{
unsigned char *SrcImageDataPtr;
long SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
/* Read image information. */
MbufInquire(MilCompositeImage, M_HOST_ADDRESS, &SrcImageDataPtr);
MbufInquire(MilCompositeImage, M_SIZE_X, &SrcImageSizeX);
MbufInquire(MilCompositeImage, M_SIZE_Y, &SrcImageSizeY);
//M_SIZE_BIT = which returns the depth per band, in bits.
MbufInquire(MilCompositeImage, M_SIZE_BIT, &SrcImageSizeBit);
MbufInquire(MilCompositeImage, M_PITCH_BYTE, &SrcImagePitchByte);
//printf("got paired Image..size x=%d, size y=%d\n", SrcImageSizeX, SrcImageSizeY);
//MimResize(MilCompositeImage, MilCompositeTargetImage, M_FILL_DESTINATION, M_FILL_DESTINATION, M_DEFAULT);
//MbufInquire(MilCompositeTargetImage, M_HOST_ADDRESS, &SrcImageDataPtr);
return SrcImageDataPtr;
}
/**************************************************************
*
* Gets the pitch of the composite image.
***************************************************************/
int getPitch()
{
long SrcImagePitchByte;
MbufInquire(MilCompositeImage, M_PITCH_BYTE, &SrcImagePitchByte);
return (SrcImagePitchByte);
}
/**************************************************************
*
* Gets the Right images.
***************************************************************/
unsigned char* getRightImage()
{
unsigned char *SrcImageDataPtr;
long SrcImageSizeX, SrcImageSizeY, SrcImageSizeBit, SrcImagePitchByte;
/* Read image information. */
MbufInquire(MilRightSubImage, M_HOST_ADDRESS, &SrcImageDataPtr);
MbufInquire(MilRightSubImage, M_SIZE_X, &SrcImageSizeX);
MbufInquire(MilRightSubImage, M_SIZE_Y, &SrcImageSizeY);
//M_SIZE_BIT = which returns the depth per band, in bits.
MbufInquire(MilRightSubImage, M_SIZE_BIT, &SrcImageSizeBit);
MbufInquire(MilRightSubImage, M_PITCH_BYTE, &SrcImagePitchByte);
//printf("got paired Image..size x=%d, size y=%d\n", SrcImageSizeX, SrcImageSizeY);
//MimResize(MilCompositeImage, MilCompositeTargetImage, M_FILL_DESTINATION, M_FILL_DESTINATION, M_DEFAULT);
//MbufInquire(MilCompositeTargetImage, M_HOST_ADDRESS, &SrcImageDataPtr);
return SrcImageDataPtr;
}
/**************************************************************
*
* Gets the pitch of the right image.
***************************************************************/
int getRightPitch()
{
long SrcImagePitchByte;
MbufInquire(MilRightSubImage, M_PITCH_BYTE, &SrcImagePitchByte);
return (SrcImagePitchByte);
}
int count=0;
void convertToAR3x4Matrix(float srcMatrix[16], double dstMatrix[3][4]) {
// Convert OpenGL style 16-element array (representing modelview matrix) to 3x4 matrix (representing transformation matrix)
int i,j;
for (i=0; i<3; i++) {
for (j=0; j<4; j++) {
dstMatrix[i][j] = *(srcMatrix+4*j+i);
}
}
}
//////////
void opengl_tick(void)
{
if(5==5) {
int j=0;
int source_offset,dest_offset;
//grab camera image
//camera_grab_bgr_blocking(CAM_NUM,cam_image,cam_width,cam_height);
//USE MATROX
cam_image = getARTagImage();
//printf("ARTag image pitch = %d\n", getARTagPitch());
//printf("composite pitch = %d\n", getPitch());
//printf("cam width, height=%d, %d\n", cam_width, cam_height);
//main ARTag function - process image for markers and 'coordframe' marker arrays
artag_find_objects(cam_image,1);
testObject->updateState();
if (testObject->IsFound())
{
fiducialVisible[TEST_OBJECT] = 1;
testObject->getPoseArray(fiducialArray[TEST_OBJECT]);
convertToAR3x4Matrix(fiducialArray[TEST_OBJECT], fiducialMatrix[TEST_OBJECT]);
} else {
fiducialVisible[TEST_OBJECT] = 0;
}
//copy camera image into 1024x1024 texture image
source_offset=0; dest_offset=0;//200*3+550*1024*3;//0;
for(j=0;j<cam_height;j++)
{
memcpy(cam_tex_img+dest_offset,cam_image+source_offset,cam_width*3);
source_offset+=cam_width*3; dest_offset+=1024*3;
}
//copy image into OpenGL buffer
glBindTexture(GL_TEXTURE_2D, camera_texID);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 1024, 1024, 0, GL_BGR_EXT, GL_UNSIGNED_BYTE,cam_tex_img);
glutPostRedisplay();
count=0;
}
namedPipeServer->update();
}
void opengl_draw(void)
{
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
// draw a quad for the background video
glLoadIdentity();
glDisable(GL_LIGHTING);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, camera_texID);
glBegin(GL_QUADS);
glColor3f(1.0f, 1.0f, 1.0f);
//draw camera texture, set with offset to aim only at cam_width x cam_height upper left bit
//normal operation (magic mirror off)
if(mirror_on==0)
{
glTexCoord2f(0.0f, (float)cam_height/1024.0);
glVertex3f(camera_opengl_dLeft, camera_opengl_dBottom, -1024);
glTexCoord2f(0.0f, 0.0f);
glVertex3f(camera_opengl_dLeft, camera_opengl_dTop, -1024);
glTexCoord2f((float)cam_width/1024.0, 0.0f);
glVertex3f(camera_opengl_dRight, camera_opengl_dTop, -1024);
glTexCoord2f((float)cam_width/1024.0, (float)cam_height/1024.0);
glVertex3f(camera_opengl_dRight, camera_opengl_dBottom, -1024);
glEnd();
}
//flip background image for "magic mirror" effect
else
{
glTexCoord2f(0.0f, (float)cam_height/1024.0);
glVertex3f(camera_opengl_dRight, camera_opengl_dBottom, -1024);
glTexCoord2f(0.0f, 0.0f);
glVertex3f(camera_opengl_dRight, camera_opengl_dTop, -1024);
glTexCoord2f((float)cam_width/1024.0, 0.0f);
glVertex3f(camera_opengl_dLeft, camera_opengl_dTop, -1024);
glTexCoord2f((float)cam_width/1024.0, (float)cam_height/1024.0);
glVertex3f(camera_opengl_dLeft, camera_opengl_dBottom, -1024);
glEnd();
}
//draw simple augmentations
if(testObject->IsFound())
{
//printf("!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!Found it!\n");
//artag_set_object_opengl_matrix(artag_object_id,mirror_on);
testObject->loadOpenGLMatrix();
glDisable(GL_TEXTURE_2D);
glBegin(GL_TRIANGLES);
glColor3f(1, 0, 0);
glVertex3f(0,0,0);
glColor3f(0, 1, 0);
glVertex3f(10,0,0);
glColor3f(0, 0, 1);
glVertex3f(10,0,10);
//
glColor3f(1, 0, 0);
glVertex3f(0,0,0);
glColor3f(0, 1, 0);
glVertex3f(10,0,0);
glColor3f(0, 0, 1);
glVertex3f(10,0,10);
glEnd();
glBegin(GL_QUADS);
glColor3f(0, 1, 0);
glVertex3f(-5,25,0);
glVertex3f(-5,35,0);
glVertex3f( 5,35,0);
glVertex3f( 5,25,0);
glEnd();
glBegin(GL_LINES);
glColor3f(1, 0, 0);
glVertex3f(0,0,0);
glColor3f(0, 0, 1);
glVertex3f(00,0,100);
glEnd();
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, pgm_texID);
glBegin(GL_QUADS);
glColor3f(1.0f, 1.0f, 1.0f);
glTexCoord2f(0.0f, 1.0f); glVertex3f(-50,50,0);
glTexCoord2f(0.0f, 0.0f); glVertex3f(-50,150,0);
glTexCoord2f(1.0f, 0.0f); glVertex3f( 50,150,0);
glTexCoord2f(1.0f, 1.0f); glVertex3f( 50,50,0);
glEnd();
} else {
//printf("no marker found..\n");
}
glutSwapBuffers();
}
void opengl_reshape(int w, int h)
{
//Setup a new viewport.
glViewport (0, 0, w, h);
}
//empty keyboard and mouse move,click functions
void opengl_mouse_moved(int x, int y)
{
int w=glutGet(GLUT_WINDOW_WIDTH),h=glutGet(GLUT_WINDOW_HEIGHT);
}
void opengl_mouse_clicked(int button, int state, int x, int y)
{
}
void opengl_key_down(unsigned char k, int x, int y)
{
if(k==27) exit(0); //escape key exits program
else if(k==' ') mirror_on=mirror_on?0:1; //space bar toggles "magic mirror" mode (hor. mirroring)
}
void terminateMe(void)
{
close_artag();
//close_camera(CAM_NUM);
}
int main(int argc, char** argv)
{
int width,height;
char version_string[1024];
//int argc = 0;
//char** argv = (char**)malloc(sizeof(char*) * 10);
//argc = 0;
//argv = (char**)malloc(sizeof(char*) * 10);
// initialize glut
glutInit(&argc, argv);
// initialize glut
//glutInit(&argc, argv);
glutInitDisplayMode(GLUT_DOUBLE | GLUT_DEPTH | GLUT_RGB | GLUT_STENCIL);
// create the window
glutInitWindowSize(SCREEN_WIDTH,SCREEN_HEIGHT);
glutInitWindowPosition(100, 100);
glutCreateWindow("OpenGL with ARTag");
glutSetWindowTitle("OpenGL with ARTag");
glClearColor(0, 0, 0, 0.0);
// setup callbacks
atexit(terminateMe);
glutDisplayFunc(opengl_draw);
glutIdleFunc(opengl_tick);
glutReshapeFunc(opengl_reshape);
glutKeyboardFunc(opengl_key_down);
glutPassiveMotionFunc(opengl_mouse_moved);
glutMouseFunc(opengl_mouse_clicked);
// set up gl parameters
glEnable(GL_DEPTH_TEST);
glBlendFunc(GL_SRC_ALPHA,GL_ONE);
glShadeModel(GL_SMOOTH);
glEnable(GL_TEXTURE_2D);
glCullFace(GL_BACK);
glFrontFace(GL_CW);
//initialize camera
//rgb_greybar=1;
//if(init_camera(CAM_NUM,640,480,rgb_greybar,&cam_width,&cam_height)) {printf("Can't start camera\n");exit(1);}
//printf("Camera started with resolution=%dx%d\n",cam_width,cam_height);
//USE MATROX
initMatrox();
cam_width = 640;
cam_height = 320;
//display version (optional)
artag_get_version(version_string);
printf("version: <%s>\n",version_string);
// ARTag initialization
if(init_artag(cam_width,cam_height,3)) {printf("ERROR: Can't start ARTag\n");exit(1);}
//set camera params
artag_set_camera_params(DEFAULT_FX,DEFAULT_FY,cam_width/2.0,cam_height/2.0);
//load coordframe file
if(load_array_file(ARRAY_FILENAME)) printf("ERROR loading <%s>\n",ARRAY_FILENAME);
else printf("Loaded array file <%s>\n",ARRAY_FILENAME);
//associate object with array named "base0"
artag_object_id=artag_associate_array("toolbar1");
printf("object id=%d", artag_object_id);
int g = artag_associate_array("base0");
printf("object id=%d", g);
//Make a tangible object where we can apply smoothing
testObject = new TangibleObject(artag_associate_array("toolbar1"), mirror_on);
testObject->setFiltering(true);
////////////////////////////NAMED PIPE/////////////////////////////
namedPipeServer = new NamedPipeServer();
//set viewing frustrum to match camera FOV
glMatrixMode (GL_PROJECTION);
glLoadIdentity ();
camera_opengl_dRight = (double)cam_width / (double)(2.0 * DEFAULT_FX);
camera_opengl_dLeft = -camera_opengl_dRight;
camera_opengl_dTop = (double)cam_height / (double)(2.0 * DEFAULT_FY);
camera_opengl_dBottom = -camera_opengl_dTop;
glFrustum(camera_opengl_dLeft, camera_opengl_dRight, camera_opengl_dBottom, camera_opengl_dTop, 1.0, 102500.0);
camera_opengl_dLeft *= 1024;
camera_opengl_dRight *= 1024;
camera_opengl_dBottom *= 1024;
camera_opengl_dTop *= 1024;
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
//rgb image
cam_image=(unsigned char*)malloc(cam_width*cam_height*4+100);
if(cam_image==NULL) {printf("failed malloc\n");exit(1);}
//camera texture
cam_tex_img=(unsigned char*)malloc(1024*1024*4+100);
if(cam_tex_img==NULL) {printf("failed malloc\n");exit(1);}
// make a texture for the video
glGenTextures(1, &camera_texID);
glBindTexture(GL_TEXTURE_2D, camera_texID);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
//load example PPM image for textured quad
glGenTextures(1, &pgm_texID);
glBindTexture(GL_TEXTURE_2D, pgm_texID);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
texpgm=demo_read_ppm("test_texture.ppm",&width,&height);
if (texpgm!=NULL)
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, width, height, 0, GL_RGB, GL_UNSIGNED_BYTE,texpgm);
else {printf("Couldn't load <test_texture.ppm>\n");exit(1);}
// show the window and start the loop
glutShowWindow();
glutMainLoop();
return 0;
}
//------------------------------------------------------------------------------------------------
//PPM files encoded byte order: red green blue
unsigned char* demo_read_ppm(char *file_name, int *width, int *height)
{
int i,size;
int max_grey;
FILE *in_file;
char c,comment[256];
unsigned char uc,*image;
in_file=fopen(file_name,"rb");
if(in_file==NULL) return NULL;
c=fgetc(in_file); if(c!='P') {printf("%s Not a PPM file\n",file_name);exit(1);}
c=fgetc(in_file); if(c!='6') {printf("%s Not a PPM file\n",file_name);exit(1);}
c=fgetc(in_file); if(c!='\n') {printf("%s Not a PPM file\n",file_name);exit(1);}
while(1)
{
fgets(comment,256,in_file);
if((comment[0]!='#')&&(comment[0]!=0x0a)&&(comment[0]!=0x0d)) break;
}
sscanf(comment,"%d %d",width,height);
fgets(comment,256,in_file);
sscanf(comment,"%d",&max_grey);
size=(*width)*(*height);
image=(unsigned char*)malloc(size*3+10);
if(image==NULL) {printf("READ_PPM() error: Couldn't malloc image\n");exit(1);}
for(i=0;i<size*3;i++)
{
uc=fgetc(in_file);
*(image+i)=uc;
}
fclose(in_file);
return image;
}
