SynthExport

Source code checked in, #70298

Project Collection Service Accounts — Mon, 01 Oct 2012 21:36:45 GMT

Upgrade: New Version of LabDefaultTemplate.xaml. To upgrade your build definitions, please visit the following link: http://go.microsoft.com/fwlink/?LinkId=254563

Source code checked in, #70297

Project Collection Service Accounts — Mon, 01 Oct 2012 21:31:58 GMT

Checked in by server upgrade

New Post: Optionally exporting the camera positions as part of the model

joshharle — Sat, 04 Dec 2010 04:39:39 GMT

Hi Everyone;

I've modified SynthExport to pull camera positions and - more interestingly - create textured camera projection maps in 3DS Max. This should make it fairly easy to repeat Greg Downing's results.

Check out http://blog.neonascent.net/archives/cameraexport-photosynth-to-camera-projection-in-3ds-max/ for the application, and videos of the process.

Kind Regards,

Josh Harle

New Post: Optionally exporting the camera positions as part of the model

joshharle — Thu, 25 Nov 2010 05:38:39 GMT

@Nate

Thanks - I'm glad they were helpful! And Greg Downing's code looks like good fun too!

Kind Regards,

Josh

New Post: Optionally exporting the camera positions as part of the model

NateLawrence — Wed, 24 Nov 2010 23:29:34 GMT

Josh, I never got around to saying it either, but great work with the video tutorials.

For everyone else on this thread: Greg Downing from xRez Studios recently posted about how he used Photosynth's exported camera parameters to create a gigapixel image of an Egyptian artifact that could not have been captured otherwise. Give it a read; I reckon you'll enjoy it. He also provides the C# he wrote to extract the camera positions.

You can also view his results on the xRez weblog.

New Post: Optionally exporting the camera positions as part of the model

eliscio — Sun, 03 Oct 2010 23:43:58 GMT

Josh,

A little late perhaps, but the videos are very well done and I am sure they have already been well received.

I have been using Astre's Toolkit now on a few projects and I try to avoid the 32 bit memory issue, by trying to keep the number of photos down to 25 or so and also shoot in 2mp resolution so I don't have to convert anything.

I am traveling at the moment, but once I get back to my office, I am going to test the 64 bit version of PMVS2.

Cheers,

Eugene

New Post: Optionally exporting the camera positions as part of the model

joshharle — Sun, 19 Sep 2010 16:04:34 GMT

Hi Guys,

I have a guide and tutorial video each for Astre's PhotoSynth Toolkit, and a repackaged BundleMatcher (Bundler/CMVS/PMVS2). I wrote them for a university workshop I hosted; I hope you find them useful!

http://blog.neonascent.net/archives/photosynth-toolkit/

http://blog.neonascent.net/archives/bundler-photogrammetry-package/

Kind Regards,

Josh

New Post: Optionally exporting the camera positions as part of the model

eliscio — Tue, 24 Aug 2010 13:43:51 GMT

I had a response back from Astre:

"CMVS can’t be used used with PhotoSynth because some information given by Bundler output are not available in the binary produced by PhotoSynth. I’m working on 2 others options:

- compiling a 64bit version of PMVS2 for windows
- create a new Bundler Matcher based on Surf to replace PhotoSynth"

So for now I will have to stick with small photo sets.

New Post: Optionally exporting the camera positions as part of the model

joshharle — Mon, 23 Aug 2010 01:12:10 GMT

@eliscio: CMVS will really help. Level 0, csize 1 was impossible for me (led to out-of-memory crashes), without splitting everything up. CMVS is fairly easy to use. Run CMVS with number of images you want for your chunks, and the number of CPUs. Then run GenOption with your PMVS2 configuration options.

From the CMVS docs:

Usage: ./cmvs prefix maximage[=100] CPU[=4]

Example: If you want to specify maximage=70 and CPU=6 with the data directory located at ./pmvs, try the following command
./cmvs ./pmvs/ 70 6

and

Usage: ./genOption prefix level[=1] csize[=2] threshold[=0.7] wsize[=7] minImageNum[=3] CPU[=8]

@Nate,

Everything that comes out of PMVS2 is orientated patch based data, but that .patch file in particular is (for me) an opaque format that I haven't used. I either reconstruct using the built-in Poisson Reconstruction of Meshlab using the .PLYs, or using the stand-alone implementation using the .PSET. I'll post an example of the whole process this evening.

New Post: Optionally exporting the camera positions as part of the model

NateLawrence — Sun, 22 Aug 2010 21:28:59 GMT

@Josh,

I was under the impression from the screenshots linked to from Building Rome In A Day and the video from Towards Internet-Scale Multi-View Stereo that oriented patches were an alternate or intermediate step between dense point clouds and meshes and that these were what the pmvs_options.txt.patch file's coordinates described.

It is, of course, conceivable that the patches are really being laid on a transparent mesh, but this seems a little strange to me.

New Post: Optionally exporting the camera positions as part of the model

eliscio — Sun, 22 Aug 2010 20:42:31 GMT

Wow, I finally got it and what a great result!

I had to reduce my set down to 22 images and make them 500kb instead of 3mb each. The settings were as follows:

level 2
csize 2
threshold 0.7
wsize 7
minImageNum 3
CPU 8
setEdge 0
useBound 0
useVisData 0
sequence -1
timages -1 0 22
oimages -3

Level 1 did not work, but even at level 2 it still took a good half hour to process everything. However the end result was impressive and really clean of any stray points or noise. I will definitely need to continue testing...Thanks!

New Post: Optionally exporting the camera positions as part of the model

eliscio — Sun, 22 Aug 2010 19:16:19 GMT

I am still working at it, but my set is 62 images. I tried at full resolution and kept changing the settings, but I have been just spinning my wheels with crashes. I have just now reduced the file sizes from 3mb each to about 150kb. I am not sure if this is still over the top, but I just had another crash. So, I am going to try and increase the levels and mess with CPU settings. Perhaps this will help.

I have downloaded the CMVS, but could use some help as to how to run.

Eugene

New Post: Optionally exporting the camera positions as part of the model

joshharle — Sun, 22 Aug 2010 16:18:53 GMT

@Eugene; the patch file is not easily usable - I believe it is mainly for debugging. The .pset files contain the vertexes with normals, in a format ready to reconstruct with Michael Misha Kazhdan and Matthew Bolitho's Poisson mesh reconstruction implementation.

See the PMVS2 documentation for other output details.

Kind Regards,

Josh

New Post: Optionally exporting the camera positions as part of the model

eliscio — Sun, 22 Aug 2010 15:47:22 GMT

Thanks for the tips guys. I was using the downloaded "thumbs" so that's the first reason and the second is that I used the default images. I am just working on another synth for which I have the high res images. I am going to crank up the settings to see what this gives me.

As far as output goes, I saw there was a ply file, but what are the other two files for?

Cheers,

Eugene

New Post: Optionally exporting the camera positions as part of the model

joshharle — Sun, 22 Aug 2010 15:40:27 GMT

I have had very nice results using a Level of 0, csize of 1, and threshold 0.9

I would suggest using CMVS first, to split your images into smaller chunks to avoid running out of memory. CMVS also creates a vis.dat file that speeds up PMVS2.

New Post: Optionally exporting the camera positions as part of the model

NateLawrence — Sun, 22 Aug 2010 14:43:50 GMT

@eliscio, what resolution are the images that you used? If you used the thumbs, then the results won't be very dense at all.

Astre is reluctant to release the tool to download the full resolution images, lest it land him in legal trouble, but if you're doing this with your own synths, you should still have the original photos on your hard drive, yes?

I don't have very much experience with PMVS yet, but in the pmvs_options.txt file, the defaults should look like this:

level 1 (Level 0 will set PMVS to use the original resolution of whatever version of the images that you have placed in the distort folder. Higher levels will use smaller and smaller copies of the photos as the numbers increase. You can use this to find less detail if you're running out of memory. Level 1 is default because most cameras do not have RGB sensors for every pixel, thus examining the original resolution might be viewed as a waste of time. This call is yours, though.)

csize 2 (Cell size determines the density at which PMVS will try to reconstruct patches. The bigger the cell size, the fewer cells per image.)

threshold 0.7 (I'm a little hazy on this one. I understand that it is the tipping point which determines whether a patch will be accepted or not, but you'll just have to read the PMVS documentation yourself on this one.)

wsize 7 (Determines how large a patch of imagery from the input photos patches will be matched to. Larger patches result in more stable reconstructions but drive the time to reconstruct up.)

minImageNum 3 (Determines the number of photos a point must appear in to be kept in the reconstruction. Photosynth's default is 3. See: "Rule of three".)

CPU 4 (Determines the number of threads or 'Virtual CPUs' that will be spun up. If you have a single processor that supports multi-threading, then choose 2. If you have a multi-processor CPU where each processor supports multi-threading then do the multiplication and fill in the number here. It is, however, suggested that if you are running out of memory, you might want to lower this number.)

setEdge 0 ()

useBound 0 ()

useVisData 0 ()

sequence -1 ()

timages Must specify (Target images. The default numbers will change per dataset. These are the images that PMVS will attempt to cover in reconstructed points.)

oimages Must specify (Other images. These will be used to verify the accuracy of the reconstruction, but need not be covered in reconstructed points.)

To me, the big hitters seem to be using the original resolution photos, choosing a low Level, and keeping the Cell Size small.

If anyone else with more experience can help clarify things, please feel free. I'm still very much a newbie at this.

New Post: Optionally exporting the camera positions as part of the model

eliscio — Sun, 22 Aug 2010 13:52:01 GMT

I just tried this out and for one synth that I did, I was expecting to see a more dense result. Can someone give some explanation as to the options and how they affect the result?

New Post: Optionally exporting the camera positions as part of the model

NateLawrence — Sat, 21 Aug 2010 18:41:17 GMT

Awesome news, Josh! This is almost exactly what I was after. This all has great timing, being yesterday was the second anniversary of Photosynth's public release.

I'm off to test the toolkit, then.

Cheers, Astre!

New Post: Optionally exporting the camera positions as part of the model

joshharle — Sat, 21 Aug 2010 18:00:11 GMT

Hi All,

Astre Henri of the Visual Experiments blog has done some great work recently. Not only has he made a GPU feature matcher for bundler which significantly speeds up matching, he's just come out with a PhotoSynth Toolkit that processes PhotoSynths into PMVS2. I haven't tried the PhotoSynth toolkit yet, but the gpuSIFT works great, so I'm sure the toolkit does too.

Well done Astre!

New Post: Optionally exporting the camera positions as part of the model

eliscio — Fri, 23 Jul 2010 01:38:13 GMT

eliscio wrote:

Just wondering if anyone has had any success at all getting the camera positions and orientations to work??

Well, it's one month later and it seems the camera positions are not as simple as would seem. Please post if there has been any further progress.