[dead2sin]

[QUOTE][u]Quote by: larkost[/u][p]Where is the world did you get a .cdr image? In any case, if you need to use this one, then your best bet is to convert it to a main-line .dmg file. For sanity reasons I have put in the check you are running into to track what base disk image is in use, and we are going to be going farther in that direction later, so it is a good idea to get lined up for that (most people should not have to worry about this, you are in a weird case).

Assuming that this is a .cdr that hdiutil can recognize (you are hosed otherwise), you can use a command line this to convert it:
[code]hdiutil convert /path/to/image.cdr -format UDBZ -o ‘Mac OS X Install DVD.dmg'[/code]

I do want to sound the note of caution here: the .cdr format is a really strange bit. It makes me think that there is something odd going on here, namely that it is a “hackintosh” image (they are the only ones using that format… for whatever reason). I would be really leery of trying anything involvine “hackintosh” images with InstaDMG. There are real chances that you are not going to get what you expect, no matter what it is you are trying to do.[/p][/QUOTE]

When you make a new image of a DVD/CD in Disk Utility and specify a ‘Image Format’ of ‘DVD/CD Master’ it spits out a .cdr file by default. I normally rip it as DVD/CD Master and then just change the extension to .dmg (Because it really IS a dmg, but it makes it a cdr for some reason…)

[dead2sin]

Alright, I did some more tests. Mounting a network volume on raid 5 turned out to be pretty bad, who would of guessed? 🙂 I clocked that at 1:11:03.

I put my two 500gb HDs in Software Raid 0 and ran that, two tests back to back I got just under 50 minutes (Around 49 minutes and change).

Here is the rather interesting bit: A single 500gb drive, is 70mb/sec r/w, where as two in raid 0 are 155mb/sec, yet the time difference between one drive and 2x in Raid 0 is roughly 11 minutes. That is around 16% faster. In thinking about it, I guess the single 500gb drive can’t max out the write on the 2 in raid 0, so for it to really work well, I’ll need 2x Raid 0 arrays for it to have a significant change in speed.

Anywho, those were the final numbers I came up with.

Nate

[dead2sin]

[QUOTE][u]Quote by: Joy[/u][p]Do I have to run a checksum on the base dmg? I moved the .cdr to the base of folder and put the .dmg on the end and it is giving me a “unable to get checksum for image …..[/p][/QUOTE]

It will checksum it automatically. It needs to be ‘Mac OS X Install DVD.dmg’ you don’t want ‘Mac OS X Install DVD.cdr.dmg’

If it is not checksumming properly, you could always try importing it using importDisk.py. Just move the file you have in BaseOS out and then run the python script. It’ll place it in BaseOS for you.

[dead2sin]

You can do it two ways. In the InstaDMG/Addons/InstaUp2Date/ Folder there is importDisk.py file. Just put the Original retail OS X Installer DVD in the drive and type ./importDisk.py –automatic –legacy and it will rip the disc.

Alternatively, you can put the OS Install DVD in and open disc utility, select the DVD and click new image. It sounds like you already did this for the Install DVD and have a .cdr file. You should be able to just make the file extension .dmg instead of .cdr and throw it in the BaseOS folder.

Nate

[dead2sin]

[QUOTE][u]Quote by: Allister[/u][p]As mentioned in the guide, I go from an SSD(housing the entire instadmg dir structure) to a hardware RAID 0 which is striped between two independent eSATA enclosures connected to a HighPoint RocketRaid 2314 card inside a MacPro. I timed it recently with the fastest SSD on the market below $800, a Crucial RealSSD we had in for testing, and it’s faster to output to the RAID than having it do the output to the same drive. I’d like to add a second RAID to go from scratch to output, but need to wait for more enclosures.

Allister [/p][/QUOTE]

Between the Raid 0 array and the SSD, which is faster and by how much? If raid 0 is close to the SSDs, then I’ll just do that because it is slightly cheaper.

Nate

[dead2sin]

[QUOTE][u]Quote by: sgstuart[/u][p]Hi Nate,
Can you let us know what the drive speeds of each of your HD’s are? Also your 11:42 & 1:07 posts seem to contradict each other on which drive was giving the 30 min speed?

Cutting the time in half is really nice, and I will probably try to do that.

Also, if we do this from a laptop, would Firewire 400 or firewire 800 give any speed increase? I understand already USB2 would not so will not even attempt that.

Thanks,
Steven Stuart[/p][/QUOTE]

Good question. I think I got confused 🙂 Let me run the build once more on each drive and I’ll see what we get. I ran it on the 80gb again last night and it took over an hour, so I must of gotten the numbers swapped around.

[dead2sin]

[QUOTE][u]Quote by: larkost[/u][p]To amplify this, InstaDMG is almost entirely limited by the speed of the disks. CPU and memory almost don’t come into play at all. An older computer with faster disks can produce an image faster than a newer computer that is more limited by its disks.

I don’t have any solid-state drives to play with at the moment, but I suspect that these would make really disks for image creation.[/p][/QUOTE]

What is the best setup then? Is there any real advantage to having 3 drives? (One for InstaDMG, one for Scratch and one for Output). I need to consider this before I upgrade the drives on my Mac Pro 🙂

[dead2sin]

I can confirm that it was due to a slow drive. My 80gb is much much faster then my 500gb drive, so that is what was causing the slowdown.

500gb as scratch = 1 hour process
80gb as scratch = 30 minute process

HUGE difference. When I go to buy drives for building images next fiscal year I am definitely going to be looking for fast drives. It makes all the difference in the world.

Thanks,

Nate

[dead2sin]

Here is some more info. You are right, I didn’t mention it, but its 4 bare drives in a Mac Pro. 3x 500gb and 1x 80gb. I use one 500gb for the InstaDMG 1.6b2 on 10.6.4, the second 500gb for the tmp cache and the 50gb for the output drive.

Here is something of interest. I ran another run, this time using ONLY –instadmg-scratch-folder pointed to the second 500gb drive and it ran much much faster. I’m going to run another test and see if the 80gb drive is just really slow or what is going on.

My numbers for the –instadmg-scratch-folder run using the 1st 500gb for the InstaDMG folder and running 10.6.4 client and the 2nd 500gb HD being used as the scratch are as follows:

0:29:08 (Two times as fast! Smokin’)

Let me do another run using the 80gb as the scratch drive and we’ll see what the numbers do. If its still slow, then it is clearly my 80gb drive that was the bottleneck. If it isn’t, then perhaps there is something funky going on when I specify both a scratch drive as well as an output drive (although this seems super unlikely). I’ll let you know…I’m 99% sure my 80gb drive is slow.

Nate

[dead2sin]

Alright, I did a quick test. I ran InstaUp2Date using –instadmg-output-folder as well as –instadmg-scratch-folder. The times listed are only the time it takes for InstaDMG to run, because InstaUp2Date times should be uneffected.

My results are as follows:

Normal InstaDMG run:

1:03:33

Cache Drive InstaDMG Run:

1:03:01

Only 32 seconds faster. Is that normal? None of the drives are Raid 0 (I’ll be setting that up as soon as I get my new HDs for the Mac Pro).

Nate

[dead2sin]

[QUOTE][u]Quote by: wilcoxjay[/u][p]Thanks for the quick response Nate. I just found instaDMG about a week ago, so I’m still catching up in terms of familiarity with the code and especially the diaspora that is the documentation provided by this forum. I’ll start working on a patch to implement this hack so that people who want to try it for testing can do so.

Thanks again,

James[/p][/QUOTE]

No worries! We have a great community here, welcome aboard 😀

Check out the google project for InstaDMG, Allister has made some documentation on the topic and there are also some videos regarding it on iTunes U.

Nate

[dead2sin]

[QUOTE][u]Quote by: wilcoxjay[/u][p]Hey all,

Two things. First, I remember seeing somewhere either on the forum or over at googlecode that there were plans to port instadmg to python. Is this still something people are interested in doing? Has any work been done on it?

Second, I’ve been looking into implementing some extra caching in instadmg, which will streamline the testing process by allowing you to create a new image in just the time it takes to install your package(s) and run instadmg’s final checks. This is huge in reducing the amount of redundant work that instadmg performs every time you build an image. (Think 10.6.4 combo update.) The way I’ve been doing it here is pretty hacky, I just make a catalog file with all the packages that I don’t need/want to have reinstalled everytime (for me, the most important one is the 10.6.4 update; it’s a good 10 minutes everytime I run a build), and run the build normally. Now the hacky part: copy the image from it’s output location into the Caches/BaseImageCache folder, with the same name as the previous base image cache (which you want to backup somehow, perhaps renaming it to .bak). Now when you run a build, don’t include any of the packages that you installed in your cache. You should notice a marked improvement in build times.

Clearly this process needs to be cleaned up and integrated into the actual code before it’s really usable, but I just wanted to check if anyone was working on this. I’d love to hear what other people do to work around this redundancy.

Best,

James[/p][/QUOTE]

#1 Karl has been working on that for a while now. He is progressively porting it iirc.

#2 Check out this post: [url]https://www.afp548.com/forum/viewtopic.php?showtopic=26947[/url] We discussed something similar a little bit, not sure if you saw that post already or not. Karl might be around to discuss this with you 🙂 I don’t do any coding myself, but I discussed the caching idea with him previously. I’d love to see this type of functionality for testing purposes.

Nate

[dead2sin]

Here are the registration files needed for CS4 to not prompt to register:

http://dl.dropbox.com/u/11466/CS4RegFiles.zip

These go in the User Template (/System/Library/User Template/English.lproj/Library/Preferences).

These registration files are for Photoshop, Illustrator, InDesign, Dreamweaver, Flash and Acrobat 9 Pro.

If anyone has questions, feel free to post.

[dead2sin]

[QUOTE][u]Quote by: larkost[/u][p]You can’t store the checksum of the final vanilla image, since there are a lot of things in the image that get the date that the image is created (the receipts for instance). So two vanilla images that are functionally equivalent are not going to have the same checksum either internally, or of the DMG (side note: I do want to create a tool that I can use to create checksums for images where two runs would result in the same checksum, and use this in validation. But that is necessarily going to be slow, and would really only serve to help me tell when I have broken something during development).

I do plan on using a checksum to do this, but one that combines the checksums of the inputs (installer dvd + installers + installer choices files, including settings and possibly an InstaDMG “compatibility number”, done in order). I have sketched this out already, but it absolutely requires something like InstaUp2Date to drive it, while at the same time requiring that the same component be driving the selection of the cached image. I am tentatively calling this idea “break-points”, and am still figuring out how they are going to work (beyond automatically setting one after the combo update).[/p][/QUOTE]

Sounds good to me! I can’t wait to see what you come up with in the future. Its not a huge deal as you said due to the fact that is is already pretty speedy, but it would be a nice feature for later down the road.

Nate

[dead2sin]

[QUOTE][u]Quote by: larkost[/u][p]There are now two questions here:

1) Why not cache the vanilla image (base image + combo + other updates)?

The answer here is that I do have a design in mind to do essentially this, but it requires that at run-time the system knows that nothing in the process up to the the point of caching has changed. Unfortunately with pure InstaDMG this idea was too complex, and I had to drop it. Making sure that the image is correct trumps speed (although the latter is also a goal). But with the merge of InstaUp2Date that I have had in the works for a long time now this becomes possible because everything is checksummed (so I can validate things), and InstaUp2Date/InstaDMG can tell explicitly when things change. I am still working out a lot of the details, but rest assured that this is planned.

For the moment though, the best thing to do is to grab a second fast external disk and use the option to use two drives durring image creation. This really drives down the time.

2) Allister saw the compression of the base image that is still hanging around in code. I tried to compress the base image a long time ago, figuring that I could make it take up a bit less room, and probably gain a bit of speed out of it (since disk I/O is what kills us). But something in it blew up every time, and at the time I was under some pressure to get the other features (base image caching) running, so it got put off. I should go back and validate that again….[/p][/QUOTE]

A Thought I had had in regards to item #1 was using checksums. Would this scenario be possible:

Store a checksum of the post-catalog cached image within the catalog for it (I.E., Vanilla catalog has a checksum in it for the cached image that has vanilla already processed). So when it processes, InstaUp2Date would look at the checksum and compare it to the cached Vanilla catalog image to make sure its still the same. Then, if you want to reprocess vanilla, when you update it, just delete the chucksum and that was instaup2date reprocesses it. That way, it ensures accuracy while adding speed and still allows you to reprocess on command.

Would that even be possible or desired?

Nate

[Author]

Posts