Tim Anderson's ITWriting

Wrestling with the Windows Installer

 

Want to reproduce this article?

If you would like to reproduce this article on your own publication or web site, please contact Tim Anderson .

Wrestling with the Windows Installer

A Visual Studio .NET Setup project looks good at first, but there are plenty of traps for the unwary, and some features require external tools. By Tim Anderson.


Visual Studio includes integrated support for creating setup applications so that you can easily deploy your .NET applications. It is disappointing that it doesn't know how to deploy the .NET Framework runtime - a problem fixed in the forthcoming 2005 release - but it still has some nice features. It is in effect a visual editor for Microsoft Installer databases, the .MSI files you will be familiar with if you work with Windows. An installation engine that understands .MSI files is built into Windows, including automatic repair and some great features for administrators rolling an application out across an entire network. The Microsoft Installer is therefore is the officially preferred approach to Windows deployment.

At first glance Visual Studio is a decent MSI editor. I did discover that the dependency detection is not reliable, and for a while I was plagued with duplicate assemblies appearing by magic, but there's a tip to fix this: remove the SearchPath, which is a property of the setup project, and dependency detection will no longer bother you. It is also important to read up on a few critical installer features. At a minimum, you need to understand the meaning of the UpgradeCode, ProductCode and PackageCode, and the implications of Version, RemovePreviousVersions and DetectNewerInstalledVersion. You learn that you never ever change the UpgradeCode, and that Version must be incremented if you want the setup to install over the top of an existing version of your app, and that you should let Visual Studio generate new ProductCode and PackageCode GUIDs when it wants to. You also learn to beware of installing things like database or configuration files with your setup. Here's how it can go wrong. You install somedata.mdb and someconfig.ini with the first cut of your application. Works great. Now here comes the next version. If you have RemovePreviousVersions set, the installer will handily delete the user's database and config files before installing brand new versions. Probably not what you want. You can get round this by setting the Permanent property to true for a particular file, which stops the Installer from removing it, and then setting a Not Installed condition so that it doesn't get overwritten. However, there are bugs in some versions of the Installer that make the Permanent setting ineffective. Bottom line - don't go there. I resorted to a little custom action that copies the files I never want to overwrite seperately from the installer engine.

Ah, custom actions! These guys let you do all kinds of things that would otherwise be impossible. Visual Studio even has a Custom Action editor, which you get to by choosing View - Editor - Custom Actions when a setup project is active. I used this to fire up my utility for copying data files, but I ran into a more difficult issue. I decided it would be nice if users had the option to run the application automatically at the end of the install. Step one was to add a dialog with a checkbox offering the option, and that worked fine. The dialog, that is. I could not get the app itself to run at the right moment, that is, right at the end of the install. The best I could manage was to have it run before the end of the install, which is horrible.

Let me introduce you to Orca. This natty utility is installed with the Windows SDK; it is another MSI editor. Unlike Visual Studio though, Orca gives you the raw unsullied view of the Installer tables. So how many tables does it take to install a few files? A lot, apparently. Another snag is that MSI is a relational database, and many of the keys are GUIDs. Orca gives a table by table view, and working out which GUID in one table applies to which file in another table is a bit of a strain. Still, anything you can do with the Installer, you can do with Orca. After a bit of Googling, and a bit of ferreting through the Installer SDK, I worked out how to create a custom action of type 210 that would fire up my application, and added a DoAction to the ControlEvent table that runs the custom action when the user clicks Close on the last dialog. There are a few other tweaks I made in Orca that I won't go into now. Enough to say that by tweaking my install with Orca I could get it running as I wanted.

Orca - no prizes for UI, but still a powerful MSI editing tool. Can you spot that the Dialog column actually has a trailing underscore?

However, there is a flaw in the Orca plan. Every time you build your setup, your changes are lost. When you get to the point where you are adding or editing 6 or 10 lines in Orca to make your install work properly, you find you have introduced a tedious and error-prone extra step. Clearly I had to find a way to automate the process. You can't automate Orca itself, but you can use the same API that Orca uses. This is an unmanaged API, so I started up VB.NET, dug out MsiQuery.h from the Platform SDK, and got to work adding PInvoke declares for the main MSI functions. Google suggests that while I am not the first to do this, there are not many others that have gone down this route. There's actually an example in MSDN, but it is not very helpful. It shows how to open an MSI database in code, and how to close it. Nothing about doing anything useful in between.

The MSI API doesn't look too bad at first. It understands SQL, so I figured all I needed to do was to fire some SQL queries and statements to make the changes I needed. I know SQL pretty well, but this didn't prove to be much advantage. First, I found I could execute SELECT * FROM sometable, but not much else. I kept getting an invalid syntax error, for queries that looked fine to me. To be more accurate, I got error 1615 which is the standard Windows error code for ERROR_BAD_QUERY_SYNTAX. Well, it turns out that MSI has some special SQL requirements. To have success, you need to surround field and table names with grave accents, as in `ControlEvent`.`Action`. All the names are case-sensitive too. Once I discovered this, things went better, but I was still getting syntax errors on a particular table. As the night wore on, I was studying the table in Orca, comparing the column names with those in my query, not finding any errors and getting more and more frustrated. Then I looked up the table in the SDK. I made the painful discovery that two of the column names have trailing underscores, as in "Dialog_". In Orca, the trailing underscore is pretty much invisible.

Most things now worked, but I was still getting syntax errors on a particular update statement. I narrowed it down to the column I was trying to update, which was VersionMin in the Upgrade table. I could update other columns such as ActionProperty without any issue, but not VersionMin. This time the error was 1627, also known as ERROR_FUNCTION_FAILED. Indeed. In desperation, I replaced one line of SQL with several. Retrieve and store the column values, delete the row, build the SQLto insert a new row, insert it. To my relief, that worked. I still don't know why the other Update kept failing. By the way, even if you open an MSI database with MSIDBOPEN_DIRECT, rather than MSIDBOPEN_TRANSACT, you still usually have to call MsiDatabaseCommit before changes are written to the file. Just thought I'd mention that.

Finally, I noticed a problem that arises if the Installer finds some files in use and prompts for a reboot. This occurs after the normal last dialog. Unless you take steps to prevent it, your app will attempt to fire up at the same time as the Installer is asking for a reboot. So I added a condition that checks the ReplacedInUseFiles property, which indicates whether the installer will ask for a reboot. Didn't work - because that property is not availalble in the UI sequence. Another smart installer feature. So I added a new global property of my own along with a custom action consisting of one line of VB Script. This action checks the Session.Mode(6) property, which also indicates if a reboot is needed. If it is, the VB code sets my new property to "Yes". I then use my global property as a further condition before executing my app.

So now I'm done. I just build the setup project, then run my VB.NET application to modify the tables. What's more, it would be trivial to make further modifications, encouraging me to explore further. But after all this, I guess the obvious question is: why did I bother? Why not just use InstallShield, or any one of several more straightforward and intuitive installation tools, some of them free?

I'm not altogether sure, but here's a few considerations. First, I like the idea of using the proper Windows installer service. Second, I tend to the view that less is better, and I'm happy to be without the extra layers that the likes of Wise and InstallShield impose on Installer packages. Third, having made this effort I'm hoping that everything will now work smoothly; certainly the build process is now very manageable. Fourth, I still enjoy a challenge.

A few questions remain. The chief one is: why does Microsoft make it so difficult? Certain tasks are easy enough, such as editing the registry, creating shortcuts, and of course copying a bunch of files. Other tasks are near-impossible, unless you take steps such as those outlined above. Another challenge is discovering which is which. Where is the document that clearly lays out what you can do with a Visual Studio setup project, and what are its limitations? Here's hoping for better things in Visual Studio 2005.

Copyright Tim Anderson 9th September 2004. All rights reserved.

Copyright ©2004 Tim Anderson


 
 
40 comments
Comment posted on 2012-02-16 22:35:50 by: LK.
Wix sucks. Don't use it.

Microsoft should get it's shit together and work on a good installer editor rather than doling out shit like WiX

Comment posted on 2010-08-30 19:49:21 by: Anna Mercken.
Good article. What good is a tool when you have delve to these levels to get it to handle your needs? Someone at Microsoft needs to rethink this tool as well as the documentation.
Comment posted on 2009-01-29 16:23:01 by: Thomas Wamenthal.
Dear Praveen Kella. Have you tried rebooting the PC? Usually this helps when I face the similar experience.
HTH.

Comment posted on 2008-06-26 19:41:06 by: Jason.
Did Visual Studio 2008 fix any of these drawbacks?
Comment posted on 2008-05-15 04:46:17 by: Steve Friedl.
Thank you for this post: finding out that `Tabname.Field` doesn't work but `Tabname`.`Field` does has put an end to a frustrating two hour debug fest. ~~ Steve
Comment posted on 2007-12-11 01:19:59 by: Sijmen Grunbauer.
Hi,

I have spent 9 years writing install programs. Do yourself a favour and give up on Visual Studio for install programs. For anything more that the simplest install, you will get much more power and flexibility with specialist install production tools such as InstallShield or WiseInstall. In the long run you will save time and money using them.

It is very disappointing that Microsoft have not put the effort into producing a decent install tool. In my opinion we would be better served if they did provide more useful tools, or spend the effort in other areas which lack functionality. What they've given us is a false belief that they have an install tool we can use to write any install program, when the reality is far from that. Even the latest version of Visual Studio 2008 has not improved much in this area.

Sijmen.

Comment posted on 2007-09-05 23:47:56 by: GIS.NET.
Hi, I am new to vb.net. I have developed a GIS application in .net. Now I am trying to develop a deployment project to install on the users machine.

Part of this task is, I need to retrieve the location of the .msi file from the deployment project programatically. The application is throwing error when I am trying to retrieve configuration properties of the deployment project(setup project). Can any help me find a solution to this?

I also want to delete "installation folder" from the userinterface or disable the browse button and options buttons and install the application on "all user" profile in "Application Data" folder. I also want this application available to all users. Here my problem is I do not know how to make the application available to all users, if I delete the "installation folder" interface from user interface editor. Since "installation folder" interface has user only option choosen by default, How can I programmatically make the application available to all users.

Comment posted on 2007-08-12 19:44:29 by: Yoni.
Markus:
Perhaps you used "Save As". Use "Save" instead.
See:
http://zeroflag.wordpress.com/2007/08/04/save-and-save-as-dont-do-the-same-thing-in-microsofts-orca/

Comment posted on 2007-07-19 19:10:18 by: Markus.
So I have a nice .msi and would like to use Orca to set the REBOOT property. fine. However, my .msi has two CA which invoke exe applications.
When I simply use Orca to open the .msi and save the file size gets smaller and the package is missing the .exe - at least it reports an error when they are invoked. Any ideas?

thanks, Markus

Comment posted on 2007-05-30 13:33:36 by: Kamal.
Hi Tim,

I have a question here. I am using Visual Studio 2005 Professional edition to create the MSI Installer. Once the installation is done, how do we get a detailed report of what all files are installed, what all modifications are done to Registry.

Regards,
Kamal

Comment posted on 2007-02-04 17:19:46 by: Paul Hoepping.
Hi,
it seems I will have to implemente something similar to what you did. For my application I have created to separate installs:

- Full install including required Microsoft components and a few database files that are mostly static. All together this package consists of about 110 MB data.

- Update install including the program files I am continuously working on. This package has a size of about 10 MB.

It works fine with just one small but really annoying problem: Both installs contain a database file that must be updated by any subsequent install. This file does not have a version number and the file date is updated every time the user runs the application. The update install must replace this file even though the target has a version of the file with a file date newer than of the file contnained in the installer package.

Using Orca I can set that this file is a companion to the main assembly of my application. After doing that the installer will replace the file if the main assembly of my application has changed.

Creating a Transform MSI .MST file does not help. With the MST I can paste in a static value but I cannot tell it to look up and copy a specific value from another row in the same table.

Thanks for the interesting article.
With best regards,
Paul Hoepping

Comment posted on 2007-02-01 10:10:01 by: Eugene.
Can you please tell me how to string a previously made installer after or before a custom installer using wix
Comment posted on 2006-12-19 08:26:58 by: Adrian.
To change the option from Just me to Everyon, in the windows installer follow these steps:
1. Download Orca
2. Edit ..\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools\Deployment\VsdDialogs\1033\VsdFolderDlg.wid
(1033 is for english language)
3. In Orca, on the Tables column, edit Property row...on the right you will see FolderFrom_AllUsers set to ME...just click on ME and type in ALL.
This should modify the template used by VS2003 to make setup.

Comment posted on 2006-12-08 07:29:38 by: Kshitiz.
Hi Tim,
The article is very good and gives a broad view about MSI.
My concern is little bit different.
I have made a MSI with VS.NET 2.0.When i update some files at my end like changes made in a file or two, now want to update this part to client end directly without any new setup(MSI).(Like creating a batch file or something by which user click once and get updated with fresh version)
How can i do this.
I would highly appreciate if i can get a best way to resolve this issue from your side

Thanks in advance
Kshitiz

Comment posted on 2006-10-12 12:52:57 by: saban.
thanks
Comment posted on 2006-10-02 17:16:28 by: Tom.
You mention losing all of your custom edits to the .msi file when you re-build you install project. I had the same problem but then realized that orca has a build in feature that fixes that problem. In Orca if you open an msi, go to the Transform menu option and select 'New Transform' now make all the changes you'd like to the msi file through orca and then select 'Generate Transform'. Orca will save all of your changes to a special .mst file. Then, in order to use the transform, open your msi again, select "Apply Transform" and pick the .mst file you created. All of your changes are made to the msi file again, and then you have the option to save the transformed msi file. This is a good solution if you need to repeatedly build you install project in Visual Studio, plus it has the benefits of not actually changing the original msi file - it will make you save the transformed version under a different name
Comment posted on 2006-07-26 06:20:02 by: jokiz.
great stuff you got here... although i want a more detailed documentation for orca if there exists one. thanks for sharing!
Comment posted on 2006-07-26 02:19:56 by: Nas Raja.
Orca stand-alone (hacked) install package:
http://blogs.msdn.com/astebner/archive/2004/07/12/180792.aspx

Comment posted on 2006-05-24 03:07:28 by: Toby Ovod-Everett.
In case you're curious, I ended up going through the entire Components table modifying the applicable conditions to say things like "1 OR <previous condition>" and putting that in a transform. Yeach! I have been forced to resort to unspeakable depravities because Microsoft can't be bothered to conform to the spirit of their own initiatives.

--Toby Ovod-Everett

Comment posted on 2006-05-24 01:20:19 by: Toby Ovod-Everett.
Can anyone confirm for me that the "standard" way the VS .NET creates an MSI with "optional" components is via this whole CustomCheck[ABC] dialog crap? I'm about ready to draw and quarter both which ever MS developer decided that they had the right to pervert MSIs in this manner and whatever idiot approved it. I've wasted the last two hours attempting to figure out what magical incanctation of an MST file will convince this VS .NET created monstrosity to install the optional components.

The whole point of MSI was that it might provide a little standardization and automation for us poor, beleagured admins. So the VS.NET geniuses decided that God had appointed them to ensure that MSIs would be useful for silent deployment of VS.NET apps . . .

Aiiiii. Too many idiots, not enough bullets.

--Toby Ovod-Everett

Comment posted on 2006-05-15 18:55:44 by: Valery.
I tried to use Orca to put more control on predefined Windows Installer dialog boxes but like you I have got the original msi installer database and my additional controls disappeared.<br />
I think best way to change default VS setup project original installer database with extended set of dialog boxes and controls.
Do you think about it?
My problem is how to add controls on existing dialog box of windows installer user interface.

Valery

Comment posted on 2006-04-05 11:58:41 by: Mick Weiss.
Whenever I have created a .WOS (virtual os) file on a clean machine my work machine declares that the virtual os file is invalid. Also when repackaging largish installations (about 600Mb and above) I reliably get knocked out by spurious sharing violation. It's very hard to obtain information regarding either of these issues.
Comment posted on 2006-03-29 17:29:05 by: a.
What I think of MSI installer packages? Videogames, a little bit like GameBoys, only crappier.
Comment posted on 2006-03-29 11:21:35 by: ram.
thank u
Comment posted on 2006-03-03 07:54:19 by: Unknown.
It's not Wrestling! It's a suicide! I couldn't handle it!
;-)

Comment posted on 2005-09-12 16:33:56 by: duncan Tamsett.
I am new to Windows Installer. This may be a trivial one, but it is bugging me at the moment. I can get a shortcut to an exe to appear on the desktop, and an entry in the list of programs from the Start button, but neither is connected to the .exe that they are supposed to be a short cut to. Nothing happens when they are activated. How do you get the shortcut/prog entry to connect to the exe? Thanks. Duncan
Comment posted on 2005-08-23 00:32:13 by: eric g..
For those having problems with MsiDatabaseCommit() not writing any data when called from a C# or VB progam, check the construction of the szPersist argmument to MsiOpenDatabase(). From the declaration in MsQuery.h it looks like it wants a string, e.g. "2", but in reality it needs an IntPtr set to the value of the desired open mode, e.g. IntPtr(2). A wrong value causes the 'commit function to be silently ignored. Looks like MS's attitude towards error handling hasn't changed much over the years.

HTH,

Eric G.

Comment posted on 2005-08-22 19:33:45 by: Eric.
Anyone have a suggestions regarding what to do when MsiDatabaseCommit() doesn't write anything and doesn't return an error? I've check the code for things like proper open mode, checking all returns for errors, closing all handles and reading back my row update to confirm that the change is in memory. It just doesn't update the .msi file and it doesn't give any error message.

TIA,

Eric G.

Comment posted on 2005-08-20 10:36:16 by: Tim Anderson.
Downloading Orca - this is from MSDN:

"Orca is a GUI .msi file editor that is provided with the Windows Installer SDK. It provides full access to the Windows Installer database tables.

"A set of Help files are provided with Orca. To obtain the orca.exe and Orca documentation you must download and install the Windows Installer SDK, which is provided as an Orca.msi file. After you install the Platform SDK Update, double click the Orca.msi file to install Orca.

"Note It is not possible to download Orca separately."

Comment posted on 2005-08-20 09:54:38 by: ShOcK3R.
where cna i download orca???
please reply

Comment posted on 2005-02-18 17:06:56 by: Robert Love.
Very useful article thanks. I adopted the same approach, except I used VBScript to modify the MSI database. I strongly recommend "The definitive guide to Windows Installer" by Phil Wilson -- contains lots of information about MSI as viewed through Orca, and generally provides excellent support and tips for this approach.
I'm happy to share my scripts. My email, which doesn't fit in the box, is robert.love@object-oriented.com

Comment posted on 2005-01-01 02:22:46 by: Nathan Schopf.
Why is this not a bigger problem in the development community?

I recently derived a strategy very similar to yours. I thought I might be searching too far out of the box for a solution. It's nice to see others comming up with the same ideas.

Comment posted on 2004-10-18 15:49:16 by: Praveen Kella.
Hi,

This is praveen kella, I am facing a problem with MSI files. I have created two setups with same Targetpath. these two setups installs the same application. But when I uninstall one setup the files in the target path are deleted. when I try to uninstall the second setup this set up searches for files in the target the fact is those files has been deleted by first setup.

please help me on this.

Comment posted on 2004-10-06 17:55:08 by: Randy.
Great article. I have shared your exact experience when it comes to the limitation of the VS.NET installer. I now use Orca to change my install after I've built it. The real question is, "Why doesn't Microsoft expose just a little bit more of the actual msi script in the VS.NET editor?" For example, the "Everyone" and "Just Me" option? This certainly does not apply in all cases. Also, the ability to prevent a reboot. Come on! Some of the installs we need to create are called by other installs. That's just a plain mistake on MS's part. Anyway, I'm probably preaching to the choir here.
Comment posted on 2004-10-05 09:38:55 by: nGear.
Maybe the ERD of the database in the .msi file, would clear up some things, it did for me (plus, it also shows all the trailing underscores): http://msdn.microsoft.com/library/default.asp?url=/library/en-us/msi/setup/about_the_installer_database.asp
Comment posted on 2004-09-27 12:07:24 by: Chris.
I think that as VersionMin in the Upgrade table is a key, it cannot be updated, which is the same for most of the columns in the ControlEvent table.
Comment posted on 2004-09-13 23:53:08 by: MaSaI Solutions.
You might also want to take a look at MaSaI Editor which is a free replacement for orca allowing direct table access ( like orca ) as well as a graphical editor ( for adding files, registry keys, etc ) and file compression. Should make things much easier for you.
Comment posted on 2004-09-11 13:15:41 by: Joe.
I use WiRunSql.vbs (from the Installer SDK) to automate post-build updates to the MSI file. Once you've worked out the SQL syntax it's easy.
Comment posted on 2004-09-10 23:03:52 by: Dennis.
Hi,

With my MSIDIFF tool you can do a compare between a original version of an MSI and one you might have manually edited to dump the differences, tweek this a bit and run it back though MAKEMSI to automate any following updates. Or if you already know what you are doing you can also of course skip the MSIDIFF step.

Comment posted on 2004-09-10 13:37:01 by: Issam Elbaytam.
Have you looked at WiX and WiXTool? Two open source projects (WiX from MS) on sourceforge.net.
http://sourceforge.net/projects/wix/
http://sourceforge.net/projects/wixtool/

Comments are closed.