Wednesday, January 24, 2007

Finding the Paths for Special Folders

Storing an application's data files in the correct place has gotten trickier under Windows Vista. Most developers simply write to INI files stored in the application folder (usually C:\Program Files\Some Folder) and store the data files in a Data subdirectory of that folder. While Microsoft strongly discouraged this practice in the past, now it's enforced; even if you're logged in as an administrator, writes to certain protected resources, such as C:\Program Files or the HKEY_LOCAL_MACHINE hive in the Registry, fail.

This would break most "legacy" apps (anything written before Vista), so to prevent that, they're run in XP compatibility mode. Through a process called virtualization, writes to protected resources are redirected by the operating system to somewhere else. For example, trying to write to a file in C:\Program Files\Some Folder causes that file to be created or updated in C:\Users\username\AppData\Local\VirtualStore\Program Files\Some Folder instead.

While it's great that Microsoft prevents our applications from breaking, this really just a short-term solution. It's better to revisit where your data is stored and use one of the preferred locations, such as C:\ProgramData for global settings and C:\Users\username\AppData\Local for user-specific data. The problem is that your shouldn't used a hard-coded location for these paths, especially if some users are on Vista and some on XP. Instead, use Windows API functions to determine those locations.

SpecialFolders.PRG is a wrapper for these functions. It returns the appropriate values regardless of whether the application is running in Vista or XP. See the comments in the header for a description of what parameters to pass. This code doesn't handle all special folders, only the more common ones, but it could easily be updated to support other folders as well.
* Program: SpecialFolders.PRG
* Purpose: Determine the path to the specified special folder
* Author: Doug Hennig
* Last revision: 01/24/2007
* Parameters: tuFolder - the folder to get the path for. Specify
* the CSIDL value of the desired folder (which can
* be obtained from:
* or use one of the following strings:
* "AppData": application-specific data
* "CommonAppData": application data for all users
* "Desktop": the user's Desktop
* "LocalAppData": data for local (nonroaming)
* applications
* "Personal": the My Documents folder
* Returns: The path for the specified folder or blank if the
* folder wasn't found
* Environment in: None
* Environment out: Error 11 occurs if tuFolder isn't specified
* properly
* Notes: This code was adapted from:
* Support for other CSIDLs can easily be added

lparameters tuFolder
local lcPath, ;
lnFolder, ;
lcFolder, ;
lnPidl, ;
lnPidlFound, ;

* Define the CSIDLs for the different folders.

#define CSIDL_APPDATA 0x1A
* Application-specific data:
* XP: C:\Documents and Settings\username\Application Data
* Vista: C:\Users\username\AppData\Roaming
* Application data for all users:
* XP: C:\Documents and Settings\All Users\Application Data
* Vista: C:\ProgramData
* The user's Desktop:
* XP: C:\Documents and Settings\username\Desktop
* Vista: C:\Users\username\Desktop
* Data for local (nonroaming) applications:
* XP: C:\Documents and Settings\username\Local Settings\Application Data
* Vista: C:\Users\username\AppData\Local
#define CSIDL_PERSONAL 0x05
* The My Documents folder:
* XP: C:\Documents and Settings\username\My Documents
* Vista: C:\Users\username\Documents

* Define some other constants.

#define MAX_PATH 260
#define NOERROR 0
#define SUCCESS 1

* Test the parameter.

do case

* If it's numeric, assume it's a valid CSIDL; if not, the API function will
* return a blank string.

case vartype(tuFolder) = 'N'
lnFolder = tuFolder

* An invalid data type or empty folder name was passed.

case vartype(tuFolder) <> 'C' or empty(tuFolder)
return ''

* If a string was passed, convert it to the appropriate CSIDL.

lcFolder = upper(tuFolder)
do case
case lcFolder = 'APPDATA'
case lcFolder = 'COMMONAPPDATA'
case lcFolder = 'DESKTOP'
case lcFolder = 'LOCALAPPDATA'
case lcFolder = 'PERSONAL'
return ''

* Declare the API functions we need.

declare long SHGetSpecialFolderLocation in shell32 long hWnd, long nFolder, ;
long @ ppidl
declare long SHGetPathFromIDList in shell32 long Pidl, string @ pszPath
declare CoTaskMemFree in ole32 long pvoid

* Initialize the variables the API functions will update.

lcPath = space(MAX_PATH)
lnPidl = 0

* Get the path of the specified folder.

lnPidlFound = SHGetSpecialFolderLocation(0, lnFolder, @lnPidl)
if lnPidlFound = NOERROR
lnFolderFound = SHGetPathFromIDList(lnPidl, @lcPath)
if lnFolderFound = SUCCESS
lcPath = left(lcPath, at(chr(0), lcPath) - 1)
endif lnFolderFound = SUCCESS
endif lnPidlFound = NOERROR
lcPath = alltrim(lcPath)
return lcPath


Anonymous said...

There's an alternative means of accessing these using the shell.application helper object:

oShell = createobject("Shell.Application")
oFolder = oShell.Namespace(CSIDL_LOCAL_APPDATA)
oFolderItem = oFolder.self
cPath = addbs(oFolderItem.path)
? cPath

Doug Hennig said...

I don't think the Shell.Application approach works on Vista; it returns the same value for me (C:\Users\username\AppData\Local) regardless of what value I pass.

Anonymous said...

What about per machine data like a checksum value for an exe? The Common Application Data folder (i.e. All Users/Application Data on XP) is no longer readable except with Admin privileges in Vista.

I can't find any way to do per machine data storage on Vista while running as a Standard User.

Doug Hennig said...

The only place I've found so far when a non-admin user can read and write is \Users\Public, but I need to do more experimenting.

Anonymous said...

Any further results on your experimenting on where a non-admin user can write, Doug?
I had hoped to put my app data in C:\ProgramData but m.mcculloch reports that the on-admin user cant read there (and obviously will be prevented write access)

My next option for my app data is c:\Users\username\AppData\Local
However, alarmed by your posting:
The only place I've found so far when a non-admin user can read and write is \Users\Public
Can the user not write to his/her own appdata\Local folder?

Doug Hennig said...

I should have been more clear, George. I meant the only common location (ie. not user-specific) is \Users\Public. A user can write to their own \Users\username folders.

Anonymous said...

Thats cleared that up Doug, and it does certainly make sense.

I'm still deciding where to put my data in distributed apps.
I would appreciate clarification on the C:\ProgramData folder.
Per m.mcculloch posting, compared to equivalent in XP, this "is no longer readable and requires Admin privileges".
My app data wise, what, if anything, can I put in there?

Doug Hennig said...

I don't recommend C;\ProgramData because you can't store read-write data there. My recommendation is still to use C:\Users\Public.

Anonymous said...

To build this into my app, I'd like to use your SpecialFolders.PRG program to return the equivalent in other Windows versions.
Is there a CSIDL associated with c:\users\Public ?

Doug Hennig said...

There isn’t a CSIDL value for this folder, but the environment variable PUBLIC points to it, so you can use GETENV('PUBLIC') to determine its location. By default, this variable contains C:\Users\Public in Windows Vista. The variable doesn't exist in Windows XP, so your code must handle the case where GETENV('PUBLIC') returns a blank value.

Alternatively, you could use the parent of the folder specified by CSIDL_COMMON_DOCUMENTS, which gives C:\Documents and Settings\All Users\Documents on XP and C:\Users\Public\Documents on Vista.

Unknown said...

I was following the link in your code's comment section and found that the CSIDL codes that were being used are being superceded in Vista by KNOWNFOLDERID values, which are supposedly in a file called knownfolders.h. However I cannot find any reference for locating the file itself and I was curious as to whether the new codes were superset of the old ones or completely different. Also, do you know if there is a code for the "shared documents" folder? Can't seem to find that either and it's a place I would like to offer as an option when a user is first initializing a database folder. Thanks!

Anonymous said...

In your June/July Advisor article you suggested that one good place to put the read-write global data might be in the parent folder of COMMON_DOCUMENTS.

Here is the problem. When using an Installer such as InstallShield or Advanced Installer to create the .msi file how would one designate the parent of a "special folder"?

Even if one went with the "Public" folder, how would one designate this within an Installer especially since there is no CSIDL value?

I assume that there is probably a way but it doesn't look very obvious when looking around the two installers mentioned earlier.



Doug Hennig said...

Ed, I've changed my mind about where to store global R/W data. I now put it into a Data subdirectory of the app folder and have the installer give R/W permissions to that folder. The reason I do that is because our app could be installed on a server or a local workstation and I need a location I can count on existing in either scenario.

Anonymous said...

On Vista instead of using SHGetSpecialFolderLocation you should make use of SHGetKnownfolderPath(). It's a bit more complicated as it eg expects some little Memory Management of the caller and if you want to display the localized path to the user you also need SHGetLocalizedName() and some more API calls.

Doug Hennig said...

Hi Olaf. What's the benefit of calling SHGetKnownfolderPath over SHGetSpecialFolderLocation?

Anonymous said...

c/user /username/????? appdat file where the heck is it???

Anonymous said...
This comment has been removed by a blog administrator.
Unknown said...

Does all of this still hold for Windows 7?

Doug Hennig said...

Hi Art.

Yes, this still applies in Windows 7.


Unknown said...

Doug, I hope you don't mind if I ask for some clarifications. I really appreciate all that you offer, and I want to make sure I do this properly.

Among "AppData" "CommonAddData" "LocalAppData" "(anything else?)", which should be used for storing R/W data...

(1) that anyone on my network can use?
(2) that anyone on my computer can use?
(3) that only I can use only on my computer?
(4) that only I can use from any computer?

One of the things my program will do (after installing) is to ask the user where the shared data is located. That piece of information would be stored in a file in #2, above.

Thanks much! Art

Doug Hennig said...

Hi Art. We do something similar: we allow the user to specify where shared data is stored and write that location to an INI file in the Data subdirectory of the program folder. Our installer makes Data writable for all users. That way, whether it's on a network or local drive, our app always knows where to look to find the INI file. So that's what I recommend for #1 and #2.

For #3, use LocalAppData.

For #4, I don't know since I haven't tried that.

Art Lieberman said...

Ok. I'm using an .ini file in "LocalAppData" for the user to specify where the data is located on the network. The data itself would be under "CommonAppData" on some other machine on the LAN. On XP, that would be "Z:\Documents and Settings\All Users\Application Data\MyApp\Data".

But the "Application Data" folder on other computers is not accessible across the LAN. (Other folders under "All Users" are accessible.) Any ideas what I'm doing wrong? A hint as to the proper CSIDL value?

Doug Hennig said...

Hi Art. Not sure. I'm not using CommonAppData in a LAN situation but rather a shared folder on a server.