Files, Directories And Disk Quotas In Statistics

For general and detailed questions about the CSL/Stat file system, including how to recover lost files, how to set permissions, how to view remaining quota space, etc., go to the CSL Documentation pages and follow the Files, File Sharing, Directories, File Systems link.

Locations of Important Directories on Linux Systems in Statistics

Your Home Directory on Linux

You can get to your home directory by entering cd ~ at the Linux command line. The full path to your home directory is:

/afs/cs.wisc.edu/u/<1st char user ID>/<2nd char user ID>/<user ID>

For example, for John Q Doe with login jqdoe working on public06, to change to his home directory, he would enter:

public06% cd ~
or
public06% cd /afs/cs.wisc.edu/u/j/q/jqdoe

For typical uses, the /afs/cs.wisc.edu prefix is not required, so the path to John Q Doe's home directory can be shortened to:

public06% cd /u/j/q/jqdoe

Limiting Access to Your AFS Directories

Permissions to access your AFS files are controlled by AFS Access Control Lists (ACLs). Permissions in this case are NOT controlled by standard Unix mode commands. Information on setting permissions can be found at the CSL AFS File System Docs under Access Control And AFS Groups. The default setting is to allow anyone to list/view the names of files in all directories (except ~/private), though other users are not generally permitted to read the contents of your files or write them (except as noted below) unless you specifically change ACLs permissions. If you don't want others to be able to list/view the names of the files in your AFS directories and subdirectories, be sure to set your AFS (ACLs) permissions for system:anyuser to "none". See additional information in the AFS File System Docs.

Your Public Directory on Linux

Users have a directory called public in their home directory where they can typically put files they want to share with other users. In this case, you may want to be sure that you allow list and read of your public directory (~/public for system:anyuser to rl) for sharing of files. This is the default setting.

Another User's Public Directory on Linux

You can get to another user's public directory by entering the cd ~/public command. For example, for you to get to John Q. Doe's public directory, use:

public06% cd ~jqdoe/public

Of course, the user jqdoe would have needed to set his AFS access ACLs for system:anyuser to rl (or leave the default setting). The CS 1000 document includes more information about Accessing Other People's Files

Your Personal Web Page Directory on Linux

An html subdirectory is typically created in the user's public directory as the standard location to put html code for the user's personal web pages. These pages are then accessed by the web address http://www.cs.wisc.edu/~/ , for example http://www.cs.wisc.edu/~jqdoe/. Be sure that you allow list and read for your web/html directory (~/public/html for system:anyuser to rl) (this is the default). Otherwise your web pages will not be accessible to the outside world.

More information about building user web pages can be found at the CSL Web Server FAQ page

Using the Local /tmp and /scratch Directories on Statistics Linux Computers

Statistics computers are configured with /tmp and /scratch directories on the local hard disks. These areas can be accessed by users logged into a specific machine for temporary storage of files on that machine. /scratch is typically larger (over 100Gbytes) than /tmp (typically 1 Gbyte). On public computers, the /scratch and /tmp directories can be used by anyone that logs into the machine (public02 for example). For "private" desk and research servers, the local /scratch and /tmp directories can only be accessed by the users assigned to that machine. These directories, especially /scratch, are useful for temporarily storing large files. Example uses include storage for CD and DVD images during CD and DVD write operations (see Writing CD-Roms On Linux Workstations) and temporary storage of large data or image files locally where original copies are available at another location.

Limiting Access to Files and Directories in /tmp and /scratch

Unlike the AFS file space, where permissions are controlled by ACLs, the /tmp and /scratch files are controlled by the standard Linux mode permissions. When creating new files and directories in /scratch (or /tmp), the access mode settings are controlled by the default setting of umask. Unfortunately, the current setting of umask is not necessarily consistent. You can see what the umask setting is by executing the command:

umask

and you may get a result like:

22

That is a "bad" default since it allows other users to both list the names and read the contents of files in directories you create in /scratch. A "good" number is 27. You can temporarily change the umask setting for the current terminal session so directories you create in /scratch have the correct restrictions. Execute the command:

umask 27

To consistently keep the umask setting at your preferred value, add the above line at the end of ~/.cshrc.local (or ~/.bashrc.local if using the bash shell)

To protect your files that you already created in these directories, you must explicitly check and set the mode for "other" users to be no access. For example, for a directory created by jqdoe at /scratch/foo with umask set to 22, the command would be:

mkdir /scratch/foo

and listing the directory might show:

ls -al /scratch

drwxr-xr-x  2 jqdoe jqdoe  4096 Jun 16 17:19 foo/

with the trailing r-x indicating that all other users have read and execute/list access to the files within the /scratch/foo directory. To limit access, jqdoe can use the command:

chmod o-rwx /scratch/foo

to remove r (read) w (write) and x (execute) access to foo for o (other) users. The new listing will look like:

ls -al /scratch

drwxr-x---  2 jqdoe jqdoe  4096 Jun 16 17:19 foo/

with the trailing --- indicating that all other users have no read, write or execute permissions in the /scratch/foo directory

Linux File System WARNINGS!

1. /scratch and /tmp files are NOT backed up, so data stored there can be lost permanently at any time.
2. If you accidentally delete a file from /scratch or /tmp, it is gone forever. There are no recovery mechanisms as are available for the AFS file systems.
3. /scratch and /tmp directory spaces can be arbitratrily cleaned out at any time if there are machine problems or if machines need to be rebuilt. This is especially true of public computers. For private computers, there may be some attempt to contact the owner before removing files, but that is not guaranteed.
4. /scratch and /tmp files are not easily accessed remotely. They are not mounted and available on the network as are the AFS file systems. They can be accessed with secure ftp from remote locations by users that have valid logins to specific machines.
5. DON'T USE /scratch or /tmp for storing anything important for which you have only one copy.

Locations of Important Directories on Windows Systems in Statistics

Your Home Directory on Windows

On Statistics Windows computers, your home directory is on the U: disk. If you open My Computer or Explorer, double click on the U: disk and you will be in your home directory.

Another way to get to your home directory is through the W: disk. If you double click on the W: disk, you will see the root directory of the CSL/Statistics AFS file system, equivalent to /afs/cs.wisc.edu/ in Linux. From there, you can double click the u folder, then double click the <1st char of your user ID> folder, then double click the <2nd char of your user ID> folder, then double click on your folder. For example, if your user ID was jqdoe:

1. Double click My Computer
2. Double click W: disk
3. Double click u folder
4. Double click j folder
5. Double click q folder
6. Double click jqdoe folder

Another User's Public Directory on Windows

Since there is no equivalent to the Linux ~ (tilde) function in Windows, you need to go the "long way around" to get to another user's public directory. For example, to get to John Q Doe's public directory, with user ID jqdoe, you would follow the steps 1 - 6 above, and then:

7. Double click public folder

Using the Local TEMP Directory on Statistics Windows Computers

The TEMP directory is located on the C: disk. You can use the C:\TEMP folder to store information locally on the Windows computer. But the information will stay there ONLY for the current session. When you log out, the information will be lost. For example, you can use C:\TEMP to store a CD or DVD image for recording, but only if you burn the CD during the current session (before logging off). If you need C:\TEMP to temporarily store images or results of computations, you MUST transfer the information out of C:\TEMP before logging out, or the information will be lost. Generally, it is not a good idea to use C:\TEMP.

An example of using C:\TEMP to write a CD or DVD in Windows is given in the Writing CD-Roms In The Public Windows 2000 Lab instructions page.

Disk Quotas in Statistics

Standard disk quotas in Statistics are:

1 Gbyte for student accounts
2 Gbyte for faculty accounts

These quotas are fixed and cannot be adjusted. However, it is possible to purchase additional AFS disk space that can be made available on the /p AFS volume. This space can then be used directly from the /p volume or it can be linked to locations in your home directory. Cost of additional AFS disk space is $6 per Gbyte per year. For more information about adding disk space to your account, please contact Denise <nop>Roder.

Freeing up disk space if you are nearing or have reached your quota

If you are nearing or have met your quota, please check your .Trash folder. If .Trash contains large, or many files, remove them by executing the following:

Change directory to your trash folder.

cd /afs/cs.wisc.edu/u/<first>/<second>/<username>/.Trash

rm -fr /afs/cs.wisc.edu/u/<first>/<second>/<username>/.Trash

Where

<first> 

<second> 

-- MikeC - 16 March 2010