NeST is a storage appliance: a flexible, scalable, high-performance file server with unique characteristics for use in Grid environments.
NeST is flexible: NeST supports a variety of access protocols so clients can access it in whatever way is most convenient for them. Currently NeST supports GSI-FTP, with some features of GridFTP, HTTP, anonymous FTP, and a custom protocol named Chirp. In the future, NeST will support a variety of back-end storage sytems.
NeST is scalable: NeST uses a unique concurrency architecture to provide high-performance file transfers to a large number of simultaneous users.
NeST is high-performance: NeST provides high-performance through it's dynamic concurrency architecture which selects the appropriate concurrency model (threads, processes, or select) on the fly in order to acheive the best performance.
NeST provides unique characteristics: NeST provides the ability to reserve chunks of disk space, known as lots, so that users have a guaranteed ability to store their data. NeST provides authentication through standard Grid mechanisms, which aids in interoperability with other Grid technologies.
Operating System: Currently nest compiles and runs only on Linux. We have tested with RedHat 6.2 and RedHat 7.2, but we expect it should run with other versions of Linux.
Disks: It is best if you give nest a complete ext2 disk or disk partition to itself. There should be no files on this disk: Nest will manage the disk completely. It is also best if the Nest information directory, where Nest stores run-time information, is not kept on the same disk as Nest manages. If you want to be able to request and enforce lots, you will need to have the quota system configured. (Information about configuring the quota system can be found in the Installing Nest section below.)
Other Programs: Before you can compile Nest, you must have Globus and ClassAds installed. You do not need to be running any of the Globus daemons, such as the Globus Gatekeeper. However, you will need to have a host security certificate installed. More information on installing Globus and ClassAds can be found below.
NeST requires a Globus 2.0 installation. NeST has been tested with the first Globus 2.0 beta release. Unfortunately, while developing NeST, we discovered a bug in the Globus 2.0 beta. We believe this bug will be fixed in an upcoming Globus release. In the meantime, you may notice that if you transfer files to a NeST with the put command using the gsincftp client, you may discover that large files are corrupted. This is a problem with the gsincftp client, not NeST--files are also corrupted when you send them with gsincftp to the Globus-modified wuftp server.
If you haven't installed Globus before, you only need to install the Essential Grid Tools bundle and the Grid API budle. . Install them according to the directions on the Globus webpage.
Once you have installed Globus, you will need a host certificate for each computer on which you want to run a NeST server, and a user certificate for each user who will want to acceess the NeST server. Obtaining these certificates is described in the Globus 2.0 beta directions on the Globus webpage. Also, make sure that you add each user to the grid-mapfile.
Nest relies on the Condor Classad library. Compiling and installing ClassAds is significantly simpler than installing Globus. Complete instructions are contained in the ClassAd tarball.
First, download NeST from the NeST web page. Unpack the distribution with tar:
tar xzf nest.tar.gzNext, run the configure script provided with NeST. You will need to know three things to do this:
cd nest/src
./configure --prefix /usr/local/nest \
--with-globus-path /usr/local/globus \
--with-globus-model gcc32dbgpthr \
--with-classad-path /usr/local/classads
Finally, build and install NeST:
make make installNeST will install several files releative to the "prefix" that you gave it:
When you installed NeST, it places a "nest_config" file in [prefix]/var. You can edit this file to modify the behavior of NeST. The configuration file is suprisingly well documented within the file itself, so we will just touch on two configuration variables that you are likely to need to configure right away.
In order to use lots, you will need to enable the quota system on the disk that Nest controls. Please ensure that no quotas are already set up on the machine in question.
/usr/local/nest/bin/nest_server -c /usr/local/nest/etc/nest_config
If you have the quota system turned on and lots enabled in the NeST configuration file, you can use lots. Each lot is a chunk of disk space that is a reservation. Users must make a lot before they can save data to a NeST server. If you want to use a protocol such as GridFTP, you need to make a lot with the NeST protocol (or the NeST client) first.
Users can have multiple lots, and a single file can span multiple lots if necessary. Lots are currently enforce using the quota system.
This section needs a lot more detail.
Currently, there is no documentation on writing NeST clients. The client API is contained in nest_speak.h, and there are two sample programs you can look at for guidance: nest_client_exe.C, which implements the nest_client program, and tests/basic_api_test.C which tests the functionality of the API.
It may be a bit difficult to compile NeST clients with version 0.9 of the NeST API, because nest_speak.h relies on too many other header files. We expect this to be greatly simplified in the next release of NeST.
For additional reading about NeST, see the NeST website or the source code.