Managing EC2 Cloud Compute Clusters
The SC2IT is an Interface Toolset that wraps around the Amazon Web Services EC2 API. SC2IT serves two purposes:
- Configuring EC2 Cloud Instances into a seamless parallel compute cluster
- Managing and accessing cloud clusters in a simple and intuitive way, using short commands (bash command line) or Java GUIs
The SC2IT.jar Java plugin
We have developed a Java package for launching and managing EC2 clusters. The SC2IT.jar can be added to any existing Java GUI project to add EC2 cloud computing capability. The package includes Javadoc, a Users Manual, and two example files.
The following functionality is offered:
- Launch a cluster with n nodes of machine type a
- Stop and Restart a cluster
- Add/Remove nodes to/from a cluster
- Upload files or directories to a cluster
- Download files or directories from a cluster
- Execute a command on a cluster
- Obtain basic statistics about a cluster, such as uptime, load, and number of nodes
- Open a SSH session in a terminal window to work on the cluster
Credentials are used to verify that the user has permission to access EC2 resources, and to determine who is billed for usage of the resources. The user must configure an AWS user account before getting started. This involves at the minimum choosing a username, setting up payment information (typically a credit card), and obtaining a pair of "access keys". Further credential files are required to use the SC2IT interface tools. They can be provided in the following ways:
- Manually specify all individual keyfiles
- Provide the name of a directory (typically the same name corresponding to an IAM user belonging to the AWS account) containing all required keyfiles with standard filenames
- Let SC2IT create a new IAM user and all required keyfiles for the provided main AWS account
- Attempt to use courtesy "demo" access to the requested resources. Such demo privileges are extended e.g. for new JFEFF users for trial purposes only.
A fairly recent installation of Java is required (1.6_U24 or newer). There are no other dependencies. Required third party libraries are bundled within the SC2IT.jar download.
A coding example is given here to illustrate the ease of using SC2IT in your own Java GUI. A few instructions are omitted for the sake of clarity.
// Launch the new cluster with “cs” specifications:
ClusterResult rl = clust.launch(cs);
// Initialize the FEFF calculation on the cloud cluster:
// Copy feff.inp:
ClusterResult rp = clust.put(LocalWorkingDir+"/feff.inp", CloudWorkingDir+"/feff.inp");
// Run the FEFF9-MPI calculation:
ClusterResult rf9 = clust.executeCommand(Feff9CommandLine,CloudOut);
// Copy the output files back to the local computer:
ClusterResult rg = clust.get(CloudWorkingDir, LocalWorkingDir);
// Terminate the cloud cluster:
ClusterResult rt = clust.terminate();