LSU HPC Policy
LSU HPC Usage Policy
▶ Table of Contents
1. Introduction
The HPC@LSU computing facilities, encompassing its hardware, software, network connections, and data, are a vital but limited resource. Therefore HPC@LSU is obligated to protect its facilities and ensure they are used properly. Allocations are used to assure equitable consumption of system resources, while personal access to the systems is controlled via user accounts. You, personally, are constrained by legal and other obligations to protect resources and the intellectual property of others on the systems.
Given the extensive nature of these resources, responsibilities are attached to the right of access. Responsible user conduct is required to assure fair access by all researchers. Failure to use these resources properly may result in various penalties, including, but not limited to, loss of access, administrative, academic, civil and/or criminal action.
Back to Top2. Requirements
2.1. Individual Account Management
You have the responsibility to protect your account from unauthorized access, and for the proper expenditure of allocated resources. Tools are provided to help you manage your resources, but you are responsible for using these tools properly.
2.1.1. Valid Contact Email
Every account has a contact email address associated with it. Email is used as the primary communication channel between LSU@HPC and the user. The user is responsible for keeping this address up to date. In the event that an address is discovered to be invalid, the associated user account will be locked.
2.1.2. No Account Sharing
Your account is for your use only. It is not to be shared with others; neither students nor other collaborators. Others who need access must request their own account. User authentication certificates, if they are used, are considered personal accounts and not to be shared.
2.1.3. Protecting Passwords
Passwords and certificates are the keys to account access. You are responsible for their protection and proper use. Protective behavior includes
- never sharing passwords
- never writing passwords down where they can be easily found
- never using applications which expose passwords on the network (e.g. telnet).
See Guidelines, below, for more information.
The private key portion of an authentication certificate is the equivalent of a password. If you use certificates, you are responsible for ensuring that file and directory permissions prevent others from reading or copying any private keys.
Back to Top2.2. Fair Use
Access to a system does not grant you blanket authorization for any and all activity. You must recognize that reasonable use may vary from machine to machine within a system, depending on the function of the machine. Machines in a cluster serve one of two basic functions.
2.2.1. Computational Nodes
These are the nodes which perform the intended heavy computational work of a cluster. They are accessed via the job scheduler (e.g. LoadLevel or PBS) through the submission of job control scripts. Computational tasks are never to be run on the cluster head nodes, as this has the potential of impairing operation of the entire system.
2.2.2. Head Nodes
Head nodes are used for the interactive work required to prepare a computational task. Appropriate work on the head nodes includes: light editing, compiling, post analysis previewing, and job script creation. Extensive post processing should be done on the compute nodes. Any processing on the head nodes which adversely impacts system operation is subject to preemptory termination. Repeated abuse may result in the locking of user accounts or other administrative action as deemed appropriate.
2.2.3. Authorized Use
Authorized use of the system is defined by the purpose outlined in the research proposal on which the usage allocation is based. Any resulting use is limited to activities reasonably required to accomplish that purpose.
2.2.4. Unacceptable Behaviors
The following activities are explicitly deemed unacceptable and are subject to the penalties outlined below:
- using, or attempting to use, computing resources without a valid allocation and user account
- usage for purposes other than those stated in the allocation proposal.
- tampering with or obstructing the operation of the facilities.
- reading, changing, distributing, or copying others' data or software without their explicit permission.
- using HPC@LSU resources to attempt to gain unauthorized access to other (non-LSU) sites.
- activities in violation of local or federal law
2.3. Reporting Suspicious Activity
You are responsible for reporting, as soon as possible, any suspicious activity you notice on your account, and exposure or compromise of passwords, passphrases, or certificates. See Section 5 for reporting procedures.
Back to Top2.4. Data Confidentiality
You are responsible to ensure the confidentiality of any data you use on HPC@LSU resources that is restricted from general public access. Technologies are provided to preserve the confidentiality of data, but it is your responsibility to use that technology appropriately. Such data may include:
- Intellectual property, such as report drafts or research in progress,
- Proprietary data, such as data owned by a specific company, or licensed applications.
- Regulated data, such as medical, personal identifying information, or student records.
It is your responsibility to be aware of any requirements on a particular data set.
Back to Top2.5. Acknowledgment
Papers, publications, and web pages of any material, whether copyrighted or not, based on or developed under LSU-supported projects must acknowledge this support by including the following statement:
- This material is based upon work supported by HPC@LSU computing resources.
2.6. Software Development
Software developed with allocations approved by HPC@LSU is subject to the guidelines published by the LSU Office of Intellectual Property, Commercialization, and Development.
Back to Top2.7. Publication
Work performed under a peer-reviewed allocation must be published in the open literature.
Back to Top2.8. Non-Academic User Requirements
Non-academic (corporate/industrial, government, etc.) users frequently have more stringent usage requirements than those that might be provided by HPC@LSU. It is the user's responsibility to assure the resources used satisfy the requirements of their organization.
Back to Top2.9. Software Licenses
All software used on HPC@LSU systems must be appropriately acquired and used according to the specified licensing. Possession or use of illegally copied software is prohibited. Likewise users shall not copy copyrighted software or materials, except as permitted by the owner or the copyright. Some installed software may require special authorization in order to be used. Users must abide by any licensing requirements and protect it from misuse.
Back to Top2.10. Final Reports
Requests for subsequent allocation awards will not be allowed until an end of project report has been received for all prior awards. It is recommended that renewals and continuing projects also include a copy of prior award final reports as an attachment to the submitted proposal.
Back to Top2.11. Additional Requirements
Individual sites may be subject to organizational policies with additional requirements beyond this policy. Those organizations will make those policies available. It is your responsibility to be aware of and abide by those policies.
Back to Top3. Penalties
Failure to abide by these policies may result in a variety of penalties imposed.
3.1 Account Suspension/Revocation
Accounts may be temporarily suspended or permanently revoked if compromised or abused. Your account may be suspended without advance notice if there is suspicion of account compromise, system compromise, or malicious or illegal activity.
Back to Top3.2. Loss of Allocation
Unauthorized behavior can result in loss of your current allocation, and may lead to the inability to obtain future allocations.
Back to Top3.3. Administrative Action
Unauthorized activity may be reported to your PI, your advisor, or LSU authorities for administrative review and action.
Back to Top3.4. Civil Penalties
Civil remedies may be pursued to recoup costs incurred from unauthorized use of resources or incident response due to compromise or malicious activity.
Back to Top3.5. Criminal Penalties
Activities in violation of university, federal, state, or local law may be reported to the appropriate authorities for investigation and prosecution.
Back to Top4. Disclaimers
4.1. Support/Diagnostic Access
HPC@LSU site personnel may review files for the purposes of aiding an individual or providing diagnostic investigation for HPC@LSU systems. User activity may be monitored as allowed under policy and law for the protection of data and resources. Any or all files on HPC@LSU systems may be intercepted, monitored, recorded, copied, audited, inspected, and disclosed to authorized site or law enforcement personnel, as well as authorized officials of other agencies, both foreign and domestic. By using HPC@LSU systems, users acknowledge and consent to this activity at the discretion of authorized site personnel.
Back to Top4.2. Access Notification
Access to user data and communications will not normally be performed without explicit authorization and/or advance notice unless exigent circumstances exist. Post-incident notification will be provided in such cases.
Back to Top5. Guidelines
The following are suggestions for helping maintain the security of your account.
Back to Top5.1. Password Management
- Do not write down your password where it can be easily found and/or associated with your account.
- Do not tell anyone your password, not even HPC@LSU support staff. Support staff will never need your password, will never ask for it, and will never send a password in e-mail, set them to a requested string, or perform any other activity which could reveal a password.
- If someone insists they need your password to do something, report it to the HPC@LSU helpdesk: sys-help@loni.org.
- Do not store your password(s) in unencrypted files or even in encrypted files if possible.
- Pick passwords that are difficult to guess. Birthdays, family names, and single dictionary words are examples of easily guessed passwords.
- Change your password periodically, even if you have no reason to believe that anyone else has it.
5.2. Password Exposure
If you think your password may have been compromised or exposed, but have no reason to believe that your account has been used, change your password immediately.
Back to Top5.3. Account Compromise or Suspicious Activity
If you believe your account has been compromised or find signs of suspicious activity, take the following actions:
- notify the HPC@LSU helpdesk immediately (sys-help@loni.org)
- do not modify files found in your account
- do not execute unknown programs you might find
- if possible, do not use your account until the issue is resolved
Some indications of account compromise include:
- files in your home directory or project areas which you did not create
- alteration or deletion of your files not done by you
- discrepancies between your allocation balance and what you think you have used
6. Contacts
The central point of contact for any problems and concerns with regards to this policy is the HPC@LSU help desk which can be reached at sys-help@loni.org. Other modes of contact are listed on the HPC@LSU web site contact us page.
Back to TopLast revised: 11 December 2009
LSU HPC Allocations Policy
▶ Table of Contents
1. System Allocations
LSU@HPC (High Performance Computing at Louisiana State University) operates several HPC systems. The machines currently include IBM Power5-575 and Power7-755 constellations, and Intel Xeon-based x86 clusters. Detailed information can be found on the HPC web site at www.hpc.lsu.edu. LSU controls access to these resources via a formal user allocation and account creation process. All decisions in these matters are made by a committee made up of faculty representatives from different discplines across LSU.
Gaining access to LSU HPC resources, be it computation time or data storage, involves a two step process. The first step requires the submission of a project proposal describing the intended usage and providing justification for the resources requested. Acceptance of the proposal results in the award of an allocation of computation time to the project. Allocations are awarded to the principle investigator (PI) indicated in the proposal. Allocations for computation time are awarded in units of Service Units (SUs - see definition below) for use on a specific machine. Storage allocations are awarded in units of Gigabytes (1 billion bytes) of disk space, and are restricted to a limited number of file systems. Storage allocations are discussed in a separate document.
The second step involves the authorization of system user accounts for the expenditure of allocated resources. In order to run a program, a user must be able to charge expenditure to an allocation. To accomplish this, tools are made available to the PI to control authorization of users. The PI associates an email address with an allocation to which all user account requests are sent for authorization. The PI is responsible for all authorizations, but may formally delegate someone to manage this process. Once a user account is authorized to use an allocation, the user is allowed to submit jobs or otherwise expend the allocation. This implies that a user must be authorized for at least one non-expired allocation before the systems can be used. There is no limit to the number of allocations a user is associated with. It becomes the joint responsibility of the PI and the authorized user to make sure the appropriate allocation is expended.
Back to Top2. Allocation Categories and Process
At the present time, allocations are awarded independent of machine, allowing users to run on any available LSU HPC platform. However, HPC resources are partitioned by category, and all standard allocation proposals go through the LSU HPC Resource Allocation Committee (LSU HPCRAC). The HPCRAC represents the two distinct approval authorities: the HPCRAC Chair, and the HPCRAC as a panel. Table 1 shows the distribution of LSU HPC resources by category:
| Allocation Category | Available Resources | Allocation Authority |
|---|---|---|
| Economic Development | 10% | Vice-Chancellor for Research and Economic Development |
| Discretionary | 10% | Center for Computation and Technology Director |
| Default Allocation (2,000 SUs) | 5% | HPC@LSU Staff |
| Startup Allocation (2,000-50,000 SUs) | 15% | HPCRAC Chair |
| Research Allocation (> 50,000 SUs) | 60% | HPCRAC |
To facilitate management of proposals, they will be assigned on of the following classes (Table 2).
| Class | Description |
|---|---|
| Economic | A proposal to assist commercial interests with the adoption of high performance computing as part of their business process. Awarded by the VCRED from the Economic Development resources. Economic allocations may be renewed by the VCRED. |
| Discretionary | A proposal determined to be of value, but lying outside the normal allocation process. Awarded by the CCT Director from the Discretionary resources. Discretionary allocations may be renewed by the director. |
| Default | Every user account, upon creation, receives a default allocation. This allows sufficient time for conventional processing on the systems, and developing information for potential formal proposals. These allocations are limited to 2,000 SUs. Default allocations are not renewable. |
| Startup | A proposal requesting an allocation for the purpose of exploring the value of high performance computing for a new project. Only 2 startup allocations may be active at any time per PI. Startup allocation of 2,000 to 50,000 SUs are awarded by the Chair of HPCRAC. |
| Research | A request for a large amount of time for a significant research project. May be awarded by simple majority agreement of the HPCRAC. Research requests are limited to 3 million SUs, and a PI may have a total of 5 million SUs active at any given time. Only faculty members and permanent research staff of LSU may serve as the PI for large Research allocations. Research allocations may be eligible for renewal by the HPCRAC. |
All allocations have a duration of one year. Default and Startup allocations may be awarded at any time during the year, and Default allocations, in particular, are awarded on creation of a user account. Research allocations are granted at the beginning of every calendar quarter. Application deadlines for Research allocations are one month prior to the start date: January 1, April 1, July 1, and October 1. A PI may request, or be requested, to make a formal presentation to the HPCRAC in conjunction with the proposal submission. At the end of any allocation, a short summary report is required by the HPCRAC. Missing reports may delay proposal processing. Any request for allocation renewal must include a summary report, including renewals for startup allocations. The summary should indicate if the goals of the proposed work were accomplished, any major results, and a list of any publications and presentations that result.
Back to Top2.1. Startup and Research Allocation Requests and Renewals
2.1.1. Startup Allocations (<50,000 SUs)
Who may apply: Any LSU student, faculty member, postdoc, or research staff member whose activities require access to HPC resources (for research or class instruction) may apply for a Startup allocation.
How to apply: Applicants should submit a web form (accounts.hpc.lsu.edu) to briefly explain (one page maximum) their need for computation time on an HPC platform, their methodology, and the current state of their codes or application. Allocation requests from students, postdocs, and research staff with term appointments must be approved by a faculty member who agrees to assume oversight responsibility for the proposed project.
Review and Awards: Startup allocations may be made at any time throughout the year. Default allocations (2,000 SUs) generally will be awarded by HPC staff, once an applicant’s identity and LSU affiliation has been verified and, as appropriate, the application has been approved by the faculty member with oversight responsibility. The computationally-driven activities of any individual whose HPC resource needs exceed 2,000 SUs in a given year may be subject to a review by the Chair of HPCRAC - following review, the individual’s startup allocation may be increased to a level not to exceed 50,000 SUs, or the user may be moved under the umbrella of an existing “Research group” and thereafter be subjected to the allocation limits of that group.
Renewals Upon request, startup allocations may be renewed annually; renewal applications should be accompanied by a summary report including a list of publications and presentations in which the use of LSU HPC resources has been acknowledged.
Back to Top2.1.2. Research Allocations (>50,000 SUs)
Who may apply: Generally, it is expected that each application for a Research allocation will be submitted by an LSU faculty member or Research Staff on behalf of a research or operational group in support of a well-defined compute-intensive project. Once awarded, such an allocation will then be shared among that group with the PI taking responsibility for ensuring that the allocation is utilized effectively (at an appropriately measured pace) throughout the year.
How to apply: A request for an allocation requires that a formal proposal be submitted via the HPC web interface to the HPCRAC (accounts.hpc.lsu.edu). The proposal is expected to be 5 pages or less, and concentrate on justifying the computational resources requested. The follow outline should be followed:
- Problem Statement – Section limited to 1 page, or less, describing the desired outcomes of the project.
- Background – Section limited to 1 page, or less, describing how the resources will be used to address the problem (i.e. student access for course work, specific models, etc).
- Methodology – Section limited to 1 page, or less, describing the computational methodology that will be used. This should include the applications required.
- Research Plan – Section limited to 1 page, or less, describing the research schedule, including the anticipated expenditure of granted resources. Allocations are assumed to be uniformly consumed over their lifetime. If this will not be the case, an estimate of expenditure by quarters is required.
- Requirements Analysis - Section limited to 2 pages, or less, detailing the basis for the requested computer time. Large allocations must exhibit an understanding of application efficiency, scaling, and provide accurate estimations of the time requirements.
Please note that the maximum proposal page limit is 5, not including summary reports in the Attachments. The page limit was chosen with an eye to making it relatively easy to compose an allocation request, or to modify and reuse a successful application made to another center. Additional information may be attached as addendums, such as copies of awarded grants which will be supported by the requested resources.
Review and Awards: Applications for a Research allocation will undergo competitive peer review and will be allocated quarterly by the HPCRAC. Requests will be granted – in whole or in part – based on the availability of HPC resources and based upon the description of the proposed activities and the appropriate use of technology, with priority given to funded projects. Allocation decisions made by the HPCRAC are deemed final. Unresolved appeals will be directed to the Vice Chancellor for Research & Economic Development (VCRED).
Renewals: Renewal allocations follow the same process as all other allocations. Proposal writers should be aware that both past usage history and submission of progress reports will be considered in the award determination. Applications for allocation renewals should ideally cite peer reviewed publications that acknowledge LSU HPC resources and only require an updated version of a previously successful application.
LSU welcomes members of other institutions that are collaborating with LSU researchers to use LSU resources, but they cannot be the PI requesting standard allocations or granting access. For the PI role, Adjunct and Visiting professors do not qualify. They must ask a collaborating fulltime professor or research staff member located at LSU to sponsor them and any additional researchers.
Back to Top2.2. Director's Discretionary Allocation Request (submit to CCT Director)
Each calendar year, up to 10% of the available HPC resources will be allocated at the discretion of the CCT Director and the Chief Information Officer (CIO). Allocations made from this pool will be for a period not to exceed one year. Awardees will be expected to work in conjunction with CCT or HPC@LSU technical staff to ensure that project implementations are in line with and strengthen the CCT's broad interdisciplinary mission. At the termination of a project (or annually, if an award is extended beyond one year) awardees shall provide a summary report of project activities along with a list of publications and presentations in which the use of LSU HPC resources has been acknowledged.
Back to Top2.3. Economic Development or 'On Demand' Allocation Request (submit to VCRED)
The VCRED, in consultation with the chair of the HPCRAC and the CCT Director, may grant Economic Development or On Demand access to HPC resources for high priority projects like render farm for economic development purposes, hurricane storm surge modeling, or other possible operational responsibilities that are deemed sufficiently different from standard research proposals. Each calendar year, up to 10% of the available HPC resources may be allocated by the VCRED. At the termination of a project (or annually, if an award is extended beyond one year) awardees shall provide a summary report of project activities along with a list of publications and presentations in which the use of LSU HPC resources has been acknowledged.
Back to Top2.4. Allocation Management
An allocation is considered valid so long as a positive resource balance remains, and the expiration date has not been exceeded. Once an allocation expires, or has been fully consumed, users accounts will be blocked from submitting work against the allocation. There is no mechanism for extending an allocation beyond one year, nor for adding resources once an allocation has been expended.
User accounts must be associated with a valid allocation, and if not, will be retained for a maximum of 1 year pending authorization against a renewed or different allocation. With these restrictions in mind, the PI is required to use the tools provided to monitor system usage and control authorization of project member accounts. PIs are strongly advised to carefully budget their usage appropriately throughout the year. Automatic reminder emails will be sent by the management system as an allocation nears expiration. PIs are ultimately responsible for assuring that a current and actively monitored management email address has been assigned to each allocation.
At the end of any allocation, a short summary report must be submitted to the allocation committee. Failure to submit this report may be used in the consideration of future allocations. The report may simply reference any formal publications and presentations that resulted from using HPC resources, or provide a high level overview of what was accomplished.
Back to Top2.5. Early Allocation Access
If a PI who has already been awarded a large Research allocation by HPCRAC puts in a new request, and there is a good reason to start the project before the next cycle, then the Chair of HPCRAC can instruct the HPC@LSU staff to award as much as 25% of the project request. Justification must be provided in the renewal proposal. The HPCRAC committee would be made aware of this action and its reasons by the Chair of HPCRAC using the "LSU HPC Allocations" email list.
Back to Top3. Machine Access Policy
3.1. Job Queueing
Various workload balancing algorithms are used to determine how jobs are assigned resources on a given machine. The way a job is handled is determined by the job queue it is submitted to. Efficient use of the queuing system requires that users request runtimes consistent with estimated runtimes of their jobs. In particular, requesting more time than is necessary for a particular job can lead to inefficient and unfair queuing. Therefore, users that routinely request more time than is needed for their jobs are subject to a priority penalty that will lower the priority of their jobs. Each system sets a maximum number of jobs that a single user may have running without special permission (see below). There is no limit to the number of jobs that a particular user may have queued. Users that wish to obtain a higher priority for their jobs may use special priority queues (see below).
The available processors are currently divided into 2 architecture specific groups: IBM P5-575 and P7-755 systems running AIX, and Intel x86 systems running Linux. The processors in each group are further subdivided into preemptory and dedicated pools. Certain mission critical applications, such as storm surge prediction during a hurricane threat, are granted immediate access to processors in the preemptory pool. Processors in the dedicated pool are used to run all other job types. The processors are accessed through different job queues. There are 5 job queues which use different combinations of the processor pools, and allow for different job characteristics.
- Preempt Queue: The preempt queue controls access to the preemptory pool. Authorized applications submitted to this queue will cause the termination of all other user applications running on preemptory nodes.
- Checkpt Queue: The checkpt queue controls access to nodes in both the dedicated and preemptory pool. Jobs running in this queue may be subject to termination by the preempt queue, thus are implicitly assumed to support restarts based on periodically saved information. No refunds of lost SUs are offered if jobs in the checkpt queue are terminated by preemption. The user running jobs without restart capability assume this risk. However, the benefits from using this queue include access to larger numbers of nodes, and/or faster throughput, depending on how busy the queue is.
- Workq: The workq queue controls access to nodes in the dedicated pool. Jobs in the work queue will run until they terminate as planned, their requested run time has expired, or they stop due to an abnormal system failure. Jobs which are terminated due to system errors beyond the user's control may be subject to refund of expended SUs. Poor planning is not considered grounds for a refund.
- Interactive Queue: The interactive queue gives real-time access to jobs for on-line analysis or debugging, but only allows very short run times. It supports development work, but not production.
- Priority Queue: The priority queue controls nodes in the dedicated pool, but allows applications to be given higher priority with prior approval. Approval may be granted during training sessions, for demonstration purposes, or other special needs. The SUs charged will be adjusted by a factor of 1.3, and no more than 20% of an allocation may be expended in the queue. Requests for priority access should be directed to sys-help@loni.org. This queue does not impact other running user jobs, but will delay the start of lower priority jobs already in the queue.
Note: The named queues do not necessarily exist on all machines, and the maximum time allowed in the queues will vary from machine to machine. For system specific information, please refer to the architecture group Running Jobs pages on the HPC@LSU documentation web site.
Back to Top3.2. Storage
Currently disk space usage is controlled via user quotas rather than on a per-project basis. Storage may become an allocated resource in the future. At such time, a request for storage space will be required in the allocation request. At the current time, an estimate of required space is requested.
Back to Top3.3. Special Requests
A request for special access to LSU HPC machines (such as usage of all nodes on a machine or exceptionally long runs) must be explicitly stated in the proposal for HPC resources. Appeals to the decision of the HPCRAC may be made to the VCRED.
Back to Top4. HPC Resources Allocation Committee Members
The proposed HPCRAC members are shown in Table 3. The CCT Director, in consultation with the VERED, appoints the Chair of HPCRAC. The HPCRAC is chartered approval authority allocating HPC resources, and charged with the task of adopting the HPC Resources Allocations Policy and reporting its activities and recommendations to VCRED.
| Name | Department | Contact Email |
|---|---|---|
| Honggao Liu (Chair) | CCT | honggao@cct.lsu.edu |
| Juana Moreno | Physics | moreno@phys.lsu.edu |
| Jim Q. Chen | Civil & Environmental Engineering | qchen@lsu.edu |
| Shawn W. Walker | Mathematics | walker@math.lsu.edu |
| Krishnaswamy Nandakumar | Chemical Engineering | nandakumar@lsu.edu |
| Jeremy Brown | Biological Sciences | jembrown@lsu.edu |
5. Terminology
Summary report: This report should be a maximum of 5 pages and should include the following information:
- Principal user information [PI] (name, status, department, phone, email, institution [if different from LSU]
- Summarize the nature of the LSU-sponsored research in a non-technical fashion, suitable for public consumption. (Approximately one page; no more than two pages)
- Describe any potential applications of the research to industry or government.
- Describe any use of this allocation to encourage education in computational science, and particularly the level of student involvement and any HPC elements incorporated into formal courses.
Definition of Service Unit (SU): Currently, one SU corresponds to one hour of wall-clock time on one processing core. A single machine will have multiple nodes (individual servers) available, and multiple cores within each node. This allows many cores to be used for a single parallel processing job, but can lead to not-so-obvious charges. For example, running for 1 hour using 8 cores on an 8-core node consumes 8 SUs. On the other hand, running for 1 hour on 1 core of the same 8 core node, which allows the single core to access all of the node memory, also consumes 8 SUs. In simple terms, the number of cores that are reserved for a job, and hence are unavailable to others, is the number used to calculate SU usage.
HPC resources at LSU will be allocated/charged according to the number of “service units (SUs)” required/used, where:
# SUs = m * #Nodes * Wall_Time,
and,
m = number of processing cores per Node;
Wall_Time = Total Wall Clock Hours.
For example, if a machine has 4 processing cores per Node (m = 4), a program that ran for 24 hours on 32 nodes required 4*32*24 = 3072 SUs.
Back to TopLast revised: 30 March 2012
LSU HPC Storage Policy
▶ Table of Contents
1. Storage Systems
Available storage on LSU@HPC high performance computing machines is divided into four file systems (Table 1).
| File System | Description |
|---|---|
| Scratch | Temporary storage on each node that is available only during job execution. |
| Work | Shared storage provided to each user for input, intermediate, and output files of a job. |
| Home | Persistant storage provided to each active user account. |
| Project | Persistant storage provided by request for a limited time to hold large amounts project-specific data. |
All of the file systems share common characteristics. When they are too full, system performance begins to suffer. Similarly, placing too many individual files in a directory degrades performance. System management activities are aim at keeping the different file systems below 85% of capacity. It is strongly recommended that users hold file counts below 1,000, and never exceed 10,000, per directory. If atypical usage begins to impact performance, individual users may be contacted and asked to help resolve the issue. When performance or capacity issues become significant, system managers may intervene by requiring users to offload files, stop jobs, or take other actions to ensure the continued operation of the system.
Management makes every effort to avoid data loss. In the event of a failure, every effort will be made to recover data. However, the volume of data housed makes it impractical to provide system-wide data backup. Users are expected to safeguard their own data by making sure that all important code, scripts, documents and data are transferred to another location in a timely manner. The use of inexpensive, high capacity external hard drives attached to a workstation is highly recommended for individual user backup.
Back to Top2. File System Details
2.1. Scratch
Scratch space is available on all systems, and to all users during the course of a job, but files are subject to deletion once a job ends. The size of this file system will vary from system to system, and possibly across nodes within a system. This is the preferred place to put any intermediate files required while a job is executing. Users should not have any expectation that files will exist after a job terminates, and are expected to move the data from Scratch to their Work or Home directory as part of the clean up process in their job script.
Back to Top2.2. Work
Work is the primary file storage area to utilize when running jobs. Work is a common file system that all system nodes have access to. It is ideal for input files, checkpoint files and output. A storage quota may be enforced per user on the Work space file system, and usage allowed will vary by system, as shown in Table 3. This does not imply that each user should intend to fully consume this amount, since the quota system uses overbooking to optimize available space. This quota serves as a hard upper limit to prevent a single user from disrupting performance of the entire system. Files on Work will persist indefinitely. Users should not have any expectation of backup. The safekeeping of data is the responsibility of the user.
On systems which do not enforce a quota on Work space, the file system is fully shared. However, if capacity approaches 85%, the files are subject to purge by management. The purge process targets files with the oldest age, and continues until the capacity drops to an acceptable level. The purge process will normal occur once per month, as necessary. On systems with no Work quota, persistent storage for large amounts of data is provided on the Project file system.
Should the need arise; users may request an increase in their Work quota for a reasonable period of time. A request containing a justification for the increased storage should be sent to sys-help@loni.org. The email must include the name of the PI, a valid contact email, a summary of any existing quotas, and a justification statement. Table 2 shows the approval authority for quota increases, based on size.
| Class | Size | Approval Authority |
|---|---|---|
| Default | 50/100GB | Default by system for each user |
| Medium | Between 100GB and 1TB | HPC@LSU Operations Manager |
| Large | Over 1TB | HPC@LSU Directory |
2.3. Home
All user home directories are located in the Home file system. Home is intended for the user to store source code, executables, and scripts. Home may be considered persistent storage. The data here should remain as long as the user has a valid account on the system. Home will always have a storage quota, and is clearly a hard limit. While Home is not subject to management activities controlling capacity and performance, it should not be considered permanent storage, as system failure may result in the loss of information. Users should arrange for backing up their own data.
Back to Top2.4. Project
The Project file system may be available on some systems, and provides space specific to a project. Project space allocations must be requested, and they are available for a limited time period. Allocations are typically 6 months or less. Shortly before an allocation expires the user will be notified of the upcoming expiration. Users may request to have the allocation extended. Renewal requests should be submitted at least 1 month prior to expiration to allow decision and planning time. Users should have no expectation that data will persist, and may be erased any time after 1 month from a project’s expiration. Thus alternate safe keeping and protection actions must be taken in advance.
Back to Top3. System Specific Information
| System | File System | Storage (Tb) |
Quota (GB) |
Purge File Limit (Million) |
|---|---|---|---|---|
| IBM P5-575 Pelican |
Work | 28 (GPFS) | N/A | 5 |
| Home | (GPFS)[1] | 5 | N/A | |
| Dell x86 Cluster Tezpur |
Work | 60(LPFS)[2] | N/A | 8 |
| Home | (NFS)[3] | 5 | N/A | |
| Project | 60 | By Request |
4. Job Use
On all systems, jobs must be run from the Work file systems, and not from the Home or Project file systems. Files should be copied from Home or Project space to Work before a job is executed, and back when a job terminates, to avoid excessive I/O during execution that degrades system performance.
Back to Top5. /project Allocation Requests
Space is allocated on the Project file system by request only for periods of 6 months. A request for initial or renewal allocation may be made by sending an email to syshelp@loni.org. The email must include the name of the PI, a valid contact email, a current allocation code, summary of any existing allocation account information (CPU or storage), and a justification statement. Allocations are divided into 3 classes, each with a separate approval authority (Table 4). All storage allocation requests are limited by the available space, and justifications must be commensurate with the amount of space required.
| Class | Size | Approval Authority |
|---|---|---|
| Small | Up to 100GB | HPC@LSU Staff |
| Medium | Between 100GB and 1TB | HPC@LSU Operations Manager |
| Large | Over 1TB | HPC@LSU Director |
[1] Global Parallel File System
[2] Lustre Parallel File System
[3] Network File System
Last revised: 11 December 2009
Users are asked to acknowledge their use of LSU HPC resources in resulting publications and reports with the following statement:
Portions of this research were conducted with high performance computational resources provided by Louisiana State University (http://www.hpc.lsu.edu).