5. Running on HPC clusters¶
This section will walk you through the steps required to run 1D-REMD and 3D-REMD examples with RepEx on HPC clusters. This guide assumes that you have already installed RepEx and cloned repex.examples repository. If you haven’t installed RepEx, please follow the steps in Installation section of this user guide. If you can’t find location of repex.examles directory, please clone repository again:
git clone https://github.com/antonst/repex.examples.git
and cd into Amber examples directory where input files recide:
cd repex.examples/examples/amber
To run examples you will need to modify resource configuration file - <resource>.json
.
Please read Getting Started section of this guide to find out about clusters supported
by RepEx and parameters you need to specify in this file. Once you have properly configured
resource configuration file, you can use it for all examples of this section.
5.1. T-REMD example (peptide ala10) with Amber kernel¶
First, we will take a look at Temperature-Exchange REMD example using peptide ala10 system
with Amber simulations kernel. You need to verify if parameters specified in t_remd_ala10.json
REMD input file satisfy your requirements. By default t_remd_ala10.json
file looks like this:
{
"remd.input": {
"re_pattern": "S",
"exchange": "T-REMD",
"number_of_cycles": "4",
"number_of_replicas": "8",
"input_folder": "t_remd_inputs",
"input_file_basename": "ala10_remd",
"amber_input": "ala10.mdin",
"amber_parameters": "ala10.prmtop",
"amber_coordinates": "ala10_minimized.inpcrd",
"replica_mpi": "False",
"replica_cores": "1",
"min_temperature": "300",
"max_temperature": "600",
"steps_per_cycle": "4000",
"download_mdinfo": "True",
"download_mdout" : "True"
}
}
In comparison with general REMD input file format discussed above this input file contains some additional parameters:
min_temperature
- minimal temperature value to be assigned to replicasmax_temperature
- maximal temperature value to be assigned to replicas (we use geometrical progression for temperature assignment)
Since we are using a cluster to run REMD simulation we increase the nuber
of replicas to use. Please set "number_of_replicas"
to number of cores in a single node
of your target resource.
To get notified about important events during the simulation please specify in terminal:
export RADICAL_REPEX_VERBOSE=info
Now you are ready to run this simulation. Type and run in terminal (substitute ‘<resource>.json’ with your resource configuration file):
``repex-amber --input='t_remd_ala10.json' --rconfig='<resource>.json'``
5.1.1. Verify output¶
If simulation has successfully finished, last three lines of terminal log should be similar to:
2015:10:11 18:49:59 6600 MainThread radical.repex.amber : [INFO ] Simulation successfully finished!
2015:10:11 18:49:59 6600 MainThread radical.repex.amber : [INFO ] Please check output files in replica_x directories.
2015:10:11 18:49:59 6600 MainThread radical.repex.amber : [INFO ] Closing session.
You should see 17 new directories in your current path:
- sixteen
replica_x
directories- one
shared_files
directory
If you want to check which replicas exchanged configurations during each cycle you can cd into
shared_files
directory and check each of four pairs_for_exchange_x.dat
files. In these files are recorded indexes of replicas exchanging configurations during each cycle.
If you want to check .mdinfo or .mdout files for some replica, you can find those files in
corresponding replica_x
directory. File format is ala10_remd_i_c.mdinfo
where:
- i is index of replica
- c is current cycle
Simulation output can similarly be verified for all other examples of this tutorial.
**Note: ** After you have successfully completed your simulation and verified simulation
output, delete or move replica_x
and shared_files
directories.
5.2. US-REMD example using Alanine Dipeptide system with Amber kernel¶
For the example we will use Alanine Dipeptide (Ace-Ala-Nme) system. In examples/amber
directory are present:
us_remd_inputs
- input files for US-REMD simulationsus_remd_ace_ala_nme.json
- REMD input file for Umbrella Sampling REMD example using Alanine Dipeptide system
To run this example you need to verify if parameters specified in us_remd_ace_ala_nme.json
REMD input file satisfy your requirements. By default us_remd_ace_ala_nme.json
file looks like this:
{
"remd.input": {
"re_pattern": "S",
"exchange": "US-REMD",
"number_of_cycles": "4",
"number_of_replicas": "8",
"input_folder": "us_remd_inputs",
"input_file_basename": "ace_ala_nme_remd",
"amber_input": "ace_ala_nme.mdin",
"amber_parameters": "ace_ala_nme.parm7",
"amber_coordinates_folder": "ace_ala_nme_coors",
"same_coordinates": "True",
"us_template": "ace_ala_nme_us.RST",
"replica_mpi": "False",
"replica_cores": "1",
"us_start_param": "120",
"us_end_param": "160",
"init_temperature": "300.0",
"steps_per_cycle": "2000",
"download_mdinfo": "True",
"download_mdout" : "True"
}
}
In comparison with general REMD input file format discussed in getting-started section this input file contains some additional parameters:
same_coordinates
- specifies if each replica should use an individual coordinates file. Options are:True
orFalse
. IfTrue
is selected, inamber_coordinates_folder
must be provided coordinate files for each replica. Format of coordinates file is:filename.inpcrd.x.y
, wherefilename
can be any valid python string,inpcrd
is required file extension,x
is index of replica in 1st dimension andy
is index of replica in second dimension. For one-dimensional REMD,y = 0
must be provided
us_template
- name of Restraints template fileus_start_param
- starting value of Umbrella intervalus_end_param
- ending value of Umbrella intervalinit_temperature
- initial temperature to use
Since we are using a cluster to run REMD simulation we increase the nuber
of replicas to use. Please set "number_of_replicas"
to number of cores in a single node
of your target resource.
Now you are ready to run this simulation. Type and run in terminal (substitute ‘<resource>.json’ with your resource configuration file):
``repex-amber --input='us_remd_ace_ala_nme.json' --rconfig='<resource>.json'``
Output verification can be done similarly as for T-REMD example.
5.3. TUU-REMD example (alanine dipeptide) with Amber kernel¶
For the example we also will use Alanine Dipeptide (Ace-Ala-Nme) system. In examples/amber
directory are present:
tuu_remd_inputs
- input files for TUU-REMD simulationstuu_remd_ace_ala_nme.json
- REMD input file for TUU-REMD usecase using Alanine Dipeptide system
To run this example you need to verify if parameters specified in tuu_remd_ace_ala_nme.json
REMD input file satisfy your requirements. By default tuu_remd_ace_ala_nme.json
file looks like this:
{
"input.MD": {
"re_pattern": "S",
"exchange": "TUU-REMD",
"number_of_cycles": "4",
"input_folder": "tuu_remd_inputs",
"input_file_basename": "ace_ala_nme_remd",
"amber_input": "ace_ala_nme.mdin",
"amber_parameters": "ace_ala_nme.parm7",
"amber_coordinates_folder": "ace_ala_nme_coors",
"us_template": "ace_ala_nme_us.RST",
"replica_mpi": "False",
"replica_cores": "1",
"steps_per_cycle": "6000"
},
"input.dim": {
"umbrella_sampling_1": {
"number_of_replicas": "4",
"us_start_param": "0",
"us_end_param": "360"
},
"temperature_2": {
"number_of_replicas": "4",
"min_temperature": "300",
"max_temperature": "600"
},
"umbrella_sampling_3": {
"number_of_replicas": "4",
"us_start_param": "0",
"us_end_param": "360"
}
}
}
In comparison to general REMD simulaiton input file, this file has the following additional parameters:
input.dim
- under this key must be specified parameters and names of individual dimensions for all multi-dimensional REMD simulations.umbrella_sampling_1
- indicates that first dimension is Umbrella potentialtemperature_2
- indicates that second dimension is Temperatureumbrella_sampling_1
- indicates that third dimension is Umbrella potentialnumber_of_replicas
- indicates number of replicas in this dimension
Now you are ready to run this simulation. Type and run in terminal (substitute ‘<resource>.json’ with your resource configuration file):
repex-amber --input='tuu_remd_ace_ala_nme.json' --rconfig='<resource>.json'
Output verification can be done similarly as for T-REMD example.