Note: These instructions should work for AiiDA v1.3 on LC systems. We are still testing v1.4, the most recent AiiDA release.
For more info, see the full AiiDA docs.
Getting started with AiiDA
Cloning and basic AiiDA Setup
$ git clone https://github.com/sphuber/aiida_core.git $ cd aiida_core/ $ git checkout fix/1074/rabbitmq-configuration $ python3 -m venv aiidavenv $ source aiidavenv/bin/activate $ pip install --upgrade pip # this takes it from pip 18.1 to 20.2 $ cd .. # go up a directory level to go outside aiida_core directory when you run the next `pip install…` command! $ pip install -e aiida_core $ LC_ALL="en_US.UTF-8" # necessary on Maya # ADD `LC_ALL=“en_US.UTF-8"` to .bash_profile as well! (For future use of Verdi commands) $ verdi --version
Creating a profile
First, run verdi setup --profile test and provide the requested information as shown below.
(aiidavenv) [janeh@maya6:~]$ verdi setup --profile test Info: enter "?" for help Info: enter "!" to ignore the default and set no value Email Address (for sharing data): herriman1@llnl.gov First name: Jane Last name: Herriman Institution: LLNL Database engine (postgresql_psycopg2) [postgresql_psycopg2]: Database backend (django, sqlalchemy) [django]: Database host: maya4.llnl.gov # Nathan will use a different one for open shift Database port [5432]: #Nathan’s will also be different Database name: aiida_janeh Database username: janeh Database password: Broker protocol (amqp, amqps) [amqp]: amqps Broker username [guest]: jane # broker things are for rabbit mq Broker password [guest]: Broker host [127.0.0.1]: jane-aiida.apps.cztest.llnl.gov Broker port [5672]: 31859 Repository directory: /g/g17/janeh/aiida_core #path to directory created with `git clone…` Success: created new profile `test`. Info: migrating the database. Success: database migration completed.
To complete setting up the profile, edit the config file. Open it with your favorite text editor
vi ~/.aiida/config.json
And add broker_virtual_host and broker_parameters after the broker_port field.
"broker_port": 31784,
"broker_virtual_host": "/aiida",
"broker_parameters": {
"no_verify_ssl": "1"
},
Testing
Now test some verdi commands to see if setting up AiiDA and a profile has worked.
verdi status verdi profile list verdi daemon start 1 verdi shell
Set up and configure a computer
Set up computer
(aiidavenv) [janeh@maya6:~]$ verdi computer setup
Info: enter "?" for help
Info: enter "!" to ignore the default and set no value
Computer label: quartzdebug
Hostname: quartz
Description []:
Transport plugin: ssh
Scheduler plugin: slurm
Shebang line (first line of each script, starting with #!) [#!/bin/bash]:
Work directory on the computer [/scratch/{username}/aiida/]: /usr/workspace/janeh
Mpirun command [mpirun -np {tot_num_mpiprocs}]: srun -n {tot_num_mpiprocs}
Default number of CPUs per machine: 2
After entering the above info, a file will open. Feel free to save without adding anything! Alternatively can add #SBATCH -p pdebug in the “prepend” section to make this a computer for only the debug queue.
Create .ssh files
This applies if you don’t already have a .ssh/id_rsa file, but be careful because this will otherwise overwrite the default.
ssh-keygen -t rsa -b 4096 # This creates ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub # Now want to copy the public key into quartz ssh-copy-id -i ~/.ssh/id_rsa janeh@quartz # Test that you can ssh into quartz from Maya ssh janeh@quartz
Configure computer
(aiidavenv) [janeh@maya6:~]$ verdi computer configure ssh quartzdebug Info: enter "?" for help Info: enter "!" to ignore the default and set no value User name [janeh]: port Nr [22]: 622 Look for keys [True]: SSH key file []: /g/g0/janeh/.ssh/id_rsa Connection timeout in s [60]: Allow ssh agent [True]: SSH proxy command []: Compress file transfers [True]: GSS auth [False]: GSS kex [False]: GSS deleg_creds [False]: GSS host [quartz]: Load system host keys [True]: Key policy (RejectPolicy, WarningPolicy, AutoAddPolicy) [RejectPolicy]: Connection cooldown time (s) [30.0]: 10.0 Info: Configuring computer quartzdebug for user herriman1@llnl.gov. Success: quartzdebug successfully configured for herriman1@llnl.gov
Test computer
verdi computer test quartzdebug # make sure all tests pass
Configure quantum espresso
Install plug ins for quantum espresso
verdi plugin list # to see options, including aiida.calculations verdi plugin list aiida.calculations # shows the plugins currently registered pip install aiida-quantumespresso # use pip to install a new plug in reentry scan # if you run verdi plugin list aiida.calculations, it won’t register the new plug in installed until after the reentry scan verdi plugin list aiida.calculations # now this shows a long list of quantum espresso related plugins, including the one we’ll use next: quantumespresso.pw
Set up quantum espresso code
(aiidavenv) [janeh@maya6:aiida_core]$ verdi code setup Info: enter "?" for help Info: enter "!" to ignore the default and set no value Label: pwdebug Description []: Default calculation input plugin: quantumespresso.pw Installed on target computer? [True]: Computer: quartzdebug Remote absolute path: /g/g0/janeh/AiiDA/pw.x
After entering the above, a text file will open. Add the following module lines to the “prepend text” section:
module purge module load intel mkl impi
and then you’ll get a message like:
Success: Code<201> pwdebug@quartzdebug created
Upload pseudo potentials
# Get pseudopotentials wget https://legacy-archive.materialscloud.org/file/2018.0001/v4/SSSP_1.1_PBE_efficiency.tar.gz # Create a place to store all pseudopotentials from tar file mkdir aiida_core/sssp tar -zxvf SSSP_1.1_PBE_efficiency.tar.gz -C aiida_core/sssp #extract tar file verdi data upf uploadfamily aiida_core/sssp sssp "sssp pseudos” # load pseudopotentials
Run quantum espresso calculation
How to run a test job
You should now be able to run a calculation via
aiida-quantumespresso calculation launch pw -X pwdebug@quartzdebug -p sssp -i -d
If you run verdi process list -a, the most recently run calculation will show up at the bottom of the list, with its identifying “pk” number in the leftmost column. You can then take that number and run verdi process show <pk> to learn more about the process. Among the outputs for this command will be information about the pk for inputs to the calculation, such as
Inputs PK Type
---------- ---- -------------
pseudos
Si 111 UpfData
code 201 Code
kpoints 252 KpointsData
parameters 253 Dict
structure 158 StructureData
Syntactic sugar for running a job
The pk values obtained from verdi process show... can be used to write a launch.py script containing info such as
from aiida.orm import load_node
from aiida.engine import submit
PwCalculation = CalculationFactory('quantumespresso.pw')
inputs = {
'code': load_code('pw@quartz'),
'structure': load_node(158),
'pseudos': {
'Si': load_node(111),
},
'parameters': load_node(253),
'kpoints': load_node(252),
'metadata': {
'options': {
'resources': {
'num_machines': 1,
},
'max_wallclock_seconds': 500,
'withmpi': True
}
}
}
submit(PwCalculation, **inputs)
which can then be run via
verdi run launch.py
Instead of aiida-quantumespresso calculation launch pw -X pwdebug@quartzdebug -p sssp -i -d. You can again learn the pk of this number by running verdi process list -a.
Troubleshooting a job
Once you have the pk number of a job, you can learn more about the calculation and its outputs via
verdi process show <pk> verdi process report <pk> verdi calcjob gotocomputer <pk>
The last option takes you to a directory where you’ll see the outputs from the job and the script actually submitted to slurm to run the desired calculations. You can also dig into the details of various inputs to the calculation via, for example, verdi data dict show <pk of theparametersDict> where the pk associated with parameters is learned from verdi process show <pk>.
If necessary, you can kill a process with verdi process kill <pk>.
Troubleshooting
QE version errors
Error messages
Both the stdout and XML output files could not be read or parsed.
parser returned exit code<305>: Both the stdout and XML output files could not be read or parsed.
Potential fix You may be using a version of Quantum Espresso that isn't supported by AiiDA. For example, the messages below were generated using QE version 6.6 too soon. Changing to a different version of Quantum espresso may resolve these error messages.
More details
+-> WARNING at 2020-08-24 16:44:52.714821+00:00 | key 'symmetries' is not present in raw output dictionary +-> ERROR at 2020-08-24 16:44:52.770446+00:00 | ERROR_OUTPUT_STDOUT_INCOMPLETE +-> ERROR at 2020-08-24 16:44:52.774858+00:00 | Both the stdout and XML output files could not be read or parsed. +-> ERROR at 2020-08-24 16:44:52.777151+00:00 | parser returned exit code<305>: Both the stdout and XML output files could not be read or parsed.
Daemon restart needed
Error messages
aiida.common.exceptions.MissingEntryPointError: Entry point 'quantumespresso.pw' not found in group 'aiida.parsers'. Try running `reentry scan` to update the entry point cache.
Potential fix Restart daemon (assuming reentry scan does not fix the problem).
More details
Below is an example of full output from verdi process report <pk>
$ verdi process report 85 *** 85: None *** (empty scheduler output file) *** Scheduler errors: The following modules were not unloaded: (Use "module --force purge" to unload all): 1) intel/19.0.4 2) mvapich2/2.3 3) texlive/2016 4) StdEnv Lmod is automatically replacing "mvapich2/2.3" with "impi/2019.8". *** 1 LOG MESSAGES: +-> REPORT at 2020-09-09 21:37:14.776260+00:00 | [85|PwCalculation|on_except]: Traceback (most recent call last): | File "/g/g12/keilbart/aiida_core/aiidavenv/lib/python3.6/site-packages/plumpy/process_states.py", line 220, in execute | result = self.run_fn(*self.args, **self.kwargs) | File "/g/g12/keilbart/aiida_core/aiida/engine/processes/calcjobs/calcjob.py", line 262, in parse | exit_code = execmanager.parse_results(self, retrieved_temporary_folder) | File "/g/g12/keilbart/aiida_core/aiida/engine/daemon/execmanager.py", line 412, in parse_results | parser_class = process.node.get_parser_class() | File "/g/g12/keilbart/aiida_core/aiida/orm/nodes/process/calculation/calcjob.py", line 489, in get_parser_class | return ParserFactory(parser_name) | File "/g/g12/keilbart/aiida_core/aiida/plugins/factories.py", line 158, in ParserFactory | entry_point = BaseFactory(entry_point_group, entry_point_name) | File "/g/g12/keilbart/aiida_core/aiida/plugins/factories.py", line 46, in BaseFactory | return load_entry_point(group, name) | File "/g/g12/keilbart/aiida_core/aiida/plugins/entry_point.py", line 202, in load_entry_point | entry_point = get_entry_point(group, name) | File "/g/g12/keilbart/aiida_core/aiida/plugins/entry_point.py", line 264, in get_entry_point | 'the entry point cache.'.format(name, group) | aiida.common.exceptions.MissingEntryPointError: Entry point 'quantumespresso.pw' not found in group 'aiida.parsers'. Try running `reentry scan` to update the entry point cache.
Note: These instructions should work for AiiDA v1.3 on LC systems. We are still testing v1.4, the most recent AiiDA release.