Difference between revisions of "EOS"
Jump to navigation
Jump to search
(8 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
+ | ---- | ||
+ | ---- | ||
+ | ---- | ||
+ | ---- | ||
+ | ==<span style="background:yellow">This page has moved to the [https://wiki.dunescience.org/wiki/EOS DUNE wiki]</span>== | ||
+ | ---- | ||
+ | ---- | ||
+ | ---- | ||
+ | ---- | ||
+ | |||
=EOS and XRootD Links= | =EOS and XRootD Links= | ||
* [http://linux.die.net/man/1/xrdcp xrdcp man page] | * [http://linux.die.net/man/1/xrdcp xrdcp man page] | ||
Line 7: | Line 17: | ||
* [https://eos.readthedocs.io/en/latest/quickstart/client/configure.html An exceptionally useful page] | * [https://eos.readthedocs.io/en/latest/quickstart/client/configure.html An exceptionally useful page] | ||
+ | |||
+ | = Mounting /eos at CERN = | ||
+ | <pre> | ||
+ | export EOS_MGM_URL="root://eos.cern.ch" | ||
+ | sudo yum install eos-client # if not already installed | ||
+ | export EOS_FUSE_MOUNTDIR=/eos/ | ||
+ | sudo systemctl start eosd | ||
+ | </pre> | ||
+ | |||
+ | For reasons not understood at the time of writing, the last line | ||
+ | needs to be substituted by the following in "some" cases: | ||
+ | <pre> | ||
+ | service eosd start | ||
+ | </pre> | ||
= URI of protoDUNE eos endpoint= | = URI of protoDUNE eos endpoint= | ||
Line 14: | Line 38: | ||
</pre> | </pre> | ||
− | An example how a custom installation of xrootd is used to | + | An example how a custom installation of xrootd is used to copy: |
<pre> | <pre> | ||
/opt/xrootd/bin/xrdcp root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testOutput/83/lxdqm_crt_tpc_1.root f1.root | /opt/xrootd/bin/xrdcp root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testOutput/83/lxdqm_crt_tpc_1.root f1.root | ||
</pre> | </pre> | ||
+ | |||
+ | Another random example (directory listing and file copy): | ||
+ | <pre> | ||
+ | xrdfs root://eospublic.cern.ch/ ls -ltr /eos/experiment/neutplatform/protodune/np04tier0/p3s/output/5ae6805c-f26f-11e7-98ed-02163e018872/ | ||
+ | xrdcp xroot://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/output/5ae6805c-f26f-11e7-98ed-02163e018872/adcprep_evt2571_ch0-2559.png . | ||
+ | </pre> | ||
+ | |||
Line 26: | Line 57: | ||
=EOS CLI ("eos")= | =EOS CLI ("eos")= | ||
− | * Setting up EOS CLI at CERN | + | ==An important note== |
+ | Sometimes recursive deletion of directories via FUSE fails with error messages like "directory not empty", while it still shows empty as reported by "ls -la. | ||
+ | Using xrdfs won't work either. However, using the EOS CLI (see command list below) will work. | ||
+ | <pre> | ||
+ | # list hidden files | ||
+ | eos root://eospublic.cern.ch ls -la /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/.sys* | ||
+ | # remove offending files | ||
+ | eos root://eospublic.cern.ch rm /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/.sys* | ||
+ | # remove the directory | ||
+ | eos root://eospublic.cern.ch rmdir /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/ | ||
+ | </pre> | ||
+ | |||
+ | ...or: | ||
+ | |||
+ | <pre> | ||
+ | eos root://eospublic.cern.ch rm -r /eos/experiment/neutplatform/protodune/np04tier0/p3s/monitor/f544b13e-7558-11e8-87d6-fa163eb4b51e | ||
+ | </pre> | ||
+ | |||
+ | ==Setting up EOS CLI at CERN== | ||
+ | source /afs/cern.ch/project/eos/installation/[atlas|cms|lhcb|alice]/etc/setup.sh | ||
+ | |||
+ | |||
<pre> | <pre> | ||
bash-4.1$ more /afs/cern.ch/project/eos/installation/atlas/etc/setup.sh | bash-4.1$ more /afs/cern.ch/project/eos/installation/atlas/etc/setup.sh | ||
Line 36: | Line 88: | ||
</pre> | </pre> | ||
+ | |||
+ | ==Commands== | ||
* Now you can use EOS CLI commands, e.g.: ls /eos/atlas/atlasdatadisk/rucio/data15_13TeV/ab/f4 | * Now you can use EOS CLI commands, e.g.: ls /eos/atlas/atlasdatadisk/rucio/data15_13TeV/ab/f4 | ||
* FullEOS CLI list of commands: | * FullEOS CLI list of commands: |
Latest revision as of 02:09, 22 June 2018
Contents
This page has moved to the DUNE wiki
EOS and XRootD Links
- xrdcp man page
- Setting up EOS
- Beginner's EOS Tutorial
- A helpful page on EOS basics
- Yet another set of examples
Mounting /eos at CERN
export EOS_MGM_URL="root://eos.cern.ch" sudo yum install eos-client # if not already installed export EOS_FUSE_MOUNTDIR=/eos/ sudo systemctl start eosd
For reasons not understood at the time of writing, the last line needs to be substituted by the following in "some" cases:
service eosd start
URI of protoDUNE eos endpoint
An example actual as of June 2017:
root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testInput
An example how a custom installation of xrootd is used to copy:
/opt/xrootd/bin/xrdcp root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testOutput/83/lxdqm_crt_tpc_1.root f1.root
Another random example (directory listing and file copy):
xrdfs root://eospublic.cern.ch/ ls -ltr /eos/experiment/neutplatform/protodune/np04tier0/p3s/output/5ae6805c-f26f-11e7-98ed-02163e018872/ xrdcp xroot://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/output/5ae6805c-f26f-11e7-98ed-02163e018872/adcprep_evt2571_ch0-2559.png .
An older ATLAS example:
xrdcp root://eosatlas//eos/atlas/atlasdatadisk/rucio/data15_13TeV/ab/01/NTUP_TRIGRATE.05805712._003841.pool.root.1 .
EOS CLI ("eos")
An important note
Sometimes recursive deletion of directories via FUSE fails with error messages like "directory not empty", while it still shows empty as reported by "ls -la. Using xrdfs won't work either. However, using the EOS CLI (see command list below) will work.
# list hidden files eos root://eospublic.cern.ch ls -la /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/.sys* # remove offending files eos root://eospublic.cern.ch rm /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/.sys* # remove the directory eos root://eospublic.cern.ch rmdir /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/
...or:
eos root://eospublic.cern.ch rm -r /eos/experiment/neutplatform/protodune/np04tier0/p3s/monitor/f544b13e-7558-11e8-87d6-fa163eb4b51e
Setting up EOS CLI at CERN
source /afs/cern.ch/project/eos/installation/[atlas|cms|lhcb|alice]/etc/setup.sh
bash-4.1$ more /afs/cern.ch/project/eos/installation/atlas/etc/setup.sh # source me alias eos="/afs/cern.ch/project/eos/installation/0.3.84-aquamarine/bin/eos.select" alias eosumount="/afs/cern.ch/project/eos/installation/0.3.84-aquamarine/bin/eos.select -b fuse umount" alias eosmount="/afs/cern.ch/project/eos/installation/0.3.84-aquamarine/bin/eos.select -b fuse mount" alias eosforceumount="killall eosfsd 2>/dev/null; killall -9 eosfsd 2>/dev/null; fusermount -u "
Commands
- Now you can use EOS CLI commands, e.g.: ls /eos/atlas/atlasdatadisk/rucio/data15_13TeV/ab/f4
- FullEOS CLI list of commands:
access Access Interface archive Archive Interface attr Attribute Interface backup Backup Interface clear Clear the terminal cd Change directory chmod Mode Interface chown Chown Interface config Configuration System console Run Error Console cp Cp command debug Set debug level dropbox Drop box exit Exit from EOS console file File Handling fileinfo File Information find Find files/directories fs File System configuration fsck File System Consistency Checking fuse Fuse Mounting group Group configuration help Display this text io IO Interface json Toggle JSON output flag for stdout license Display Software License ls List a directory map Path mapping interface mkdir Create a directory motd Message of the day node Node configuration ns Namespace Interface pwd Print working directory quit Exit from EOS console quota Quota System configuration reconnect Forces a re-authentication of the shell recycle Recycle Bin Functionality rmdir Remove a directory rm Remove a file role Set the client role rtlog Get realtime log output from mgm & fst servers silent Toggle silent flag for stdout space Space configuration stat Run 'stat' on a file or directory test Run performance test timing Toggle timing flag for execution time measurement transfer Transfer Interface version Verbose client/server version vid Virtual ID System Configuration vst Virtual Storage Interface whoami Determine how we are mapped on server side who Statistics about connected users ? Synonym for `help' .q Exit from EOS console