Difference between revisions of "EOS"

From DUNE
Jump to navigation Jump to search
 
(9 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
----
 +
----
 +
----
 +
----
 +
==<span style="background:yellow">This page has moved to the [https://wiki.dunescience.org/wiki/EOS DUNE wiki]</span>==
 +
----
 +
----
 +
----
 +
----
 +
 
=EOS and XRootD Links=
 
=EOS and XRootD Links=
 
* [http://linux.die.net/man/1/xrdcp xrdcp man page]
 
* [http://linux.die.net/man/1/xrdcp xrdcp man page]
Line 7: Line 17:
  
 
* [https://eos.readthedocs.io/en/latest/quickstart/client/configure.html An exceptionally useful page]
 
* [https://eos.readthedocs.io/en/latest/quickstart/client/configure.html An exceptionally useful page]
 +
 +
= Mounting /eos at CERN =
 +
<pre>
 +
export EOS_MGM_URL="root://eos.cern.ch"
 +
sudo yum install eos-client # if not already installed
 +
export EOS_FUSE_MOUNTDIR=/eos/
 +
sudo systemctl  start eosd
 +
</pre>
 +
 +
For reasons not understood at the time of writing, the last line
 +
needs to be substituted by the following in "some" cases:
 +
<pre>
 +
service eosd start
 +
</pre>
  
 
= URI of protoDUNE eos endpoint=
 
= URI of protoDUNE eos endpoint=
Line 13: Line 37:
 
root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testInput
 
root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testInput
 
</pre>
 
</pre>
 +
 +
An example how a custom installation of xrootd is used to copy:
 +
<pre>
 +
/opt/xrootd/bin/xrdcp root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testOutput/83/lxdqm_crt_tpc_1.root f1.root
 +
</pre>
 +
 +
Another random example (directory listing and file copy):
 +
<pre>
 +
xrdfs root://eospublic.cern.ch/ ls -ltr /eos/experiment/neutplatform/protodune/np04tier0/p3s/output/5ae6805c-f26f-11e7-98ed-02163e018872/
 +
xrdcp xroot://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/output/5ae6805c-f26f-11e7-98ed-02163e018872/adcprep_evt2571_ch0-2559.png .
 +
</pre>
 +
 +
  
 
An older ATLAS example:
 
An older ATLAS example:
Line 20: Line 57:
  
 
=EOS CLI ("eos")=
 
=EOS CLI ("eos")=
* Setting up EOS CLI at CERN: source /afs/cern.ch/project/eos/installation/[atlas|cms|lhcb|alice]/etc/setup.sh
+
==An important note==
 +
Sometimes recursive deletion of directories via FUSE fails with error messages like "directory not empty", while it still shows empty as reported by "ls -la.
 +
Using xrdfs won't work either. However, using the EOS CLI (see command list below) will work.
 +
<pre>
 +
# list hidden files
 +
eos root://eospublic.cern.ch ls -la /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/.sys*
 +
# remove offending files
 +
eos root://eospublic.cern.ch rm /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/.sys*
 +
# remove the directory
 +
eos root://eospublic.cern.ch rmdir /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/
 +
</pre>
 +
 
 +
...or:
 +
 
 +
<pre>
 +
eos root://eospublic.cern.ch rm -r /eos/experiment/neutplatform/protodune/np04tier0/p3s/monitor/f544b13e-7558-11e8-87d6-fa163eb4b51e
 +
</pre>
 +
 
 +
==Setting up EOS CLI at CERN==
 +
source /afs/cern.ch/project/eos/installation/[atlas|cms|lhcb|alice]/etc/setup.sh
 +
 
 +
 
 
<pre>
 
<pre>
 
bash-4.1$ more /afs/cern.ch/project/eos/installation/atlas/etc/setup.sh
 
bash-4.1$ more /afs/cern.ch/project/eos/installation/atlas/etc/setup.sh
Line 30: Line 88:
 
</pre>
 
</pre>
  
 +
 +
==Commands==
 
* Now you can use EOS CLI commands, e.g.: ls /eos/atlas/atlasdatadisk/rucio/data15_13TeV/ab/f4
 
* Now you can use EOS CLI commands, e.g.: ls /eos/atlas/atlasdatadisk/rucio/data15_13TeV/ab/f4
 
* FullEOS CLI list of commands:
 
* FullEOS CLI list of commands:

Latest revision as of 02:09, 22 June 2018





This page has moved to the DUNE wiki





EOS and XRootD Links

Mounting /eos at CERN

export EOS_MGM_URL="root://eos.cern.ch"
sudo yum install eos-client # if not already installed
export EOS_FUSE_MOUNTDIR=/eos/
sudo systemctl  start eosd

For reasons not understood at the time of writing, the last line needs to be substituted by the following in "some" cases:

service eosd start

URI of protoDUNE eos endpoint

An example actual as of June 2017:

root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testInput

An example how a custom installation of xrootd is used to copy:

/opt/xrootd/bin/xrdcp root://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/testOutput/83/lxdqm_crt_tpc_1.root f1.root

Another random example (directory listing and file copy):

xrdfs root://eospublic.cern.ch/ ls -ltr /eos/experiment/neutplatform/protodune/np04tier0/p3s/output/5ae6805c-f26f-11e7-98ed-02163e018872/
xrdcp xroot://eospublic.cern.ch//eos/experiment/neutplatform/protodune/np04tier0/p3s/output/5ae6805c-f26f-11e7-98ed-02163e018872/adcprep_evt2571_ch0-2559.png .


An older ATLAS example:

xrdcp root://eosatlas//eos/atlas/atlasdatadisk/rucio/data15_13TeV/ab/01/NTUP_TRIGRATE.05805712._003841.pool.root.1 .

EOS CLI ("eos")

An important note

Sometimes recursive deletion of directories via FUSE fails with error messages like "directory not empty", while it still shows empty as reported by "ls -la. Using xrdfs won't work either. However, using the EOS CLI (see command list below) will work.

# list hidden files
eos root://eospublic.cern.ch ls -la /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/.sys*
# remove offending files
eos root://eospublic.cern.ch rm /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/.sys*
# remove the directory
eos root://eospublic.cern.ch rmdir /eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/52be3f92-4412-11e8-9bbe-97cfa366db4d/

...or:

eos root://eospublic.cern.ch rm -r /eos/experiment/neutplatform/protodune/np04tier0/p3s/monitor/f544b13e-7558-11e8-87d6-fa163eb4b51e

Setting up EOS CLI at CERN

source /afs/cern.ch/project/eos/installation/[atlas|cms|lhcb|alice]/etc/setup.sh


bash-4.1$ more /afs/cern.ch/project/eos/installation/atlas/etc/setup.sh
# source me
alias eos="/afs/cern.ch/project/eos/installation/0.3.84-aquamarine/bin/eos.select"
alias eosumount="/afs/cern.ch/project/eos/installation/0.3.84-aquamarine/bin/eos.select -b fuse umount"
alias eosmount="/afs/cern.ch/project/eos/installation/0.3.84-aquamarine/bin/eos.select -b fuse mount"
alias eosforceumount="killall eosfsd 2>/dev/null; killall -9 eosfsd 2>/dev/null; fusermount -u "


Commands

  • Now you can use EOS CLI commands, e.g.: ls /eos/atlas/atlasdatadisk/rucio/data15_13TeV/ab/f4
  • FullEOS CLI list of commands:
access               Access Interface
archive              Archive Interface
attr                 Attribute Interface
backup               Backup Interface
clear                Clear the terminal
cd                   Change directory
chmod                Mode Interface
chown                Chown Interface
config               Configuration System
console              Run Error Console
cp                   Cp command
debug                Set debug level
dropbox              Drop box
exit                 Exit from EOS console
file                 File Handling
fileinfo             File Information
find                 Find files/directories
fs                   File System configuration
fsck                 File System Consistency Checking
fuse                 Fuse Mounting
group                Group configuration
help                 Display this text
io                   IO Interface
json                 Toggle JSON output flag for stdout
license              Display Software License
ls                   List a directory
map                  Path mapping interface
mkdir                Create a directory
motd                 Message of the day
node                 Node configuration
ns                   Namespace Interface
pwd                  Print working directory
quit                 Exit from EOS console
quota                Quota System configuration
reconnect            Forces a re-authentication of the shell
recycle              Recycle Bin Functionality
rmdir                Remove a directory
rm                   Remove a file
role                 Set the client role
rtlog                Get realtime log output from mgm & fst servers
silent               Toggle silent flag for stdout
space                Space configuration
stat                 Run 'stat' on a file or directory
test                 Run performance test
timing               Toggle timing flag for execution time measurement
transfer             Transfer Interface
version              Verbose client/server version
vid                  Virtual ID System Configuration
vst                  Virtual Storage Interface
whoami               Determine how we are mapped on server side
who                  Statistics about connected users
?                    Synonym for `help'
.q                   Exit from EOS console