DUNE LAr Software Releases

Introduction

This page describes recent releases of DUNE software for simulation and reconstruction of event data from the LAr TPCs and associated photon and muon detectors. The software is released in the dunetpc package. Earlier releases in the lbnecode package are described on this other page.

Finding and downloading DUNE releases

Releases of dunetpc (and other DUNE products) are installed at FNAL at

/grid/fermiapp/products/dune

and on CVMFS at

/cvmfs/dune.opensciencegrid.org/products/dune

A release and all its dependencies may be installed on SLF and Mac machines using the pullProducts command:

./pullProducts ~/ups slf6 dune-v04_36_01 e9 prof   # Scientific Linux 6
./pullProducts ~/ups d14 dune-v04_36_01 e9 prof    # mac Yosemite
./pullProducts ~/ups u14 dune-v04_36_01 e9 prof    # ubuntu

The first three arguments are the directory where the products are to be installed, the OS specifier and the product name and version. The last two are the UPS qualifiers. The last is the compiler option (prof is nearly optimized but retains the code line numbers) and the first is a colon-separated list of all other qualifiers (e.g. e7:s15, not e7-s15). The available OS and qualifiers for a given release may be deduced from the list of manifests found in the manifest directory for the desired dune release in the dune bundle listing.

The above command will copy many tar files to the current directory and sot it is best to run it in a dedicated directory and not in the product installation area. E.g. something like:

mkdir ~/upstars
cd ~/upstars
wget http://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts
chmod +x ./pullProducts
./pullProducts ~/ups slf6 dune-v04_21_00 e7 prof

Tar files for individual product versions (including dunetpc) may also be downloaded directly from the SciSoft product area. To install a product, simply untar these in the local products area:

cd ~/ups
tar xf ~/Downloads/lbnecode-04.20.00-slf6-x86_64-e7-prof.tar.bz2

If ups is not installed on you machine, then download and unpack the ups product tar file in this manner. Follow the link to the latest version from the SciSoft ups package list and then choose the version appropriate to your machine, e.g. ups-5.2.0-Darwin64bit+14.tar.bz2 for mac Yosemite.

Using DUNE releases

Scripts to set up the FNAL and CVMFS releases are provided. To set up to use release 4.21 from the FNAL installation:

source /grid/fermiapp/products/dune/setup_dune.sh
setup dunetpc v04_21_00 -q e7:prof

To use the CVMFS installation of the same release:

source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup dunetpc v04_21_00 -q e7:prof

For local installation of all products in ~/ups, set up with something like the following:

source ~/ups/setup      # Or source ~/ups/setups
setup git
setup gitflow
setup mrb
setup dunetpc v04_21_00 -q e7:prof

In all cases, the ups command can be used to find out which versions of dunetpc are available:

fnal:lbne> ups list -aK+ dunetpc 
"dunetpc" "v04_20_00" "Linux64bit+2.6-2.12" "e7:prof" "" 
"dunetpc" "v04_20_00" "Linux64bit+2.6-2.12" "debug:e7" "" 
"dunetpc" "v04_21_00" "Linux64bit+2.6-2.12" "e7:prof" "" 
"dunetpc" "v04_21_00" "Linux64bit+2.6-2.12" "debug:e7" ""

Building and installing dune releases

Build and installation instruction may be found on the Redmine CVMFS page. In particular, this page has the instructions for synchronizing the CVMFS releases with those in the FNAL grid area.

The code I (D. Adams) use to build releases is dune-larrel. See README.md in that package or use "larrel -H" for instructions. The product of the build is a tar file and manifest for each dune release combination of OS, version, qualifier, etc.

Once the release tag is made, then the product must be built for each combination of platform (SLF6, OSX-10.10, ...) and optimization level (prof, debug). An alternative for supported platforms (at presnt SLF6, OSX-10.9 and OSX-10.10) is to use Jenkins: load a KCA certificate in your browser (e.g. with get-cert and import), go to https://buildmaster.fnal.gov/view/LBNE/job/dune-release-build/, click on "Rebuild Last", change the release and submit. The script ~dladams/bin/copyFromJenkins or modified ~dladams/bin/jenkinscp can then be used to copy the product tar files and manifests to the (empty) current directory.

To install the SLF6 products in the FNAL grid area:

ssh dune@dunegpvm03.fnal.gov
cd /grid/fermiapp/products/dune
tar xf /afs/fnal.gov/files/home/room1/dladams/reltars/dunecode-04.20.00-slf6-x86_64-e7-prof.tar.bz2

To synchronize the CVMFS area with the grid area:

ssh -l cvmfsdune oasiscfs02.fnal.gov
dune-sync dune/dunetpc

Replace dunetpc with dune_pardata, duneutil or lbne_raw_data to update any of those products. The script dune-sync is installed on the oasis server and may also be found at https://github.com/dladams/dune-larrel/blob/master/dune-sync.

Each synchronization generates a log in $HOME/logs and a CVMFS server tag. Periodically remove these. The tags may be listed with

cvmfs_server tag dune.opensciencegrid.org

and tags removed with

cvmfs_server tag -r TAG dune.opensciencegrid.org

To copy each product tar file to SciSoft and install the corresponding manifest:

 copyToSciSoft /afs/fnal.gov/files/home/room1/dladams/reltars/dunecode-04.20.00-slf6-x86_64-e6-prof.tar.bz2
 copyToSciSoft workdir/manifests/dune-04.21.00-Linux64bit+2.6-2.12-e7-prof_MANIFEST.txt

The command script may be found at /afs/fnal.gov/files/home/room1/dladams/bin/copyToSciSoft or http://scisoft.fnal.gov/scisoft/bundles/tools/copyToSciSoft. In case of error, log in to scisoftportal.fnal.gov and (very carefully) remove invalid files from /nasroot/SciSoft/bundles/....

More information

DUNE:

• dunetpc wiki
• dunetpc issues
• SciSoft dune bundles
• SciSoft packages
  • dunetpc
  • duneutil
  • lbne_raw_data
• Units are cm and ns (Tom, January 2015).
• LBNE/DUNE job submission (talk by Tingjjun)
• Debugging DUNE far detector SW

LArSoft:

• LArSoft wiki
• Getting LArSoft
• SciSoft bundles
• Issues
• LArSoft mailing list.

Art:

• Wiki
• Redmine code browser
• LXR
• art workbook
• August 2015 art/LArSoft course
• Message facility
• Services wiki
• Issues
• cetbuildtools defines many useful CMake commands

Building:

• LBNE tutorial
• My package to simplify building is dune-dev
• art instructions for cmake. Up-to-date instructions may be found at the top of $ART_DIR/Modules/ArtMake.cmake and friends.
• Building with cetbuildtools but without mrb.

FHICL:

• FHICL wiki
• Quick start guide

Release building:

• Instructions for any product.
• Jenkins
• Instructions for getting a KCA certificate.
• Build qualifiers.
• mrb reference guide

35 ton prototype:

• Wiki
• Splitting and stitching data
• ZS and compression algorithms
• List of 35-ton data files. Get the URL for anyn of these at FNAL with "~dladams/bin/find35t NAME".
• Files in run control

LBNE 35-ton Data Model and Reconstruction

DUNE and LarSoft geometry

LBNE code repositories and their browsers

LBNE jobs may be submitted using the package lbneutil described in a talk by Tingjun.

LBNE geometries are summarized here.

The reconstruction used in the 35-ton simulation is specified in lbnecode/source/fcl/lbne35t/reco/standard_reco_lbne35t.fcl.

Data production

• The 35-ton MC data samples are described here.
• MCC5

GENIE event generator

• 2015 user's manual
• 2010 user's manual
• developer's manual.

Results from LbTupler