Difference between revisions of "DUNE LAr Software Releases"

From DUNE
Jump to navigation Jump to search
 
(183 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
 +
----
 +
----
 +
----
 +
----
 +
==<span style="background:yellow">This page has moved to the [https://wiki.dunescience.org/wiki/DUNE_LAr_Software_Releases DUNE wiki]</span>==
 +
Below is an out-of-date snapshot of the page.
 +
----
 +
----
 +
----
 +
----
 +
 
==Introduction==
 
==Introduction==
  
−
This page describes recent releases of DUNE software for simulation and reconstruction of event data from the LAr TPCs and associated photon and muon detectors. The software is released in the dunetpc package. Earlier releases in the lbnecode package are described on [[LBNE_Far_Detector_Software_Releases | this page]].
+
This page describes recent releases of DUNE software for simulation and reconstruction of event data from the LAr TPCs and associated photon and muon detectors. The software is released in the dunetpc package. Earlier releases in the lbnecode package are described on [[LBNE_Far_Detector_Software_Releases | this other page]].
−
 
 
−
*** This page was copied from the lbnecode page and is currently under construction. ***
 
  
 
==Finding and downloading DUNE releases==
 
==Finding and downloading DUNE releases==
Line 12: Line 22:
 
: <code>/cvmfs/dune.opensciencegrid.org/products/dune</code>
 
: <code>/cvmfs/dune.opensciencegrid.org/products/dune</code>
  
−
A release and all its dependencies may be installed on any machine using the [http://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts pullProducts] command:
+
A release and all its dependencies may be installed on SLF and Mac machines using the [http://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts pullProducts] command:
 +
<code>
 +
./pullProducts ~/ups slf6 dune-v06_58_00 e14 prof  # Scientific Linux 6
 +
./pullProducts ~/ups slf7 dune-v06_58_00 e14 prof  # Scientific Linux 7
 +
./pullProducts ~/ups d15 dune-v06_58_00 e14 prof    # mac OS X 10.11 El Capitan
 +
./pullProducts ~/ups d16 dune-v06_58_00 e14 prof    # macOS 10.12 Sierra
 +
</code>
 +
The first three arguments are the directory where the products are to be installed, the OS specifier and the product name and version. The last two are the UPS [https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/AboutQualifiers qualifiers]. The last is the compiler option (prof is nearly optimized but retains the code line numbers) and the first is a colon-separated list of all other qualifiers (e.g. e7:s15, not e7-s15). The available OS and qualifiers for a given release may be deduced from the list of manifests found in the manifest directory for the desired dune release in the [http://scisoft.fnal.gov/scisoft/bundles/dune dune bundle listing].
 +
 
 +
Note that versions above are the latest available at the time of writing (typically updated each time a release is made). Older builds are available for some platforms that are no longer supported and the latest of those are:
 +
<code>
 +
./pullProducts ~/ups d13 dune-v06_13_00 e10 prof    # mac OS X 10.9 Mavericks
 +
./pullProducts ~/ups d14 dune-v06-56_00 e14 prof    # mac OS X 10.10 Yosemite
 +
./pullProducts ~/ups u14 dune-v06_06_01 e10 prof    # Ubuntu 14
 +
</code>
 +
These are likely too far out of date to be useful.
 +
 
 +
The above commands will copy many tar files to the current directory and so it is best to run it in a dedicated directory and not in the product installation area. E.g. something like:
 +
 
 +
Before first installation of UPS products, [http://scisoft.fnal.gov/scisoft/bundles/tools/checkPrerequisites checkPrerequisites] can be run to see if all prerequisite packages are installed.
 +
 
 
<code>
 
<code>
−
  ./pullProducts ~/ups slf6 dune-v04_21_00 e7 prof
+
mkdir ~/upstars
 +
cd ~/upstars
 +
wget http://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts
 +
chmod +x ./pullProducts
 +
  ./pullProducts ~/ups slf6 dune-v06_58_00 e14 prof
 
</code>
 
</code>
−
The first three arguments are the directory where the products are to be installed, the OS specifier and the product name and version. The last two are the UPS qualifier and compiler optimization level (prof is nearly optimized but retains the code line numbers). The available OS, qualifiers and optimizations for given release may be deduced from the list of manifests found in the manifest directory for the desired dune release in the [http://scisoft.fnal.gov/scisoft/bundles/dune dune bundle listing].
 
  
−
Tar files for individual package versions (including dunetpc) may be downloaded from the [http://scisoft.fnal.gov/scisoft/packages SciSoft product area]. To install a product, simply untar these in the local products area, e.g.
+
Tar files for individual product versions (including dunetpc) may also be downloaded directly from the [http://scisoft.fnal.gov/scisoft/packages SciSoft product area]. To install a product, simply untar these in the local products area:
  
 
<code>
 
<code>
 
  cd ~/ups
 
  cd ~/ups
−
  tar xf ~/Downloads/lbnecode-04.20.00-slf6-x86_64-e7-prof.tar.bz2
+
  tar xf ~/Downloads/duneutil-06.56.01-slf6-x86_64-e14-prof.tar.bz2
 
</code>
 
</code>
 +
 +
Users of dunetpc develop at remote sites will sometimes need to do this when the dunetpc dependencies have advanced beyond those in the last dunetpc build. For example the above is needed on 20nov2017.
 +
 +
If ups is not installed on your machine, then download and unpack the ups product tar file in this manner. Follow the link to the latest version from the [http://scisoft.fnal.gov/scisoft/packages/ups SciSoft ups package list] and then choose the version appropriate to your machine, e.g. [http://scisoft.fnal.gov/scisoft/packages/ups/v6_0_5/ups-6.0.5-Darwin64bit%2B16.tar.bz2 ups-6.0.5-Darwin64bit+16.tar.bz2] for mac Sierra (Nov 2017).
 +
 +
Note that SIP must be disabled and openssl installed (keg-only) for mac El Capitan and Sierra. For more information, see the [https://cdcvs.fnal.gov/redmine/projects/larsoft/wiki/_Supported_Platforms_ LArSoft supported platforms page].
 +
 +
UPS can be very picky about the OS version for Ubuntu. To install and use the above Ubuntu 14 releases on an Ubuntu 16 or different Ubuntu 14 sub-version machine, first set the UPS version overrride in one of the following ways:
 +
 +
* Set the UPS_OVERRIDE environmental variable:
 +
<code>
 +
export UPS_OVERRIDE="-H Linux64bit+3.19-2.19"
 +
</code>
 +
 +
* Add an override file to the product directory:
 +
<code>
 +
echo "-H Linux64bit+3.19-2.19" > <product_directory>/ups_OVERRIDE.`hostname`
 +
</code>
 +
This is specific to each host. If you have a shared volume, you will need to do this for each machine.
  
 
==Using DUNE releases==
 
==Using DUNE releases==
−
Scripts to set up the FNAL and CVMFS releases are provided, e.g. for the FNAL installation:
+
Scripts to set up the FNAL and CVMFS releases are provided. To set up to use release 4.21 from the FNAL installation:
 +
<code>
 +
source /grid/fermiapp/products/dune/setup_dune.sh
 +
setup dunetpc v06_04_00 -q e14:prof
 +
</code>
 +
 
 +
To use the CVMFS installation of the same release:
 +
<code>
 +
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
 +
setup dunetpc v06_46_00 -q e14:prof
 +
</code>
 +
 
 +
For local installation of all products in ~/ups, set up with something like the following:
 +
<code>
 +
source ~/ups/setup      # Or source ~/ups/setups
 +
setup git
 +
setup gitflow
 +
setup mrb
 +
setup dunetpc v06_55_00 -q e14:prof
 +
</code>
 +
 
 +
Note that ~/ups/setups will not work with a UPS override file (env override is OK). In that case, use ~/ups/setup.
 +
 
 +
In all cases, the ups command can be used to find out which versions of dunetpc are available:
 +
<code>
 +
fnal:lbne> ups list -aK+ dunetpc
 +
"dunetpc" "v04_20_00" "Linux64bit+2.6-2.12" "e7:prof" ""
 +
"dunetpc" "v04_20_00" "Linux64bit+2.6-2.12" "debug:e7" ""
 +
"dunetpc" "v04_21_00" "Linux64bit+2.6-2.12" "e7:prof" ""
 +
"dunetpc" "v04_21_00" "Linux64bit+2.6-2.12" "debug:e7" ""
 +
</code>
 +
 
 +
Useful commands:
 
<code>
 
<code>
−
   source /grid/fermiapp/products/dune/setup_dune.sh
+
   lar -c someconfig.fcl          # Use art event loop to process data according to the instructions in someconfig.fcl
 +
  fcldump someconfig.fcl 5      # Display the highest five levels of configuration in someconfig.fcl
 +
  listlibs type pattern          # Display the full path for all libraries of type "type" (e.g. service or module) whose full path name includes "pattern"
 
</code>
 
</code>
−
A similar file for CVMFS will be added soon. These are not complicated can be used as a template for setting up a local installation. The latter will be even simpler if all products are installed in the same directory, e.g. at <code>$PRODUCTS</code>.
 
  
−
==Building and installing lbnecode releases==
+
If you want to develop code, i.e. modify the code in dunetpc, larsoft, etc. or add code from any other repositories, you can use the dune-dev package. Instructions may be found at https://github.com/dladams/dune-dev/blob/master/README.md.
 +
 
 +
==Building and installing dune releases==
  
 
Build and installation instruction may be found on the [https://cdcvs.fnal.gov/redmine/projects/cvmfs/wiki/CVMFS_Tasks|'''Redmine CVMFS page''']. In particular, this page has the instructions for synchronizing the CVMFS releases with those in the FNAL grid area.
 
Build and installation instruction may be found on the [https://cdcvs.fnal.gov/redmine/projects/cvmfs/wiki/CVMFS_Tasks|'''Redmine CVMFS page''']. In particular, this page has the instructions for synchronizing the CVMFS releases with those in the FNAL grid area.
  
−
The code I (D. Adams) use to build releases is [https://github.com/dladams/dune-larrel dune-larrel].
+
The code I (D. Adams) use to build releases is [https://github.com/dladams/dune-larrel dune-larrel]. See [https://github.com/dladams/dune-larrel/blob/master/README.md README.md] in that package or use "<code>larrel -H</code>" for instructions. The (optional) products of the build include a release tag of dunetpc (in the repository) and a tar file and manifest for each dune release combination of OS, version, qualifier, etc. The latter can be installed in SciSoft as described below.
  
−
To install in FNAL grid area:
+
Once the release tag is made, then the product must be built for each combination of platform (SLF6, OSX-10.10, ...) and optimization level (prof, debug). An alternative for supported platforms (at presnt SLF6, OSX-10.9 and OSX-10.10) is to use Jenkins: load a KCA certificate in your browser (e.g. with get-cert and import), go to https://buildmaster.fnal.gov/view/LBNE/job/dune-release-build/, click on "Rebuild Last", change the release and submit. The script ~dladams/bin/copyFromJenkins or modified ~dladams/bin/jenkinscp can then be used to copy the product tar files and manifests to the (empty) current directory.
 +
 
 +
One or more builds often fail on Jenkins. To retry the failed builds, click on the build date to get to the page for that build and then click on "Matrix Reloaded" and then "Rebuild".
 +
 
 +
The above link to Jenkins will only work from inside the FNAL domain. I have not been able to get things to work properly with VPN or an ssh tunnel. I start a browser on an FNAL machine and send the window over the network. The commands I use:
 +
<code>
 +
get-cert -i
 +
firefox --no-remote
 +
</code>
 +
or use ~dladams/bin/jenkins.
 +
 
 +
A third option for building the product tar file and manifest is to use the script larsoft/larutils/buildScripts/buildDUNE.sh.
 +
 
 +
To install the SLF6 products in the FNAL grid area:
 
<code>
 
<code>
−
  ssh dune@lbnegpvm03.fnal.gov
+
  ssh dune@dunegpvm03.fnal.gov
 
  cd /grid/fermiapp/products/dune
 
  cd /grid/fermiapp/products/dune
−
  tar xf /afs/fnal.gov/files/home/room1/dladams/reltars/dunecode-04.20.00-slf6-x86_64-e7-prof.tar.bz2
+
  tar xf ~dladams/reltars/dunetpc-05.05.00-slf6-x86_64-e9-prof.tar.bz2
 
</code>
 
</code>
  
 
To synchronize the CVMFS area with the grid area:
 
To synchronize the CVMFS area with the grid area:
 
<code>
 
<code>
−
get-cert
+
  ssh -l cvmfsdune oasiscfs.fnal.gov
−
voms-proxy-init -noregen -voms lbne:/lbne/Role=Analysis
 
−
  ssh -l cvmfsdune oasiscfs02.fnal.gov
 
 
  dune-sync dune/dunetpc
 
  dune-sync dune/dunetpc
 
</code>
 
</code>
Line 64: Line 162:
 
</code>
 
</code>
  
−
To copy a tar file to SciSoft:
+
To copy each product tar file to SciSoft and install the corresponding manifest:
 
<code>
 
<code>
 
   copyToSciSoft /afs/fnal.gov/files/home/room1/dladams/reltars/dunecode-04.20.00-slf6-x86_64-e6-prof.tar.bz2
 
   copyToSciSoft /afs/fnal.gov/files/home/room1/dladams/reltars/dunecode-04.20.00-slf6-x86_64-e6-prof.tar.bz2
 +
  copyToSciSoft workdir/manifests/dune-04.21.00-Linux64bit+2.6-2.12-e7-prof_MANIFEST.txt
 
</code>
 
</code>
 
The command script may be found at /afs/fnal.gov/files/home/room1/dladams/bin/copyToSciSoft or http://scisoft.fnal.gov/scisoft/bundles/tools/copyToSciSoft.
 
The command script may be found at /afs/fnal.gov/files/home/room1/dladams/bin/copyToSciSoft or http://scisoft.fnal.gov/scisoft/bundles/tools/copyToSciSoft.
 +
In case of error, log in to scisoftportal.fnal.gov and (very carefully) remove invalid files from <code>/nasroot/SciSoft/bundles/...</code>.
 +
 +
After installation, the release should be announced on dune-computing-news AT fnal.gov.
  
 
==More information==
 
==More information==
  
−
Instructions for running and building lbnecode and larsoft at FNAL using mrb, git, etc. my be found [https://cdcvs.fnal.gov/redmine/projects/lbnecode/wiki/_Tutorial_ here].
+
DUNE:
 +
* [https://cdcvs.fnal.gov/redmine/projects/dunetpc/wiki dunetpc wiki]
 +
* [https://cdcvs.fnal.gov/redmine/projects/dunetpc dunetpc issues]
 +
* [http://scisoft.fnal.gov/scisoft/bundles/dune SciSoft dune bundles]
 +
* SciSoft packages
 +
** [http://scisoft.fnal.gov/scisoft/packages/dunetpc dunetpc]
 +
** [http://scisoft.fnal.gov/scisoft/packages/duneutil duneutil]
 +
** [http://scisoft.fnal.gov/scisoft/packages/lbne_raw_data lbne_raw_data]
 +
** [http://scisoft.fnal.gov/scisoft/packages/dune_pardata dune_pardata]
 +
* [https://cdcvs.fnal.gov/redmine/projects/dunetpc/repository dunetpc browser]
 +
* Units are cm and ns (Tom, January 2015).
 +
* [http://lbne2-docdb.fnal.gov:8080/cgi-bin/RetrieveFile?docid=10280&filename=projectpy.pdf LBNE/DUNE job submission (talk by Tingjjun)]
 +
* [[Debugging_DUNE_far_detector_SW | Debugging DUNE far detector SW]]
  
−
A package intended to simplify these procedure and allow running and development at other sites is [https://github.com/dladams/lbne-dev lbne-dev].
+
LArSoft:
 +
* [https://cdcvs.fnal.gov/redmine/projects/larsoft/wiki LArSoft wiki]
 +
* [https://cdcvs.fnal.gov/redmine/projects/larsoft/wiki/Getting_LArSoft Getting LArSoft]
 +
* [http://scisoft.fnal.gov/scisoft/bundles/larsoft SciSoft bundles]
 +
* [https://cdcvs.fnal.gov/redmine/projects/larsoft/wiki/LArSoft_release_list Release notes]
 +
* [https://cdcvs.fnal.gov/redmine/projects/larsoft/issues Issues]
 +
* [mailto:larsoft@fnal.gov LArSoft mailing list].
 +
* [https://cdcvs.fnal.gov/redmine/projects/larsoft/wiki/LArSoft_git_Guidelines Git guidelines]
 +
* [https://cdcvs.fnal.gov/redmine/projects/artextensions/wiki/SeedService SeedService] (See also the [https://indico.fnal.gov/getFile.py/access?contribId=7&resId=0&materialId=slides&confId=11915 April 2016 talk by Gianluca])
 +
* [http://larsoft.org/list LArSoft algorithm documentation]
 +
* LArSoft documents in INSPIRE: [http://larsoft.org/links-to-documents Instructions], [https://inspirehep.net/youraccount/login Log in], [https://inspirehep.net/search?ln=en&cc=LArSoft+Notes&p=&action_search=Search, Search]
  
−
The DUNE LArSoft-based SW lbnecode may be downloaded from [http://scisoft.fnal.gov/scisoft/packages/lbnecode on SciSoft]. It may browsed using [https://cdcvs.fnal.gov/redmine/projects/lbnecode/repository/revisions/develop/show/lbne Redmine].
+
Art:
 +
* [https://cdcvs.fnal.gov/redmine/projects/art/wiki Wiki]
 +
* [https://cdcvs.fnal.gov/redmine/projects/art/repository/revisions/master/show/art Redmine code browser]
 +
* [http://cdcvs.fnal.gov/lxr/art LXR]
 +
* [https://web.fnal.gov/project/artdoc/shared%20documents/art-documentation.pdf art workbook]
 +
* [https://indico.fnal.gov/conferenceTimeTable.py?confId=9928#all.detailed August 2015 art/LArSoft course]
 +
* [https://cdcvs.fnal.gov/redmine/projects/messagefacility/wiki/Using_MessageFacility Message facility]
 +
* [https://cdcvs.fnal.gov/redmine/projects/art/wiki/Guide_to_writing_and_using_services Services wiki]
 +
* [https://cdcvs.fnal.gov/redmine/projects/art/wiki/Guide_to_writing_and_using_tools Art tools]
 +
* [https://cdcvs.fnal.gov/redmine/projects/art/issues Issues]
 +
* [https://cdcvs.fnal.gov/redmine/projects/cetbuildtools/wiki cetbuildtools defines many useful CMake commands]
  
−
LArSoft is documented on the [https://cdcvs.fnal.gov/redmine/projects/larsoft/wiki LArSoft wiki]. See the [https://cdcvs.fnal.gov/redmine/projects/larsoft/wiki/Getting_LArSoft Getting LArSoft link] on that page for instructions on downloading LArSoft from [http://scisoft.fnal.gov/scisoft/bundles/ scisoft] or accessing it via CVMFS. For issues, follow the menu links "Issues" or "New issue" on the LArSoft [https://cdcvs.fnal.gov/redmine/projects/larsoft Redmine page]. Questions about LArSoft may be directed to the [mailto:larsoft@fnal.gov LArSoft mailing list].
+
Building:
 +
* [https://cdcvs.fnal.gov/redmine/projects/mrb/wiki/MrbRefereceGuide mrb reference guide]
 +
* [https://cdcvs.fnal.gov/redmine/projects/lbnecode/wiki/_Tutorial_ LBNE tutorial]
 +
* My package to simplify building is [https://github.com/dladams/dune-dev dune-dev]
 +
* [https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/Art_cmake_modules art instructions for cmake]. Up-to-date instructions may be found at the top of $ART_DIR/Modules/ArtMake.cmake and friends.
 +
* [https://cdcvs.fnal.gov/redmine/projects/mrb/wiki/WithoutMRB Building with cetbuildtools but without mrb].
  
−
[https://cdcvs.fnal.gov/redmine/projects/art/wiki Art] code may be browsed with [https://cdcvs.fnal.gov/redmine/projects/art/repository/revisions/master/show/art Redmine] or [http://cdcvs.fnal.gov/lxr/art LXR]. Also, see the [https://web.fnal.gov/project/artdoc/shared%20documents/art-documentation.pdf art workbook].
+
FHICL:
 +
* [https://cdcvs.fnal.gov/redmine/projects/fhicl/wiki FHICL wiki]
 +
* [https://cdcvs.fnal.gov/redmine/attachments/download/29136/quick_start_v3.pdf Quick start guide]
  
−
The art instructions for cmake are described in [https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/Art_cmake_modules redmine]. Upd-to-date instructions may be found at the top of $ART_DIR/Modules/ArtMake.cmake and friends
+
Release building:
 +
* [https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/BuildBinaryDistributionManifest Instructions for any product].
 +
* [https://buildmaster.fnal.gov Jenkins]
 +
* [http://docdb.fnal.gov/ILC-public/DocDB/DocDBInstructions?set=kca Instructions for getting a KCA certificate].
 +
* [https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/AboutQualifiers Build qualifiers].
 +
* [https://cdcvs.fnal.gov/redmine/projects/mrb/wiki/MrbRefereceGuide mrb reference guide]
  
−
The qualifiers used to specify compilers and other products are described [https://cdcvs.fnal.gov/redmine/projects/cet-is-public/wiki/AboutQualifiers here].
+
35 ton prototype:
−
 
+
* [https://cdcvs.fnal.gov/redmine/projects/35ton/wiki Wiki]
−
Here are [https://cdcvs.fnal.gov/redmine/projects/uboonecode/wiki/Tagging tagging instructions for uBooNE]
+
* [https://cdcvs.fnal.gov/redmine/projects/dunetpc/wiki/Using_the_stitchersplitter_module Splitting and stitching data]
 +
* [https://cdcvs.fnal.gov/redmine/projects/35ton/wiki/TSU_Counter_Locations Trigger counter indices and locations]
 +
* [https://cdcvs.fnal.gov/redmine/projects/35ton/wiki/Data_compression_and_zero_suppression ZS and compression algorithms]
 +
* [http://dune-data.fnal.gov/data/35t/testfiles/filelist.txt List of 35-ton data files.] Get the URL for any of these at FNAL with "~dladams/bin/find35t NAME".
 +
* [http://dune-moni.umd.edu/lbnerc Files in run control]
 +
* [https://cdcvs.fnal.gov/redmine/projects/35ton/wiki/Channel_map Channel map]
 +
* [https://cdcvs.fnal.gov/redmine/projects/35ton/wiki/TPC_Specific_FEMB_Configuration_Setting Gains and shaping times]
 +
* [http://dbweb5.fnal.gov:8080/ECL/lbne_35t/E/index Logbook]
 +
* Monitoring
 +
** [http://lbne-dqm.fnal.gov/OnlineMonitoring Online monitoring]
 +
** [http://lbne-dqm.fnal.gov/NearlineMonitoring Nearline monitoring]
 +
** [http://lbne-dqm.fnal.gov/EventDisplay Online event display]
 +
* [http://dbdata0vm.fnal.gov:8116/LBNE35tCon/app/get?table=dune35t.pedestals&type=data&t=11507&columns=mean,rms Example pedestal URL]
 +
* [https://web.fnal.gov/collaboration/DUNE/LAr%2035t%20Prototype%20Documents/Shared%20Documents/Graphics Approved plots]
  
 
[[LBNE_Far_Detector_Software_Data_Model | LBNE 35-ton Data Model and Reconstruction]]
 
[[LBNE_Far_Detector_Software_Data_Model | LBNE 35-ton Data Model and Reconstruction]]
Line 93: Line 252:
  
 
[[LBNE_code_repositories | LBNE code repositories and their browsers]]
 
[[LBNE_code_repositories | LBNE code repositories and their browsers]]
−
 
−
Many files are needed to create a new package. The mrb newProduct command can be used to create the skeleton for a new package. See the [https://cdcvs.fnal.gov/redmine/projects/mrb/wiki/MrbRefereceGuide mrb reference guide].
 
−
 
−
Units are cm and ns (Tom, January 2015).
 
  
 
LBNE jobs may be submitted using the package lbneutil described in [http://lbne2-docdb.fnal.gov:8080/cgi-bin/RetrieveFile?docid=10280&filename=projectpy.pdf a talk by Tingjun].
 
LBNE jobs may be submitted using the package lbneutil described in [http://lbne2-docdb.fnal.gov:8080/cgi-bin/RetrieveFile?docid=10280&filename=projectpy.pdf a talk by Tingjun].
Line 104: Line 259:
 
The reconstruction used in the 35-ton simulation is specified in lbnecode/source/fcl/lbne35t/reco/standard_reco_lbne35t.fcl.
 
The reconstruction used in the 35-ton simulation is specified in lbnecode/source/fcl/lbne35t/reco/standard_reco_lbne35t.fcl.
  
−
The 35-ton MC data samples are described [https://cdcvs.fnal.gov/redmine/projects/lbne-fd-sim/wiki/LBNE35T4APASamples here].
+
Data production
 +
* The 35-ton MC data samples are described [https://cdcvs.fnal.gov/redmine/projects/lbne-fd-sim/wiki/LBNE35T4APASamples here].
 +
* [https://indico.fnal.gov/getFile.py/access?contribId=0&resId=0&materialId=slides&confId=11076 MCC5]
 +
 
 +
GENIE event generator
 +
* [http://genie.hepforge.org/manuals/GENIE_PhysicsAndUserManual_v2.10.00a.pdf 2015 user's manual]
 +
* [http://projects-docdb.fnal.gov/cgi-bin/ShowDocument?docid=753 2010 user's manual]
 +
* [http://home.fnal.gov/~perdue/DeveloperManual_v0.2.html developer's manual].
  
−
For information about the GENIE event generator, see the [http://genie.hepforge.org/manuals/GENIE_PhysicsAndUserManual_v2.10.00a.pdf 2015 users's manual] ([http://projects-docdb.fnal.gov/cgi-bin/ShowDocument?docid=753 2010 user's manual]) and the [http://home.fnal.gov/~perdue/DeveloperManual_v0.2.html developer's manual].
 
  
−
[[Debugging_DUNE_far_detector_SW | Debugging DUNE far detector SW]]
 
  
−
[[LbTupler]]
+
[[Results from LbTupler]]

Latest revision as of 15:21, 12 December 2017

This page has moved to the DUNE wiki

Below is an out-of-date snapshot of the page.

Introduction

This page describes recent releases of DUNE software for simulation and reconstruction of event data from the LAr TPCs and associated photon and muon detectors. The software is released in the dunetpc package. Earlier releases in the lbnecode package are described on this other page.

Finding and downloading DUNE releases

Releases of dunetpc (and other DUNE products) are installed at FNAL at

/grid/fermiapp/products/dune

and on CVMFS at

/cvmfs/dune.opensciencegrid.org/products/dune

A release and all its dependencies may be installed on SLF and Mac machines using the pullProducts command: 

./pullProducts ~/ups slf6 dune-v06_58_00 e14 prof   # Scientific Linux 6
./pullProducts ~/ups slf7 dune-v06_58_00 e14 prof   # Scientific Linux 7
./pullProducts ~/ups d15 dune-v06_58_00 e14 prof    # mac OS X 10.11 El Capitan
./pullProducts ~/ups d16 dune-v06_58_00 e14 prof    # macOS 10.12 Sierra

The first three arguments are the directory where the products are to be installed, the OS specifier and the product name and version. The last two are the UPS qualifiers. The last is the compiler option (prof is nearly optimized but retains the code line numbers) and the first is a colon-separated list of all other qualifiers (e.g. e7:s15, not e7-s15). The available OS and qualifiers for a given release may be deduced from the list of manifests found in the manifest directory for the desired dune release in the dune bundle listing.

Note that versions above are the latest available at the time of writing (typically updated each time a release is made). Older builds are available for some platforms that are no longer supported and the latest of those are: 

./pullProducts ~/ups d13 dune-v06_13_00 e10 prof    # mac OS X 10.9 Mavericks
./pullProducts ~/ups d14 dune-v06-56_00 e14 prof    # mac OS X 10.10 Yosemite
./pullProducts ~/ups u14 dune-v06_06_01 e10 prof    # Ubuntu 14

These are likely too far out of date to be useful.

The above commands will copy many tar files to the current directory and so it is best to run it in a dedicated directory and not in the product installation area. E.g. something like:

Before first installation of UPS products, checkPrerequisites can be run to see if all prerequisite packages are installed. 

mkdir ~/upstars
cd ~/upstars
wget http://scisoft.fnal.gov/scisoft/bundles/tools/pullProducts
chmod +x ./pullProducts
./pullProducts ~/ups slf6 dune-v06_58_00 e14 prof

Tar files for individual product versions (including dunetpc) may also be downloaded directly from the SciSoft product area. To install a product, simply untar these in the local products area: 

cd ~/ups
tar xf ~/Downloads/duneutil-06.56.01-slf6-x86_64-e14-prof.tar.bz2

Users of dunetpc develop at remote sites will sometimes need to do this when the dunetpc dependencies have advanced beyond those in the last dunetpc build. For example the above is needed on 20nov2017.

If ups is not installed on your machine, then download and unpack the ups product tar file in this manner. Follow the link to the latest version from the SciSoft ups package list and then choose the version appropriate to your machine, e.g. ups-6.0.5-Darwin64bit+16.tar.bz2 for mac Sierra (Nov 2017).

Note that SIP must be disabled and openssl installed (keg-only) for mac El Capitan and Sierra. For more information, see the LArSoft supported platforms page.

UPS can be very picky about the OS version for Ubuntu. To install and use the above Ubuntu 14 releases on an Ubuntu 16 or different Ubuntu 14 sub-version machine, first set the UPS version overrride in one of the following ways:

  • Set the UPS_OVERRIDE environmental variable:

export UPS_OVERRIDE="-H Linux64bit+3.19-2.19"

  • Add an override file to the product directory:

echo "-H Linux64bit+3.19-2.19" > <product_directory>/ups_OVERRIDE.`hostname`

This is specific to each host. If you have a shared volume, you will need to do this for each machine.

Using DUNE releases

Scripts to set up the FNAL and CVMFS releases are provided. To set up to use release 4.21 from the FNAL installation: 

source /grid/fermiapp/products/dune/setup_dune.sh
setup dunetpc v06_04_00 -q e14:prof

To use the CVMFS installation of the same release: 

source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
setup dunetpc v06_46_00 -q e14:prof

For local installation of all products in ~/ups, set up with something like the following: 

source ~/ups/setup      # Or source ~/ups/setups
setup git
setup gitflow
setup mrb
setup dunetpc v06_55_00 -q e14:prof

Note that ~/ups/setups will not work with a UPS override file (env override is OK). In that case, use ~/ups/setup.

In all cases, the ups command can be used to find out which versions of dunetpc are available: 

fnal:lbne> ups list -aK+ dunetpc 
"dunetpc" "v04_20_00" "Linux64bit+2.6-2.12" "e7:prof" "" 
"dunetpc" "v04_20_00" "Linux64bit+2.6-2.12" "debug:e7" "" 
"dunetpc" "v04_21_00" "Linux64bit+2.6-2.12" "e7:prof" "" 
"dunetpc" "v04_21_00" "Linux64bit+2.6-2.12" "debug:e7" ""

Useful commands: 

 lar -c someconfig.fcl          # Use art event loop to process data according to the instructions in someconfig.fcl
 fcldump someconfig.fcl 5       # Display the highest five levels of configuration in someconfig.fcl
 listlibs type pattern          # Display the full path for all libraries of type "type" (e.g. service or module) whose full path name includes "pattern"

If you want to develop code, i.e. modify the code in dunetpc, larsoft, etc. or add code from any other repositories, you can use the dune-dev package. Instructions may be found at https://github.com/dladams/dune-dev/blob/master/README.md.

Building and installing dune releases

Build and installation instruction may be found on the Redmine CVMFS page. In particular, this page has the instructions for synchronizing the CVMFS releases with those in the FNAL grid area.

The code I (D. Adams) use to build releases is dune-larrel. See README.md in that package or use "larrel -H" for instructions. The (optional) products of the build include a release tag of dunetpc (in the repository) and a tar file and manifest for each dune release combination of OS, version, qualifier, etc. The latter can be installed in SciSoft as described below.

Once the release tag is made, then the product must be built for each combination of platform (SLF6, OSX-10.10, ...) and optimization level (prof, debug). An alternative for supported platforms (at presnt SLF6, OSX-10.9 and OSX-10.10) is to use Jenkins: load a KCA certificate in your browser (e.g. with get-cert and import), go to https://buildmaster.fnal.gov/view/LBNE/job/dune-release-build/, click on "Rebuild Last", change the release and submit. The script ~dladams/bin/copyFromJenkins or modified ~dladams/bin/jenkinscp can then be used to copy the product tar files and manifests to the (empty) current directory.

One or more builds often fail on Jenkins. To retry the failed builds, click on the build date to get to the page for that build and then click on "Matrix Reloaded" and then "Rebuild".

The above link to Jenkins will only work from inside the FNAL domain. I have not been able to get things to work properly with VPN or an ssh tunnel. I start a browser on an FNAL machine and send the window over the network. The commands I use: get-cert -i firefox --no-remote or use ~dladams/bin/jenkins.

A third option for building the product tar file and manifest is to use the script larsoft/larutils/buildScripts/buildDUNE.sh.

To install the SLF6 products in the FNAL grid area: 

ssh dune@dunegpvm03.fnal.gov
cd /grid/fermiapp/products/dune
tar xf ~dladams/reltars/dunetpc-05.05.00-slf6-x86_64-e9-prof.tar.bz2

To synchronize the CVMFS area with the grid area: 

ssh -l cvmfsdune oasiscfs.fnal.gov
dune-sync dune/dunetpc

Replace dunetpc with dune_pardata, duneutil or lbne_raw_data to update any of those products. The script dune-sync is installed on the oasis server and may also be found at https://github.com/dladams/dune-larrel/blob/master/dune-sync.

Each synchronization generates a log in $HOME/logs and a CVMFS server tag. Periodically remove these. The tags may be listed with 

cvmfs_server tag dune.opensciencegrid.org

and tags removed with 

cvmfs_server tag -r TAG dune.opensciencegrid.org

To copy each product tar file to SciSoft and install the corresponding manifest: 

 copyToSciSoft /afs/fnal.gov/files/home/room1/dladams/reltars/dunecode-04.20.00-slf6-x86_64-e6-prof.tar.bz2
 copyToSciSoft workdir/manifests/dune-04.21.00-Linux64bit+2.6-2.12-e7-prof_MANIFEST.txt

The command script may be found at /afs/fnal.gov/files/home/room1/dladams/bin/copyToSciSoft or http://scisoft.fnal.gov/scisoft/bundles/tools/copyToSciSoft. In case of error, log in to scisoftportal.fnal.gov and (very carefully) remove invalid files from /nasroot/SciSoft/bundles/....

After installation, the release should be announced on dune-computing-news AT fnal.gov.

More information

DUNE:

LArSoft:

Art:

Building:

FHICL:

Release building:

35 ton prototype:

LBNE 35-ton Data Model and Reconstruction

DUNE and LarSoft geometry

LBNE code repositories and their browsers

LBNE jobs may be submitted using the package lbneutil described in a talk by Tingjun.

LBNE geometries are summarized here.

The reconstruction used in the 35-ton simulation is specified in lbnecode/source/fcl/lbne35t/reco/standard_reco_lbne35t.fcl.

Data production

  • The 35-ton MC data samples are described here.
  • MCC5

GENIE event generator


Results from LbTupler

Retrieved from "https://dune.bnl.gov/w/index.php?title=DUNE_LAr_Software_Releases&oldid=2190"