Difference between revisions of "DQM payloads"

From DUNE
Jump to navigation Jump to search
(Created page with "= DQM payloads = == "Original Plan" == See DocDB references for prompt processing, p3s etc (cf. DocDB 1861). The short list of DQM items includes * ADC/FFT with variety of agg...")
 
 
(24 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
= DQM payloads =
+
= Overview =
−
== "Original Plan" ==
+
Historical notes on the design and planning have been moved to the page "[[DQM notes]]".
−
See DocDB references for prompt processing, p3s etc (cf. DocDB 1861). The short list of DQM items includes
+
User-level documents will be updated and kept on GitHub, in the "documents"
−
* ADC/FFT with variety of aggregation modes (e.g. APA, board, power supply etc)
+
folder of the p3s repo: https://github.com/DUNE/p3s.
−
* signal processing (filtering, de-noising, deconvolution etc)
+
 
−
* some (unspecified) fast recontruction and event display
+
The remainder of this page serves to keep record of current testing for the experts
 +
and can be expected to change periodically.
 +
 
 +
DQM Mailing list: DUNE-PROTO-SP-DQM
 +
<pre>
 +
dladams@BNL.GOV        David Adams
 +
baller@FNAL.GOV        Bruce R Baller
 +
Nectarios.Benekos@CERN.CH Nektarios Benekos
 +
georgios.christodoulou@CERN.CH Georgios Christodoulou
 +
zalesak@FZU.CZ        Zalesak Jaroslav
 +
trj@FNAL.GOV          Thomas R Junk
 +
kettell@BNL.GOV        Steve Kettell
 +
kutter@PHYS.LSU.EDU    Thomas Kutter
 +
mrmooney@COLOSTATE.EDU Michael Mooney
 +
potekhin@BNL.GOV      Maxim Potekhin
 +
bv@BNL.GOV            Brett Viren
 +
mworcester@BNL.GOV    Matthew Worcester
 +
tjyang@FNAL.GOV        Tingjun Yang
 +
</pre>
 +
 
 +
=Setting up=
 +
For (mostly) official instructions see:
 +
 
 +
https://wiki.dunescience.org/wiki/DUNE_Computing/Analysis_Module_From_Example_August2017
 +
 
 +
Check versions at CERN:
 +
 
 +
<pre>
 +
source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh
 +
ups list -aK+ dunetpc
 +
</pre>
 +
 
 +
Recall that when you run larsoft with the "-c" option which tells it the name of the FCL file, said file will be searched for in the path contained in $FHICL_FILE_PATH which typically starts with a "." (current directory).
 +
 
 +
= Current Work =
 +
References to JSON, FCL and .sh files correspond to what is in the current repo, available on GitHub.
 +
The "lxvm" prefix in filenames was dropped for brevity in the following notes.
 +
 
 +
==v06==
 +
Works: purity_5.json
 +
<pre>
 +
[
 +
    {
 +
"name": "Purity:Main",
 +
"timeout": "1000",
 +
        "jobtype": "purity",
 +
        "payload": "/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/purity/lxdqm_purity_3.sh",
 +
        "priority": "1",
 +
"state": "defined",
 +
"env": {
 +
    "P3S_VENV":"/afs/cern.ch/user/m/mxp/vp3s",
 +
    "P3S_LARSOFT_HOME":"/afs/cern.ch/user/d/dostefan/public/v06/",
 +
    "P3S_LARSOFT_RELEASE":"localProducts_larsoft_v06_45_01_e14_prof",
 +
    "P3S_NEVENTS":"10",
 +
    "P3S_FCL_DIR":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/purity",
 +
    "P3S_FCL":"lxdqm_purity_1.fcl",
 +
    "P3S_INPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/input",
 +
    "P3S_OUTPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/output/",
 +
    "P3S_OUTPUT_FILE":"PurityOutput.root"}
 +
    }
 +
]
 +
</pre>
 +
 
 +
 
 +
 
 +
Works: evdisp_4.json
 +
<pre>
 +
[
 +
    {
 +
"name": "EvDisp:Main",
 +
"timeout": "1000",
 +
        "jobtype": "evdisp",
 +
        "payload": "/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/evdisp/lxdqm_evdisp_4.sh",
 +
        "priority": "1",
 +
"state": "defined",
 +
"env": {
 +
    "P3S_VENV":"/afs/cern.ch/user/m/mxp/vp3s",
 +
    "P3S_LARSOFT_HOME":"/afs/cern.ch/user/d/dostefan/public/v06/",
 +
    "P3S_LARSOFT_RELEASE":"localProducts_larsoft_v06_45_01_e14_prof",
 +
    "P3S_NEVENTS":"2",
 +
    "P3S_FCL_DIR":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/evdisp",
 +
    "P3S_FCL":"lxdqm_evdisp_4.fcl",
 +
    "P3S_INPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/input/",
 +
    "P3S_OUTPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/output/",
 +
    "P3S_EVDISP_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/",
 +
    "P3S_USED_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/used/",
 +
    "P3S_OUTPUT_FILE":"evdisp.root"}
 +
    }
 +
]
 +
</pre>
 +
 
 +
 
 +
Works: crt_tpc_3.json
 +
 
 +
<pre>
 +
[
 +
    {
 +
"name": "CRT:Main",
 +
"timeout": "1000",
 +
        "jobtype": "crt2tpc",
 +
        "payload": "/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/crt/lxdqm_crt_tpc_3.sh",
 +
        "priority": "1",
 +
"state": "defined",
 +
"env": {
 +
    "P3S_VENV":"/afs/cern.ch/user/m/mxp/vp3s",
 +
    "P3S_LARSOFT_HOME":"/afs/cern.ch/user/d/dostefan/public/v06/",
 +
    "P3S_LARSOFT_RELEASE":"localProducts_larsoft_v06_45_01_e14_prof",
 +
    "P3S_NEVENTS":"50",
 +
    "P3S_FCL":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/crt/lxdqm_crt_tpc_1.fcl",
 +
    "P3S_CRT_TXT":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/crt/CRTs.txt",
 +
    "P3S_OUTPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/output",
 +
    "P3S_INPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/input",
 +
    "P3S_DATA":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/"
 +
}
 +
    }
 +
]
 +
</pre>
 +
 
 +
==v06_57==
 +
 
 +
Works: evdisp_5.json
 +
<pre>
 +
[
 +
    {
 +
"name": "EvDisp:Main",
 +
"timeout": "1000",
 +
        "jobtype": "evdisp",
 +
        "payload": "/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/evdisp/lxdqm_evdisp_5.sh",
 +
        "priority": "1",
 +
"state": "defined",
 +
"env": {
 +
    "P3S_VENV":"/afs/cern.ch/user/m/mxp/vp3s",
 +
    "P3S_LARSOFT_HOME":"/afs/cern.ch/user/d/dostefan/public/v06_57_00/",
 +
    "P3S_LARSOFT_RELEASE":"localProducts_larsoft_v06_57_00_e14_prof",
 +
    "P3S_NEVENTS":"2",
 +
    "P3S_FCL_DIR":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/evdisp",
 +
    "P3S_FCL":"lxdqm_evdisp_4.fcl",
 +
    "P3S_INPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/input/",
 +
    "P3S_OUTPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/output/",
 +
    "P3S_EVDISP_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/",
 +
    "P3S_USED_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/used/",
 +
    "P3S_OUTPUT_FILE":"evdisp.root"}
 +
    }
 +
]
 +
</pre>

Latest revision as of 12:28, 1 June 2018

Overview

Historical notes on the design and planning have been moved to the page "DQM notes". User-level documents will be updated and kept on GitHub, in the "documents" folder of the p3s repo: https://github.com/DUNE/p3s.

The remainder of this page serves to keep record of current testing for the experts and can be expected to change periodically.

DQM Mailing list: DUNE-PROTO-SP-DQM 

dladams@BNL.GOV        David Adams
baller@FNAL.GOV        Bruce R Baller
Nectarios.Benekos@CERN.CH Nektarios Benekos
georgios.christodoulou@CERN.CH Georgios Christodoulou
zalesak@FZU.CZ         Zalesak Jaroslav
trj@FNAL.GOV           Thomas R Junk
kettell@BNL.GOV        Steve Kettell
kutter@PHYS.LSU.EDU    Thomas Kutter
mrmooney@COLOSTATE.EDU Michael Mooney
potekhin@BNL.GOV       Maxim Potekhin
bv@BNL.GOV             Brett Viren
mworcester@BNL.GOV     Matthew Worcester
tjyang@FNAL.GOV        Tingjun Yang

Setting up

For (mostly) official instructions see:

https://wiki.dunescience.org/wiki/DUNE_Computing/Analysis_Module_From_Example_August2017

Check versions at CERN: 

source /cvmfs/dune.opensciencegrid.org/products/dune/setup_dune.sh 
ups list -aK+ dunetpc

Recall that when you run larsoft with the "-c" option which tells it the name of the FCL file, said file will be searched for in the path contained in $FHICL_FILE_PATH which typically starts with a "." (current directory).

Current Work

References to JSON, FCL and .sh files correspond to what is in the current repo, available on GitHub. The "lxvm" prefix in filenames was dropped for brevity in the following notes.

v06

Works: purity_5.json 

[
    {
	"name":		"Purity:Main",
	"timeout":	"1000",
        "jobtype":	"purity",
        "payload":	"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/purity/lxdqm_purity_3.sh",
        "priority":	"1",
	"state":	"defined",
	"env":		{
	    "P3S_VENV":"/afs/cern.ch/user/m/mxp/vp3s",
	    "P3S_LARSOFT_HOME":"/afs/cern.ch/user/d/dostefan/public/v06/",
	    "P3S_LARSOFT_RELEASE":"localProducts_larsoft_v06_45_01_e14_prof",
	    "P3S_NEVENTS":"10",
	    "P3S_FCL_DIR":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/purity",
	    "P3S_FCL":"lxdqm_purity_1.fcl",
	    "P3S_INPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/input",
	    "P3S_OUTPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/output/",
	    "P3S_OUTPUT_FILE":"PurityOutput.root"}
    }
]


Works: evdisp_4.json 

[
    {
	"name":		"EvDisp:Main",
	"timeout":	"1000",
        "jobtype":	"evdisp",
        "payload":	"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/evdisp/lxdqm_evdisp_4.sh",
        "priority":	"1",
	"state":	"defined",
	"env":		{
	    "P3S_VENV":"/afs/cern.ch/user/m/mxp/vp3s",
	    "P3S_LARSOFT_HOME":"/afs/cern.ch/user/d/dostefan/public/v06/",
	    "P3S_LARSOFT_RELEASE":"localProducts_larsoft_v06_45_01_e14_prof",
	    "P3S_NEVENTS":"2",
	    "P3S_FCL_DIR":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/evdisp",
	    "P3S_FCL":"lxdqm_evdisp_4.fcl",
	    "P3S_INPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/input/",
	    "P3S_OUTPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/output/",
	    "P3S_EVDISP_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/",
	    "P3S_USED_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/used/",
	    "P3S_OUTPUT_FILE":"evdisp.root"}
    }
]


Works: crt_tpc_3.json 

[
    {
	"name":		"CRT:Main",
	"timeout":	"1000",
        "jobtype":	"crt2tpc",
        "payload":	"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/crt/lxdqm_crt_tpc_3.sh",
        "priority":	"1",
	"state":	"defined",
	"env":	{
	    "P3S_VENV":"/afs/cern.ch/user/m/mxp/vp3s",
	    "P3S_LARSOFT_HOME":"/afs/cern.ch/user/d/dostefan/public/v06/",
	    "P3S_LARSOFT_RELEASE":"localProducts_larsoft_v06_45_01_e14_prof",
	    "P3S_NEVENTS":"50",
	    "P3S_FCL":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/crt/lxdqm_crt_tpc_1.fcl",
	    "P3S_CRT_TXT":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/crt/CRTs.txt",
	    "P3S_OUTPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/output",
	    "P3S_INPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/input",
	    "P3S_DATA":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/"
	}
    }
]

v06_57

Works: evdisp_5.json 

[
    {
	"name":		"EvDisp:Main",
	"timeout":	"1000",
        "jobtype":	"evdisp",
        "payload":	"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/evdisp/lxdqm_evdisp_5.sh",
        "priority":	"1",
	"state":	"defined",
	"env":		{
	    "P3S_VENV":"/afs/cern.ch/user/m/mxp/vp3s",
	    "P3S_LARSOFT_HOME":"/afs/cern.ch/user/d/dostefan/public/v06_57_00/",
	    "P3S_LARSOFT_RELEASE":"localProducts_larsoft_v06_57_00_e14_prof",
	    "P3S_NEVENTS":"2",
	    "P3S_FCL_DIR":"/afs/cern.ch/user/m/mxp/projects/p3s/inputs/larsoft/evdisp",
	    "P3S_FCL":"lxdqm_evdisp_4.fcl",
	    "P3S_INPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/input/",
	    "P3S_OUTPUT_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/output/",
	    "P3S_EVDISP_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/evdisp/",
	    "P3S_USED_DIR":"/eos/experiment/neutplatform/protodune/np04tier0/p3s/used/",
	    "P3S_OUTPUT_FILE":"evdisp.root"}
    }
]
Retrieved from "https://dune.bnl.gov/w/index.php?title=DQM_payloads&oldid=2308"