Project

General

Profile

2 Questions: Shower Reco and EndPoint2D finding

Added by Ryan Linehan almost 6 years ago

Hello everybody,

Thanks for starting this! I have two questions, but I should probably start by saying that my general goal is to be able to (hopefully) identify and reconstruct single charge exchange events, where a pi0 ends up decaying into 2 showering gammas. I know that's a tall order, so I'll enter with these questions:

/////////////////////////////////////////////
///WARNING: BIG BLOCK OF TEXT INCOMING///
/////////////////////////////////////////////

1.) I'm simulating and reconstructing particles using the template given in the "let's make/reconstruct some muons" section of the wiki. I think I understand how things work there, and so have expanded to try to do some other kinds of reconstruction. What kinds of objects does one need to produce to reconstruct showers (simplest option for now)? There is the obvious need for calibrated data and hits, and it seems like the showeranglecluster module needs to be run before running the standard_shower module (ShowerReco), but for some reason, I keep getting output from the showeranglecluster module (really the cluster matching algorithm) saying that there are 0 matched clusters. I have done this for a few different kinds of events, some of which are really evident showers (primary=5GeV electron), so I am assuming I'm doing something wrong. I also run the dbcluster method before the showeranglecluster method. (Note: my understanding of how things work in general is only from reading the wiki, and while the wiki is very helpful, it's not omniscient, so I could definitely be wrong in my interpretation/methods, too.)

2.) I am also interested in using the EndPoint2D module and its output. I am trying to run the EndPointModule_module.cc in ClusterFinder (in the $SRT_PUBLIC_CONTEXT directory) and get some funny results. Both with my own analysis module and with the event display (see attached picture), I can get some of these endpoints, but it seems like for every event, they only appear for a single cluster in a given plane. I am familiar with how the endpoint-finding algorithm works in general (harris corner finding), and I know there should be more EndPoint2Ds than what's shown. Does anyone have an idea as to what I might be doing wrong?

Thanks for the help!


Replies (22)

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Jason St. John almost 6 years ago

Hi, Ryan.

In case no one else chimes in about this, you may already be the LArIAT expert on shower reconstruction. Luckily, MicroBooNE has a large working group on this. We may need to collaborate outside of our collaboration. (In fact, by using LArsoft, we already do!) I'll send you more in an email.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

Hi Ryan

I don't think we use the dbcluster method any more, and folks are not developing with that. You could try the fuzzy cluster module to produce an output cluster. At one point, Andrzej and Roberto were working on shower reconstruction, so you might want to talk to them.

as for your 2.) - you shouldn't have a $SRT_PUBLIC_CONTEXT directory at all, we are now using CMake and mrb for our build system. In which case, you should be setting up to use the ups products for the bits of LArSoft that you are not actively developing. The current LArSoft documentation is at https://cdcvs.fnal.gov/redmine/projects/larsoft/wiki so you want to be sure you are not looking at the legacy site. Also, no one has used the EndPoint2D code for a long time, so it is possible that it is not working correctly.

Brian

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Hmm. Welp, okay. That's certainly good to know. I have been using the legacy site, so that might explain some of the issues I have been having. Now I know! Thanks for the help!

- P.S. I don't remember seeing one, so maybe I missed it, but in case there isn't a disclaimer on the legacy site redirecting newbies like me to the updated one, it might be good to put one there. I'm sure that would save someone in the future a little bit of frustration :)

- P.P.S. I'll be back to the rest of the questions/advice in a day or two.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

I just put a disclaimer on the legacy site, sorry about the confusion.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Thanks! Another quick question. I'm a little bit confused about the new page. I believe I have done everything correctly in the setup (although for some reason when I try to use: "mrbsetenv" I get the following error:

ERROR: Version conflict -- dependency tree requires versions conflicting with current setup of product larsim: version v03_02_04 vs v03_00_00
ERROR: setup of required products has failed

I also tried to setup LArSoft with a version 03_02_04, but the suggested setup method doesn't allow me to use this one. It doesn't seem to be listed in the available versions of larsoft).

However, that's not my main question. The "getting started with an analysis task" page suggests that one should already have a root file filled with simulated/reconstructed events, ready for analysis. I am at the moment trying to get my bearings, and am interested in how different reconstruction algorithms can spit out different clusters and whatnot. Is there a way to simulate/reconstruct events with this newer environment, or am I stuck with simulating/reconstructing via the method outlined on the legacy site? (Do I need to have mrbsetenv work to do so?)

Pardon the ignorance, and thanks a lot for the help, everybody.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

Hmmm...that looks like you might have had a previous version setup and may need to do an unsetup. Do you get that version conflict error from a fresh login? Are you setting anything up from a .bash_profile or .login or similar script automatically?

Once you have setup larsoft, which is done for you when you run the develop_lariat function described at

https://cdcvs.fnal.gov/redmine/projects/lardbt/wiki/Setting_up_the_Offline_Software

Then you can run the same command indicated on the legacy site, for example

lar -c prodsingle.fcl -n 10

should produce you a file with 10 events, each containing 1 muon. You can copy the prodsingle.fcl file into your local area doing

cp $LARSIM_DIR/job/prodsingle.fcl prodsingle.fcl

Notice that you did not need the full path to run it out of the box, that is because the location of all LArSoft related .fcl files are in $FHICL_FILE_PATH and the lar executable looks through all the locations to find the one specified. The $PWD is also in that path.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Brian,

Thanks for the help! As for the mrbsetenv error, I haven't touched my .login (and can't even find my .bash_profile), and after looking in at least my .login, .profile, and .bashrc, it doesn't seem like there's a funky setup of a particular version. (I'm on lariatgpvm01.) And yes, the error happens even with a fresh login. (I have to do the setup for my larsoft version, which I changed to v03_02_00, but immediately once I am able to use mrbsetenv, it isn't agreeing with my version.) Even with this error, though, build still seems to work, and with your last piece of advice, I can successfully run prodsingle.fcl and some other job files, so LArSoft seems to work for now. I think I'll stick to baby steps and declare victory for now with my general (i.e. non-experiment-specific) installation of LArSoft. If I run into any strange problems later, I'll bring them up.

On that topic, what is the general difference between using the default LArSoft package (which uses the argoneut geometry, I believe?) and LArIATSoft? If I worked with the default LArSoft for now, what sort of things would I be missing out on?

Thanks a ton!

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

Hi Ryan

You are very welcome.

If you run the develop_lariat function, then that will setup larsoft for you. By setting it up before setting up lariatsoft, you are causing yourself a version skew. Did you follow the instructions on that wiki page I pointed you to? It sounds like you need to create a .bash_profile file and then populate it with the develop_lariat function on that web page.

You definitely want to setup lariatsoft, because that not only gets you the correct geometry for the experiment, it also gets you the correct electronics response. If you were to use the default LArSoft, you would not be simulating muons as they are expected to appear in LArIAT.

At any rate, some day you will have to setup the LArIATSoft environment, may as well be today. You will be more productive in the long run by figuring it out now.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Brian,

Thanks for the insight! So I followed your instructions for setting up LArIATSoft outside of my LArSoft area (a different subdirectory in my lariatgpvm home directory) but I get the error below (disk quota exceeded):

-----------Checking Repository Out-----------------
git clone: clone lariatsoft at /afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop/srcs
NOTICE: Running git clone ssh:///cvs/projects/lariatsoft
Cloning into 'lariatsoft'...
remote: Counting objects: 436, done.
remote: Compressing objects: 100% (429/429), done.
remote: Total 436 (delta 210), reused 0 (delta 0)
Receiving objects: 100% (436/436), 11.17 MiB | 5.89 MiB/s, done.
Resolving deltas: 100% (210/210), done.
fatal: fsync error on '/afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop/srcs/lariatsoft/.git/objects/pack/tmp_pack_4DZC1g': Disk quota exceeded
fatal: index-pack failed
ERROR: The git command failed!
---------------Building Code----------------------
The working build directory is /afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop/build_slf6.x86_64
The source code directory is /afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop/srcs
----------- check this block for errors -----------------------
DIAGNOSTICS: there are no packages in /afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop/srcs
DIAGNOSTICS: nothing to build
DIAGNOSTICS: there are no packages in /afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop/srcs
DIAGNOSTICS: nothing to build
----------------------------------------------------------------
/afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop/build_slf6.x86_64
ERROR: you must "mrbsetenv" in /afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop/build_slf6.x86_64
Ready to develop LArIATSoft version in /afs/fnal.gov/files/home/room3/linehan3/LArIATSoft/develop

Since what I have read about disk quotas makes me think that only system administrators can manage what each of us is allotted, I don't know if I have the access/power to do anything about it. If I can, advice would be much appreciated!

Thanks again! I know this is a bit of a bothersome process, but I really appreciate the help!

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

Hi Ryan

see that line with all the !!!!!!!!!!!!!!!! in it in the example function? It tells you to set the value of LARIATDEVDIR to a place where you want the build to happen. Because the build takes up more than the meager quota you have in the afs home area, change the value of LARIATDEVDIR to someplace other than your home area that has enough space to build the software, i.e.

/lariat/app/users/<your username here>

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Hi Brian!

Okay. That makes sense, and seemed to work, and now I have LArIATSoft up and running. So, as a final set of two questions before I jump out of the nest and see if my wings work:

1.) This is more of a check to see if the process of checking out and tinkering with things is correct in my head. Now that I have LArIATSoft up and running, if I wanted to develop a package, say, AnalysisExample (or if I wanted to play with a clustering module), all I would have to do is follow the directions given on the general LArSoft page ("cd srcs, mrb getCheckout <package> , etc.), make sure that the includes referenced in the files I check out are for lariat, and that's it, right?

2.) This is a bit of a problem that I encountered while trying to run "lar -c prodsingle_lariat.fc -n 10l" out of the box (right after successful installation of LArIATSoft):

%MSG-i MF_INIT_OK: lar 13-Nov-2014 22:41:44 CST JobSetup
Messagelogger initialization complete.
%MSG
%MSG-s ArtException: lar 13-Nov-2014 22:41:50 CST JobSetup
cet::exception caught in art
---- OtherArt BEGIN
ServiceCreation
---- NotFound BEGIN
Service unable to find requested service with compiler type name 'util::TimeService'.
---- NotFound END
cet::exception caught during construction of service type util::DetectorProperties:
---- OtherArt END
%MSG
Art has completed and will exit with status 1.

The legacy wiki suggests that there is a missing include statement (or a missing definition) to the "util::TimeService" service, which I could follow as far as an include statement to "detectorproperties.fcl." However, I imagine there might be a quick installation-related fix that wouldn't involve me going around and mucking up a bunch of my .fcl files. Do you have an idea of what the remedy might be?

Sorry for leaning on you a lot today, but it is starting to make some sense, I think. Once again, thanks a ton.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

Hi Ryan

no worries about all the questions - the quicker you come up to speed, the more productive you can be and the sooner you can help others.

To answer your 1.), you can checkout any LArSoft product into your srcs area for LArIAT by following the instructions at

https://cdcvs.fnal.gov/redmine/projects/lardbt/wiki/Setting_up_the_Offline_Software#Option-2-Developing-LArIATSoft-within-your-LArSoft-development-area

That link tells you how to check out LArIATSoft into a LArSoft development area, but the reverse also works, you just have to make the appropriate substitutions. Once you have those checked out, then mrb and CMake will handle getting the include flags for compilation for you. You still have to put the #include statements in any code you right, but nothing tricker than that...and those statements only need the relative path from within LArSoft, you don't need to try to put in an absolute path.

As for 2.), I think I led you slightly astray earlier. I forgot that there is a LArIAT version of prodsingle.fcl in LArIATSoft in the JobConfigurations package called, unsurprisingly, prodsingle_lariat.fcl. That includes configurations from the LArSoftConfigurations package of LArIATSoft and I believe those configurations should have everything set up that you need. Please give prodsingle_lariat.fcl a try and see if it works. If not, then we should only need to update the LArSoftConfigurations/detectorproperties_lariat.fcl file.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Hi Brian,

Thanks! So yes, I followed your top paragraph's advice, and successfully checked out the AnalysisExample package (larexamples), just using "mrb g larexamples". However, when I tried to build, I found another version inconsistency error:

bash-4.1$ cd ../build_slf6.x86_64/
bash-4.1$ mrb z
Removing everything in /lariat/app/users/linehan3/lariatsoft/develop/build_slf6.x86_64
You must now run the following:
mrbsetenv
bash-4.1$ mrbsetenv
The working build directory is /lariat/app/users/linehan3/lariatsoft/develop/build_slf6.x86_64
The source code directory is /lariat/app/users/linehan3/lariatsoft/develop/srcs
----------- check this block for errors -----------------------
ERROR: Version conflict -- dependency tree requires versions conflicting with current setup of product cetbuildtools: version v4_03_03 vs v4_02_02
ERROR: Version conflict -- dependency tree requires versions conflicting with current setup of product larsim: version v03_02_04 vs v03_02_01
ERROR: setup of required products has failed

Should I be passing something other than the "larexamples" argument to the gitCheckout command?

As for your second comment, what I did last time was in fact the prodsingle_lariat.fcl job file being run. Actually, the normal prodsingle.fcl (defaulted to argoneut) works fine. So I have the old one working and the lariat one missing a service definition somewhere.

Thanks!

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Brian,

UPDATE: I am not sure if I just glazed over the command a bajillion times or if it was recently added, but it looks like I missed the "mrb uc" command in the package downloading. With that, I am no longer getting version inconsistency errors. So that works. Yay!

One note: the build/install procedure used on repositories/packages checked out works, but also takes ~10-20 min per build/install. Since modifying code, debugging, and whatnot is easiest with a short compilation/build time, is there a way to reduce this?

However, prodsingle_lariat.fcl (and anything that relies on detectorproperties_lariat.fcl) still has the problem/error mentioned. I took a look in the detectorproperties_bo.fcl and there is more in that than in the lariat one, so I assume that maybe something about timing should be included in the lariat one? I'm not exactly sure what is supposed to be there, though.

Thanks again!

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Andrzej Szelc almost 6 years ago

Hey,

as for Shower reconstruction,

the process should be running dbcluster or fuzzycluster,
then ShowerAngleCluster and then the ShowerReco code in that sequence.

I think this chain should work if the relevant labels are set correctly.
Now, the most recent version of the ShowerCode lives in the analysis framework developed in MicroBooNE which,
is now called LArLite, and porting the newest version of the code into LArSoft is on my to do list, but cntinues to get bumped down.
I hope we can get this done this week during the shower workshop. Feel free to bug me about this, as you have via email. ;-)

Andrzej

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Johnny Ho almost 6 years ago

Hi Ryan,

After the initial build/install (mrb i -j8), any modifications to the existing source code (that was initially built/installed using mrb i -j8) can be built/installed by running make install in the $MRB_BUILDDIR directory.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

Hi Ryan

I will try to look at the lariat detector properties .fcl file tomorrow and see if I can bring it up to date.

Brian

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

Hi Ryan

I just committed a file and a change to another file that should solve your problem with running the single particle generation for lariat. Can you give it another try?

We needed to have a timeservice_lariat.fcl file and to get our services list to use it so that it would be available when detector properties needed it.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Brian,

I updated my version with "git pull origin develop", but there still seems to be no file called timeservices_lariat.fcl in the lariatsoft package, and that's preventing a successful run. Where is it supposed to be? (I'm guessing in the LArSoftConfigurations directory?)

Maybe I'm updating my packages wrong, too.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Brian Rebel almost 6 years ago

Hi Ryan

looks like it didn't get pushed to the main repository. Should be there now.

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Ryan Linehan almost 6 years ago

Hi Brian,

It looks like it's up and running! Thanks a ton for the help, everybody! Wooo!

Last note: I found some errors when I tried to run the lariat event display on my reconstructed/simulated events, but found solutions to them (there were points in the evdservices_lariat.fcl file that were a little wonky). I am not super familiar/confident with git pushing (although I can learn, of course) or the protocols by which our group allows pushing, so I haven't committed them or anything yet, but in case anyone is interested in general, here are the changes that need to be made to run the event display:

IN THE FILE evdservices_lariat.fcl:

1.) Add the line "DrawShowerCalor: 0" to the lariat_analysisdrawingopt block
2.) Remove the "s" from the end of CalorTemplateFileNames and remove the "[" and "]" from the that key's value.
3.) Add the line " TimeService: @local::lariat_timeservice" to the lariat_disp block
4.) Add the include statement: #include "timeservice_lariat.fcl" at the top

That seemed to work for me, so if people are having problems, that might be it. Also, let me know if I should try to commit this myself :)

RE: 2 Questions: Shower Reco and EndPoint2D finding - Added by Jason St. John almost 6 years ago

Hi, Ryan,

I think those changes should be committed and pushed to develop. But if you want to be more cautious, create a feature branch with an appropriate name, commit and push to that, and then .... I don't know how we build the consensus about merging a feature branch into develop. But at least the central repository has your changes where anyone can easily pull them down and try them.

    (1-22/22)