GMBrowser Update – Uses all gates now

  • Previously GMBrowser that shifters look at only uses a fraction of the gates, because the early processing stages (particularly DecodeRawEvent) were slow.
  • Now that we have a faster version of DecodeRawEvent, and we modified GMBrowser to use all gates
  • Modified following parameters in NearlineCurrent.opts in Tools/DaqRecv/options on mnvonlinelogger, to be 100 percent:
    • PdstlPrescaler.PercentPass          = 25;
    • LinjcPrescaler.PercentPass          = 25;
    • NumibPrescaler.PercentPass       = 20;
  • Ran the “” script in all Nearline Machines to get the update
    • mnvnearline1
    • mnvnearline2
    • mnvnearline3
    • mnvnearline4
  • Informed Current Shifter about the update and started GMBrowser at Tufts UROC
    • Will investigate the behavior for some time, until we make this change permanent.

Control Room Computer Update to v10r9p1

  • Updated minerva-cr-01 and minerva-cr-02 to v10r9p1
  • Modified setupFiles for initiating setup with v10r9p1
    • .bashrc
  • Modified for v10r9p1
    • Python was a problem for v10r9p1 setup script
    • Created an alias to use the local version python instead of Framework Version
      • alias python=”/usr/bin/python”
  • Tested and Tagged ControlRoomTools under v10r9p1 as “stable_v10r9p1″
  • Updated the .bashrc for using the setup file under v10r9p1
  • Linked .k5login with “cmtuser/Minerva_v10r9p1/Tools/ControlRoomTools/authenticate/k5login-master”
  • Updated the documentation on wiki: “Control_Room_Setup_Manual”

GMBrowser Problem

  • GMBrowser Live works but shifter can not access to the previous runs and subruns
    • GMBrowser -r xx -s xxx does not work
  • The files are not copied automatically to the /minerva/data/online_processing/swap_area
  • I copied the files manually:
    • Connect to the mnvnearline1 – it has the BlueArc /minerva/ mount
    • Necessary files located: /scratch/nearonline/var/job_dumb
  • This solved the issue for non copied files.
  • I checked the log file for script under /scratch/nearonline/var/logs
    • After sometime the auto-script seems to be working.
  • Currently, we are not running any runs, I will check the status again tomorrow

Minerva Software Installed on new CR Machines (ROC-West)

MINERvA Software Installation on minerva-cr-01 and minerva-cr-02

  • Firefox configured for Shifter Bookmarks
  • Special Kerberos Principal Installed
  • ROOT 5.34/21 installed and tested
  • ControlRoomTools Installed and Configured
    • file and .k5login file configured with ControlRoomTools
    • GMBrowser installed and tested
  • mnvdaqrunscripts installed and configured
    • registered new hostnames(minerva-cr-01 and minerva-cr-02) into “mnvdaqrunscripts/”
    • committed and tagged changes as “oaltinok_2014_09_17″
  • mnvruncontrol installed and configured
    •  I tried connecting using run control but could not connected. Will test again tomorrow.
  • MINOS Software is NOT installed

Major Update to RunControl Software (v6r1)

  1. Killed all processes
  2. Jeremy updated the
  3. I updated the remaining Control Room Computers
  4. Testing the Updates
    1. Successful Test on Control Room Computers
    2. Successful Test on Rochester UROC
    3. Successful Test on Tufts UROCs
  5. I updated the script and notified UROC Users

Fermilab Power Failure

  • On Sunday 03:30 am there was a power failure affecting MINOS and MINERvA underground machines
  • Control Room Computers lost network mount to /minerva/data/
    • GMBrowser needs /minerva/data mounted and it was not working
    • minerva-evd is used by UROCs to mount /minerva/data and they are also affected.
    • Carrie opened a service ticket to ask Computer Division Help for Control Room Computers
    • Computer Division solved the incident and all machines and UROCs working properly.
  • machine is still down and we have no access to Veto Wall HV Monitoring.
  • e-Checklist can be used either one of the following servers: (minerva-wbm was down due to power glitch)

minerva-om Update

  • minerva-om no longer support MINERvA Software
  • minerva-om has latest MINOS RMS Installed for om near check
  • .bashrc script modified to prompt users to use the “” command to start the minos om GUI
  • Nothing removed and every file and software are recoverable.

UROC and Control Room Update

  • Heather and Joel has a new GUI for Veto Wall HV Monitoring
  • I have written a script: RunVetoHVMonitor
    • Script is added to Tools/ControlRoomTools/launchers and Tools/ControlRoomTools/
    • Tagged the version as oaltinok_2014_05_13
    • updated to get the latest version of ControlRoomTools
  • Send an e-mail to UROC_LIST to inform users to update their UROCs
  • Updated the Control Room Computers: minerva-bm, -evd,-om-02, -rc
  • There were some local modifications on minerva-rc
    • MakeDST
    • gmbrowser/macros/OccupancyPlots.C
  • Sent and e-mail to inform Control Room People to know the updates

UROC and Control Room Update

  • Phil Rodrigues updated GMBrowser

    • GMBrowser to make it skip empty plots when it’s cycling. This way, it doesn’t spend most of the time showing the shifter empty canvases.
  • Version Tag: rodriges_2014-05-01
  • Updated the with the tag
  • Committed to CVS and tagged as CURRENT
  • Tested update on Tufts UROC and UROC02
  • Sent and e-mail to UROC-LIST to let people know the update
  • Minerba also updated Control Room Computers

No Power on FNAL Control Room – April 19, 2014

  • UROCs need a mount to minerva-evd for GMBrowser
  • Tools/ControlRoomTools/authenticate/MountRemoteFS script responsible for mounting UROCs to minerva-evd
  • is the Emergency Server incase there is a problem with minerva-evd
  • New Script: MountRemoteFS_Emergency
    • This script uses the as the Bluearc Mount point
  • Tested on both UROCs at Tufts and GMBrowser works with a mount to
  • Updates committed to CVS and UROC Users informed with the update.
  • • Successfully tested Emergency Script: MountRemoteFS_Emergency

Warning Messages

  • WARNING messages disappeared after some time: Slave Nodes need time to be fed with the Modified Nearline.opts
  • Committed new version to CVS, follow this procedure:
  1. Before editing a file always: cvs update -A
  2.  Modify the file
  3. Commit the file: cvs commit -m “I modified the … … ….” DaqRecv/options/Nearline.opts
  4. Tag latest version: cvs tag -F nearline_03_26 Nearline.opts

Warning Messages

Warnings for each run:


WARNING # (216,1): Reassignment of option ‘MessageSvc.OutputLevel’ .

Previously defined at /scratch/nearonline/mirror/mnvsoft/v10r7p3/minerva/MINERVA/

MINERVA_v10r7p3/Top/MinervaConf/options/MinervaApplication.opts: (53, 1).


WARNING # (281,1): Reassignment of option ‘ElectronicsMapAlg.AuditExecute’ .

Previously defined at /scratch/nearonline/mirror/cmtuser/Minerva_v10r7p3/Tools/DaqRecv/options/Nearline.opts: (266, 1).

  • modified the Tools/DaqRecv/options/Nearline.opts on near line machines in order to stop the WARNING messages.
  • Changes I made:
    • – I have disabled the MessageSvc.OutputLevel = 4, because it is previously assigned by Top/MinervaConf/options/MinervaApplication.opts
    • – ElectronicsMapAlg.AuditExecute assigned two different lines. I have removed the duplicate line, which causes the WARNING message.
  • If the WARNING messages disappear, I will commit the newest version of Nearline.opts to CVS Script on minerva-rc

  • script installed on minerva-rc under $HOME/bin/
  • script checks any running process in the name of and opens RunControl GUI if it is not running on minerva-rc
  • script creates a log file $HOME/watchdog.log, which records the time and date when the RunControl runned automatically
  • crontab edited to run this script every minute.


  • make script executable using:
    • chmod +x
  • In order to edit the crontab use:
    • crontab -e
  • crontab environment is not same with the script environment
  • dump local environment to a temp file:
    • env > localenv.output
  • Copy all the lines in “localenv.output” to the crontab (before your command)
  • For running commands which have GUI use
    • export DISPLAY=:0 (or localenv has this value)
    • Carefull with remote connections. Remote connection Displays are different than local displays
  • To run a command every minute write the following line in crontab:
    • * * * * * . $HOME/bin/

Control Room Update

  • 3 Components updated
  • mnvdaqscripts
    • cvs update –r v3r6
  • mnvruncontrol
    • cvs update –r v5r7p2
  • Tools/ControlRoomTools
    • cvs update –r nearline_cfg_2013_11_13
  • Updates Completed on 10.00 a.m.

Problems & Troubleshooting

  • On minerva-rc run control does not show up after update
    • Jeremy solved the problem
    • Cause of problem is due to someone locally modified the
    • Always check for CVS messages after update
    • Solve all conflicts