The automated “nearline_bluearc_copy.sh” script on mnvnearline1 fails to copy necessary files from local_dump_area to online_processing/swap_area
(from /scratch/nearonline/var/job_dump to /minerva/data/online_processing/swap_area)
- Investigating mnvnearline1:scripts/nearline_bluearc_copy.sh
- Script runs automatically every 5 minutes.
- Log file for the script: /scratch/var/nearonline/logs
- Local copy from HEAD to following folders WORKS
- NEARLINE_DUMP_AREA /scratch/nearonline/var/job_dump
- NEARLINE_LOCAL_GMPLOTTER_LOCATION /scratch/nearonline/var/gmplotter
- The problem is with the python script “filechecklist.py”
- It does not generate the file list for files from the following folders:
- It works for the following folder
- Since there is no file list generated by the python script “filechecklist.py”, NO files copied to the swap_area
- I modified the script to use rsync command.
- Now it synchronizes the local_dump_area and online_processing/swap_area
- Inside the script Jeremy notes that, “using rsync for this stage incurs a lot of overhead on the BlueArc disk”, therefor, he writes a more efficient script “filechecklist.py” for this task
- The Problem is confirmed.
- .fileindex under /scratch/nearonline/var/job_dump got corrupted and causing “file checklist.py” to crash for that folder
- Using rsync manually fixed the .fileindex
- Software sync between mnvonlinelogger and mnvnearline1 updates the nearline_bluearc_copy.sh script to the original version
- Now everything works as before. The near ine_bluearc_copy.sh script copies the changed files to bluearc area using “file checklist.py”