fflGen.pl

by Livio Salconi — last modified 2008-08-07 10:54

fflGen.pl script: howto

The fflGen.pl script (v2.80)

The fflGen.pl script runs like a deamon on the machine STOL2 at the port 9010 on the storage subnet (stol2rear:9010).

- the executable is located at /root/bin/fflGen.pl
- the working directory with data and configuration files is: /var/fflgen

• Short description about it's functionalities:

The fflGen.pl script:

- receives information from all the dataStorage.pl modules, it has an internal data base about all replicas on storage farm disks. It update/generate the file raw.ffl after an evan like: a new file is come, a file is going to be deleted, a file is moved to another location.

- send infos about new incoming files to the rawdata dataSender.pl module;

- send infos about new incoming files to the rawdata dataBackup.pl module;

- receive infos about the storage status from the dataManager.pl and include/exclude available/unavailable volumes from the raw.ffl list;

- receive infos about the files replica (like md5checksum, ecc..) from dataStorage.pl modules and other:

- send emails to the operators when there are missing frames on a file, recovered files and for all other exceptions;

• Configuration parameters and files

The following files must be present into the working directory:

this files can be manually edited:
- /var/fflgen/local_host_name : usually stol2rear, is the local host name (mapped on the storage subnet);
- /var/fflgen/datasend_host_name : usually datagwrear, is the name of the machine that hosts the module dataSender.pl;

this files can be edited due to manage an exception:
- /var/fflgen/host_path_table : contains a list of couples "host::volume" matched with the replica priority (usually "0" or "1") and the buffer location under the /data/rawdata structure;

- other files like .db .list and .lock files must be leaved untouched, changing or deleting them can cause system inconsistency.

All files are automatically replicated each hour by a crond script into the directory /var/dataSoft.bkp/fflgen.stol2 located on the datasw machine.

The following variables are located into the head of the script: to change their value the script must be stopped and restarted:

- set this variable to 1 or 2 to increment the log level detail:
# verbose level (1/2)
our $verbose = 1;

- set this services variable to 0 (disable) or 1 (enable):
# active services (0/1)
our $ffl_print = 1;
our $file_send = 0;
our $file_backup = 1;

... other variable must not be changed!

• Start the script

In order to start the script like a deamon, follow this rules:

- check if no instance are running:

# ps -edaf | grep fflGen

if the list is not empty and there are processes like fflGen.pl:main, please wait until they will finish before restart the script like a deamon. If not, a port conflict will occour. To start the script on the STOL2 machine (from the root account), you must:

# cd /var/fflgen

# nohup /root/bin/fflGen.pl &

then check the log or/and the nohup.out file:

# tail -f /var/log/fflGen.pl.log

If you are starting the script after a total shutdown, remember that the correct script activation sequence is the following:

- the fflGen.pl on stol2rear;

- the dataBackup.pl on datagwrear;

- all the dataStorage.pl on stXXrear;

- the dataFinder.pl on stol2rear;

- the dataManager.pl on stol1rear;

• Stop the script

From any machine in the storage farm network simply type:

# dataCommand.pl stol2rear:9010 "stop:<reasons>"

please note that the ":" is mandatory even no <reasons> are specified, than take a look to the log file or to the nohup.out:

# tail -f /var/log/fflGen.pl.log

• Commands recognized by the script

Like other modules, is possible to send a command direct to the fflGen.pl module:

- force a volume to be online(1) or offline(0), use this command carefully: all unavailable volumes will be removed (temporarily) from raw.ffl and will be not accessible by the users:

# dataCommand.pl stol2rear:9020 "avaliable:host::vol;0/1"

- remove a single file from the raw.ffl (permanently):

# dataCommand.pl stol2rear:9010 "remove:fileBaseName"

- remove a full volume (and all files related to) from the raw.ffl (permanently):

# dataCommand.pl stol2rear:9010 "delete:hostName::hostVolume"

- print into the log file the location (related to the /data/rawdata/buffervXX structure):

# dataCommand.pl stol2rear:9010 "location:fileBaseName"

- start (1) or stop (0) to send infos about new incoming files file to the dataSender.pl module, scheduling for data transfer (rawdata only):

# dataCommand.pl stol2rear:9010 "sender:0/1"

- start (1) or stop (0) to send infos about new incoming files file to the dataBackup.pl module, scheduling for CA backup (rawdata only):

# dataCommand.pl stol2rear:9010 "backup:0/1"

- ask for a dump of the md5 data base in a txt file with a given path/name:

# dataCommand.pl stol2rear:9010 "md5dump:fullPathDumpFileName"

- ask for a md5 value for a single file:

# dataCommand.pl stol2rear:9010 "md5ask:fileBaseName"

… other commands present into the fflGen.pl code are not intended to be execute manually.

• Other scripts related to the fflGen.pl module:

There are other perl scripts software modules used to interact with the fflGen.pl module, they are:

- the fflAddDir.pl module: must be run only from the data storage farm nodes (called ST4 - ST13), in particular from the machine that hosts the directory that we must add directly into the raw.ffl file. All files in the directory will be controlled an analyzed by the dataStorage.pl module running on that host and, for incomplete files, a query for a better replica will be sent to the dataFinder.pl module running on stol2.

# fflAddDir.pl hostDir hostName::volumeName

for example, from st12 to register all the v001 volume, you must type:

# fflAddDir.pl /storage/v001/data/DAQ/rawdata st12rear::v003

- the fflAddFile.pl module: must be run only from the data storage farm nodes (called ST4 - ST13), in particular from the machine that hosts the file that we must add directly into the raw.ffl file. A single file will be controlled an analyzed by the dataStorage.pl module running on that host and, if the file is incomplete, a query for a better replica will be sent to the dataFinder.pl module running on stol2.

# fflAddFile.pl fileFullName hostName::volumeName

for example, from st12 to register a file on the v001 volume, you must type:

# fflAddFile.pl /storage/v001/data/DAQ/rawdata/V-raw-869587440-240.gwf st12rear::v001

- the fflCheck.pl module: must be run from any machine that has access by automounter to the /data/rawdata directory and to the /virgoData/ffl directory. The script will check the raw.ffl consistency, it will:
- search for each entry if the related file is accessible or not from the /data/rawdata area;

- check for "holes" into the ffl: a list of files that are present into the backup stream (STOL2) but not into the offline buffers;

errors will be printed into the log and sent by email to operators.

# fflCheck.pl ; cat /var/log/fflAllCheck.pl | less