KaliVeda
Toolkit for HIC analysis
|
Base class for interface to a batch job management system.
The different "batch" systems available are described in the $KVROOT/KVFiles/.kvrootrc or the user's $HOME/.kvrootrc, as in the following example:
BatchSystem: Xterm Xterm.BatchSystem.Title: Execute task in an X-terminal window Xterm.BatchSystem.DefaultJobOptions: -T #JobName# -e Xterm.BatchSystem.JobSubCmd: xterm Xterm.BatchSystem.JobScript: "$KVROOT/bin/KaliVedaAnalysis -b -n"&
For each named batch system, there may be a corresponding plugin which defines the actual class to use, if it is necessary to extend the functionality of the KVBatchSystem base class, as for example:
Plugin.KVBatchSystem: Xterm KVRootBatch KVMultiDet "KVRootBatch(const Char_t*)"
The "default" batch system is defined as follows:
#Default batch system Default.BatchSystem: Xterm
It can be accessed via the batch system manager (see KVBatchSystemManager):
Global pointer gBatchSystem is 0 if analysis task is not run in batch mode If it is non-zero, it gives access to the batch system used to run the task. Therefore, if the user puts
in her analysis code, she can then use gBatchSystem as follows:
Definition at line 78 of file KVBatchSystem.h.
#include <KVBatchSystem.h>
Public Member Functions | |
KVBatchSystem (const Char_t *name) | |
virtual | ~KVBatchSystem () |
Destructor. | |
virtual void | AlterJobs (TGWindow *, TList *) |
void | cd () |
Make this the default batch system. | |
virtual Bool_t | CheckJobParameters () |
Checks the job and ask for the job name if needed. | |
virtual void | Clear (Option_t *opt="") |
virtual void | GetBatchSystemParameterList (KVNameValueList &) |
virtual const Char_t * | GetDefaultJobOptions () const |
virtual const Char_t * | GetJobName () const |
virtual const Char_t * | GetJobSubCmd () const |
virtual const Char_t * | GetJobSubCmdLine () |
virtual KVList * | GetListOfJobs () |
KVNameValueList & | GetParameters () |
virtual Bool_t | MultiJobsMode () const |
virtual void | Print (Option_t *="") const |
virtual void | PrintJobs (Option_t *="") |
virtual void | ReadBatchEnvFile (TEnv *) |
virtual void | Run () |
virtual void | SanitizeJobName () const |
virtual void | SetAnalyser (KVDataAnalyser *da) |
virtual void | SetBatchSystemParameters (const KVNameValueList &) |
Use the parameters in the list to set all relevant parameters for batch system. | |
virtual void | SetDefaultJobOptions (const Char_t *opt) |
virtual void | SetJobName (const Char_t *name) |
virtual void | SetJobScript (const Char_t *path) |
virtual void | SubmitJob () |
virtual void | SubmitTask (KVDataAnalyser *da) |
virtual void | WriteBatchEnvFile (TEnv *) |
Public Member Functions inherited from KVBase | |
KVBase () | |
Default constructor. | |
KVBase (const Char_t *name, const Char_t *title="") | |
Ctor for object with given name and type. | |
KVBase (const KVBase &) | |
copy ctor | |
virtual | ~KVBase () |
virtual void | Copy (TObject &) const |
Make a copy of this object. | |
const Char_t * | GetLabel () const |
UInt_t | GetNumber () const |
UInt_t | GetNumberOfObjects () const |
virtual TObject * | GetObject () const |
virtual const Char_t * | GetType () const |
Bool_t | HasLabel () const |
virtual Bool_t | IsCalled (const Char_t *name) const |
Bool_t | IsLabelled (const Char_t *l) const |
virtual Bool_t | IsType (const Char_t *typ) const |
virtual void | List () |
KVBase & | operator= (const KVBase &) |
copy assignment operator | |
Double_t | ProtectedGetX (const TF1 *func, Double_t val, int &status, Double_t xmin=0.0, Double_t xmax=0.0) const |
void | SetLabel (const Char_t *lab) |
virtual void | SetNumber (UInt_t num) |
virtual void | SetType (const Char_t *str) |
Public Member Functions inherited from TNamed | |
TNamed () | |
TNamed (const char *name, const char *title) | |
TNamed (const TNamed &named) | |
TNamed (const TString &name, const TString &title) | |
virtual | ~TNamed () |
void | Clear (Option_t *option="") override |
TObject * | Clone (const char *newname="") const override |
Int_t | Compare (const TObject *obj) const override |
void | Copy (TObject &named) const override |
virtual void | FillBuffer (char *&buffer) |
const char * | GetName () const override |
const char * | GetTitle () const override |
ULong_t | Hash () const override |
TClass * | IsA () const override |
Bool_t | IsSortable () const override |
void | ls (Option_t *option="") const override |
TNamed & | operator= (const TNamed &rhs) |
void | Print (Option_t *option="") const override |
virtual void | SetName (const char *name) |
virtual void | SetNameTitle (const char *name, const char *title) |
virtual void | SetTitle (const char *title="") |
virtual Int_t | Sizeof () const |
void | Streamer (TBuffer &) override |
void | StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b) |
Public Member Functions inherited from TObject | |
TObject () | |
TObject (const TObject &object) | |
virtual | ~TObject () |
void | AbstractMethod (const char *method) const |
virtual void | AppendPad (Option_t *option="") |
virtual void | Browse (TBrowser *b) |
ULong_t | CheckedHash () |
virtual const char * | ClassName () const |
virtual void | Delete (Option_t *option="") |
virtual Int_t | DistancetoPrimitive (Int_t px, Int_t py) |
virtual void | Draw (Option_t *option="") |
virtual void | DrawClass () const |
virtual TObject * | DrawClone (Option_t *option="") const |
virtual void | Dump () const |
virtual void | Error (const char *method, const char *msgfmt,...) const |
virtual void | Execute (const char *method, const char *params, Int_t *error=nullptr) |
virtual void | Execute (TMethod *method, TObjArray *params, Int_t *error=nullptr) |
virtual void | ExecuteEvent (Int_t event, Int_t px, Int_t py) |
virtual void | Fatal (const char *method, const char *msgfmt,...) const |
virtual TObject * | FindObject (const char *name) const |
virtual TObject * | FindObject (const TObject *obj) const |
virtual Option_t * | GetDrawOption () const |
virtual const char * | GetIconName () const |
virtual char * | GetObjectInfo (Int_t px, Int_t py) const |
virtual Option_t * | GetOption () const |
virtual UInt_t | GetUniqueID () const |
virtual Bool_t | HandleTimer (TTimer *timer) |
Bool_t | HasInconsistentHash () const |
virtual void | Info (const char *method, const char *msgfmt,...) const |
virtual Bool_t | InheritsFrom (const char *classname) const |
virtual Bool_t | InheritsFrom (const TClass *cl) const |
virtual void | Inspect () const |
void | InvertBit (UInt_t f) |
Bool_t | IsDestructed () const |
virtual Bool_t | IsEqual (const TObject *obj) const |
virtual Bool_t | IsFolder () const |
R__ALWAYS_INLINE Bool_t | IsOnHeap () const |
R__ALWAYS_INLINE Bool_t | IsZombie () const |
void | MayNotUse (const char *method) const |
virtual Bool_t | Notify () |
void | Obsolete (const char *method, const char *asOfVers, const char *removedFromVers) const |
void | operator delete (void *ptr) |
void | operator delete (void *ptr, void *vp) |
void | operator delete[] (void *ptr) |
void | operator delete[] (void *ptr, void *vp) |
void * | operator new (size_t sz) |
void * | operator new (size_t sz, void *vp) |
void * | operator new[] (size_t sz) |
void * | operator new[] (size_t sz, void *vp) |
TObject & | operator= (const TObject &rhs) |
virtual void | Paint (Option_t *option="") |
virtual void | Pop () |
virtual Int_t | Read (const char *name) |
virtual void | RecursiveRemove (TObject *obj) |
void | ResetBit (UInt_t f) |
virtual void | SaveAs (const char *filename="", Option_t *option="") const |
virtual void | SavePrimitive (std::ostream &out, Option_t *option="") |
void | SetBit (UInt_t f) |
void | SetBit (UInt_t f, Bool_t set) |
virtual void | SetDrawOption (Option_t *option="") |
virtual void | SetUniqueID (UInt_t uid) |
void | StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b) |
virtual void | SysError (const char *method, const char *msgfmt,...) const |
R__ALWAYS_INLINE Bool_t | TestBit (UInt_t f) const |
Int_t | TestBits (UInt_t f) const |
virtual void | UseCurrentStyle () |
virtual void | Warning (const char *method, const char *msgfmt,...) const |
virtual Int_t | Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0) |
virtual Int_t | Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0) const |
Static Public Member Functions | |
static KVBatchSystem * | GetBatchSystem (const Char_t *plugin) |
Static Public Member Functions inherited from KVBase | |
static Bool_t | AreEqual (Double_t x, Double_t y, Long64_t maxdif=1) |
Comparison between two 64-bit floating-point values. | |
static void | BackupFileWithDate (const Char_t *path) |
static void | CombineFiles (const Char_t *file1, const Char_t *file2, const Char_t *newfilename, Bool_t keep=kTRUE) |
static void | Deprecated (const char *method, const char *advice) |
static Bool_t | FindClassSourceFiles (const Char_t *class_name, KVString &imp_file, KVString &dec_file, const Char_t *dir_name=".") |
static Bool_t | FindExecutable (TString &exec, const Char_t *path="$(PATH)") |
static const Char_t * | FindFile (const Char_t *search, TString &wfil) |
static const Char_t * | GetBINDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetDATABASEFilePath () |
static const Char_t * | GetDATADIRFilePath (const Char_t *namefile="") |
static Bool_t | GetDataSetEnv (const Char_t *dataset, const Char_t *type, Bool_t defval) |
static const Char_t * | GetDataSetEnv (const Char_t *dataset, const Char_t *type, const Char_t *defval) |
static Double_t | GetDataSetEnv (const Char_t *dataset, const Char_t *type, Double_t defval) |
static const Char_t * | GetETCDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetExampleFilePath (const Char_t *library, const Char_t *namefile) |
Return full path to example file for given library (="KVMultiDet", "BackTrack", etc.) | |
static const Char_t * | GetINCDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetKVBuildDate () |
Returns KaliVeda build date. | |
static const Char_t * | GetKVBuildDir () |
Returns top-level directory used for build. | |
static const Char_t * | GetKVBuildTime () |
Returns KaliVeda build time. | |
static const Char_t * | GetKVBuildType () |
Returns KaliVeda build type (cmake build: Release, Debug, RelWithDebInfo, ...) | |
static const Char_t * | GetKVBuildUser () |
Returns username of person who performed build. | |
static const Char_t * | GetKVSourceDir () |
Returns top-level directory of source tree used for build. | |
static const Char_t * | GetKVVersion () |
Returns KaliVeda version string. | |
static const Char_t * | GetLIBDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetListOfPlugins (const Char_t *base) |
static const Char_t * | GetListOfPluginURIs (const Char_t *base) |
static const Char_t * | GetPluginURI (const Char_t *base, const Char_t *plugin) |
static void | GetTempFileName (TString &base) |
static const Char_t * | GetTEMPLATEDIRFilePath (const Char_t *namefile="") |
static const Char_t * | GetWORKDIRFilePath (const Char_t *namefile="") |
static const Char_t * | gitBranch () |
Returns git branch of sources. | |
static const Char_t * | gitCommit () |
Returns last git commit of sources. | |
static void | InitEnvironment () |
static bool | is_gnuinstall () |
static Bool_t | IsThisAPlugin (const TString &uri, TString &base) |
static TPluginHandler * | LoadPlugin (const Char_t *base, const Char_t *uri="0") |
static Bool_t | OpenContextMenu (const char *method, TObject *obj, const char *alt_method_name="") |
static void | OpenTempFile (TString &base, std::ofstream &fp) |
static void | PrintSplashScreen () |
Prints welcome message and infos on version etc. | |
static Bool_t | SearchAndOpenKVFile (const Char_t *name, KVSQLite::database &dbfile, const Char_t *kvsubdir="") |
static Bool_t | SearchAndOpenKVFile (const Char_t *name, std::ifstream &file, const Char_t *kvsubdir="", KVLockfile *locks=0) |
static Bool_t | SearchAndOpenKVFile (const Char_t *name, std::ofstream &file, const Char_t *kvsubdir="", KVLockfile *locks=0) |
static Bool_t | SearchKVFile (const Char_t *name, TString &fullpath, const Char_t *kvsubdir="") |
static const Char_t * | WorkingDirectory () |
Static Public Member Functions inherited from TNamed | |
static TClass * | Class () |
static const char * | Class_Name () |
static constexpr Version_t | Class_Version () |
static const char * | DeclFileName () |
Static Public Member Functions inherited from TObject | |
static TClass * | Class () |
static const char * | Class_Name () |
static constexpr Version_t | Class_Version () |
static const char * | DeclFileName () |
static Longptr_t | GetDtorOnly () |
static Bool_t | GetObjectStat () |
static void | SetDtorOnly (void *obj) |
static void | SetObjectStat (Bool_t stat) |
Private Member Functions | |
virtual void | ChangeDefJobOpt (KVDataAnalyser *da) |
Private Attributes | |
KVDataAnalyser * | fAnalyser |
the analyser object which requests job submission, it has all details on the job | |
KVString | fCurrJobName |
name of current job being submitted | |
KVNumberList | fCurrJobRunList |
runlist for (multi job mode) job being submitted | |
KVString | fDefOpt |
default options for job submission command | |
KVString | fJobName |
base job name | |
KVString | fJobScript |
full path of shell script to be executed by batch system | |
KVString | fJobSubCmd |
shell command for submitting job | |
KVNameValueList | fParList |
list of parameters/switches to be passed on job submission command line | |
Additional Inherited Members | |
Public Types inherited from KVBase | |
enum | EKaliVedaBits { kIsKaliVedaObject = BIT(23) } |
Public Types inherited from TObject | |
enum | EDeprecatedStatusBits |
enum | EStatusBits |
Public Attributes inherited from TObject | |
kBitMask | |
kCanDelete | |
kCannotPick | |
kHasUUID | |
kInconsistent | |
kInvalidObject | |
kIsOnHeap | |
kIsReferenced | |
kMustCleanup | |
kNoContextMenu | |
kNotDeleted | |
kObjInCanvas | |
kOverwrite | |
kSingleKey | |
kWriteDelete | |
kZombie | |
Protected Member Functions inherited from TObject | |
virtual void | DoError (int level, const char *location, const char *fmt, va_list va) const |
void | MakeZombie () |
Protected Attributes inherited from TNamed | |
TString | fName |
TString | fTitle |
Protected Attributes inherited from TObject | |
kOnlyPrepStep | |
KVBatchSystem::KVBatchSystem | ( | const Char_t * | name | ) |
Constructor with name of batch system. Name will be used to retrieve resources from configuration file, e.g. batch system title, job submission command, name of batch script, default job options, runs per job in multijob mode:
[batch system name].BatchSystem.Title: [batch system name].BatchSystem.JobSubCmd: [batch system name].BatchSystem.JobScript: [batch system name].BatchSystem.DefaultJobOptions: set title of batch system
Definition at line 36 of file KVBatchSystem.cpp.
|
virtual |
Destructor.
Definition at line 68 of file KVBatchSystem.cpp.
Definition at line 162 of file KVBatchSystem.h.
void KVBatchSystem::cd | ( | ) |
Make this the default batch system.
Definition at line 181 of file KVBatchSystem.cpp.
|
privatevirtual |
PRIVATE method called by SubmitTask() at moment of job submission. Depending on the current environment, the default job submission options may be changed by this method. For example, default options may be changed depending on the analysis task to be performed. We look for an environment variable of the form:
[batch system name].BatchSystem.DefaultJobOptions.[analysis task name]
and if found we use the options defined by it. If not, we use the default job options defined by [batch system name].BatchSystem.DefaultJobOptions
Reimplemented in KV_CCIN2P3_GE, and KV_CCIN2P3_Slurm.
Definition at line 297 of file KVBatchSystem.cpp.
|
virtual |
Checks the job and ask for the job name if needed.
Reimplemented in KV_CCIN2P3_GE, and KV_CCIN2P3_Slurm.
Definition at line 378 of file KVBatchSystem.cpp.
Clear previously set parameters in order to create a new job submission command (default options are not affected: use SetDefaultJobOptions to change them)
Reimplemented from KVBase.
Reimplemented in KV_CCIN2P3_GE, and KV_CCIN2P3_Slurm.
Definition at line 165 of file KVBatchSystem.cpp.
|
static |
Create batch system object defined as a plugin in .kvrootrc If no plugin is found, we create a new KVBatchSystem base object which will be initialised from resources defined in .kvrootrc using its name. check and load plugin library
Definition at line 243 of file KVBatchSystem.cpp.
|
virtual |
Fill the list with all relevant parameters for batch system, set to their default values.
Parameters defined here are: JobName [string] AutoJobName [bool] AutoJobNameFormat [string]
Reimplemented in KV_CCIN2P3_GE, KV_CCIN2P3_Slurm, KVPROOFLiteBatch, and KVXtermBatch.
Definition at line 477 of file KVBatchSystem.cpp.
|
inlinevirtual |
Definition at line 127 of file KVBatchSystem.h.
|
virtual |
Returns name of batch job, either during submission of batch jobs or when an analysis task is running in batch mode (access through gBatchSystem global pointer).
In multi-job mode, the job name is generated from the base name set by SetJobName() plus the extension "_Rxxxx-yyyy" with "xxxx" and "yyyy" the number of the first and last run which will be analysed by the current job.
Depending on the batch system, some sanitization of the jobname may be required e.g. to remove "illegal" characters from the jobname. This is done by SanitizeJobName() before the jobname is returned.
Reimplemented in KV_CCIN2P3_Slurm.
Definition at line 339 of file KVBatchSystem.cpp.
|
inlinevirtual |
Definition at line 103 of file KVBatchSystem.h.
|
virtual |
Builds and returns static string containing full command line necessary to submit a batch job. This is constructed as follows:
[JobSubCmd] [default options] [par1 val1] [par2 val2] ... [JobScript]
The 'default options' can be set with SetDefaultJobOptions().
The 'par1,val1' pairs are the named parameters held in the fParList KVParameterList. For each parameter in the list, the name of the parameter is used for 'par1' and the value of the parameter is used for 'val1' etc. etc. e.g. if the job submission command requires an option to be passed with the following syntax [JobSubCmd] -Noption one of the parameters in the list should be called "-N" and its value should be 'option'
In order to add a simple flag such as [JobSubCmd] -V add a parameter called "-V" with value "".
The special variable #JobName# can be used anywhere and will be replaced by the jobname returned by GetJobName() at the moment of job submission.
The special variable #tmpDIR# can be used anywhere and will be replaced by the full path to the system temporary directory at the moment of job submission.
The special variable #launchDIR# can be used anywhere and will be replaced by the full path to the working directory at the moment of job submission.
Definition at line 106 of file KVBatchSystem.cpp.
|
virtual |
Create and fill list with KVBatchJob objects, one for each job currently handled by the batch system.
Needs to be implemented for specific systems in child classes. This method returns 0x0.
Reimplemented in KV_CCIN2P3_GE.
Definition at line 456 of file KVBatchSystem.cpp.
|
inline |
Definition at line 98 of file KVBatchSystem.h.
|
inlinevirtual |
Reimplemented in KV_CCIN2P3_GE, KV_CCIN2P3_Slurm, and KVXtermBatch.
Definition at line 143 of file KVBatchSystem.h.
if option="log", print infos for batch log file if option="all", print detailed info on batch system
Reimplemented from KVBase.
Reimplemented in KV_CCIN2P3_GE, and KV_CCIN2P3_Slurm.
Definition at line 428 of file KVBatchSystem.cpp.
Reimplemented in KV_CCIN2P3_GE, and KV_CCIN2P3_Slurm.
Definition at line 109 of file KVBatchSystem.h.
Read any useful information on batch system from the TEnv (this method is used by KVDataAnalyser::ReadBatchEnvFile)
Reimplemented in KV_CCIN2P3_GE, and KV_CCIN2P3_Slurm.
Definition at line 414 of file KVBatchSystem.cpp.
|
virtual |
Processes the job requests for the batch system. In normal mode, this submits one job for the data analyser fAnalyser In multijobs mode, this submits one job for each run in the runlist associated to fAnalyser
Reimplemented in KV_CCIN2P3_GE, KV_CCIN2P3_Slurm, and KVXtermBatch.
Definition at line 223 of file KVBatchSystem.cpp.
|
inlinevirtual |
Reimplemented in KV_CCIN2P3_GE, and KV_CCIN2P3_Slurm.
Definition at line 158 of file KVBatchSystem.h.
|
inlinevirtual |
Definition at line 152 of file KVBatchSystem.h.
|
virtual |
Use the parameters in the list to set all relevant parameters for batch system.
Reimplemented in KV_CCIN2P3_GE, KV_CCIN2P3_Slurm, KVPROOFLiteBatch, and KVXtermBatch.
Definition at line 499 of file KVBatchSystem.cpp.
fDefOpt.Form(opt,gSystem->GetGroupInfo()->fGroup.Data());
Definition at line 122 of file KVBatchSystem.h.
Set the job name. In MultiJobsMode this will be used as the base name for all jobs; each individual job will have the name 'basejobname_Rxxxx", with xxxx=run number for job.
The job name can be generated automatically by replacing certain special symbols in the name given here depending on the characteristics of the job. See KVDataAnalyser::ExpandAutoBatchName for allowed symbols.
Definition at line 521 of file KVBatchSystem.cpp.
Definition at line 117 of file KVBatchSystem.h.
|
virtual |
Submits a job to batch system, i.e. executes the string formed by GetJobSubCmdLine, if all necessary parameters have been given (any missing ones will be asked for). Parameters specific to a given batch system can be added my modifying the CheckJobParameters() method for the associated child class. set environment variables required by KaliVedaAnalysis batch executable
Definition at line 197 of file KVBatchSystem.cpp.
|
virtual |
Submit data analysis task described by KVDataAnalyser object to the batch system.
Note that the default options for this batch system may be changed just before job submission depending on the current environment, see ChangeDefJobOpt().
Reimplemented in KVPROOFLiteBatch.
Definition at line 267 of file KVBatchSystem.cpp.
Store any useful information on batch system in the TEnv (this method is used by KVDataAnalyser::WriteBatchEnvFile)
Reimplemented in KV_CCIN2P3_GE, and KV_CCIN2P3_Slurm.
Definition at line 400 of file KVBatchSystem.cpp.
|
private |
the analyser object which requests job submission, it has all details on the job
Definition at line 82 of file KVBatchSystem.h.
|
mutableprivate |
name of current job being submitted
Definition at line 88 of file KVBatchSystem.h.
|
private |
runlist for (multi job mode) job being submitted
Definition at line 89 of file KVBatchSystem.h.
|
private |
default options for job submission command
Definition at line 87 of file KVBatchSystem.h.
|
private |
base job name
Definition at line 84 of file KVBatchSystem.h.
|
private |
full path of shell script to be executed by batch system
Definition at line 86 of file KVBatchSystem.h.
|
private |
shell command for submitting job
Definition at line 85 of file KVBatchSystem.h.
|
private |
list of parameters/switches to be passed on job submission command line
Definition at line 83 of file KVBatchSystem.h.