Detailed Description

Base class for transferring data between data repositories.

Created by KVClassFactory on Mon Sep 18 14:27:21 2006 Author: franklan

If the user has defined two or more repositories in her $HOME/.kvrootrc file, then it is possible to transfer files from one to the other by running one of the following commands on the local machine:

KVDataTransfer::NewTransfer("ccali", "default")->Run() // transfer from ccali to local machine

KVDataTransfer::NewTransfer("default", "ccali")->Run() // transfer from local machine to ccali

KVDataTransfer::Run

virtual void Run()

Definition KVDataTransfer.cpp:161

KVDataTransfer::NewTransfer

static KVDataTransfer * NewTransfer(const Char_t *source_rep, const Char_t *target_rep)

Definition KVDataTransfer.cpp:72

The same menu-driven approach as for submitting analysis tasks is used to allow the user to choose among the available datasets, systems and runs to transfer. The transferred files are copied in to the target repository, creating any missing directories which may be needed.

It is also possible to use a KVDataTransfer object interactively, in this case you should use:

KVDataTransfer* trans = KVDataTransfer::NewTransfer("ccali", "default");
trans->SetDataSet(...);
trans->SetDataType(...);
trans->SetRuns(...);
trans->TransferRuns();

To use the BBFTP transfer facility, a bbftp client must be installed on the user's machine If the client is not located in the user's PATH, he should give the full path to the executable in the configuration, e.g. on WinXP with cygwin version of bbftp client:

ccali.DataRepository.FileTransfer.type: c:\cygwin\bin\bbftp.exe

c

#define c(i)

Definition at line 51 of file KVDataTransfer.h.

#include <KVDataTransfer.h>

Inheritance diagram for KVDataTransfer:

[legend]

Public Member Functions
	KVDataTransfer ()

virtual	~KVDataTransfer ()
	Destructor.

virtual void	Run ()

virtual void	TransferRuns ()

Public Member Functions inherited from KVDataSetAnalyser
	KVDataSetAnalyser ()
	Default constructor.

virtual	~KVDataSetAnalyser ()
	Destructor.

virtual void	AddJobDescriptionList (TList *)

void	ChooseDataSet ()

void	ChooseDataType ()

void	ChooseRuns (KVDBSystem system=nullptr, const Char_t data_type="")

void	ChooseSystem (const Char_t *data_type="")

void	ClearRunList ()

TString	ExpandAutoBatchName (const Char_t *format) const

KVDBRun *	GetAnalysedRun () const

KVDBSystem *	GetAnalysedSystem () const

const KVDataSet *	GetDataSet () const

const KVNumberList &	GetFullRunList () const

const KV2Body *	GetKinematics () const

Int_t	GetNumberOfFilesToAnalyse () const

const Char_t *	GetRecognisedAutoBatchNameKeywords () const

KVString	GetRootDirectoryOfDataToAnalyse () const
	Returns path to data to be analysed.

const KVNumberList &	GetRunList () const

Int_t	GetRunNumberFromFileName (const Char_t *)
	Get the run number from the filename.

const KVDBSystem *	GetSystem () const

Bool_t	IsChooseDataSet () const

Bool_t	IsChooseRuns () const

Bool_t	IsChooseSystem () const

Bool_t	IsChooseTask () const

Bool_t	ReadBatchEnvFile (const TString &filename)

void	Reset ()

void	SetChooseDataSet (Bool_t yes=kTRUE)

void	SetChooseRuns (Bool_t yes=kTRUE)

void	SetChooseSystem (Bool_t yes=kTRUE)

void	SetChooseTask (Bool_t yes=kTRUE)

void	SetDataSet (const Char_t *name)

void	SetDataSet (KVDataSet *ds)

void	SetFullRunList (const KVNumberList &nl)

void	SetRuns (const KVNumberList &nl, Bool_t check=kTRUE)

void	SetSystem (KVDBSystem *syst)
	Set the System used in the analysis.

TString	SystemBatchName () const

void	WriteBatchEnvFile (const TString &jobname, Bool_t save=kTRUE)

Public Member Functions inherited from KVDataAnalyser
	KVDataAnalyser ()
	Default constructor.

virtual	~KVDataAnalyser ()
	Default destructor.

Bool_t	BatchMode () const

virtual Bool_t	CheckStatusUpdateInterval (Long64_t nevents) const

virtual Bool_t	CheckTaskVariables ()

void	ChooseNbEventToRead ()
	Ask user to set number of events to read.

void	ChooseRunningMode ()

void	ChooseUserClass ()
	Choose the user's analysis class.

void	DeleteBatchStatusFile () const
	Delete batch status file (and backup - '.bak') for batch job.

void	DoStatusUpdate (Long64_t nevents) const
	Print infos on events treated, disk usage, memory usage.

Bool_t	DoUserClassFilesExist ()

virtual TTree *	GetAnalysedTree () const

KVDataAnalysisTask *	GetAnalysisTask () const

const Char_t *	GetBatchName ()

const Char_t *	GetBatchStatusFileName () const
	Returns full path to file used to store status of running batch jobs.

virtual const KVBatchSystem *	GetBatchSystem ()

const KVString &	GetDataType () const

const Char_t *	GetLaunchDirectory () const

Long64_t	GetNbEventToRead (void) const

KVDataAnalyser *	GetParent () const

TString	GetPathToFileInLaunchDirectory (const TString &) const

EProofMode	GetProofMode () const

Double_t	GetStatusUpdateInterval () const

virtual Long64_t	GetTotalEntriesToRead () const

const Char_t *	GetUserClass ()

const KVString &	GetUserClassDec () const

const KVString &	GetUserClassImp () const

const KVString &	GetUserClassOptions () const

const KVString &	GetUserIncludes () const

const KVString &	GetUserLibraries () const

Bool_t	IsMenus () const

Bool_t	IsQuit () const

Bool_t	IsSubmit () const

Bool_t	IsUserClassValid () const

virtual void	postAnalysis ()

virtual void	postEndAnalysis ()

virtual void	postEndRun ()

virtual void	postInitAnalysis ()

virtual void	postInitRun ()

virtual void	preAnalysis ()

virtual void	preEndAnalysis ()

virtual void	preEndRun ()

virtual void	preInitAnalysis ()

virtual void	preInitRun ()

virtual void	RegisterUserClass (TObject *)

virtual void	RunMenus ()
	Run data analyser in menu-driven mode.

Bool_t	RunningInLaunchDirectory ()

virtual void	SetAnalysedTree (TTree *)

void	SetAnalysisTask (KVDataAnalysisTask *at)

virtual void	SetAuxFileList (TList *)

void	SetBatchMode (Bool_t on=kTRUE)

void	SetBatchName (const Char_t *batchname)

void	SetBatchSystem (KVBatchSystem *bs)

void	SetDataType (const Char_t *name)

virtual void	SetFileList (TList *)

void	SetMenus (Bool_t on=kTRUE)

void	SetNbEventToRead (Long64_t nb=0)

void	SetParent (KVDataAnalyser *da)

void	SetProofMode (EProofMode e)

void	SetQuit (Bool_t yes=kTRUE)

void	SetStatusUpdateInterval (double t)

void	SetSubmit (Bool_t yes=kTRUE)

virtual void	SetTriggerConditionsForRun (int)

void	SetUserClass (const Char_t *kvs, Bool_t check=kTRUE)

void	SetUserClassOptions (const Char_t *o="")

void	SetUserIncludes (const Char_t *incDirs=0)

void	SetUserLibraries (const Char_t *libs=0)

void	UpdateBatchStatusFile (Int_t totev, Int_t evread, TString disk) const
	Update infos in batch status file.

void	UseBaseClassSubmitTask ()

void	WriteBatchInfo (TTree *)

Public Member Functions inherited from TObject
	TObject ()

	TObject (const TObject &object)

virtual	~TObject ()

void	AbstractMethod (const char *method) const

virtual void	AppendPad (Option_t *option="")

virtual void	Browse (TBrowser *b)

ULong_t	CheckedHash ()

virtual const char *	ClassName () const

virtual void	Clear (Option_t *="")

virtual TObject *	Clone (const char *newname="") const

virtual Int_t	Compare (const TObject *obj) const

virtual void	Copy (TObject &object) const

virtual void	Delete (Option_t *option="")

virtual Int_t	DistancetoPrimitive (Int_t px, Int_t py)

virtual void	Draw (Option_t *option="")

virtual void	DrawClass () const

virtual TObject *	DrawClone (Option_t *option="") const

virtual void	Dump () const

virtual void	Error (const char method, const char msgfmt,...) const

virtual void	Execute (const char method, const char params, Int_t *error=nullptr)

virtual void	Execute (TMethod method, TObjArray params, Int_t *error=nullptr)

virtual void	ExecuteEvent (Int_t event, Int_t px, Int_t py)

virtual void	Fatal (const char method, const char msgfmt,...) const

virtual TObject *	FindObject (const char *name) const

virtual TObject *	FindObject (const TObject *obj) const

virtual Option_t *	GetDrawOption () const

virtual const char *	GetIconName () const

virtual const char *	GetName () const

virtual char *	GetObjectInfo (Int_t px, Int_t py) const

virtual Option_t *	GetOption () const

virtual const char *	GetTitle () const

virtual UInt_t	GetUniqueID () const

virtual Bool_t	HandleTimer (TTimer *timer)

virtual ULong_t	Hash () const

Bool_t	HasInconsistentHash () const

virtual void	Info (const char method, const char msgfmt,...) const

virtual Bool_t	InheritsFrom (const char *classname) const

virtual Bool_t	InheritsFrom (const TClass *cl) const

virtual void	Inspect () const

void	InvertBit (UInt_t f)

virtual TClass *	IsA () const

Bool_t	IsDestructed () const

virtual Bool_t	IsEqual (const TObject *obj) const

virtual Bool_t	IsFolder () const

R__ALWAYS_INLINE Bool_t	IsOnHeap () const

virtual Bool_t	IsSortable () const

R__ALWAYS_INLINE Bool_t	IsZombie () const

virtual void	ls (Option_t *option="") const

void	MayNotUse (const char *method) const

virtual Bool_t	Notify ()

void	Obsolete (const char method, const char asOfVers, const char *removedFromVers) const

void	operator delete (void *ptr)

void	operator delete (void ptr, void vp)

void	operator delete[] (void *ptr)

void	operator delete[] (void ptr, void vp)

void *	operator new (size_t sz)

void *	operator new (size_t sz, void *vp)

void *	operator new[] (size_t sz)

void *	operator new[] (size_t sz, void *vp)

TObject &	operator= (const TObject &rhs)

virtual void	Paint (Option_t *option="")

virtual void	Pop ()

virtual void	Print (Option_t *option="") const

virtual Int_t	Read (const char *name)

virtual void	RecursiveRemove (TObject *obj)

void	ResetBit (UInt_t f)

virtual void	SaveAs (const char filename="", Option_t option="") const

virtual void	SavePrimitive (std::ostream &out, Option_t *option="")

void	SetBit (UInt_t f)

void	SetBit (UInt_t f, Bool_t set)

virtual void	SetDrawOption (Option_t *option="")

virtual void	SetUniqueID (UInt_t uid)

virtual void	Streamer (TBuffer &)

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

virtual void	SysError (const char method, const char msgfmt,...) const

R__ALWAYS_INLINE Bool_t	TestBit (UInt_t f) const

Int_t	TestBits (UInt_t f) const

virtual void	UseCurrentStyle ()

virtual void	Warning (const char method, const char msgfmt,...) const

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0)

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0) const

Static Public Member Functions
static KVDataTransfer *	NewTransfer (const Char_t source_rep, const Char_t target_rep)

Static Public Member Functions inherited from KVDataAnalyser
static Bool_t	AbortProcessingLoop ()

static KVDataAnalyser *	GetAnalyser (const Char_t *plugin)

static Bool_t	IsRunningBatchAnalysis ()

static void	RunAnalyser (const Char_t *plugin="")

static void	SetAbortProcessingLoop (Bool_t now=kTRUE)

Static Public Member Functions inherited from TObject
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

static Longptr_t	GetDtorOnly ()

static Bool_t	GetObjectStat ()

static void	SetDtorOnly (void *obj)

static void	SetObjectStat (Bool_t stat)

Private Member Functions
virtual void	CheckTargetRepository ()

virtual void	ChooseAnalysisTask ()

virtual void	ExecuteCommand ()=0

virtual void	init ()
	Initialisation of data transfer.

virtual void	set_dataset_name (const Char_t *name)

virtual void	set_dataset_pointer (KVDataSet *ds)

virtual void	SetTransferExec (const Char_t *path)

virtual void	SubmitTask ()

virtual void	WriteTransferScript ()=0

Private Attributes
TString	fCmdFile
	name of command file given to transfer agent

Bool_t	fOK
	may be set to kFALSE by init(), in which case Run() will abort

KVDataRepository *	fSourceRep
	repository containing source files

KVDataRepository *	fTargetRep
	repository where files will be copied

TString	fTransferExec
	full path to transfer client executable

Additional Inherited Members
Public Types inherited from KVDataAnalyser
enum	EProofMode { None , Lite , Proof }

Public Types inherited from TObject
enum	EDeprecatedStatusBits

enum	EStatusBits

Public Attributes inherited from TObject
	kBitMask

	kCanDelete

	kCannotPick

	kHasUUID

	kInconsistent

	kInvalidObject

	kIsOnHeap

	kIsReferenced

	kMustCleanup

	kNoContextMenu

	kNotDeleted

	kObjInCanvas

	kOverwrite

	kSingleKey

	kWriteDelete

	kZombie

Protected Member Functions inherited from TObject
virtual void	DoError (int level, const char location, const char fmt, va_list va) const

void	MakeZombie ()

Protected Attributes inherited from TObject
	kOnlyPrepStep

Constructor & Destructor Documentation

◆ KVDataTransfer()

KVDataTransfer::KVDataTransfer ( )

Default constructor. Use KVDataTransfer::NewTransfer to create a new data transfer object with the correct properties to transfer files between two repositories.

Definition at line 27 of file KVDataTransfer.cpp.

◆ ~KVDataTransfer()

KVDataTransfer::~KVDataTransfer ( )

virtual

Destructor.

Definition at line 42 of file KVDataTransfer.cpp.

Member Function Documentation

◆ CheckTargetRepository()

void KVDataTransfer::CheckTargetRepository ( )

privatevirtual

Make sure that the target repository has the necessary dataset/subdirectory to receive the transferred files. If not, we create the new dataset/subdir.

Definition at line 269 of file KVDataTransfer.cpp.

◆ ChooseAnalysisTask()

virtual void KVDataTransfer::ChooseAnalysisTask ( )

inlineprivatevirtual

Reimplemented from KVDataSetAnalyser.

Definition at line 62 of file KVDataTransfer.h.

◆ ExecuteCommand()

virtual void KVDataTransfer::ExecuteCommand ( )

privatepure virtual

Implemented in KVDataTransferDMS, and KVDataTransferXRD.

◆ init()

void KVDataTransfer::init ( )

privatevirtual

Initialisation of data transfer.

Definition at line 298 of file KVDataTransfer.cpp.

◆ NewTransfer()

KVDataTransfer * KVDataTransfer::NewTransfer	(	const Char_t *	source_rep,
		const Char_t *	target_rep
	)

static

Creates a new data transfer object to transfer files between the two named data repositories.

The type of the created object depends on the values of the environment variables (defined in .kvrootrc file):

source_rep.DataRepository.FileTransfer.type

target_rep.DataRepository.FileTransfer.type

If either of these = bbftp, we create a KVDataTransferBBFTP object.

If source_rep.DataRepository.FileTransfer.type = xrd, we create a KVDataTransferXRD object.

By default (no type given), we use a KVDataTransferSFTP object. In fact, the type of object created is defined in .kvrootrc by the following plugins:

Plugins for data transfer between repositories

Used by KVDataTransfer::NewTransfer

Plugin.KVDataTransfer: sftp KVDataTransferSFTP KVMultiDet "KVDataTransferSFTP()" +Plugin.KVDataTransfer: bbftp KVDataTransferBBFTP KVMultiDet "KVDataTransferBBFTP()" +Plugin.KVDataTransfer: xrd KVDataTransferXRD KVMultiDet "KVDataTransferXRD()"

Definition at line 72 of file KVDataTransfer.cpp.

◆ Run()

void KVDataTransfer::Run ( )

virtual

Perform file transfer The user selects runs which are available in the source repository in the same way as when performing data analysis Note that this will make the 'source' repository the current active repository (i.e. gDataRepository and gDataSetManager will correspond to source repository after execution of this command).

Reimplemented from KVDataAnalyser.

Definition at line 161 of file KVDataTransfer.cpp.

◆ set_dataset_name()

void KVDataTransfer::set_dataset_name ( const Char_t * name )

privatevirtual

Set dataset to be analysed. If 'name' is not the name of a valid and available dataset in the 'source' data repository an error message is printed.

Reimplemented from KVDataSetAnalyser.

Definition at line 311 of file KVDataTransfer.cpp.

◆ set_dataset_pointer()

void KVDataTransfer::set_dataset_pointer ( KVDataSet * ds )

privatevirtual

Set dataset to be used for transfer. If the chosen dataset is not available, an error message is printed Only datasets which are available in the source repository can be transferred If the pointer actually corresponds to a dataset in the target repository, we replace it with a pointer to the dataset with the same name in the source repository. allow user to reset dataset pointer to 0

Reimplemented from KVDataSetAnalyser.

Definition at line 339 of file KVDataTransfer.cpp.

◆ SetTransferExec()

virtual void KVDataTransfer::SetTransferExec ( const Char_t * path )

inlineprivatevirtual

Definition at line 58 of file KVDataTransfer.h.

◆ SubmitTask()

virtual void KVDataTransfer::SubmitTask ( )

inlineprivatevirtual

Reimplemented from KVDataAnalyser.

Definition at line 65 of file KVDataTransfer.h.

◆ TransferRuns()

void KVDataTransfer::TransferRuns ( )

virtual

Based on information gathered from user (see Run()), perform the transfer of files. Any missing directories in the target repository are created beforehand. After transfer, the available runlist for the target repository is updated

Definition at line 217 of file KVDataTransfer.cpp.

◆ WriteTransferScript()

virtual void KVDataTransfer::WriteTransferScript ( )

privatepure virtual

Implemented in KVDataTransferDMS, and KVDataTransferXRD.

Member Data Documentation

◆ fCmdFile

TString KVDataTransfer::fCmdFile

private

name of command file given to transfer agent

Definition at line 55 of file KVDataTransfer.h.

◆ fOK

Bool_t KVDataTransfer::fOK

private

may be set to kFALSE by init(), in which case Run() will abort

Definition at line 75 of file KVDataTransfer.h.

◆ fSourceRep

KVDataRepository* KVDataTransfer::fSourceRep

private

repository containing source files

Definition at line 53 of file KVDataTransfer.h.

◆ fTargetRep

KVDataRepository* KVDataTransfer::fTargetRep

private

repository where files will be copied

Definition at line 54 of file KVDataTransfer.h.

◆ fTransferExec

TString KVDataTransfer::fTransferExec

private

full path to transfer client executable

Definition at line 56 of file KVDataTransfer.h.

Detailed Description

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ KVDataTransfer()

◆ ~KVDataTransfer()

Member Function Documentation

◆ CheckTargetRepository()

◆ ChooseAnalysisTask()

◆ ExecuteCommand()

◆ init()

◆ NewTransfer()

Plugins for data transfer between repositories

Used by KVDataTransfer::NewTransfer

◆ Run()

◆ set_dataset_name()

◆ set_dataset_pointer()

◆ SetTransferExec()

◆ SubmitTask()

◆ TransferRuns()

◆ WriteTransferScript()

Member Data Documentation

◆ fCmdFile

◆ fOK

◆ fSourceRep

◆ fTargetRep

◆ fTransferExec