Detailed Description

Base class for managing repositories of experimental data.

Data repositories are defined in the user's $HOME/.kvrootrc; some typical examples are given in $KVROOT/KVFiles/.kvrootrc:

DataRepository: home
home.DataRepository.RootDir: /root/Data

<p<blockquote>‍

This is the minimum requirement to define a data repository: give a name ("home") and the full path to the top level directory ("...RootDir: $(HOME)/Data"). In this case the top level directory must be directly accessible to the user's machine as a locally-mounted disk. Note that environment variables can be used, if enclosed between "$(...)".

DataRepository: ccali
ccali.DataRepository.Type: remote
ccali.DataRepository.RootDir:       cchpssindra:/hpss/in2p3.fr/group/indra
ccali.DataRepository.ReadProtocol:     root
ccali.DataRepository.XRDServer:      ccxroot:1999
ccali.DataRepository.XRDRootDir:       /hpss/in2p3.fr/group/indra
ccali.DataRepository.XRDTunnel.host:       ccali.in2p3.fr
ccali.DataRepository.XRDTunnel.port:          10000
ccali.DataRepository.XRDTunnel.user:
ccali.DataRepository.RemoteAvailableRuns.protocol:  curl
ccali.DataRepository.RemoteAvailableRuns.url:   http://indra.in2p3.fr/KaliVedaDoc
ccali.DataRepository.FileTransfer.type:    bbftp
ccali.DataRepository.FileTransfer.server:    ccbbftp.in2p3.fr
ccali.DataRepository.FileTransfer.user:

<p<blockquote>‍

This is the definition of a "remote" data repository. It will be handled by an object of the class KVRemoteDataRepository.

A remote data repository is principally characterised by the fact that access to the data files is via some non-local protocol: in this case it is xrootd ("...ReadProtocol: root"). The host name and port of the xrootd server are given ("...XRDServer: ccxroot:1999"), as well as the root directory to be used ("...XRDRootDir: /hpss/in2p3.fr/group/indra").

In order to use a remote repository, some way to access the database of available data files for each run type must be defined. Here it is the curl programme which is used to read them via a website ("...RemoteAvailableRuns.protocol: curl"; "...RemoteAvailableRuns.url: http://indra.in2p3.fr/KaliVedaDoc").

Another particularity of remote data repositories is that some additional stuff may be needed in order to be able to access the data. This is handled by KVRemoteDataRepository::IsConnected(). In the present example, an SSH tunnel is set up in order to allow secure connection to the xrootd server ("...XRDTunnel.*:").

Finally, transfer of data between data repositories is handled by KVDataTransfer and child classes. In the present example, this is configured to use bbftp ("...FileTransfer.type: bbftp"), for which the required details are given.

Default data repository

If more than one repository is defined, which one will be "active" (gDataRepository) after initialisation of the data repository manager ? You can define the default repository by setting the variable

DataRepository.Default:   [name]

in your .kvrootrc file. If you don't, the default repository "by default" will be either: the data repository named "default" if there is one; or the last one defined in your .kvrootrc.

Definition at line 99 of file KVDataRepository.h.

#include <KVDataRepository.h>

Inheritance diagram for KVDataRepository:

[legend]

Public Member Functions
	KVDataRepository ()
	Default constructor.

virtual	~KVDataRepository ()
	Destructor.

virtual Bool_t	CanWrite () const

void	cd ()

virtual Bool_t	CheckFileStatus (const KVDataSet dataset, const Char_t datatype, const Char_t *runfile)

virtual Bool_t	CheckSubdirExists (const Char_t dir, const Char_t subdir=0)

virtual void	CommitFile (TFile file, const Char_t datatype, const KVDataSet *dataset)

virtual int	CopyFile (const char f, const char t, Bool_t overwrite=kFALSE)

virtual void	CopyFileFromRepository (const KVDataSet dataset, const Char_t datatype, const Char_t filename, const Char_t destination)

virtual int	CopyFileToRepository (const Char_t source, const KVDataSet dataset, const Char_t datatype, const Char_t filename)

void	CreateAllNeededSubdirectories (const KVDataSet DataSet, const Char_t DataType)

virtual TFile *	CreateNewFile (const KVDataSet dataset, const Char_t datatype, const Char_t *filename)

virtual void	DeleteFile (const KVDataSet dataset, const Char_t datatype, const Char_t *filename, Bool_t confirm=kTRUE)

TSystem *	FindHelper (const char path, void dirptr=0)

virtual KVDataSetManager *	GetDataSetManager () const
	Return pointer to data set manager for this repository.

virtual KVUniqueNameList *	GetDirectoryListing (const KVDataSet dataset, const Char_t datatype="", const Char_t *subdir="")

virtual Bool_t	GetFileInfo (const KVDataSet dataset, const Char_t datatype, const Char_t *runfile, FileStat_t &fs)

virtual const Char_t *	GetFileTransferExec () const
	returns full path to executable used for remote file transfer

virtual const Char_t *	GetFileTransferServer () const
	returns server url used for remote file transfer

virtual const Char_t *	GetFileTransferType () const
	returns protocol used for remote file transfer

virtual const Char_t *	GetFileTransferUser () const
	returns user name used for remote file transfer

virtual const Char_t *	GetFullPathToOpenFile (const KVDataSet dataset, const Char_t datatype, const Char_t *runfile)

virtual const Char_t *	GetFullPathToTransferFile (const KVDataSet dataset, const Char_t datatype, const Char_t *runfile)

virtual const Char_t *	GetRootDirectory () const
	returns root directory of data repository (fLocalrootdir)

Bool_t	HelperIsConsistentWith (TSystem helper, const char path, void *dirptr=0)

virtual Bool_t	Init ()

virtual Bool_t	IsConnected ()
	Always returns kTRUE for local repositories.

virtual Bool_t	IsRemote () const
	Returns kTRUE for remote repositories, kFALSE for local repositories.

virtual void	MakeSubdirectory (const KVDataSet dataset, const Char_t datatype="")

virtual KVAvailableRunsFile *	NewAvailableRunsFile (const Char_t , const KVDataSet )

virtual TObject *	OpenDataSetRunFile (const KVDataSet ds, const Char_t type, Int_t run, Option_t *opt="")

virtual void	Print (Option_t *opt="") const
	Print info on repository.

virtual void	PrintAvailableDatasetsUpdateWarning () const

Public Member Functions inherited from KVBase
	KVBase ()
	Default constructor.

	KVBase (const Char_t name, const Char_t title="")
	Ctor for object with given name and type.

	KVBase (const KVBase &)
	copy ctor

virtual	~KVBase ()

virtual void	Clear (Option_t *opt="")
	Clear object properties : name, type/title, number, label.

virtual void	Copy (TObject &) const
	Make a copy of this object.

const Char_t *	GetLabel () const

UInt_t	GetNumber () const

UInt_t	GetNumberOfObjects () const

virtual TObject *	GetObject () const

virtual const Char_t *	GetType () const

Bool_t	HasLabel () const

virtual Bool_t	IsCalled (const Char_t *name) const

Bool_t	IsLabelled (const Char_t *l) const

virtual Bool_t	IsType (const Char_t *typ) const

virtual void	List ()

KVBase &	operator= (const KVBase &)
	copy assignment operator

Double_t	ProtectedGetX (const TF1 *func, Double_t val, int &status, Double_t xmin=0.0, Double_t xmax=0.0) const

void	SetLabel (const Char_t *lab)

virtual void	SetNumber (UInt_t num)

virtual void	SetType (const Char_t *str)

Public Member Functions inherited from TNamed
	TNamed ()

	TNamed (const char name, const char title)

	TNamed (const TNamed &named)

	TNamed (const TString &name, const TString &title)

virtual	~TNamed ()

void	Clear (Option_t *option="") override

TObject *	Clone (const char *newname="") const override

Int_t	Compare (const TObject *obj) const override

void	Copy (TObject &named) const override

virtual void	FillBuffer (char *&buffer)

const char *	GetName () const override

const char *	GetTitle () const override

ULong_t	Hash () const override

TClass *	IsA () const override

Bool_t	IsSortable () const override

void	ls (Option_t *option="") const override

TNamed &	operator= (const TNamed &rhs)

void	Print (Option_t *option="") const override

virtual void	SetName (const char *name)

virtual void	SetNameTitle (const char name, const char title)

virtual void	SetTitle (const char *title="")

virtual Int_t	Sizeof () const

void	Streamer (TBuffer &) override

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

Public Member Functions inherited from TObject
	TObject ()

	TObject (const TObject &object)

virtual	~TObject ()

void	AbstractMethod (const char *method) const

virtual void	AppendPad (Option_t *option="")

virtual void	Browse (TBrowser *b)

ULong_t	CheckedHash ()

virtual const char *	ClassName () const

virtual void	Delete (Option_t *option="")

virtual Int_t	DistancetoPrimitive (Int_t px, Int_t py)

virtual void	Draw (Option_t *option="")

virtual void	DrawClass () const

virtual TObject *	DrawClone (Option_t *option="") const

virtual void	Dump () const

virtual void	Error (const char method, const char msgfmt,...) const

virtual void	Execute (const char method, const char params, Int_t *error=nullptr)

virtual void	Execute (TMethod method, TObjArray params, Int_t *error=nullptr)

virtual void	ExecuteEvent (Int_t event, Int_t px, Int_t py)

virtual void	Fatal (const char method, const char msgfmt,...) const

virtual TObject *	FindObject (const char *name) const

virtual TObject *	FindObject (const TObject *obj) const

virtual Option_t *	GetDrawOption () const

virtual const char *	GetIconName () const

virtual char *	GetObjectInfo (Int_t px, Int_t py) const

virtual Option_t *	GetOption () const

virtual UInt_t	GetUniqueID () const

virtual Bool_t	HandleTimer (TTimer *timer)

Bool_t	HasInconsistentHash () const

virtual void	Info (const char method, const char msgfmt,...) const

virtual Bool_t	InheritsFrom (const char *classname) const

virtual Bool_t	InheritsFrom (const TClass *cl) const

virtual void	Inspect () const

void	InvertBit (UInt_t f)

Bool_t	IsDestructed () const

virtual Bool_t	IsEqual (const TObject *obj) const

virtual Bool_t	IsFolder () const

R__ALWAYS_INLINE Bool_t	IsOnHeap () const

R__ALWAYS_INLINE Bool_t	IsZombie () const

void	MayNotUse (const char *method) const

virtual Bool_t	Notify ()

void	Obsolete (const char method, const char asOfVers, const char *removedFromVers) const

void	operator delete (void *ptr)

void	operator delete (void ptr, void vp)

void	operator delete[] (void *ptr)

void	operator delete[] (void ptr, void vp)

void *	operator new (size_t sz)

void *	operator new (size_t sz, void *vp)

void *	operator new[] (size_t sz)

void *	operator new[] (size_t sz, void *vp)

TObject &	operator= (const TObject &rhs)

virtual void	Paint (Option_t *option="")

virtual void	Pop ()

virtual Int_t	Read (const char *name)

virtual void	RecursiveRemove (TObject *obj)

void	ResetBit (UInt_t f)

virtual void	SaveAs (const char filename="", Option_t option="") const

virtual void	SavePrimitive (std::ostream &out, Option_t *option="")

void	SetBit (UInt_t f)

void	SetBit (UInt_t f, Bool_t set)

virtual void	SetDrawOption (Option_t *option="")

virtual void	SetUniqueID (UInt_t uid)

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

virtual void	SysError (const char method, const char msgfmt,...) const

R__ALWAYS_INLINE Bool_t	TestBit (UInt_t f) const

Int_t	TestBits (UInt_t f) const

virtual void	UseCurrentStyle ()

virtual void	Warning (const char method, const char msgfmt,...) const

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0)

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0) const

Static Public Member Functions
static KVDataRepository *	NewRepository (const Char_t *type)

Static Public Member Functions inherited from KVBase
static Bool_t	AreEqual (Double_t x, Double_t y, Long64_t maxdif=1)
	Comparison between two 64-bit floating-point values.

static void	BackupFileWithDate (const Char_t *path)

static void	CombineFiles (const Char_t file1, const Char_t file2, const Char_t *newfilename, Bool_t keep=kTRUE)

static void	Deprecated (const char method, const char advice)

static Bool_t	FindClassSourceFiles (const Char_t class_name, KVString &imp_file, KVString &dec_file, const Char_t dir_name=".")

static Bool_t	FindExecutable (TString &exec, const Char_t *path="$(PATH)")

static const Char_t *	FindFile (const Char_t *search, TString &wfil)

static const Char_t *	GetBINDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetDATABASEFilePath ()

static const Char_t *	GetDATADIRFilePath (const Char_t *namefile="")

static Bool_t	GetDataSetEnv (const Char_t dataset, const Char_t type, Bool_t defval)

static const Char_t *	GetDataSetEnv (const Char_t dataset, const Char_t type, const Char_t *defval)

static Double_t	GetDataSetEnv (const Char_t dataset, const Char_t type, Double_t defval)

static const Char_t *	GetETCDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetExampleFilePath (const Char_t library, const Char_t namefile)
	Return full path to example file for given library (="KVMultiDet", "BackTrack", etc.)

static const Char_t *	GetINCDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetKVBuildDate ()
	Returns KaliVeda build date.

static const Char_t *	GetKVBuildDir ()
	Returns top-level directory used for build.

static const Char_t *	GetKVBuildTime ()
	Returns KaliVeda build time.

static const Char_t *	GetKVBuildType ()
	Returns KaliVeda build type (cmake build: Release, Debug, RelWithDebInfo, ...)

static const Char_t *	GetKVBuildUser ()
	Returns username of person who performed build.

static const Char_t *	GetKVSourceDir ()
	Returns top-level directory of source tree used for build.

static const Char_t *	GetKVVersion ()
	Returns KaliVeda version string.

static const Char_t *	GetLIBDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetListOfPlugins (const Char_t *base)

static const Char_t *	GetListOfPluginURIs (const Char_t *base)

static const Char_t *	GetPluginURI (const Char_t base, const Char_t plugin)

static void	GetTempFileName (TString &base)

static const Char_t *	GetTEMPLATEDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetWORKDIRFilePath (const Char_t *namefile="")

static const Char_t *	gitBranch ()
	Returns git branch of sources.

static const Char_t *	gitCommit ()
	Returns last git commit of sources.

static void	InitEnvironment ()

static bool	is_gnuinstall ()

static Bool_t	IsThisAPlugin (const TString &uri, TString &base)

static TPluginHandler *	LoadPlugin (const Char_t base, const Char_t uri="0")

static Bool_t	OpenContextMenu (const char method, TObject obj, const char *alt_method_name="")

static void	OpenTempFile (TString &base, std::ofstream &fp)

static void	PrintSplashScreen ()
	Prints welcome message and infos on version etc.

static Bool_t	SearchAndOpenKVFile (const Char_t name, KVSQLite::database &dbfile, const Char_t kvsubdir="")

static Bool_t	SearchAndOpenKVFile (const Char_t name, std::ifstream &file, const Char_t kvsubdir="", KVLockfile *locks=0)

static Bool_t	SearchAndOpenKVFile (const Char_t name, std::ofstream &file, const Char_t kvsubdir="", KVLockfile *locks=0)

static Bool_t	SearchKVFile (const Char_t name, TString &fullpath, const Char_t kvsubdir="")

static const Char_t *	WorkingDirectory ()

Static Public Member Functions inherited from TNamed
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

Static Public Member Functions inherited from TObject
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

static Longptr_t	GetDtorOnly ()

static Bool_t	GetObjectStat ()

static void	SetDtorOnly (void *obj)

static void	SetObjectStat (Bool_t stat)

Private Member Functions
virtual int	Chmod (const char *file, UInt_t mode)

virtual const Char_t *	GetReadProtocol (const Char_t dataset, const Char_t datatype)

virtual KVDataSetManager *	NewDataSetManager ()
	Create and return pointer to new data set manager.

TObject *	OpenDataSetFile (const KVDataSet ds, const Char_t type, const TString &fname, Option_t *opt="")

virtual void	PrepareXRDTunnel ()

void	SetFullPath (TString &path, const Char_t *protocol)

Private Attributes
TString	fAccessprotocol

TString	fAccessroot

Bool_t	fCanWrite

const KVDataSet *	fCommitDataSet
	used by CreateNewFile and CommitFile

TString	fCommitDataType

TString	fCommitFileName

KVDataSetManager *	fDSM
	handles datasets in repository

TSeqCollection *	fHelpers
	List of helper classes for alternative file/directory access.

TString	fLocalrootdir

TString	fReadprotocol

TString	fReadroot

TString	fRfiorootdir

TString	fRfioserver

TString	fTransferExec

TString	fTransferserver

TString	fTransfertype

TString	fTransferuser

TString	fXRDtunHost

Bool_t	fXRDtunnel

Int_t	fXRDtunPort

Int_t	fXRDtunRetry

TString	fXRDtunSpec

TString	fXRDtunUser

TString	fXrootdrootdir

TString	fXrootdserver

Additional Inherited Members
Public Types inherited from KVBase
enum	EKaliVedaBits { kIsKaliVedaObject = BIT(23) }

Public Types inherited from TObject
enum	EDeprecatedStatusBits

enum	EStatusBits

Public Attributes inherited from TObject
	kBitMask

	kCanDelete

	kCannotPick

	kHasUUID

	kInconsistent

	kInvalidObject

	kIsOnHeap

	kIsReferenced

	kMustCleanup

	kNoContextMenu

	kNotDeleted

	kObjInCanvas

	kOverwrite

	kSingleKey

	kWriteDelete

	kZombie

Protected Member Functions inherited from TObject
virtual void	DoError (int level, const char location, const char fmt, va_list va) const

void	MakeZombie ()

Protected Attributes inherited from TNamed
TString	fName

TString	fTitle

Protected Attributes inherited from TObject
	kOnlyPrepStep

Constructor & Destructor Documentation

◆ KVDataRepository()

KVDataRepository::KVDataRepository ( )

Default constructor.

Definition at line 198 of file KVDataRepository.cpp.

◆ ~KVDataRepository()

KVDataRepository::~KVDataRepository ( )

virtual

Destructor.

Definition at line 213 of file KVDataRepository.cpp.

Member Function Documentation

◆ CanWrite()

virtual Bool_t KVDataRepository::CanWrite ( ) const

inlinevirtual

Returns kTRUE if new files can be created and written directly in the repository, returns kFALSE if they have to be written locally then copied to repository This is set by the property [repository name].DataRepository.CanWrite in the repository configuration (see $KVROOT/KVFiles/.kvrootrc)

Definition at line 178 of file KVDataRepository.h.

◆ cd()

void KVDataRepository::cd ( )

Make this the 'active' or 'default' data repository, i.e. gDataRepository points to this and gDataSetManager points to the associated data set manager.

Definition at line 47 of file KVDataRepository.cpp.

◆ CheckFileStatus()

Bool_t KVDataRepository::CheckFileStatus	(	const KVDataSet *	dataset,
		const Char_t *	datatype,
		const Char_t *	runfile
	)

virtual

Checks if the run file of given type is physically present in dataset subdirectory, i.e. (schematically), if

/root_of_data_repository/[datasetdir]/[datatypedir]/[runfile]

exists. If it does, the returned value is kTRUE (=1).

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 327 of file KVDataRepository.cpp.

◆ CheckSubdirExists()

Bool_t KVDataRepository::CheckSubdirExists	(	const Char_t *	dir,
		const Char_t *	subdir = `0`
	)

virtual

Returns kTRUE if the following path is valid /root_of_data_repository/dir/[subdir]

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 234 of file KVDataRepository.cpp.

◆ Chmod()

int KVDataRepository::Chmod	(	const char *	file,
		UInt_t	mode
	)

privatevirtual

Used to change file access permissions in the repository do we need a special helper for this filesystem ?

Reimplemented in KVDMSDataRepository.

Definition at line 917 of file KVDataRepository.cpp.

◆ CommitFile()

void KVDataRepository::CommitFile	(	TFile *	file,
		const Char_t *	datatype,
		const KVDataSet *	dataset
	)

virtual

Add this file (previously created by a call to CreateNewFile) to the repository. Any objects should be written to the file before calling this method, either by calling the Write() method of each object, or by calling file->Write(). No file->Write() is done here: we only close (delete) the TFile.

For repositories in which files can be created and written directly (i.e. if CanWrite() = kTRUE), we just have to close the file. For repositories where CanWrite() = kFALSE, we close the file, then copy it to the correct place in the repository, using the previously recorded values of fCommitDataSetDir, fCommitDataType, and fCommitFileName. Then we remove the local copy from disk.

NB: after calling this method, the TFile pointer 'file' must not be used!!! close ROOT file if (fCommitFileName == "") {

Definition at line 730 of file KVDataRepository.cpp.

◆ CopyFile()

int KVDataRepository::CopyFile	(	const char *	f,
		const char *	t,
		Bool_t	overwrite = `kFALSE`
	)

virtual

Method to copy any file from anywhere to anywhere (if the necessary TFile and TSystem plugins are available). Copy a file. If overwrite is true and file already exists the file will be overwritten. Returns 0 when successful, -1 in case of failure, -2 in case the file already exists and overwrite was false.

Definition at line 936 of file KVDataRepository.cpp.

◆ CopyFileFromRepository()

void KVDataRepository::CopyFileFromRepository	(	const KVDataSet *	dataset,
		const Char_t *	datatype,
		const Char_t *	filename,
		const Char_t *	destination
	)

virtual

Copy file [datasetdir]/[datatypedir]/[filename] from the repository to [destination] We check if the file to copy exists.

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 504 of file KVDataRepository.cpp.

◆ CopyFileToRepository()

int KVDataRepository::CopyFileToRepository	(	const Char_t *	source,
		const KVDataSet *	dataset,
		const Char_t *	datatype,
		const Char_t *	filename
	)

virtual

Copy file [source] to [datasetdir]/[datatypedir]/[filename] in the repository The file access permissions are set to '664 (u:rw, g:rw, o:r) Returns status of file transfer command

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 531 of file KVDataRepository.cpp.

◆ CreateAllNeededSubdirectories()

void KVDataRepository::CreateAllNeededSubdirectories	(	const KVDataSet *	DataSet,
		const Char_t *	DataType
	)

Ensure that all required subdirectories exist and any missing ones are created in order to store a runfile of given DataType for the given DataSet

Definition at line 1366 of file KVDataRepository.cpp.

◆ CreateNewFile()

TFile * KVDataRepository::CreateNewFile	(	const KVDataSet *	dataset,
		const Char_t *	datatype,
		const Char_t *	filename
	)

virtual

This will create and open a new ROOT runfile for the dataset in the repository, ready to be written.

If the subdirectory for 'datatype' does not exist, it will be created.

In fact, if the repository is not one in which files can be created and written directly (i.e. if CanWrite() = kFALSE), the file will be created in the local working directory, and only copied into the repository when CommitFile is called.

Definition at line 677 of file KVDataRepository.cpp.

◆ DeleteFile()

void KVDataRepository::DeleteFile	(	const KVDataSet *	dataset,
		const Char_t *	datatype,
		const Char_t *	filename,
		Bool_t	confirm = `kTRUE`
	)

virtual

Delete repository file [datasetdir]/[datatype]/[filename]

By default (confirm=kTRUE) we ask for confirmation before deleting. Set confirm=kFALSE to delete without confirmation (DANGER!!!)

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 790 of file KVDataRepository.cpp.

◆ FindHelper()

TSystem * KVDataRepository::FindHelper	(	const char *	path,
		void *	dirptr = `0`
	)

Create helper TSystem to handle file and directory operations that might be special for remote file access, like via rfiod or rootd.

Definition at line 852 of file KVDataRepository.cpp.

◆ GetDataSetManager()

KVDataSetManager * KVDataRepository::GetDataSetManager ( ) const

virtual

Return pointer to data set manager for this repository.

Definition at line 827 of file KVDataRepository.cpp.

◆ GetDirectoryListing()

KVUniqueNameList * KVDataRepository::GetDirectoryListing	(	const KVDataSet *	dataset,
		const Char_t *	datatype = `""`,
		const Char_t *	subdir = `""`
	)

virtual

Use the access protocol defined by DataRepository.AccessProtocol (=local by default) in order to open the directory

/root_of_data_repository/[datasetdir]/[datatype]/[subdir] /root_of_data_repository/[datasetdir]/[datatype] (if subdir="", default value) /root_of_data_repository/[datasetdir] (if datatype="", default value)

and fill a TList with one KVBase object for each entry in the directory, excluding "." and ".." User must delete the KVUniqueNameList after use (list will delete its members)

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 607 of file KVDataRepository.cpp.

◆ GetFileInfo()

Bool_t KVDataRepository::GetFileInfo	(	const KVDataSet *	dataset,
		const Char_t *	datatype,
		const Char_t *	runfile,
		FileStat_t &	fs
	)

virtual

Checks if the run file of given type is physically present in dataset subdirectory, i.e. (schematically), if

/root_of_data_repository/[datasetdir]/[datatypedir]/[runfile]

exists. If it does, the returned value is kTRUE (=1), in which case the FileStat_t object contains information about the file.

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 295 of file KVDataRepository.cpp.

◆ GetFileTransferExec()

virtual const Char_t * KVDataRepository::GetFileTransferExec ( ) const

inlinevirtual

returns full path to executable used for remote file transfer

Definition at line 214 of file KVDataRepository.h.

◆ GetFileTransferServer()

virtual const Char_t * KVDataRepository::GetFileTransferServer ( ) const

inlinevirtual

returns server url used for remote file transfer

Definition at line 219 of file KVDataRepository.h.

◆ GetFileTransferType()

virtual const Char_t * KVDataRepository::GetFileTransferType ( ) const

inlinevirtual

returns protocol used for remote file transfer

Definition at line 209 of file KVDataRepository.h.

◆ GetFileTransferUser()

virtual const Char_t * KVDataRepository::GetFileTransferUser ( ) const

inlinevirtual

returns user name used for remote file transfer

Definition at line 224 of file KVDataRepository.h.

◆ GetFullPathToOpenFile()

const Char_t * KVDataRepository::GetFullPathToOpenFile	(	const KVDataSet *	dataset,
		const Char_t *	datatype,
		const Char_t *	runfile
	)

virtual

Returns the full path (including protocol specification) needed by TFile::Open or TChain::Add to open a runfile belonging to the given dataset. The protocol depends on the value of fReadprotocol. If a special protocol has been defined for the given dataset and/or datatype, as in this example for "raw" data:

ccali.DataRepository.ReadProtocol.raw: rfio

it will be used instead of the default (fReadprotocol). If a given data type is not accessible for a repository, use "none":

ccali.DataRepository.ReadProtocol.raw: none

If this is the case, an error message will be printed and an empty string returned.

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 365 of file KVDataRepository.cpp.

◆ GetFullPathToTransferFile()

const Char_t * KVDataRepository::GetFullPathToTransferFile	(	const KVDataSet *	dataset,
		const Char_t *	datatype,
		const Char_t *	runfile
	)

virtual

Used by KVDataTransfer. Returns the full path needed to transfer a runfile belonging to the given dataset either from or to the repository, using sftp or bbftp etc. This is just a concatenation of the repository root directory with the dataset subdirectories and filename.

Reimplemented in KVRemoteDataRepository.

Definition at line 421 of file KVDataRepository.cpp.

◆ GetReadProtocol()

const Char_t * KVDataRepository::GetReadProtocol	(	const Char_t *	dataset,
		const Char_t *	datatype
	)

privatevirtual

Returns string containing protocol for reading files of the given datatype belonging to the dataset whose name is given. Protocol = "local", "root", "rfio" , "none" (if "none", the data can not be read) If a specific protocol for the dataset and/or datatype is not defined in $KVROOT/KVFiles/.kvrootrc, the default protocol (=fReadprotocol) is returned. Format for defining specific protocols: name_of_repository.DataRepository.ReadProtocol.dataset_name.data_type: protocol name_of_repository.DataRepository.ReadProtocol.dataset_name: protocol name_of_repository.DataRepository.ReadProtocol.data_type: protocol

Definition at line 458 of file KVDataRepository.cpp.

◆ GetRootDirectory()

virtual const Char_t * KVDataRepository::GetRootDirectory ( ) const

inlinevirtual

returns root directory of data repository (fLocalrootdir)

Definition at line 229 of file KVDataRepository.h.

◆ HelperIsConsistentWith()

Bool_t KVDataRepository::HelperIsConsistentWith	(	TSystem *	helper,
		const char *	path,
		void *	dirptr = `0`
	)

Copy of (protected) method TSystem::ConsistentWith Check consistency of this helper with the one required by 'path' or 'dirptr'

Definition at line 1035 of file KVDataRepository.cpp.

◆ Init()

Bool_t KVDataRepository::Init ( )

virtual

Initialises data repository based on information in .kvrootrc file. Each line beginning "name.DataRepository" is used for this data repository, where "name" is the name of this repository.

Then we create and initialise the data set manager for this repository.

returns kTRUE if all goes well. returns kFALSE if not (i.e. data set manager cannot be initialised, no available datasets)

Definition at line 69 of file KVDataRepository.cpp.

◆ IsConnected()

virtual Bool_t KVDataRepository::IsConnected ( )

inlinevirtual

Always returns kTRUE for local repositories.

Reimplemented in KVRemoteDataRepository.

Definition at line 170 of file KVDataRepository.h.

◆ IsRemote()

virtual Bool_t KVDataRepository::IsRemote ( ) const

inlinevirtual

Returns kTRUE for remote repositories, kFALSE for local repositories.

Reimplemented in KVRemoteDataRepository.

Definition at line 165 of file KVDataRepository.h.

◆ MakeSubdirectory()

void KVDataRepository::MakeSubdirectory	(	const KVDataSet *	dataset,
		const Char_t *	datatype = `""`
	)

virtual

Create a new subdirectory in the repository: /root_of_data_repository/[datasetdir] or with 'datatype' given: /root_of_data_repository/[datasetdir]/[datatypedir] Set access permissions to 775 (u:rwx, g:rwx, o:rx)

Reimplemented in KVDMSDataRepository, and KVRemoteDataRepository.

Definition at line 569 of file KVDataRepository.cpp.

◆ NewAvailableRunsFile()

KVAvailableRunsFile * KVDataRepository::NewAvailableRunsFile	(	const Char_t *	data_type,
		const KVDataSet *	ds
	)

virtual

Create new instance of class derived from KVAvailableRunsFile. Actual class of object depends on the type of the repository, which is used to select one of the Plugin.KVDataAvailableRunsFile's defined in .kvrootrc. check and load plugin library

Definition at line 1157 of file KVDataRepository.cpp.

◆ NewDataSetManager()

KVDataSetManager * KVDataRepository::NewDataSetManager ( )

privatevirtual

Create and return pointer to new data set manager.

Reimplemented in KVRemoteDataRepository.

Definition at line 839 of file KVDataRepository.cpp.

◆ NewRepository()

KVDataRepository * KVDataRepository::NewRepository ( const Char_t * type )

static

Create new instance of class derived from KVDataRepository. Actual class of object depends on 'type' which is used to select one of the Plugin.KVDataRepository's defined in .kvrootrc. check and load plugin library

Definition at line 1132 of file KVDataRepository.cpp.

◆ OpenDataSetFile()

TObject * KVDataRepository::OpenDataSetFile	(	const KVDataSet *	ds,
		const Char_t *	type,
		const TString &	fname,
		Option_t *	opt = `""`
	)

private

Open a file using plugin defined in $KVROOT/KVFiles/.kvrootrc fname is the full path required to open the file The default base classes for each type are defined as in this example:

Default base classes for reading runfiles of different types

DataSet.RunFileClass.raw: KVRawDataReader DataSet.RunFileClass.recon: TFile DataSet.RunFileClass.ident: TFile DataSet.RunFileClass.root: TFile

A different base class can be defined for a specific dataset/datatype by adding a line to your $HOME/.kvrootrc like this:

name_of_dataset.DataSet.RunFileClass.data_type: BaseClassName

The actual class to be used is then defined by plugins in $KVROOT/KVFiles/.kvrootrc, for example

name_of_dataset.DataSet.RunFileClass.data_type: plugin_name

where 'plugin_name' is one of the known plugins for the default base class for the given data_type, i.e. if data_type="raw" we could use one of either "GANIL" or "MFM" as plugin_name:

Plugin.KVRawDataReader: GANIL KVGANILDataReader KVMultiDetexp_events "KVGANILDataReader()" +Plugin.KVRawDataReader: MFM KVMFMDataFileReader KVMultiDetdaq_cec "KVMFMDataFileReader()"

If no plugin is found for the base class defined by DataSet.RunFileClass, the base class is used.

To actually open the file, each base class & plugin must define a method static BaseClass* Open(const Char_t* path, Option_t* opt="", ...) which takes the full path to the file as argument (any other arguments taking default options) and returns a pointer of the BaseClass type to the created object which can be used to read the file. check connection to repository (e.g. SSH tunnel) in case of remote repository

Definition at line 1276 of file KVDataRepository.cpp.

◆ OpenDataSetRunFile()

TObject * KVDataRepository::OpenDataSetRunFile	(	const KVDataSet *	ds,
		const Char_t *	type,
		Int_t	run,
		Option_t *	opt = `""`
	)

virtual

Open a file using plugin defined in $KVROOT/KVFiles/.kvrootrc The default base classes for each type are defined as in this example:

Default base classes for reading runfiles of different types

DataSet.RunFileClass.raw: KVRawDataReader DataSet.RunFileClass.recon: TFile DataSet.RunFileClass.ident: TFile DataSet.RunFileClass.root: TFile

A different base class can be defined for a specific dataset/datatype by adding a line to your $HOME/.kvrootrc like this:

name_of_dataset.DataSet.RunFileClass.data_type: BaseClassName

The actual class to be used is then defined by plugins in $KVROOT/KVFiles/.kvrootrc, for example

Plugin.KVRawDataReader: raw.INDRA* KVINDRARawDataReader KVIndra "KVINDRARawDataReader()"

which defines the plugin for raw data for all datasets whose name begins with "INDRA" If no plugin is found for the base class defined by DataSet.RunFileClass, the base class is used.

To actually open the file, each base class & plugin must define a method static BaseClass* Open(const Char_t* path, Option_t* opt="", ...) which takes the full path to the file as argument (any other arguments taking default options) and returns a pointer of the BaseClass type to the created object which can be used to read the file.

Reimplemented in KVDMSDataRepository.

Definition at line 1204 of file KVDataRepository.cpp.

◆ PrepareXRDTunnel()

void KVDataRepository::PrepareXRDTunnel ( )

privatevirtual

PRIVATE method, called by constructor. Called if an SSH tunnel is to be used to connect to an xrootd server, e.g. if the repository configuration includes the following lines (or similar):

#ccali.DataRepository.XRDTunnel.host: ccali.in2p3.fr #ccali.DataRepository.XRDTunnel.port: 10000 #ccali.DataRepository.XRDTunnel.retry: 5 #ccali.DataRepository.XRDTunnel.user:

In this case we need to replace the real name & port of the xrootd server:

ccali.DataRepository.XRDServer: ccxrdsn012:1999

with the name & port of the SSH tunnel

ccali.DataRepository.XRDServer: localhost:10000

which will be used for path names in e.g. GetFullPathToOpenFile. The tunnel is not created here; it should be opened when needed by calling IsConnected()

Definition at line 1081 of file KVDataRepository.cpp.

◆ Print()

void KVDataRepository::Print ( Option_t * opt = "" ) const

virtual

Print info on repository.

Reimplemented from KVBase.

Definition at line 166 of file KVDataRepository.cpp.

◆ PrintAvailableDatasetsUpdateWarning()

virtual void KVDataRepository::PrintAvailableDatasetsUpdateWarning ( ) const

inlinevirtual

Reimplemented in IRODSDataRepository.

Definition at line 242 of file KVDataRepository.h.

◆ SetFullPath()

void KVDataRepository::SetFullPath	(	TString &	path,
		const Char_t *	protocol
	)

private

Will replace contents of 'path' with the full base path needed for the given protocol.

protocol = "local" : path = fLocalrootdir protocol = "root" : path = "root://" + fXrootdserver + "/" + fXrootdrootdir protocol = "rfio" : path = "rfio:" + fRfioserver + ":" + fRfiorootdir

Definition at line 262 of file KVDataRepository.cpp.

Member Data Documentation

◆ fAccessprotocol

TString KVDataRepository::fAccessprotocol

private

Definition at line 104 of file KVDataRepository.h.

◆ fAccessroot

TString KVDataRepository::fAccessroot

private

Definition at line 116 of file KVDataRepository.h.

◆ fCanWrite

Bool_t KVDataRepository::fCanWrite

private

kTRUE if new files can be created and written directly in the repository; kFALSE if they have to be written locally then copied to repository

Definition at line 122 of file KVDataRepository.h.

◆ fCommitDataSet

const KVDataSet* KVDataRepository::fCommitDataSet

private

used by CreateNewFile and CommitFile

Definition at line 124 of file KVDataRepository.h.

◆ fCommitDataType

TString KVDataRepository::fCommitDataType

private

Definition at line 125 of file KVDataRepository.h.

◆ fCommitFileName

TString KVDataRepository::fCommitFileName

private

Definition at line 126 of file KVDataRepository.h.

◆ fDSM

KVDataSetManager* KVDataRepository::fDSM

private

handles datasets in repository

Definition at line 127 of file KVDataRepository.h.

◆ fHelpers

TSeqCollection* KVDataRepository::fHelpers

private

List of helper classes for alternative file/directory access.

Definition at line 135 of file KVDataRepository.h.

◆ fLocalrootdir

TString KVDataRepository::fLocalrootdir

private

Definition at line 103 of file KVDataRepository.h.

◆ fReadprotocol

TString KVDataRepository::fReadprotocol

private

Definition at line 105 of file KVDataRepository.h.

◆ fReadroot

TString KVDataRepository::fReadroot

private

Definition at line 117 of file KVDataRepository.h.

◆ fRfiorootdir

TString KVDataRepository::fRfiorootdir

private

Definition at line 115 of file KVDataRepository.h.

◆ fRfioserver

TString KVDataRepository::fRfioserver

private

Definition at line 114 of file KVDataRepository.h.

◆ fTransferExec

TString KVDataRepository::fTransferExec

private

Definition at line 119 of file KVDataRepository.h.

◆ fTransferserver

TString KVDataRepository::fTransferserver

private

Definition at line 120 of file KVDataRepository.h.

◆ fTransfertype

TString KVDataRepository::fTransfertype

private

Definition at line 118 of file KVDataRepository.h.

◆ fTransferuser

TString KVDataRepository::fTransferuser

private

Definition at line 121 of file KVDataRepository.h.

◆ fXRDtunHost

TString KVDataRepository::fXRDtunHost

private

Definition at line 109 of file KVDataRepository.h.

◆ fXRDtunnel

Bool_t KVDataRepository::fXRDtunnel

private

Definition at line 108 of file KVDataRepository.h.

◆ fXRDtunPort

Int_t KVDataRepository::fXRDtunPort

private

Definition at line 113 of file KVDataRepository.h.

◆ fXRDtunRetry

Int_t KVDataRepository::fXRDtunRetry

private

Definition at line 112 of file KVDataRepository.h.

◆ fXRDtunSpec

TString KVDataRepository::fXRDtunSpec

private

Definition at line 111 of file KVDataRepository.h.

◆ fXRDtunUser

TString KVDataRepository::fXRDtunUser

private

Definition at line 110 of file KVDataRepository.h.

◆ fXrootdrootdir

TString KVDataRepository::fXrootdrootdir

private

Definition at line 107 of file KVDataRepository.h.

◆ fXrootdserver

TString KVDataRepository::fXrootdserver

private

Definition at line 106 of file KVDataRepository.h.

Detailed Description

Default data repository

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ KVDataRepository()

◆ ~KVDataRepository()

Member Function Documentation

◆ CanWrite()

◆ cd()

◆ CheckFileStatus()

◆ CheckSubdirExists()

◆ Chmod()

◆ CommitFile()

◆ CopyFile()

◆ CopyFileFromRepository()

◆ CopyFileToRepository()

◆ CreateAllNeededSubdirectories()

◆ CreateNewFile()

◆ DeleteFile()

◆ FindHelper()

◆ GetDataSetManager()

◆ GetDirectoryListing()

◆ GetFileInfo()

◆ GetFileTransferExec()

◆ GetFileTransferServer()

◆ GetFileTransferType()

◆ GetFileTransferUser()

◆ GetFullPathToOpenFile()

◆ GetFullPathToTransferFile()

◆ GetReadProtocol()

◆ GetRootDirectory()

◆ HelperIsConsistentWith()

◆ Init()

◆ IsConnected()

◆ IsRemote()

◆ MakeSubdirectory()

◆ NewAvailableRunsFile()

◆ NewDataSetManager()

◆ NewRepository()

◆ OpenDataSetFile()

Default base classes for reading runfiles of different types

◆ OpenDataSetRunFile()

Default base classes for reading runfiles of different types

◆ PrepareXRDTunnel()

◆ Print()

◆ PrintAvailableDatasetsUpdateWarning()

◆ SetFullPath()

Member Data Documentation

◆ fAccessprotocol

◆ fAccessroot

◆ fCanWrite

◆ fCommitDataSet

◆ fCommitDataType

◆ fCommitFileName

◆ fDSM

◆ fHelpers

◆ fLocalrootdir

◆ fReadprotocol

◆ fReadroot

◆ fRfiorootdir

◆ fRfioserver

◆ fTransferExec

◆ fTransferserver

◆ fTransfertype

◆ fTransferuser

◆ fXRDtunHost

◆ fXRDtunnel

◆ fXRDtunPort

◆ fXRDtunRetry

◆ fXRDtunSpec

◆ fXRDtunUser

◆ fXrootdrootdir

◆ fXrootdserver