Detailed Description

A repository for experimental datasets.

This kind of data repository contains data described by KVDataSet objects in a KVDataSetManager.

Definition at line 15 of file KVDataSetRepository.h.

#include <KVDataSetRepository.h>

Inheritance diagram for KVDataSetRepository:

[legend]

Public Member Functions
	KVDataSetRepository ()=default
	handles datasets in repository More...

	~KVDataSetRepository ()
	Just to allow std::unique_ptr<KVDataSetManager> member with incomplete (forward) declaration of KVDataSetManager. More...

void	cd () override

Bool_t	CheckFileStatus (const KVDataSet *dataset, const TString &datatype, const TString &runfile)

void	CommitFile (TFile file, const TString &datatype, const KVDataSet dataset)

void	CopyFileFromRepository (const KVDataSet *dataset, const TString &datatype, const TString &filename, const TString &destination)

int	CopyFileToRepository (const TString &source, const KVDataSet *dataset, const TString &datatype, const TString &filename)

void	CreateAllNeededSubdirectories (const KVDataSet DataSet, const Char_t DataType)

TFile *	CreateNewFile (const KVDataSet *dataset, const TString &datatype, const TString &filename)

void	DeleteFile (const KVDataSet *ds, const TString &datatype, const TString &runfile, Bool_t confirm)

KVDataSetManager *	GetDataSetManager () const
	Return pointer to data set manager for this repository. More...

KVUniqueNameList	GetDirectoryListing (const KVDataSet *dataset, const TString &datatype="", const TString &subdir="")

Bool_t	GetFileInfo (const KVDataSet *dataset, const TString &datatype, const TString &runfile, FileStat_t &fs) const

TString	GetFullPathToTransferFile (const KVDataSet *dataset, const TString &datatype, const TString &runfile)

Bool_t	Init () override

void	MakeSubdirectory (const KVDataSet *dataset, const TString &datatype="")

virtual KVAvailableRunsFile *	NewAvailableRunsFile (const Char_t , const KVDataSet )

void	PrintAvailableDatasetsUpdateWarning () const

Public Member Functions inherited from KVDataRepository
	KVDataRepository ()
	Default constructor. More...

	KVDataRepository (const TString &name, const TString &root_dir, const TString &access_backend="local")

Bool_t	CanWrite () const

template<typename... Paths>
Bool_t	CheckFileStatus (const TString &filename, const Paths &... paths) const

template<typename... Paths>
Bool_t	CheckSubdirExists (const TString &path1, const Paths &... paths) const

template<typename... Paths>
void	CommitFile (TFile *file, const Paths &... paths)

template<typename... Paths>
void	CopyFileFromRepository (const TString &filename, const TString &destination, const Paths &... paths) const

template<typename... Paths>
int	CopyFileToRepository (const TString &filename, const TString &source, const Paths &... paths) const

template<typename... Paths>
TFile *	CreateNewFile (const TString &filename, const Paths &... paths) const

template<typename... Paths>
void	DeleteFile (const TString &filename, Bool_t confirm, const Paths &... paths) const

template<typename... Paths>
KVUniqueNameList	GetDirectoryListing (const Paths &... paths) const

template<typename... Paths>
std::optional< FileStat_t >	GetFileInfo (const TString &filename, const Paths &... paths) const

TString	GetFileTransferExec () const
	returns full path to executable used for remote file transfer More...

TString	GetFileTransferServer () const
	returns server url used for remote file transfer More...

TString	GetFileTransferType () const
	returns protocol used for remote file transfer More...

TString	GetFileTransferUser () const
	returns user name used for remote file transfer More...

template<typename... Paths>
TString	GetFullPathToOpenFile (const TString &filename, const Paths &... paths) const

template<typename... Paths>
TString	GetFullPathToTransferFile (const TString &filename, const Paths &... paths) const

template<typename... Paths>
TString	GetRepositoryPath (const Paths &... paths) const

TString	GetRootDirectory () const
	returns root directory of data repository (fLocalrootdir) More...

const Char_t *	GetType () const override

template<typename... Paths>
void	MakeSubdirectory (const TString &path1, const Paths &... paths) const

template<typename... Paths>
TObject *	OpenFile (const TString &filename, const std::function< TObject *(const TString &)> &open_file, const Paths &... paths)

template<typename... Paths>
TFile *	OpenROOTFile (const TString &filename, const Paths &... paths)

void	Print (Option_t *opt="") const override
	Print info on repository. More...

void	ReadWithXrootd (const TString &server, const TString &rootdir)

void	SetCanWrite (Bool_t yes=kTRUE)

Public Member Functions inherited from KVBase
	KVBase ()
	Default constructor. More...

	KVBase (const Char_t name, const Char_t title="")
	Ctor for object with given name and type. More...

	KVBase (const KVBase &)
	copy ctor More...

void	Clear (Option_t *opt="") override
	Clear object properties : name, type/title, number, label. More...

void	Copy (TObject &) const override
	Make a copy of this object. More...

void	Error (const char method, const char msgfmt,...) const override
	colourised errors (red) ! More...

const Char_t *	GetLabel () const

UInt_t	GetNumber () const

virtual TObject *	GetObject () const

Bool_t	HasLabel () const

virtual Bool_t	IsCalled (const Char_t *name) const

Bool_t	IsLabelled (const Char_t *l) const

virtual Bool_t	IsType (const Char_t *typ) const

virtual void	List ()

KVBase &	operator= (const KVBase &)
	copy assignment operator More...

void	SetLabel (const Char_t *lab)

virtual void	SetNumber (UInt_t num)

virtual void	SetType (const Char_t *str)

void	Warning (const char method, const char msgfmt,...) const override
	colourised warnings (orange) ! More...

Public Member Functions inherited from TNamed
	TNamed ()

	TNamed (const char name, const char title)

	TNamed (const TNamed &named)

	TNamed (const TString &name, const TString &title)

virtual	~TNamed ()

TObject *	Clone (const char *newname="") const override

Int_t	Compare (const TObject *obj) const override

virtual void	FillBuffer (char *&buffer)

const char *	GetName () const override

const char *	GetTitle () const override

ULong_t	Hash () const override

TClass *	IsA () const override

Bool_t	IsSortable () const override

void	ls (Option_t *option="") const override

TNamed &	operator= (const TNamed &rhs)

virtual void	SetName (const char *name)

virtual void	SetNameTitle (const char name, const char title)

virtual void	SetTitle (const char *title="")

virtual Int_t	Sizeof () const

void	Streamer (TBuffer &) override

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

Public Member Functions inherited from TObject
	TObject ()

	TObject (const TObject &object)

virtual	~TObject ()

void	AbstractMethod (const char *method) const

virtual void	AppendPad (Option_t *option="")

virtual void	Browse (TBrowser *b)

ULong_t	CheckedHash ()

virtual const char *	ClassName () const

virtual void	Delete (Option_t *option="")

virtual Int_t	DistancetoPrimitive (Int_t px, Int_t py)

virtual void	Draw (Option_t *option="")

virtual void	DrawClass () const

virtual TObject *	DrawClone (Option_t *option="") const

virtual void	Dump () const

virtual void	Execute (const char method, const char params, Int_t *error=nullptr)

virtual void	Execute (TMethod method, TObjArray params, Int_t *error=nullptr)

virtual void	ExecuteEvent (Int_t event, Int_t px, Int_t py)

virtual void	Fatal (const char method, const char msgfmt,...) const

virtual TObject *	FindObject (const char *name) const

virtual TObject *	FindObject (const TObject *obj) const

virtual Option_t *	GetDrawOption () const

virtual const char *	GetIconName () const

virtual char *	GetObjectInfo (Int_t px, Int_t py) const

virtual Option_t *	GetOption () const

virtual UInt_t	GetUniqueID () const

virtual Bool_t	HandleTimer (TTimer *timer)

Bool_t	HasInconsistentHash () const

virtual void	Info (const char method, const char msgfmt,...) const

virtual Bool_t	InheritsFrom (const char *classname) const

virtual Bool_t	InheritsFrom (const TClass *cl) const

virtual void	Inspect () const

void	InvertBit (UInt_t f)

Bool_t	IsDestructed () const

virtual Bool_t	IsEqual (const TObject *obj) const

virtual Bool_t	IsFolder () const

R__ALWAYS_INLINE Bool_t	IsOnHeap () const

R__ALWAYS_INLINE Bool_t	IsZombie () const

void	MayNotUse (const char *method) const

virtual Bool_t	Notify ()

void	Obsolete (const char method, const char asOfVers, const char *removedFromVers) const

void	operator delete (void *ptr)

void	operator delete (void ptr, void vp)

void	operator delete[] (void *ptr)

void	operator delete[] (void ptr, void vp)

void *	operator new (size_t sz)

void *	operator new (size_t sz, void *vp)

void *	operator new[] (size_t sz)

void *	operator new[] (size_t sz, void *vp)

TObject &	operator= (const TObject &rhs)

virtual void	Paint (Option_t *option="")

virtual void	Pop ()

virtual Int_t	Read (const char *name)

virtual void	RecursiveRemove (TObject *obj)

void	ResetBit (UInt_t f)

virtual void	SaveAs (const char filename="", Option_t option="") const

virtual void	SavePrimitive (std::ostream &out, Option_t *option="")

void	SetBit (UInt_t f)

void	SetBit (UInt_t f, Bool_t set)

virtual void	SetDrawOption (Option_t *option="")

virtual void	SetUniqueID (UInt_t uid)

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

virtual void	SysError (const char method, const char msgfmt,...) const

R__ALWAYS_INLINE Bool_t	TestBit (UInt_t f) const

Int_t	TestBits (UInt_t f) const

virtual void	UseCurrentStyle ()

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0)

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0) const

Static Public Member Functions
static KVDataSetRepository *	NewRepository (const TString &local_or_irods)

Static Public Member Functions inherited from KVBase
static std::optional< TString >	AbsoluteUnixPath (const TString &)

static Bool_t	AreEqual (Double_t x, Double_t y, Long64_t maxdif=1)
	Comparison between two 64-bit floating-point values. More...

static void	BackupFileWithDate (const Char_t *path)

static void	CombineFiles (const Char_t file1, const Char_t file2, const Char_t *newfilename, Bool_t keep=kTRUE)

static void	Deprecated (const char method, const char advice)

static Bool_t	FindClassSourceFiles (const KVString &class_name, KVString &imp_file, KVString &dec_file, const KVString &dir_name=".")

static Bool_t	FindExecutable (TString &exec, const Char_t *path="$(PATH)")

static const Char_t *	FindFile (const Char_t *search, TString &wfil)

static const Char_t *	GetBINDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetDATABASEFilePath ()

static const Char_t *	GetDATADIRFilePath (const Char_t *namefile="")

static KVString	GetDataSetEnv (const KVString &dataset, const KVString &type, const char *defval)

template<typename ValType >
static ValType	GetDataSetEnv (const KVString &dataset, const KVString &type, const ValType &defval)

static const Char_t *	GetETCDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetExampleFilePath (const Char_t library, const Char_t namefile)
	Return full path to example file for given library (="KVMultiDet", "BackTrack", etc.) More...

static const Char_t *	GetINCDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetKVBuildDate ()
	Returns KaliVeda build date. More...

static const Char_t *	GetKVBuildDir ()
	Returns top-level directory used for build. More...

static const Char_t *	GetKVBuildTime ()
	Returns KaliVeda build time. More...

static const Char_t *	GetKVBuildType ()
	Returns KaliVeda build type (cmake build: Release, Debug, RelWithDebInfo, ...) More...

static const Char_t *	GetKVBuildUser ()
	Returns username of person who performed build. More...

static int	GetKVMajorVersion ()

static int	GetKVMinorVersion ()

static int	GetKVPatchVersion ()

static const Char_t *	GetKVSourceDir ()
	Returns top-level directory of source tree used for build. More...

static const Char_t *	GetKVVersion ()
	Returns KaliVeda version string. More...

static const Char_t *	GetLIBDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetListOfPlugins (const Char_t *base)

static const Char_t *	GetListOfPluginURIs (const Char_t *base)

static const Char_t *	GetPluginURI (const Char_t base, const Char_t plugin)

static void	GetTempFileName (TString &base)

static const Char_t *	GetTEMPLATEDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetWORKDIRFilePath (const Char_t *namefile="")

static const Char_t *	gitBranch ()
	Returns git branch of sources. More...

static const Char_t *	gitCommit ()
	Returns last git commit of sources. More...

static void	InitEnvironment ()

static bool	is_gnuinstall ()

static Bool_t	IsEnvInit ()

static Bool_t	IsThisAPlugin (const TString &uri, TString &base)

static TPluginHandler *	LoadPlugin (const Char_t base, const Char_t uri="0")

static Bool_t	OpenContextMenu (const char method, TObject obj, const char *alt_method_name="")

static void	OpenTempFile (TString &base, std::ofstream &fp)

static void	PrintSplashScreen ()
	Prints welcome message and infos on version etc. More...

static Double_t	ProtectedGetX (const TF1 &func, Double_t val, int &status, Double_t xmin=0.0, Double_t xmax=0.0)

static Double_t	ProtectedGetX (const TF1 *func, Double_t val, int &status, Double_t xmin=0.0, Double_t xmax=0.0)

static Bool_t	SearchAndOpenKVFile (const Char_t name, KVSQLite::database &dbfile, const Char_t kvsubdir="")

static Bool_t	SearchAndOpenKVFile (const Char_t name, std::ifstream &file, const Char_t kvsubdir="", KVLockfile *locks=0)

static Bool_t	SearchAndOpenKVFile (const Char_t name, std::ofstream &file, const Char_t kvsubdir="", KVLockfile *locks=0)

static Bool_t	SearchKVFile (const Char_t name, TString &fullpath, const Char_t kvsubdir="")

static const Char_t *	WorkingDirectory ()

Static Public Member Functions inherited from TNamed
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

Static Public Member Functions inherited from TObject
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

static Longptr_t	GetDtorOnly ()

static Bool_t	GetObjectStat ()

static void	SetDtorOnly (void *obj)

static void	SetObjectStat (Bool_t stat)

Private Attributes
std::unique_ptr< KVDataSetManager >	fDSM

Additional Inherited Members
Public Types inherited from KVBase
enum	EKaliVedaBits { kIsKaliVedaObject = BIT(23) }

Public Types inherited from TObject
enum	EDeprecatedStatusBits

enum	EStatusBits

Public Attributes inherited from TObject
	kBitMask

	kCanDelete

	kCannotPick

	kHasUUID

	kInconsistent

	kInvalidObject

	kIsOnHeap

	kIsReferenced

	kMustCleanup

	kNoContextMenu

	kNotDeleted

	kObjInCanvas

	kOverwrite

	kSingleKey

	kWriteDelete

	kZombie

Protected Member Functions inherited from TObject
virtual void	DoError (int level, const char location, const char fmt, va_list va) const

void	MakeZombie ()

Protected Attributes inherited from TNamed
TString	fName

TString	fTitle

Protected Attributes inherited from TObject
	kOnlyPrepStep

Constructor & Destructor Documentation

◆ KVDataSetRepository()

KVDataSetRepository::KVDataSetRepository ( )

default

handles datasets in repository

◆ ~KVDataSetRepository()

KVDataSetRepository::~KVDataSetRepository ( )

Just to allow std::unique_ptr<KVDataSetManager> member with incomplete (forward) declaration of KVDataSetManager.

Definition at line 267 of file KVDataSetRepository.cpp.

Member Function Documentation

◆ cd()

void KVDataSetRepository::cd ( )

overridevirtual

Reimplemented from KVDataRepository.

Definition at line 276 of file KVDataSetRepository.cpp.

◆ CheckFileStatus()

Bool_t KVDataSetRepository::CheckFileStatus	(	const KVDataSet *	dataset,
		const TString &	datatype,
		const TString &	runfile
	)

Checks if the run file of given type is physically present in dataset subdirectory, i.e. (schematically), if

/root_of_data_repository/[datasetdir]/[datatypedir]/[runfile]

exists. If it does, the returned value is kTRUE (=1).

Definition at line 41 of file KVDataSetRepository.cpp.

◆ CommitFile()

void KVDataSetRepository::CommitFile	(	TFile *	file,
		const TString &	datatype,
		const KVDataSet *	dataset
	)

Add this file (previously created by a call to CreateNewFile) to the repository. Any objects should be written to the file before calling this method, either by calling the Write() method of each object, or by calling file->Write(). No file->Write() is done here: we only close (delete) the TFile.

For repositories in which files can be created and written directly (i.e. if CanWrite() = kTRUE), we just have to close the file. For repositories where CanWrite() = kFALSE, we close the file, then copy it to the correct place in the repository, using the previously recorded values of fCommitDataSetDir, fCommitDataType, and fCommitFileName. Then we remove the local copy from disk.

NB: after calling this method, the TFile pointer 'file' must not be used!!!

Definition at line 175 of file KVDataSetRepository.cpp.

◆ CopyFileFromRepository()

void KVDataSetRepository::CopyFileFromRepository	(	const KVDataSet *	dataset,
		const TString &	datatype,
		const TString &	filename,
		const TString &	destination
	)

Copy file [datasetdir]/[datatypedir]/[filename] from the repository to [destination] We check if the file to copy exists.

Definition at line 109 of file KVDataSetRepository.cpp.

◆ CopyFileToRepository()

int KVDataSetRepository::CopyFileToRepository	(	const TString &	source,
		const KVDataSet *	dataset,
		const TString &	datatype,
		const TString &	filename
	)

Copy file [source] to [datasetdir]/[datatypedir]/[filename] in the repository The file access permissions are set to '664 (u:rw, g:rw, o:r) Returns status of file transfer command

Definition at line 124 of file KVDataSetRepository.cpp.

◆ CreateAllNeededSubdirectories()

void KVDataSetRepository::CreateAllNeededSubdirectories	(	const KVDataSet *	DataSet,
		const Char_t *	DataType
	)

Ensure that all required subdirectories exist and any missing ones are created in order to store a runfile of given DataType for the given DataSet

Definition at line 227 of file KVDataSetRepository.cpp.

◆ CreateNewFile()

TFile * KVDataSetRepository::CreateNewFile	(	const KVDataSet *	dataset,
		const TString &	datatype,
		const TString &	filename
	)

This will create and open a new ROOT runfile for the dataset in the repository, ready to be written.

If the subdirectory for 'datatype' does not exist, it will be created.

In fact, if the repository is not one in which files can be created and written directly (i.e. if CanWrite() = kFALSE), the file will be created in the local working directory, and only copied into the repository when CommitFile is called.

Definition at line 145 of file KVDataSetRepository.cpp.

◆ DeleteFile()

void KVDataSetRepository::DeleteFile	(	const KVDataSet *	ds,
		const TString &	datatype,
		const TString &	runfile,
		Bool_t	confirm
	)

Delete file from repository.

If confirm=kTRUE, ask for confirmation first (=kFALSE, no confirmation required)

Definition at line 242 of file KVDataSetRepository.cpp.

◆ GetDataSetManager()

KVDataSetManager * KVDataSetRepository::GetDataSetManager ( ) const

Return pointer to data set manager for this repository.

Definition at line 256 of file KVDataSetRepository.cpp.

◆ GetDirectoryListing()

KVUniqueNameList KVDataSetRepository::GetDirectoryListing	(	const KVDataSet *	dataset,
		const TString &	datatype = `""`,
		const TString &	subdir = `""`
	)

Use the access protocol defined by DataRepository.AccessProtocol (=local by default) in order to open the directory

/root_of_data_repository/[datasetdir]/[datatype]/[subdir] /root_of_data_repository/[datasetdir]/[datatype] (if subdir="", default value) /root_of_data_repository/[datasetdir] (if datatype="", default value)

and fill a KVUniqueNameList with one KVBase object for each entry in the directory, excluding "." and ".."

Definition at line 88 of file KVDataSetRepository.cpp.

◆ GetFileInfo()

Bool_t KVDataSetRepository::GetFileInfo	(	const KVDataSet *	dataset,
		const TString &	datatype,
		const TString &	runfile,
		FileStat_t &	fs
	)		const

Checks if the run file of given type is physically present in dataset subdirectory, i.e. (schematically), if

/root_of_data_repository/[datasetdir]/[datatypedir]/[runfile]

exists. If it does, the returned value is kTRUE (=1), in which case the FileStat_t object contains information about the file.

Definition at line 15 of file KVDataSetRepository.cpp.

◆ GetFullPathToTransferFile()

TString KVDataSetRepository::GetFullPathToTransferFile	(	const KVDataSet *	dataset,
		const TString &	datatype,
		const TString &	runfile
	)

Used by KVDataTransfer.

Returns the full path needed to transfer a runfile belonging to the given dataset either from or to the repository, using sftp or bbftp etc. This is just a concatenation of the repository root directory with the dataset subdirectories and filename.

Definition at line 63 of file KVDataSetRepository.cpp.

◆ Init()

Bool_t KVDataSetRepository::Init ( void )

overridevirtual

Reimplemented from KVDataRepository.

Definition at line 286 of file KVDataSetRepository.cpp.

◆ MakeSubdirectory()

void KVDataSetRepository::MakeSubdirectory	(	const KVDataSet *	dataset,
		const TString &	datatype = `""`
	)

Create a new subdirectory in the repository:

/root_of_data_repository/[datasetdir]

or with 'datatype' given:

/root_of_data_repository/[datasetdir]/[datatypedir]

Set access permissions to 775 (u:rwx, g:rwx, o:rx)

Definition at line 206 of file KVDataSetRepository.cpp.

◆ NewAvailableRunsFile()

KVAvailableRunsFile * KVDataSetRepository::NewAvailableRunsFile	(	const Char_t *	data_type,
		const KVDataSet *	ds
	)

virtual

Create new instance of class derived from KVAvailableRunsFile.

Actual class of object depends on the type of the repository back-end which is used to select one of the Plugin.KVDataAvailableRunsFile's defined in .kvrootrc. check and load plugin library

Definition at line 306 of file KVDataSetRepository.cpp.

◆ NewRepository()

KVDataSetRepository * KVDataSetRepository::NewRepository ( const TString & local_or_irods )

static

Create new instance of KVDataSetRepository

local_or_irods = "local" => KVDataRepositoryImpl or "irods" => IRODSDataRepositoryImpl

Definition at line 331 of file KVDataSetRepository.cpp.

◆ PrintAvailableDatasetsUpdateWarning()

void KVDataSetRepository::PrintAvailableDatasetsUpdateWarning ( ) const

inline

Definition at line 27 of file KVDataSetRepository.h.

Member Data Documentation

◆ fDSM

std::unique_ptr<KVDataSetManager> KVDataSetRepository::fDSM

private

Definition at line 16 of file KVDataSetRepository.h.