Detailed Description

Base class for managing repositories of data.

A data repository is a collection of files, possibly organised into different directories, which contain data of some kind. This collection may be located on a local or network-mounted disk, i.e. one which can be accessed with the usual Unix/Linux commands, or it may be on a remote server, accessible using the iRODS service, and possibly allowing to read the files using xrootd.

Setting up a data repository

Local repository

Call the constructor giving just a name for the repository and the full path to its root directory, e.g.:

KVDataRepository dr{"DataRepository1", "/home/user/data_repository"};

KVDataRepository

Base class for managing repositories of data.

Definition: KVDataRepository.h:126

iRODS/xrootd repository

In addition to the name and full path to its root directory, give the type "irods":

KVDataRepository dr_irods{"DataRepository2", "/irods_root/my_repository", "irods"};

The root directory is the same as the path you would use with ils in order to view the contents of the iRODS collection:

$ ils /irods_root
/irods_root:
  C- /irods_root/experiments
  C- /irods_root/calibrations
  C- /irods_root/other_stuff
etc.

Note that the root directory (collection) must already exist in the iRODS catalogue: KVDataRepository cannot create it.

Creating a new ROOT file in the repository

In order to open a new file for writing, call the method CreateNewFile():

auto f = dr.CreateNewFile("my_new_file.root", "path1", "path2");

f

#define f(i)

with the name of the new file and any necessary path elements (which will be added to the root directory).

Depending on the kind of repository ("local" or "irods") this may either directly create the file inside the repository with the given path, or create a file in the current local working directory on disk.

When all objects to be written in the file have been written, call the method CommitFile():

auto h = new TH1F("h", "histo", 100, -5, 5);
h->FillRandom("gaus");
h->Write();
dr.CommitFile(f, "path1", "path2");

with the TFile pointer returned by CreateNewFile() and giving all necessary path elements. This will close the file (the pointer is no longer valid afterwards) and, in the case of an "irods" repository, copy the file into the repository.

In both cases ("local" or "irods"), all necessary subdirectories which do not already exist will be created.

Browsing the contents of the repository

Call method GetDirectoryListing() to retrieve a KVUniqueNameList containing objects describing the contents of any given directory defined by its path:

auto ldir = dr.GetDirectoryListing();
ldir.ls();
OBJ: KVUniqueNameList   KVSeqCollection_5 Optimised list in which objects with the same name can only be placed once : 0
 OBJ: KVBase   path1  : 0 at: 0x5d95aac47940
 
ldir = dr.GetDirectoryListing("path1");
ldir.ls();
OBJ: KVUniqueNameList   KVSeqCollection_7 Optimised list in which objects with the same name can only be placed once : 0
 OBJ: KVBase   path2  : 0 at: 0x5d95acb85510
 
ldir = dr.GetDirectoryListing("path1", "path2");
ldir.ls();
OBJ: KVUniqueNameList   KVSeqCollection_9 Optimised list in which objects with the same name can only be placed once : 0
 OBJ: KVBase   my_new_file.root   : 0 at: 0x5d95acb85060

Opening files in the repository

For general files, see OpenFile(). For ROOT files, use method OpenROOTFile():

auto f = dr.OpenROOTFile("my_new_file.root", "path1", "path2");
f->ls();
TFile**     /home/user/data_repository/path1/path2/my_new_file.root
 TFile*     /home/user/data_repository/path1/path2/my_new_file.root
  KEY: TH1F h;1   histo

For iRODS-enabled data repositories, this will first download the file to the local temporary directory (gSystem->TempDirectory()) if not already done, and then open the local copy. Subsequent calls to open the same file will directly open the local copy.

Opening files using xrootd

If the repository allows to read files using an xrootd server, just call the method

dr.ReadWithXrootd("[url of server]", "[xrootd root directory]");

before calling OpenFile() or OpenROOTFile(). The root directory given for xrootd will be used instead of the root directory given to the constructor when opening files via the xrootd server.

Definition at line 126 of file KVDataRepository.h.

#include <KVDataRepository.h>

Inheritance diagram for KVDataRepository:

[legend]

Classes
struct	PathConcatenator

Public Member Functions
	KVDataRepository ()
	Default constructor. More...

	KVDataRepository (const TString &name, const TString &root_dir, const TString &access_backend="local")

Bool_t	CanWrite () const

virtual void	cd ()

template<typename... Paths>
Bool_t	CheckFileStatus (const TString &filename, const Paths &... paths) const

template<typename... Paths>
Bool_t	CheckSubdirExists (const TString &path1, const Paths &... paths) const

template<typename... Paths>
void	CommitFile (TFile *file, const Paths &... paths)

template<typename... Paths>
void	CopyFileFromRepository (const TString &filename, const TString &destination, const Paths &... paths) const

template<typename... Paths>
int	CopyFileToRepository (const TString &filename, const TString &source, const Paths &... paths) const

template<typename... Paths>
TFile *	CreateNewFile (const TString &filename, const Paths &... paths) const

template<typename... Paths>
void	DeleteFile (const TString &filename, Bool_t confirm, const Paths &... paths) const

template<typename... Paths>
KVUniqueNameList	GetDirectoryListing (const Paths &... paths) const

template<typename... Paths>
std::optional< FileStat_t >	GetFileInfo (const TString &filename, const Paths &... paths) const

TString	GetFileTransferExec () const
	returns full path to executable used for remote file transfer More...

TString	GetFileTransferServer () const
	returns server url used for remote file transfer More...

TString	GetFileTransferType () const
	returns protocol used for remote file transfer More...

TString	GetFileTransferUser () const
	returns user name used for remote file transfer More...

template<typename... Paths>
TString	GetFullPathToOpenFile (const TString &filename, const Paths &... paths) const

template<typename... Paths>
TString	GetFullPathToTransferFile (const TString &filename, const Paths &... paths) const

template<typename... Paths>
TString	GetRepositoryPath (const Paths &... paths) const

TString	GetRootDirectory () const
	returns root directory of data repository (fLocalrootdir) More...

const Char_t *	GetType () const override

virtual Bool_t	Init ()

template<typename... Paths>
void	MakeSubdirectory (const TString &path1, const Paths &... paths) const

template<typename... Paths>
TObject *	OpenFile (const TString &filename, const std::function< TObject *(const TString &)> &open_file, const Paths &... paths)

template<typename... Paths>
TFile *	OpenROOTFile (const TString &filename, const Paths &... paths)

void	Print (Option_t *opt="") const override
	Print info on repository. More...

void	ReadWithXrootd (const TString &server, const TString &rootdir)

void	SetCanWrite (Bool_t yes=kTRUE)

Public Member Functions inherited from KVBase
	KVBase ()
	Default constructor. More...

	KVBase (const Char_t name, const Char_t title="")
	Ctor for object with given name and type. More...

	KVBase (const KVBase &)
	copy ctor More...

void	Clear (Option_t *opt="") override
	Clear object properties : name, type/title, number, label. More...

void	Copy (TObject &) const override
	Make a copy of this object. More...

void	Error (const char method, const char msgfmt,...) const override
	colourised errors (red) ! More...

const Char_t *	GetLabel () const

UInt_t	GetNumber () const

virtual TObject *	GetObject () const

Bool_t	HasLabel () const

virtual Bool_t	IsCalled (const Char_t *name) const

Bool_t	IsLabelled (const Char_t *l) const

virtual Bool_t	IsType (const Char_t *typ) const

virtual void	List ()

KVBase &	operator= (const KVBase &)
	copy assignment operator More...

void	SetLabel (const Char_t *lab)

virtual void	SetNumber (UInt_t num)

virtual void	SetType (const Char_t *str)

void	Warning (const char method, const char msgfmt,...) const override
	colourised warnings (orange) ! More...

Public Member Functions inherited from TNamed
	TNamed ()

	TNamed (const char name, const char title)

	TNamed (const TNamed &named)

	TNamed (const TString &name, const TString &title)

virtual	~TNamed ()

TObject *	Clone (const char *newname="") const override

Int_t	Compare (const TObject *obj) const override

virtual void	FillBuffer (char *&buffer)

const char *	GetName () const override

const char *	GetTitle () const override

ULong_t	Hash () const override

TClass *	IsA () const override

Bool_t	IsSortable () const override

void	ls (Option_t *option="") const override

TNamed &	operator= (const TNamed &rhs)

virtual void	SetName (const char *name)

virtual void	SetNameTitle (const char name, const char title)

virtual void	SetTitle (const char *title="")

virtual Int_t	Sizeof () const

void	Streamer (TBuffer &) override

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

Public Member Functions inherited from TObject
	TObject ()

	TObject (const TObject &object)

virtual	~TObject ()

void	AbstractMethod (const char *method) const

virtual void	AppendPad (Option_t *option="")

virtual void	Browse (TBrowser *b)

ULong_t	CheckedHash ()

virtual const char *	ClassName () const

virtual void	Delete (Option_t *option="")

virtual Int_t	DistancetoPrimitive (Int_t px, Int_t py)

virtual void	Draw (Option_t *option="")

virtual void	DrawClass () const

virtual TObject *	DrawClone (Option_t *option="") const

virtual void	Dump () const

virtual void	Execute (const char method, const char params, Int_t *error=nullptr)

virtual void	Execute (TMethod method, TObjArray params, Int_t *error=nullptr)

virtual void	ExecuteEvent (Int_t event, Int_t px, Int_t py)

virtual void	Fatal (const char method, const char msgfmt,...) const

virtual TObject *	FindObject (const char *name) const

virtual TObject *	FindObject (const TObject *obj) const

virtual Option_t *	GetDrawOption () const

virtual const char *	GetIconName () const

virtual char *	GetObjectInfo (Int_t px, Int_t py) const

virtual Option_t *	GetOption () const

virtual UInt_t	GetUniqueID () const

virtual Bool_t	HandleTimer (TTimer *timer)

Bool_t	HasInconsistentHash () const

virtual void	Info (const char method, const char msgfmt,...) const

virtual Bool_t	InheritsFrom (const char *classname) const

virtual Bool_t	InheritsFrom (const TClass *cl) const

virtual void	Inspect () const

void	InvertBit (UInt_t f)

Bool_t	IsDestructed () const

virtual Bool_t	IsEqual (const TObject *obj) const

virtual Bool_t	IsFolder () const

R__ALWAYS_INLINE Bool_t	IsOnHeap () const

R__ALWAYS_INLINE Bool_t	IsZombie () const

void	MayNotUse (const char *method) const

virtual Bool_t	Notify ()

void	Obsolete (const char method, const char asOfVers, const char *removedFromVers) const

void	operator delete (void *ptr)

void	operator delete (void ptr, void vp)

void	operator delete[] (void *ptr)

void	operator delete[] (void ptr, void vp)

void *	operator new (size_t sz)

void *	operator new (size_t sz, void *vp)

void *	operator new[] (size_t sz)

void *	operator new[] (size_t sz, void *vp)

TObject &	operator= (const TObject &rhs)

virtual void	Paint (Option_t *option="")

virtual void	Pop ()

virtual Int_t	Read (const char *name)

virtual void	RecursiveRemove (TObject *obj)

void	ResetBit (UInt_t f)

virtual void	SaveAs (const char filename="", Option_t option="") const

virtual void	SavePrimitive (std::ostream &out, Option_t *option="")

void	SetBit (UInt_t f)

void	SetBit (UInt_t f, Bool_t set)

virtual void	SetDrawOption (Option_t *option="")

virtual void	SetUniqueID (UInt_t uid)

void	StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b)

virtual void	SysError (const char method, const char msgfmt,...) const

R__ALWAYS_INLINE Bool_t	TestBit (UInt_t f) const

Int_t	TestBits (UInt_t f) const

virtual void	UseCurrentStyle ()

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0)

virtual Int_t	Write (const char *name=nullptr, Int_t option=0, Int_t bufsize=0) const

Private Member Functions
void	CreateAllNeededSubdirectories () const

template<typename... Paths>
void	CreateAllNeededSubdirectories (const TString &path1, const Paths &... paths) const

template<typename... Paths>
TString	get_path_with_access_root (const Paths &... paths) const

template<typename... Paths>
TString	get_path_with_arbitrary_root (const TString &arb_root, const Paths &... paths) const

template<typename... Paths>
TString	get_path_with_read_root (const Paths &... paths) const

void	set_impl_backend (const TString &impl_type)

void	SetFullPath (TString &path, const TString &protocol)

Private Attributes
std::unique_ptr< KVDataRepositoryImpl >	_impl

TString	fAccessprotocol = "local"

TString	fAccessroot

Bool_t	fCanWrite = kTRUE

TString	fLocalrootdir

TString	fReadprotocol = "local"

TString	fReadroot

TString	fRfiorootdir

TString	fRfioserver

TString	fTransferExec

TString	fTransferserver

TString	fTransfertype

TString	fTransferuser

TString	fXrootdrootdir

TString	fXrootdserver

Additional Inherited Members
Public Types inherited from KVBase
enum	EKaliVedaBits { kIsKaliVedaObject = BIT(23) }

Public Types inherited from TObject
enum	EDeprecatedStatusBits

enum	EStatusBits

Static Public Member Functions inherited from KVBase
static std::optional< TString >	AbsoluteUnixPath (const TString &)

static Bool_t	AreEqual (Double_t x, Double_t y, Long64_t maxdif=1)
	Comparison between two 64-bit floating-point values. More...

static void	BackupFileWithDate (const Char_t *path)

static void	CombineFiles (const Char_t file1, const Char_t file2, const Char_t *newfilename, Bool_t keep=kTRUE)

static void	Deprecated (const char method, const char advice)

static Bool_t	FindClassSourceFiles (const KVString &class_name, KVString &imp_file, KVString &dec_file, const KVString &dir_name=".")

static Bool_t	FindExecutable (TString &exec, const Char_t *path="$(PATH)")

static const Char_t *	FindFile (const Char_t *search, TString &wfil)

static const Char_t *	GetBINDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetDATABASEFilePath ()

static const Char_t *	GetDATADIRFilePath (const Char_t *namefile="")

static KVString	GetDataSetEnv (const KVString &dataset, const KVString &type, const char *defval)

template<typename ValType >
static ValType	GetDataSetEnv (const KVString &dataset, const KVString &type, const ValType &defval)

static const Char_t *	GetETCDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetExampleFilePath (const Char_t library, const Char_t namefile)
	Return full path to example file for given library (="KVMultiDet", "BackTrack", etc.) More...

static const Char_t *	GetINCDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetKVBuildDate ()
	Returns KaliVeda build date. More...

static const Char_t *	GetKVBuildDir ()
	Returns top-level directory used for build. More...

static const Char_t *	GetKVBuildTime ()
	Returns KaliVeda build time. More...

static const Char_t *	GetKVBuildType ()
	Returns KaliVeda build type (cmake build: Release, Debug, RelWithDebInfo, ...) More...

static const Char_t *	GetKVBuildUser ()
	Returns username of person who performed build. More...

static int	GetKVMajorVersion ()

static int	GetKVMinorVersion ()

static int	GetKVPatchVersion ()

static const Char_t *	GetKVSourceDir ()
	Returns top-level directory of source tree used for build. More...

static const Char_t *	GetKVVersion ()
	Returns KaliVeda version string. More...

static const Char_t *	GetLIBDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetListOfPlugins (const Char_t *base)

static const Char_t *	GetListOfPluginURIs (const Char_t *base)

static const Char_t *	GetPluginURI (const Char_t base, const Char_t plugin)

static void	GetTempFileName (TString &base)

static const Char_t *	GetTEMPLATEDIRFilePath (const Char_t *namefile="")

static const Char_t *	GetWORKDIRFilePath (const Char_t *namefile="")

static const Char_t *	gitBranch ()
	Returns git branch of sources. More...

static const Char_t *	gitCommit ()
	Returns last git commit of sources. More...

static void	InitEnvironment ()

static bool	is_gnuinstall ()

static Bool_t	IsEnvInit ()

static Bool_t	IsThisAPlugin (const TString &uri, TString &base)

static TPluginHandler *	LoadPlugin (const Char_t base, const Char_t uri="0")

static Bool_t	OpenContextMenu (const char method, TObject obj, const char *alt_method_name="")

static void	OpenTempFile (TString &base, std::ofstream &fp)

static void	PrintSplashScreen ()
	Prints welcome message and infos on version etc. More...

static Double_t	ProtectedGetX (const TF1 &func, Double_t val, int &status, Double_t xmin=0.0, Double_t xmax=0.0)

static Double_t	ProtectedGetX (const TF1 *func, Double_t val, int &status, Double_t xmin=0.0, Double_t xmax=0.0)

static Bool_t	SearchAndOpenKVFile (const Char_t name, KVSQLite::database &dbfile, const Char_t kvsubdir="")

static Bool_t	SearchAndOpenKVFile (const Char_t name, std::ifstream &file, const Char_t kvsubdir="", KVLockfile *locks=0)

static Bool_t	SearchAndOpenKVFile (const Char_t name, std::ofstream &file, const Char_t kvsubdir="", KVLockfile *locks=0)

static Bool_t	SearchKVFile (const Char_t name, TString &fullpath, const Char_t kvsubdir="")

static const Char_t *	WorkingDirectory ()

Static Public Member Functions inherited from TNamed
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

Static Public Member Functions inherited from TObject
static TClass *	Class ()

static const char *	Class_Name ()

static constexpr Version_t	Class_Version ()

static const char *	DeclFileName ()

static Longptr_t	GetDtorOnly ()

static Bool_t	GetObjectStat ()

static void	SetDtorOnly (void *obj)

static void	SetObjectStat (Bool_t stat)

Public Attributes inherited from TObject
	kBitMask

	kCanDelete

	kCannotPick

	kHasUUID

	kInconsistent

	kInvalidObject

	kIsOnHeap

	kIsReferenced

	kMustCleanup

	kNoContextMenu

	kNotDeleted

	kObjInCanvas

	kOverwrite

	kSingleKey

	kWriteDelete

	kZombie

Protected Member Functions inherited from TObject
virtual void	DoError (int level, const char location, const char fmt, va_list va) const

void	MakeZombie ()

Protected Attributes inherited from TNamed
TString	fName

TString	fTitle

Protected Attributes inherited from TObject
	kOnlyPrepStep

Constructor & Destructor Documentation

◆ KVDataRepository() [1/2]

KVDataRepository::KVDataRepository ( )

Default constructor.

Definition at line 170 of file KVDataRepository.cpp.

◆ KVDataRepository() [2/2]

KVDataRepository::KVDataRepository	(	const TString &	name,
		const TString &	root_dir,
		const TString &	access_backend = `"local"`
	)

Create a new data repository with characteristics given by argument values

Parameters

[in]	name	name of repository
[in]	root_dir	full path to root directory of data repository
[in]	access_backend	protocol to access repository. Can be "local" (default), or "irods"

Definition at line 185 of file KVDataRepository.cpp.

Member Function Documentation

◆ CanWrite()

Bool_t KVDataRepository::CanWrite ( ) const

inline

Returns: kTRUE if new files can be created and written directly in the repository,; kFALSE if they have to be written locally then copied to repository This is set by the property [repository name].DataRepository.CanWrite in the repository configuration

Definition at line 310 of file KVDataRepository.h.

◆ cd()

void KVDataRepository::cd ( )

virtual

Make this the 'active' or 'default' data repository, i.e. gDataRepository points to this

Reimplemented in KVDataSetRepository.

Definition at line 39 of file KVDataRepository.cpp.

◆ CheckFileStatus()

template<typename... Paths>

Bool_t KVDataRepository::CheckFileStatus	(	const TString &	filename,
		const Paths &...	paths
	)		const

inline

Given an arbitrary number of paths, returns kTRUE if the file exists in the repository

Definition at line 262 of file KVDataRepository.h.

◆ CheckSubdirExists()

template<typename... Paths>

Bool_t KVDataRepository::CheckSubdirExists	(	const TString &	path1,
		const Paths &...	paths
	)		const

inline

Returns: kTRUE if the arbitrary path
/root_of_data_repository/path1/path2/.../pathN

is valid and exists

Definition at line 242 of file KVDataRepository.h.

◆ CommitFile()

template<typename... Paths>

void KVDataRepository::CommitFile	(	TFile *	file,
		const Paths &...	paths
	)

inline

close ROOT file

create file in local repository - make sure subdirectory exists!

delete local file

Definition at line 387 of file KVDataRepository.h.

◆ CopyFileFromRepository()

template<typename... Paths>

void KVDataRepository::CopyFileFromRepository	(	const TString &	filename,
		const TString &	destination,
		const Paths &...	paths
	)		const

inline

Copy file [path1]/[path2]/.../[filename] from the repository to [destination]

We check if the file to copy exists.

Definition at line 333 of file KVDataRepository.h.

◆ CopyFileToRepository()

template<typename... Paths>

int KVDataRepository::CopyFileToRepository	(	const TString &	filename,
		const TString &	source,
		const Paths &...	paths
	)		const

inline

Copy file [source] to [path1]/[path2]/.../[filename] in the repository

If all necessary subdirectories do not exist, they will be created.

The file access permissions are set to '664 (u:rw, g:rw, o:r)

Returns: status of file transfer command

change file access permissions to 0664

Definition at line 344 of file KVDataRepository.h.

◆ CreateAllNeededSubdirectories() [1/2]

void KVDataRepository::CreateAllNeededSubdirectories ( ) const

inlineprivate

Definition at line 226 of file KVDataRepository.h.

◆ CreateAllNeededSubdirectories() [2/2]

template<typename... Paths>

void KVDataRepository::CreateAllNeededSubdirectories	(	const TString &	path1,
		const Paths &...	paths
	)		const

inlineprivate

expand any environment variables in path name

Definition at line 210 of file KVDataRepository.h.

◆ CreateNewFile()

template<typename... Paths>

TFile* KVDataRepository::CreateNewFile	(	const TString &	filename,
		const Paths &...	paths
	)		const

inline

This will create and open a new ROOT file in the repository, ready to be written.

If all necessary subdirectories do not exist, they will be created.

Note: if the repository is not one in which files can be created and written directly (i.e. if CanWrite() = kFALSE), the file will be created in the local working directory, and only copied into the repository when CommitFile() is called.

files cannot be created and written directly in the repository create local file

Definition at line 367 of file KVDataRepository.h.

◆ DeleteFile()

template<typename... Paths>

void KVDataRepository::DeleteFile	(	const TString &	filename,
		Bool_t	confirm,
		const Paths &...	paths
	)		const

inline

Given an arbitrary number of paths, physically delete the file from the repository

param[in] confirm if true, ask for confirmation first

Definition at line 420 of file KVDataRepository.h.

◆ get_path_with_access_root()

template<typename... Paths>

TString KVDataRepository::get_path_with_access_root ( const Paths &... paths ) const

inlineprivate

expand any environment variables in path name

Definition at line 179 of file KVDataRepository.h.

◆ get_path_with_arbitrary_root()

template<typename... Paths>

TString KVDataRepository::get_path_with_arbitrary_root	(	const TString &	arb_root,
		const Paths &...	paths
	)		const

inlineprivate

expand any environment variables in path name

Definition at line 197 of file KVDataRepository.h.

◆ get_path_with_read_root()

template<typename... Paths>

TString KVDataRepository::get_path_with_read_root ( const Paths &... paths ) const

inlineprivate

expand any environment variables in path name

Definition at line 188 of file KVDataRepository.h.

◆ GetDirectoryListing()

template<typename... Paths>

KVUniqueNameList KVDataRepository::GetDirectoryListing ( const Paths &... paths ) const

inline

Use the access protocol defined by DataRepository.AccessProtocol (=local by default) in order to open the directory

/root_of_data_repository/[path1]/[path2]/.../[path3]

and fill a KVUniqueNameList with one KVBase object for each entry in the directory, excluding "." and ".."

Definition at line 320 of file KVDataRepository.h.

◆ GetFileInfo()

template<typename... Paths>

std::optional<FileStat_t> KVDataRepository::GetFileInfo	(	const TString &	filename,
		const Paths &...	paths
	)		const

inline

Given an arbitrary number of paths, if the file exists in the repository the std::optional<FileStat_t> object contains information about the file.

Definition at line 252 of file KVDataRepository.h.

◆ GetFileTransferExec()

TString KVDataRepository::GetFileTransferExec ( ) const

inline

returns full path to executable used for remote file transfer

Definition at line 481 of file KVDataRepository.h.

◆ GetFileTransferServer()

TString KVDataRepository::GetFileTransferServer ( ) const

inline

returns server url used for remote file transfer

Definition at line 486 of file KVDataRepository.h.

◆ GetFileTransferType()

TString KVDataRepository::GetFileTransferType ( ) const

inline

returns protocol used for remote file transfer

Definition at line 476 of file KVDataRepository.h.

◆ GetFileTransferUser()

TString KVDataRepository::GetFileTransferUser ( ) const

inline

returns user name used for remote file transfer

Definition at line 491 of file KVDataRepository.h.

◆ GetFullPathToOpenFile()

template<typename... Paths>

TString KVDataRepository::GetFullPathToOpenFile	(	const TString &	filename,
		const Paths &...	paths
	)		const

inline

Given an arbitrary number of paths, returns the full path required to open the file:

GetFullPathToOpenFile("filename", "path1", "path2") => "/root_directory/path1/path2/filename" etc.

KVDataRepository::GetFullPathToOpenFile

TString GetFullPathToOpenFile(const TString &filename, const Paths &... paths) const

Definition: KVDataRepository.h:283

Definition at line 283 of file KVDataRepository.h.

◆ GetFullPathToTransferFile()

template<typename... Paths>

TString KVDataRepository::GetFullPathToTransferFile	(	const TString &	filename,
		const Paths &...	paths
	)		const

inline

Used by KVDataTransfer.

Returns the full path needed to transfer a file either from or to the repository, using sftp or bbftp etc.

Definition at line 298 of file KVDataRepository.h.

◆ GetRepositoryPath()

template<typename... Paths>

TString KVDataRepository::GetRepositoryPath ( const Paths &... paths ) const

inline

Given an arbitrary number of paths, returns the path within the repository, i.e. the full path but without the root directory of the repository

GetFullPathInRepository("path1", "path2") => "path1/path2" etc.

Definition at line 269 of file KVDataRepository.h.

◆ GetRootDirectory()

TString KVDataRepository::GetRootDirectory ( ) const

inline

returns root directory of data repository (fLocalrootdir)

Definition at line 496 of file KVDataRepository.h.

◆ GetType()

const Char_t* KVDataRepository::GetType ( ) const

inlineoverridevirtual

'type' of repository is actually the type of the underlying back-end, i.e. 'local' or 'dms'

this is used by KVDataSetRepository::NewAvailableRunsFile() in order to select the right plugin for KVAvailableRunsFile

Reimplemented from KVBase.

Definition at line 233 of file KVDataRepository.h.

◆ Init()

Bool_t KVDataRepository::Init ( void )

virtual

Initialises data repository based on information in .kvrootrc file. Each line beginning "name.DataRepository" is used for this data repository, where "name" is the name of this repository.

Then we create and initialise the data set manager for this repository.

returns kTRUE if all goes well. returns kFALSE if not (i.e. data set manager cannot be initialised, no available datasets)

Reimplemented in KVDataSetRepository.

Definition at line 59 of file KVDataRepository.cpp.

◆ MakeSubdirectory()

template<typename... Paths>

void KVDataRepository::MakeSubdirectory	(	const TString &	path1,
		const Paths &...	paths
	)		const

inline

Create a new subdirectory in the repository:

/root_of_data_repository/path1/[path2]/.../[pathN]

and set access permissions to 775 (u:rwx, g:rwx, o:rx)

Definition at line 409 of file KVDataRepository.h.

◆ OpenFile()

template<typename... Paths>

TObject* KVDataRepository::OpenFile	(	const TString &	filename,
		const std::function< TObject *(const TString &)> &	open_file,
		const Paths &...	paths
	)

inline

open file given arbitrary number of paths, return handle to open file as TObject* pointer

param[in] open_file a callable object with signature 'TObject* (const TString&)'. when called with the full path to the file to be opened, opens the file and returns its handle as a TObject* pointer.

If files need to be downloaded to a local temporary directory before opening, this is handled here.

look for file in temp dir

copy file to temp dir if not already there

Definition at line 444 of file KVDataRepository.h.

◆ OpenROOTFile()

template<typename... Paths>

TFile* KVDataRepository::OpenROOTFile	(	const TString &	filename,
		const Paths &...	paths
	)

inline

open ROOT file given arbitrary number of paths, return handle to open file as TFile* pointer

Definition at line 464 of file KVDataRepository.h.

◆ Print()

void KVDataRepository::Print ( Option_t * opt = "" ) const

overridevirtual

Print info on repository.

Reimplemented from KVBase.

Definition at line 144 of file KVDataRepository.cpp.

◆ ReadWithXrootd()

void KVDataRepository::ReadWithXrootd	(	const TString &	server,
		const TString &	rootdir
	)

Call this method if you can read files in the repository using an xrootd server.

Parameters

[in]	server	complete URL of xrootd server (without any network transport specification i.e. 'root://', 'https://', etc.)
[in]	rootdir	the full path of the root directory used to access the repository through the xrootd server (replaces root directory given to constructor)

Definition at line 209 of file KVDataRepository.cpp.

◆ set_impl_backend()

void KVDataRepository::set_impl_backend ( const TString & impl_type )

private

Initialises the back-end used to access/interrogate the repository, accessed through member pointer std::unique_ptr<KVDataRepositoryImpl> _impl

Parameters

[in] impl_type type of back-end. can be "local" (repository on local disk) or "irods" (access through iRODs client)

if impl_type!="local", we set CanWrite() = false

Definition at line 260 of file KVDataRepository.cpp.

◆ SetCanWrite()

void KVDataRepository::SetCanWrite ( Bool_t yes = kTRUE )

inline

Definition at line 314 of file KVDataRepository.h.

◆ SetFullPath()

void KVDataRepository::SetFullPath	(	TString &	path,
		const TString &	tmp
	)

private

Will replace contents of 'path' with the full base path needed for the given protocol:

protocol = "local" : path = fLocalrootdir
protocol = "root" : path = "root://" + fXrootdserver + "/" + fXrootdrootdir
protocol = "rfio" : path = "rfio:" + fRfioserver + ":" + fRfiorootdir

Definition at line 231 of file KVDataRepository.cpp.

Member Data Documentation

◆ _impl

std::unique_ptr<KVDataRepositoryImpl> KVDataRepository::_impl

private

Definition at line 207 of file KVDataRepository.h.

◆ fAccessprotocol

TString KVDataRepository::fAccessprotocol = "local"

private

Definition at line 129 of file KVDataRepository.h.

◆ fAccessroot

TString KVDataRepository::fAccessroot

private

Definition at line 135 of file KVDataRepository.h.

◆ fCanWrite

Bool_t KVDataRepository::fCanWrite = kTRUE

private

Definition at line 141 of file KVDataRepository.h.

◆ fLocalrootdir

TString KVDataRepository::fLocalrootdir

private

Definition at line 128 of file KVDataRepository.h.

◆ fReadprotocol

TString KVDataRepository::fReadprotocol = "local"

private

Definition at line 130 of file KVDataRepository.h.

◆ fReadroot

TString KVDataRepository::fReadroot

private

Definition at line 136 of file KVDataRepository.h.

◆ fRfiorootdir

TString KVDataRepository::fRfiorootdir

private

Definition at line 134 of file KVDataRepository.h.

◆ fRfioserver

TString KVDataRepository::fRfioserver

private

Definition at line 133 of file KVDataRepository.h.

◆ fTransferExec

TString KVDataRepository::fTransferExec

private

Definition at line 138 of file KVDataRepository.h.

◆ fTransferserver

TString KVDataRepository::fTransferserver

private

Definition at line 139 of file KVDataRepository.h.

◆ fTransfertype

TString KVDataRepository::fTransfertype

private

Definition at line 137 of file KVDataRepository.h.

◆ fTransferuser

TString KVDataRepository::fTransferuser

private

Definition at line 140 of file KVDataRepository.h.

◆ fXrootdrootdir

TString KVDataRepository::fXrootdrootdir

private

Definition at line 132 of file KVDataRepository.h.

◆ fXrootdserver

TString KVDataRepository::fXrootdserver

private

Definition at line 131 of file KVDataRepository.h.

Detailed Description

Setting up a data repository

Local repository

iRODS/xrootd repository

Creating a new ROOT file in the repository

Browsing the contents of the repository

Opening files in the repository

Opening files using xrootd

Classes

Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ KVDataRepository() [1/2]

◆ KVDataRepository() [2/2]

Member Function Documentation

◆ CanWrite()

◆ cd()

◆ CheckFileStatus()

◆ CheckSubdirExists()

◆ CommitFile()

◆ CopyFileFromRepository()

◆ CopyFileToRepository()

◆ CreateAllNeededSubdirectories() [1/2]

◆ CreateAllNeededSubdirectories() [2/2]

◆ CreateNewFile()

◆ DeleteFile()

◆ get_path_with_access_root()

◆ get_path_with_arbitrary_root()

◆ get_path_with_read_root()

◆ GetDirectoryListing()

◆ GetFileInfo()

◆ GetFileTransferExec()

◆ GetFileTransferServer()

◆ GetFileTransferType()

◆ GetFileTransferUser()

◆ GetFullPathToOpenFile()

◆ GetFullPathToTransferFile()

◆ GetRepositoryPath()

◆ GetRootDirectory()

◆ GetType()

◆ Init()

◆ MakeSubdirectory()

◆ OpenFile()

◆ OpenROOTFile()

◆ Print()

◆ ReadWithXrootd()

◆ set_impl_backend()

◆ SetCanWrite()

◆ SetFullPath()

Member Data Documentation

◆ _impl

◆ fAccessprotocol

◆ fAccessroot

◆ fCanWrite

◆ fLocalrootdir

◆ fReadprotocol

◆ fReadroot

◆ fRfiorootdir

◆ fRfioserver

◆ fTransferExec

◆ fTransferserver

◆ fTransfertype

◆ fTransferuser

◆ fXrootdrootdir

◆ fXrootdserver