The ezFileSystem provides high-level functionality to manage files in a virtual file system. More...

#include <FileSystem.h>

Classes
struct	FileEvent
	The event data that is broadcast by the ezFileSystem upon certain file operations. More...

struct	FileEventType
	Describes the type of events that are broadcast by the ezFileSystem. More...

Static Public Member Functions
static ezEventSubscriptionID	RegisterEventHandler (ezEvent< const FileEvent & >::Handler handler)
	Registers an Event Handler that will be informed about all the events that the file system broadcasts.

static void	UnregisterEventHandler (ezEvent< const FileEvent & >::Handler handler)
	Unregisters a previously registered Event Handler.

static void	UnregisterEventHandler (ezEventSubscriptionID subscriptionId)

static ezResult	CreateDirectoryStructure (ezStringView sPath)

static void	DeleteFile (ezStringView sFile)
	Deletes the given file from all data directories, if possible. More...

static bool	ExistsFile (ezStringView sFile)
	Checks whether the given file exists in any data directory. More...

static ezResult	GetFileStats (ezStringView sFileOrFolder, ezFileStats &out_stats)
	Tries to get the ezFileStats for the given file. Typically should give the same results as ezOSFile::GetFileStats, but some data dir implementations may not support retrieving all data (e.g. GetFileStats on folders might not always work).

static ezResult	ResolvePath (ezStringView sPath, ezStringBuilder out_pAbsolutePath, ezStringBuilder out_pDataDirRelativePath, const ezDataDirectoryInfo **out_pDataDir=nullptr)
	Tries to resolve the given path and returns the absolute and relative path to the final file. More...

static ezResult	FindFolderWithSubPath (ezStringBuilder &ref_sResult, ezStringView sStartDirectory, ezStringView sSubPath, ezStringView sRedirectionFileName={})
	Starts at szStartDirectory and goes up until it finds a folder that contains the given sub folder structure. More...

static bool	ResolveAssetRedirection (ezStringView sPathOrAssetGuid, ezStringBuilder &out_sRedirection)
	Returns true, if any data directory knows how to redirect the given path. Otherwise the original string is returned in out_sRedirection.

static ezStringView	MigrateFileLocation (ezStringView sOldLocation, ezStringView sNewLocation)
	Migrates a file from an old location to a new one, and returns the path that should be used to open it (either the old or the new path). More...

Special Directories
static ezResult	DetectSdkRootDirectory (ezStringView sExpectedSubFolder="Data/Base")
	Searches for a directory to use as the "Sdk" special directory. More...

static void	SetSdkRootDirectory (ezStringView sSdkDir)
	the special directory ">Sdk" is the root folder of the SDK data, it is often used as the main reference from where other data directories are found. For higher level code (e.g. ezApplication) it is often vital that this is set early at startup. More...

static ezStringView	GetSdkRootDirectory ()
	Returns the previously set Sdk root directory. More...

static void	SetSpecialDirectory (ezStringView sName, ezStringView sReplacement)
	Special directories are used when mounting data directories as basic references. More...

static ezResult	ResolveSpecialDirectory (ezStringView sDirectory, ezStringBuilder &out_sPath)
	Returns the absolute path to szDirectory. More...

Misc
static ezMutex &	GetMutex ()
	Returns the (recursive) mutex that is used internally by the file system which can be used to guard bundled operations on the file system.

Friends
class	ezDataDirectoryReaderWriterBase

class	ezFileReaderBase

class	ezFileWriterBase

Data Directory Modifications
All functions that add / remove data directories are not thread safe and require that this is done on a single thread with no other thread accessing anything in ezFileSystem simultaneously.
using	ezDataDirFactory = ezDataDirectoryType ()(ezStringView, ezStringView, ezStringView, ezDataDirUsage)
	This factory creates a data directory type, if it can handle the given data directory. Otherwise it returns nullptr. More...

static void	RegisterDataDirectoryFactory (ezDataDirFactory factory, float fPriority=0)
	This function allows to register another data directory factory, which might be invoked when a new data directory is to be added.

static void	ClearAllDataDirectoryFactories ()
	Will remove all known data directory factories.

static ezResult	AddDataDirectory (ezStringView sDataDirectory, ezStringView sGroup={}, ezStringView sRootName={}, ezDataDirUsage usage=ezDataDirUsage::ReadOnly)
	Adds a data directory. It will try all the registered factories to find a data directory type that can handle the given path. More...

static bool	RemoveDataDirectory (ezStringView sRootName)
	Searches for a data directory with the given root name and removes it. More...

static ezUInt32	RemoveDataDirectoryGroup (ezStringView sGroup)
	Removes all data directories that belong to the given group. Returns the number of data directories that were removed.

static void	ClearAllDataDirectories ()
	Removes all data directories.

static const ezDataDirectoryInfo *	FindDataDirectoryWithRoot (ezStringView sRootName)
	If a data directory with the given root name already exists, it will be returned, nullptr otherwise.

static ezUInt32	GetNumDataDirectories ()
	Returns the number of currently active data directories.

static ezDataDirectoryType *	GetDataDirectory (ezUInt32 uiDataDirIndex)
	Returns the n-th currently active data directory.

static const ezDataDirectoryInfo &	GetDataDirectoryInfo (ezUInt32 uiDataDirIndex)
	Returns the info about the n-th currently active data directory.

static void	ReloadAllExternalDataDirectoryConfigs ()
	Calls ezDataDirectoryType::ReloadExternalConfigs() on all active data directories.

Detailed Description

The ezFileSystem provides high-level functionality to manage files in a virtual file system.

There are two sides at which the file system can be extended: Data directories are the 'sources' of data. These can be simple folders, zip files, data-bases, HTTP servers, etc. Different ezDataDirectoryType's can implement these different 'decoding methods', i.e. they handle how to actually access the data and they use their own readers/writers to implement a common interface for passing data streams to and from the data directory. On the other end there are the actual file readers/writers, which implement policies how to optimize these reads/writes. The default ezFileReader and ezFileWriter implement a buffering policy, i.e. they use an internal cache to only sporadically read or write to the actual data stream. A 'threaded' or 'parallel' file reader/writer could implement a different policy, where a file is read/written in a thread and thus allows to have non-blocking file accesses.

Which policy to use is defined by the user every time he needs to access a file, by simply using the desired reader/writer class. How to mount data directories (i.e. with which ezDataDirectoryType) is defined by the 'DataDirFactories', which are functions that create ezDataDirectoryType's. This way one can mount the same data directory (e.g. "MyTestDir") differently, depending on which Factories have been registered previously. This allows to easily configure how to set up data directories. E.g. by default ordinary folders will be mounted to be read from the local file system. However, by registering a different Factory, the same directory could also be mounted over a network on a remote file serving machine.

Additionally ezFileSystem also broadcasts events about which files are (about to be) accessed. This allows to hook into the system and implement stuff like automatic asset transformations before/after certain file accesses, checking out files from revision control systems, or simply logging all file activity.

All operations that go through the ezFileSystem are protected by a mutex, which means that opening, closing, deleting files, as well as adding or removing data directories etc. will be synchronized and cannot happen in parallel. Reading/writing file streams can happen in parallel, only the administrative tasks need to be protected. File events are broadcast as they occur, that means they will be executed on whichever thread triggered them. Since they are executed from within the filesystem mutex, they cannot occur in parallel.

Member Typedef Documentation

◆ ezDataDirFactory

using ezFileSystem::ezDataDirFactory = ezDataDirectoryType* (*)(ezStringView, ezStringView, ezStringView, ezDataDirUsage)

This factory creates a data directory type, if it can handle the given data directory. Otherwise it returns nullptr.

Every time a data directory is supposed to be added, the file system will query its data dir factories, which one can successfully create an ezDataDirectoryType. In this process the last factory added has the highest priority. Once a factory is found that was able to create a ezDataDirectoryType, that one is used. Different factories can be used to mount different types of data directories. But the same directory can also be mounted in different ways. For example a simple folder could be mounted on the local system, or via a HTTP server over a network (lets call it a 'FileServer'). Thus depending on which type of factories are registered, the file system can provide data from very different sources.

Member Function Documentation

◆ AddDataDirectory()

ezResult ezFileSystem::AddDataDirectory	(	ezStringView	sDataDirectory,
		ezStringView	sGroup = `{}`,
		ezStringView	sRootName = `{}`,
		ezDataDirUsage	usage = `ezDataDirUsage::ReadOnly`
	)

static

Adds a data directory. It will try all the registered factories to find a data directory type that can handle the given path.

If Usage is ReadOnly, writing to the data directory is not allowed. This is independent of whether the data directory type COULD write anything. szGroup defines to what 'group' of data directories this data dir belongs. This is only used in calls to RemoveDataDirectoryGroup, to remove all data directories of the same group. You could use groups such as 'Base', 'Project', 'Settings', 'Level', 'Temp' to distinguish between different sets of data directories. You can also specify the exact same string as szDataDirectory for szGroup, and thus uniquely identify the data dir, to be able to remove just that one. szRootName is optional for read-only data dirs, but mandatory for writable ones. It has to be unique to clearly identify a file within that data directory. It must be used when writing to a file in this directory. For instance, if a data dir root name is "mydata", then the path ":mydata/SomeFile.txt" can be used to write to the top level folder of this data directory. The same can be used for reading exactly that file and ignoring the other data dirs.

◆ DeleteFile()

void ezFileSystem::DeleteFile ( ezStringView sFile )

static

Deletes the given file from all data directories, if possible.

The path must be absolute or rooted, to uniquely identify which file to delete. For example ":appdata/SomeData.txt", assuming a writable data directory has been mounted with the "appdata" root name.

◆ DetectSdkRootDirectory()

ezResult ezFileSystem::DetectSdkRootDirectory ( ezStringView sExpectedSubFolder = "Data/Base" )

static

Searches for a directory to use as the "Sdk" special directory.

It does so by starting at the directory where the application binary is located and then goes up until it finds a folder that contains the given sub-folder. The sub-folder is usually where the engine loads the most basic data from, so it should exist.

Additionally the 'redirection file' feature of ezFileSystem::FindFolderWithSubPath() is used to allow finding a relocated SDK folder. To do that, place a file called 'ezSdkRoot.txt' in your top level folder. It should contain the relative path pointing to the SDK folder.

Upon success SetSdkRootDirectory() is called with the resulting path.

Note: If the Sdk root directory has been set before, this function does nothing! It will not override a previously set value. If that is desired, call SetSdkRootDirectory("") first.

See also: ezFileSystem::FindFolderWithSubPath()

◆ ExistsFile()

bool ezFileSystem::ExistsFile ( ezStringView sFile )

static

Checks whether the given file exists in any data directory.

The search can be restricted to directories of certain categories (see AddDataDirectory).

◆ FindFolderWithSubPath()

ezResult ezFileSystem::FindFolderWithSubPath	(	ezStringBuilder &	ref_sResult,
		ezStringView	sStartDirectory,
		ezStringView	sSubPath,
		ezStringView	sRedirectionFileName = `{}`
	)

static

Starts at szStartDirectory and goes up until it finds a folder that contains the given sub folder structure.

Returns EZ_FAILURE if nothing is found. Otherwise result is the absolute path to the existing folder that has a given sub-folder.

Parameters

result	If successful, this contains the folder path in which szSubPath exists.
szStartDirectory	The directory in which to start the search and iterate upwards.
szSubPath	the relative path to look for in each visited directory. The function succeeds if such a file or folder is found.
szRedirectionFileName	An optional file name for a redirection file. If in any visited folder a file with this name is found, it will be opened, read entirely, and appended to the current search path, and it is checked whether szSubPath can be found there. This step is not recursive and can't result in an endless loop. It allows to relocate the SDK folder and still have it found, by placing such a redirection file. A common use case, is when ezEngine is used as a Git submodule and therefore the overall file structure is slightly different.

◆ GetSdkRootDirectory()

ezStringView ezFileSystem::GetSdkRootDirectory ( )

static

Returns the previously set Sdk root directory.

Note: Asserts that the path is not empty!

See also: SetSdkRootDirectory; DetectSdkRootDirectory

◆ MigrateFileLocation()

ezStringView ezFileSystem::MigrateFileLocation	(	ezStringView	sOldLocation,
		ezStringView	sNewLocation
	)

static

Migrates a file from an old location to a new one, and returns the path that should be used to open it (either the old or the new path).

If the file does not exist in the old location, nothing is done, and the new location is returned. Otherwise, it is attempted to move the file from the old location to the new location. In case that fails (target not writeable or so), the old path is returned, so that code that needs to read that file, finds it in the correct location. If it succeeds, the new location is returned. Afterwards, the file does not exist in the old location anymore.

◆ RemoveDataDirectory()

bool ezFileSystem::RemoveDataDirectory ( ezStringView sRootName )

static

Searches for a data directory with the given root name and removes it.

Returns true, if one was found and removed, false if no such data dir existed.

◆ ResolvePath()

ezResult ezFileSystem::ResolvePath	(	ezStringView	sPath,
		ezStringBuilder *	out_pAbsolutePath,
		ezStringBuilder *	out_pDataDirRelativePath,
		const ezDataDirectoryInfo **	out_pDataDir = `nullptr`
	)

static

Tries to resolve the given path and returns the absolute and relative path to the final file.

If the given path is a rooted path, for instance something like ":appdata/UserData.txt", (which is necessary for writing to files), the path can be converted easily and the file does not need to exist. Only the data directory with the given root name must be mounted.

If the path is relative, it is attempted to open the specified file, which means it is searched in all available data directories. The path to the file that is found will be returned.

Parameters

sPath	can be a relative, an absolute or a rooted path. This can also be used to find the relative location to the data directory that would handle it.
out_sAbsolutePath	will contain the absolute path to the file. Can be nullptr.
out_sDataDirRelativePath	will contain the relative path to the file (from the data directory in which it might end up in). Can be nullptr.
out_ppDataDir	If not null, it will be set to the data directory that would handle this path.

Returns: The function will return EZ_FAILURE if it was not able to determine any location where the file could be read from or written to.

Todo:: We might also need the none-redirected path as an output

Todo:: We might also need the none-redirected path as an output

◆ ResolveSpecialDirectory()

ezResult ezFileSystem::ResolveSpecialDirectory	(	ezStringView	sDirectory,
		ezStringBuilder &	out_sPath
	)

static

Returns the absolute path to szDirectory.

If the path starts with a known special directory marker (">marker/") it is replaced accordingly. See SetSpecialDirectory() for setting custom special directories.

Built-in special directories (always available) are:

">sdk/" - Resolves to what GetSdkRootDirectory() returns. ">user/" - Resolves to what ezOSFile::GetUserDataFolder() returns. ">temp/" - Resolves to what ezOSFile::GetTempDataFolder() returns. ">appdir/" - Resolves to what ezOSFile::GetApplicationDirectory() returns.

Returns EZ_FAILURE if szDirectory starts with an unknown special directory.

◆ SetSdkRootDirectory()

void ezFileSystem::SetSdkRootDirectory ( ezStringView sSdkDir )

static

the special directory ">Sdk" is the root folder of the SDK data, it is often used as the main reference from where other data directories are found. For higher level code (e.g. ezApplication) it is often vital that this is set early at startup.

See also: DetectSdkRootDirectory()

◆ SetSpecialDirectory()

void ezFileSystem::SetSpecialDirectory	(	ezStringView	sName,
		ezStringView	sReplacement
	)

static

Special directories are used when mounting data directories as basic references.

They are indicated with a ">", ie. ">sdk/Test", but using them is only allowed in few places, e.g. in AddDataDirectory(). Special directories are needed to be able to set up other paths relative to them and to be able to use different ones on different PCs. For instance when using file-serve functionality, the special directories may be different on the host and client machines, but the paths used to mount data directories can stay the same because of this.

The documentation for this class was generated from the following files:

/home/runner/work/ezEngine/ezEngine/Code/Engine/Foundation/IO/FileSystem/FileSystem.h
/home/runner/work/ezEngine/ezEngine/Code/Engine/Foundation/IO/FileSystem/Implementation/FileSystem.cpp

Classes

Static Public Member Functions

Friends

Data Directory Modifications

Detailed Description

Member Typedef Documentation

◆ ezDataDirFactory

Member Function Documentation

◆ AddDataDirectory()

◆ DeleteFile()

◆ DetectSdkRootDirectory()

◆ ExistsFile()

◆ FindFolderWithSubPath()

◆ GetSdkRootDirectory()

◆ MigrateFileLocation()

◆ RemoveDataDirectory()

◆ ResolvePath()

◆ ResolveSpecialDirectory()

◆ SetSdkRootDirectory()

◆ SetSpecialDirectory()