#include <data_source.h>

Public Member Functions
template<data_source_compatible_type T>
	data_source (const T &source)
	Constructs a data_source from a compatible type.
template<data_source_compatible_type T>
	data_source (T &&source)
	Constructs a data_source by moving from a compatible type.
template<data_source_compatible_type T>
	data_source (const T &source, file_extension file_extension)
	Constructs a data_source with an explicit file extension.
template<data_source_compatible_type T>
	data_source (T &&source, file_extension file_extension)
	Constructs a data_source by moving, with an explicit file extension.
template<data_source_compatible_type T>
	data_source (const T &source, mime_type mime_type, confidence mime_type_confidence)
	Constructs a data_source with an initial MIME type and confidence.
template<data_source_compatible_type T>
	data_source (T &&source, mime_type mime_type, confidence mime_type_confidence)
	Constructs a data_source by moving, with an initial MIME type and confidence.
std::span< const std::byte >	span (std::optional< length_limit > limit=std::nullopt) const
	Returns the content as a span of bytes.
std::string	string (std::optional< length_limit > limit=std::nullopt) const
	Returns the content as a string.
std::string_view	string_view (std::optional< length_limit > limit=std::nullopt) const
	Returns the content as a string_view.
std::shared_ptr< std::istream >	istream () const
	Returns an input stream for reading the data.
std::optional< std::filesystem::path >	path () const
	Returns the file path if the source is a file, otherwise std::nullopt.
std::optional< df0::file_extension >	file_extension () const
	Returns the file extension if available.
unique_identifier	id () const
	Returns the unique identifier for this data source.
std::optional< std::pair< mime_type, confidence > >	highest_confidence_mime_type_info () const
	Returns the MIME type with the highest confidence and its confidence level.
std::optional< mime_type >	highest_confidence_mime_type () const
	Returns the MIME type with the highest confidence.
confidence	highest_mime_type_confidence () const
	Returns the highest confidence level found among detected MIME types.
bool	has_highest_confidence_mime_type_in (const std::vector< mime_type > &mts) const
	Checks if the highest confidence mime type is present in the given list.
void	assert_not_encrypted () const
	Asserts that the data source is not encrypted.
confidence	mime_type_confidence (mime_type mt) const
	Returns the confidence level for a specific MIME type.
void	add_mime_type (mime_type mt, confidence c)
	Adds a mime type with a confidence level.

Public Attributes
std::unordered_map< mime_type, confidence >	mime_types
	Map of detected MIME types and their confidence levels.

Detailed Description

The class below represents a binary data source for data processing. It can be initialized with a file path, memory buffer, input stream or other data source. All popular C++ data sources are supported. Document parsers and 3rdparty libraries needs to have access to the data in preferred way like memory buffer or file path or stream or range, because of their implementation and it cannot be changed. Sometimes one method is faster than other, and parser needs to know about state of data source to decide. Converting data from one storage form to other should be possible in all combinations but performed only as required (lazy) and cached inside the class, for example file should be read to memory only once. Performance is very important, for example we should not duplicate memory buffer that is passed to class.

Definition at line 135 of file data_source.h.

Constructor & Destructor Documentation

◆ data_source() [1/6]

template<data_source_compatible_type T>

df0::data_source::data_source ( const T & source )

inlineexplicit

Constructs a data_source from a compatible type.

Parameters

source The data source (e.g., path, string, vector<byte>).

Definition at line 143 of file data_source.h.

◆ data_source() [2/6]

template<data_source_compatible_type T>

df0::data_source::data_source ( T && source )

inlineexplicit

Constructs a data_source by moving from a compatible type.

Parameters

source The data source to move from.

Definition at line 152 of file data_source.h.

◆ data_source() [3/6]

template<data_source_compatible_type T>

df0::data_source::data_source	(	const T &	source,
		file_extension	file_extension )

inlineexplicit

Constructs a data_source with an explicit file extension.

Parameters

source	The data source.
file_extension	The file extension to associate with the data.

Definition at line 162 of file data_source.h.

◆ data_source() [4/6]

template<data_source_compatible_type T>

df0::data_source::data_source	(	T &&	source,
		file_extension	file_extension )

inlineexplicit

Constructs a data_source by moving, with an explicit file extension.

Parameters

source	The data source to move from.
file_extension	The file extension to associate with the data.

Definition at line 172 of file data_source.h.

◆ data_source() [5/6]

template<data_source_compatible_type T>

df0::data_source::data_source	(	const T &	source,
		mime_type	mime_type,
		confidence	mime_type_confidence )

inlineexplicit

Constructs a data_source with an initial MIME type and confidence.

Parameters

source	The data source.
mime_type	The initial MIME type.
mime_type_confidence	The confidence level for the initial MIME type.

Definition at line 183 of file data_source.h.

◆ data_source() [6/6]

template<data_source_compatible_type T>

df0::data_source::data_source	(	T &&	source,
		mime_type	mime_type,
		confidence	mime_type_confidence )

inlineexplicit

Constructs a data_source by moving, with an initial MIME type and confidence.

Parameters

source	The data source to move from.
mime_type	The initial MIME type.
mime_type_confidence	The confidence level for the initial MIME type.

Definition at line 196 of file data_source.h.

Member Function Documentation

◆ add_mime_type()

void df0::data_source::add_mime_type	(	mime_type	mt,
		confidence	c )

inline

Adds a mime type with a confidence level.

Parameters

mt	The mime type to add.
c	The confidence level.

Definition at line 308 of file data_source.h.

◆ assert_not_encrypted()

void df0::data_source::assert_not_encrypted ( ) const

inline

Asserts that the data source is not encrypted.

Definition at line 497 of file data_source.h.

◆ file_extension()

std::optional< df0::file_extension > df0::data_source::file_extension ( ) const

inline

Returns the file extension if available.

Definition at line 480 of file data_source.h.

◆ has_highest_confidence_mime_type_in()

bool df0::data_source::has_highest_confidence_mime_type_in ( const std::vector< mime_type > & mts ) const

inline

Checks if the highest confidence mime type is present in the given list.

Parameters

mts	The list of mime types to check against.

Definition at line 490 of file data_source.h.

◆ highest_confidence_mime_type()

std::optional< mime_type > df0::data_source::highest_confidence_mime_type ( ) const

inline

Returns the MIME type with the highest confidence.

Definition at line 266 of file data_source.h.

◆ highest_confidence_mime_type_info()

std::optional< std::pair< mime_type, confidence > > df0::data_source::highest_confidence_mime_type_info ( ) const

inline

Returns the MIME type with the highest confidence and its confidence level.

Note: Because a file extension can map to multiple valid MIME type aliases with the same confidence level, this method uses a deterministic alphabetical tie-breaker (e.g., application/xml wins over text/xml) to guarantee consistent cross-platform behavior.

Definition at line 248 of file data_source.h.

◆ highest_mime_type_confidence()

confidence df0::data_source::highest_mime_type_confidence ( ) const

inline

Returns the highest confidence level found among detected MIME types.

Definition at line 276 of file data_source.h.

◆ id()

unique_identifier df0::data_source::id ( ) const

inline

Returns the unique identifier for this data source.

Definition at line 236 of file data_source.h.

◆ istream()

std::shared_ptr< std::istream > df0::data_source::istream ( ) const

inline

Returns an input stream for reading the data.

Definition at line 467 of file data_source.h.

◆ mime_type_confidence()

confidence df0::data_source::mime_type_confidence ( mime_type mt ) const

inline

Returns the confidence level for a specific MIME type.

Definition at line 294 of file data_source.h.

◆ path()

std::optional< std::filesystem::path > df0::data_source::path ( ) const

inline

Returns the file path if the source is a file, otherwise std::nullopt.

Definition at line 472 of file data_source.h.

◆ span()

std::span< const std::byte > df0::data_source::span ( std::optional< length_limit > limit = std::nullopt ) const

inline

Returns the content as a span of bytes.

Parameters

limit Optional limit on the number of bytes to return.

Returns: A span over the data.

Definition at line 371 of file data_source.h.

◆ string()

std::string df0::data_source::string ( std::optional< length_limit > limit = std::nullopt ) const

inline

Returns the content as a string.

Parameters

limit Optional limit on the number of characters to return.

Returns: A string containing the data.

Definition at line 404 of file data_source.h.

◆ string_view()

std::string_view df0::data_source::string_view ( std::optional< length_limit > limit = std::nullopt ) const

inline

Returns the content as a string_view.

This method avoids memory allocation if the underlying source is already in memory (e.g. string, vector<byte>). If the source is a stream or file, it may load data.

Parameters

limit Optional limit on the number of characters to return.

Definition at line 447 of file data_source.h.

Member Data Documentation

◆ mime_types

std::unordered_map<mime_type, confidence> df0::data_source::mime_types

Map of detected MIME types and their confidence levels.

Definition at line 316 of file data_source.h.

The documentation for this class was generated from the following file:

data_source.h

Public Member Functions

Public Attributes

Detailed Description

Constructor & Destructor Documentation

◆ data_source() [1/6]

◆ data_source() [2/6]

◆ data_source() [3/6]

◆ data_source() [4/6]

◆ data_source() [5/6]

◆ data_source() [6/6]

Member Function Documentation

◆ add_mime_type()

◆ assert_not_encrypted()

◆ file_extension()

◆ has_highest_confidence_mime_type_in()

◆ highest_confidence_mime_type()

◆ highest_confidence_mime_type_info()

◆ highest_mime_type_confidence()

◆ id()

◆ istream()

◆ mime_type_confidence()

◆ path()

◆ span()

◆ string()

◆ string_view()

Member Data Documentation

◆ mime_types