Introduction

Any problem in Computer Science can be solved with another level of indirection

Layered file protocols is a high-level interface for byte-oriented file handles with layering. File descriptors are composed and stacked at run-time, adding features such as pre-fetching, indices, cloud access, and file format support at each level.

Layered file protocols, lfp, provides

• A simple interface for application and library developers

• Multiple protocols for different file implementations and formats

• A developer interface for adding new protocols with C++

https://github.com/equinor/layered-file-protocols/

License

Layered file protocols is licensed under the LGPL. See the LICENSE file for details.

INSTALLATION

• Installation
• Changelog

USAGE

• Getting started
  • First example - cat.c
  • Building with cmake
  • Second example - tif-cat.c

API REFERENCE

• API design
  • Namespacing
  • Layers visualization
  • Function signatures
  • Status codes and backwards compatibility
• Functions
  • lfp_close()
  • lfp_readinto()
  • lfp_seek()
  • lfp_tell()
  • lfp_ptell()
  • lfp_peel()
  • lfp_peek()
  • lfp_eof()
  • lfp_errormsg()
• Status codes
  • lfp_status
    • lfp_status::LFP_OK
    • lfp_status::LFP_OKINCOMPLETE
    • lfp_status::LFP_NOTIMPLEMENTED
    • lfp_status::LFP_LEAF_PROTOCOL
    • lfp_status::LFP_NOTSUPPORTED
    • lfp_status::LFP_UNHANDLED_EXCEPTION
    • lfp_status::LFP_IOERROR
    • lfp_status::LFP_RUNTIME_ERROR
    • lfp_status::LFP_INVALID_ARGS
    • lfp_status::LFP_PROTOCOL_FATAL_ERROR
    • lfp_status::LFP_PROTOCOL_TRYRECOVERY
    • lfp_status::LFP_PROTOCOL_FAILEDRECOVERY
    • lfp_status::LFP_EOF
    • lfp_status::LFP_UNEXPECTED_EOF

PROTOCOLS

• cfile
  • lfp_cfile()
• rp66
  • lfp_rp66_open()
• tapeimage
  • lfp_tapeimage_open()

PROTOCOL DEVELOPMENT

• Writing protocols
• lfp_protocol interface
  • lfp_protocol
    • lfp_protocol::close()
    • lfp_protocol::readinto()
    • lfp_protocol::seek()
    • lfp_protocol::tell()
    • lfp_protocol::ptell()
    • lfp_protocol::peel()
    • lfp_protocol::peek()
    • lfp_protocol::errmsg()
    • lfp_protocol::errmsg()
• Exceptions
  • error
  • not_implemented()
  • leaf_protocol()
  • not_supported()
  • io_error()
  • runtime_error()
  • invalid_args()
  • protocol_fatal()
  • protocol_failed_recovery()
  • unexpected_eof()
• Automation
  • unique_lfp
    • unique_lfp::operator lfp_protocol*()
    • unique_lfp::operator const lfp_protocol*()
    • unique_lfp::operator->()
    • unique_lfp::operator->()
    • unique_lfp::close()
    • unique_lfp::release()
    • unique_lfp::get()
    • unique_lfp::operator bool()
  • advance()
  • advance()

Why not use FILE?

Layered file protocols was design to scratch our own itch - we want to deal with files that sometimes use the tapeimage protocol, but not always. With lfp, our applications can have a simple and straight-forward code path, and not really notice the presence of tape markers and other such nonsense. There is no standard way to add new drivers to FILE, except really being on the inside of libc.

Technical details

The interface is deliberately different from the C file library. While it does make porting from FILE slightly more work, there are subtle differences in guarantees and usage, which means it is necessary for lfp to detach itself from the FILE interface.

The consumer interface is a C API. This is to provide the best and most ergonomic experience for creating bindings for other languages, and to have an easier time maintaing binary compatibility. Public functions will never be removed, but may be deprecated.

Protocols are layered with dependency injection, and are implemented in C++. For an API reference, a description, and rationale, see the Writing protocols section.

Many protocols are provided and bundled with lfp, and while it is sometimes useful for very application-specific protocols to be implemented and maintained in-application, the most sustainable approach is to contribute them upstream and maintain them in the lfp tree.