Multiscale Electrophysiology Format
Multiscale Electrophysiology Format (MEF) was developed to handle the large amounts of data produced by large-scale electrophysiology in human and animal subjects. MEF can store any time series data up to 24 bits in length, and employs lossless range encoded difference compression. Subject identifying information in the file header can be encrypted using 128-bit AES encryption in order to comply with HIPAA requirements for patient privacy when transmitting data across an open network.
Compressed data is stored in independent blocks to allow direct access to the data, facilitate parallel processing and limit the effects of potential damage to files. Data fidelity is ensured by a 32-bit cyclic redundancy check in each compressed data block using the Koopman polynomial (0xEB31D82E), which has a Hamming distance of from 4 to 114 kbits.
A formal specification [1] and source code [2] are available online.
See also
References
Sources
- Martin, GNN. Range encoding: an algorithm for removing redundancy from a digitised message. Video & Data Recoding Conference, Southampton, 1979.
- Koopman, P. 32-Bit Cyclic Redundancy Codes for Internet Applications. The International Conference on Dependable Systems and Networks (June 2002). 459.
- Brinkmann, BH; et al. (2009). "Large-scale electrophysiology: acquisition, compression, encryption, and storage of big data". Journal of Neuroscience Methods. 180: 185–192. doi:10.1016/j.jneumeth.2009.03.022. PMC 2720128
. PMID 19427545.