framed.std.serialization
Utilities for reading/writing formatted data to disk
In general:
reader-like - Anything that can be coerced to an open java.io.Reader
using clojure.java.io/reader
writer-like - Anything that can be coerced to an open java.io.Writer
using clojure.java.io/writer
istream-like - Anything that can be coerced to an open java.io.InputStream
using clojure.java.io/input-stream
ostream-like - Anything that can be coerced to an open java.io.OutputStream
using clojure.java.io/output-streamavro-schema
Alias over Abracad for specifying an Avro RecordSchema
See https://github.com/damballa/abracad for format documentation
Ex:
(def schema
(avro-schema {:type "record"
:name "User"
:fields [{:name "age" :type "long"}
{:name "email" :type "string"}]})coll->FressianSeq
(coll->FressianSeq coll)
Construct a reified view of coll (encoded in Fressian) that is
is both seqable and accessible as a file on disk
Ex:
(def f (coll->FressianSeq [1 2 3 4 5]))
(seq f) ; => '(1 2 3 4 5)
(clojure.java.io/file f) ; => #<File /tmp/...>
coll->NippySeq
(coll->NippySeq coll)
Construct a reified view of coll (encoded in Nippy) that is
is both seqable and accessible as a file on disk
Ex:
(def n (coll->NippySeq [1 2 3 4 5]))
(seq n) ; => '(1 2 3 4 5)
(clojure.java.io/file n) ; => #<File /tmp/...>
decode-avro
:: schema -> bytes -> record
default-transit-encoding
The default encoding used by read-transit and write-transit.
Defaults to :msgpack.
encode-avro
:: schema -> record -> bytes
file->FressianSeq
(file->FressianSeq x)
Construct a FressianSeq directly from file-like x
file->NippySeq
(file->NippySeq x)
Construct a NippySeq directly from file-like x
read-avro
(read-avro x)
Return a lazy seq of values from Avro-encoded File or InputStream
TODO: if passed an InputStream, will consume entire stream
in-memory (arbitrarily large files are safe)
read-csv
(read-csv reader-like)(read-csv drop-header reader-like)
Return a lazy sequence of rows as vectors of strings
drop-header - Boolean, whether or not to ignore header row (default: false)
read-edn
(read-edn reader-like)
Parse the contents of reader-like as a EDN value
read-fressian
(read-fressian istream-like)
Return a lazy seq of values from Fressian-encoded input
read-json
(read-json reader-like)
Parse the contents of reader-like as a JSON value
read-jsonl
(read-jsonl reader-like)
Return a lazy sequence of objects from newline-delimited JSON input
(JSON Lines)
See http://jsonlines.org/
read-nippy
(read-nippy istream-like)
Return a lazy seq of values from Nippy-encoded input
read-transit
(read-transit istream-like)(read-transit encoding istream-like)
Parse the contents of reader-like as a Transit value
write-avro
(write-avro ostream-like schema coll)
Write coll of records each conforming to schema to ostream-like
Ex:
(def schema
(avro-schema {:type "record"
:name "User"
:fields [{:name "age" :type "long"}
{:name "email" :type "string"}]})
(write-avro "test.avro" schema [{:age 27 :email "foo@example.com"}
{:age 32 :email "bar@example.com"}])
; => #<File test.avro>write-csv
(write-csv writer-like labels rows)
labels - Seq of column labels, ex ['user_id' 'user_email'] (can be empty)
Note that commas and newlines/carriage returns in labels will be
replaced with semicolons
rows - Seq of row data seqswrite-edn
(write-edn ostream-like data)
Write arbitrary data as EDN to ostream-like and return ostream-like
write-fressian
(write-fressian ostream-like coll)
Write a coll of values as Fressian to ostream-like and return ostream-like
write-json
(write-json writer-like data)
Write arbitrary data as JSON to writer-like and return writer-like
write-jsonl
(write-jsonl writer-like coll)
Write arbitrary data as JSONL to writer-like and return writer-like
write-nippy
(write-nippy ostream-like coll)
Write a coll of values as Nippy to ostream-like and return ostream-like
write-transit
(write-transit ostream-like data)(write-transit ostream-like encoding data)
encoding - one of :json, :msgpack