taoensso.faraday
Clojure DynamoDB client. Originally adapted from Rotary by James Reeves.
Ref. https://github.com/weavejester/rotary (Rotary),
http://goo.gl/22QGA (DynamoDBv2 API)
Definitions:
* Attribute - [<name> <val>] pair
* Item - Collection of attributes
* Table - Collection of items
* Primary key - Partition (hash) key or
Partition (hash) AND sort (range) key
*attr-multi-vs?*
dynamic
Treat attribute vals as expansions rather than literals?
nil => use `attr-multi-vs?-default` (currently `true` though this may be
changed in future to better support DDB's new collection types,
Ref. https://github.com/ptaoussanis/faraday/issues/63).
batch-get-item
(batch-get-item client-opts requests & [{:keys [return-cc? span-reqs attr-multi-vs?], :as opts}])
Retrieves a batch of items in a single request.
Limits apply, Ref. http://goo.gl/Bj9TC.
(batch-get-item client-opts
{:users {:prim-kvs {:name "alice"}}
:posts {:prim-kvs {:id [1 2 3]}
:attrs [:timestamp :subject]
:consistent? true}
:friends {:prim-kvs [{:catagory "favorites" :id [1 2 3]}
{:catagory "recent" :id [7 8 9]}]}})
:span-reqs - {:max _ :throttle-ms _} allows a number of requests to
automatically be stitched together (to exceed throughput limits, for example).
batch-write-item
(batch-write-item client-opts requests & [{:keys [return-cc? span-reqs attr-multi-vs?], :as opts}])
Executes a batch of Puts and/or Deletes in a single request.
Limits apply, Ref. http://goo.gl/Bj9TC. No transaction guarantees are
provided, nor conditional puts. Request execution order is undefined.
(batch-write-item client-opts
{:users {:put [{:user-id 1 :username "sally"}
{:user-id 2 :username "jane"}]
:delete [{:user-id [3 4 5]}]}})
:span-reqs - {:max _ :throttle-ms _} allows a number of requests to
automatically be stitched together (to exceed throughput limits, for example).
create-table
(create-table client-opts table-name hash-keydef & [{:keys [block?], :as opts}])
Creates a table with options:
hash-keydef - [<name> <#{:s :n :ss :ns :b :bs}>].
:range-keydef - [<name> <#{:s :n :ss :ns :b :bs}>].
:throughput - {:read <units> :write <units>}.
:billing-mode - :provisioned | :pay-per-request ; defaults to provisioned
:block? - Block for table to actually be active?
:lsindexes - [{:name _ :range-keydef _
:projection <#{:all :keys-only [<attr> ...]}>}].
:gsindexes - [{:name _ :hash-keydef _ :range-keydef _
:projection <#{:all :keys-only [<attr> ...]}>
:throughput _}].
:stream-spec - {:enabled? <default true if spec is present>
:view-type <#{:keys-only :new-image :old-image :new-and-old-images}>}
delete-item
(delete-item client-opts table prim-kvs & [{:keys [return expected return-cc?], :as opts}])
Deletes an item from a table by its primary key.
See `put-item` for option docs.
delete-table
(delete-table client-opts table)
Deletes a table, go figure.
describe-stream
(describe-stream client-opts stream-arn & [{:keys [limit start-shard-id], :as opts}])
Returns a map describing a stream, or nil if the stream doesn't exist.
describe-table
(describe-table client-opts table)
Returns a map describing a table, or nil if the table doesn't exist.
describe-ttl
(describe-ttl client-opts table-name)
Returns a map describing the TTL configuration for a table, or nil if
the table does not exist.
ensure-table
(ensure-table client-opts table-name hash-keydef & [opts])
Creates a table iff it doesn't already exist.
ensure-ttl
(ensure-ttl client-opts table-name key-name)
Activates TTL with the given key-name iff the TTL configuration is
not already set in this way.
ex
DynamoDB API exceptions. Use #=(ex _) for `try` blocks, etc.
freeze
(freeze x)
(freeze x opts)
Forces argument of any type to be subject to automatic de/serialization with
Nippy.
get-item
(get-item client-opts table prim-kvs & [opts])
Retrieves an item from a table by its primary key with options:
prim-kvs - {<hash-key> <val>} or {<hash-key> <val> <range-key> <val>}.
:attrs - Attrs to return, [<attr> ...].
:proj-expr - Projection expression as a string
:expr-attr-names - Map of strings for ExpressionAttributeNames
:consistent? - Use strongly (rather than eventually) consistent reads?
get-stream-records
(get-stream-records client-opts shard-iterator & [{:keys [limit], :as opts}])
ISerializable
protocol
Extensible protocol for mapping Clojure vals to AttributeValue objects.
members
serialize
(serialize this)
items-by-attrs
(items-by-attrs attr-or-attrs item-or-items)
Groups one or more items by one or more attributes, returning a map of form
{<attr-val> <item> ...} or {{<attr> <val> ...} <item>}.
Good for collecting batch or query/scan items results into useable maps,
indexed by their 'primary key' (which, remember, may consist of 1 OR 2 attrs).
list-streams
(list-streams client-opts & [{:keys [start-arn table-name limit], :as opts}])
Returns a lazy sequence of stream descriptions. Each item is a map of:
{:stream-arn - Stream identifier string
:table-name - The table name for the stream}
Options:
:start-arn - The stream identifier to start listing from (exclusive)
:table-name - List only the streams for <table-name>
:limit - Retrieve streams in batches of <limit> at a time
list-tables
(list-tables client-opts)
Returns a lazy sequence of table names.
put-item
(put-item client-opts table item & [{:keys [return expected return-cc? cond-expr], :as opts}])
Adds an item (Clojure map) to a table with options:
:return - e/o #{:none :all-old}
:cond-expr - "attribute_exists(attr_name) AND|OR ..."
:expr-attr-names - {"#attr_name" "name"}
:expr-attr-vals - {":attr_value" "value"}
:expected - DEPRECATED in favor of `:cond-expr`,
{<attr> <#{:exists :not-exists [<comparison-operator> <value>] <value>}> ...}
With comparison-operator e/o #{:eq :le :lt :ge :gt :begins-with :between}.
query
(query client-opts table prim-key-conds & [{:keys [last-prim-kvs query-filter span-reqs return index order limit consistent? return-cc?], :as opts}])
Retrieves items from a table (indexed) with options:
prim-key-conds - {<key-attr> [<comparison-operator> <val-or-vals>] ...}.
:last-prim-kvs - Primary key-val from which to eval, useful for paging.
:query-filter - {<key-attr> [<comparison-operator> <val-or-vals>] ...}.
:proj-expr - Projection expression string
:filter-expr - Filter expression string
:expr-attr-names - Expression attribute names, as a map of {"#attr_name" "name"}
:expr-attr-vals - Expression attribute values, as a map {":attr_value" "value"}
:span-reqs - {:max _ :throttle-ms _} controls automatic multi-request
stitching.
:return - e/o #{:all-attributes :all-projected-attributes :count
[<attr> ...]}.
:index - Name of a local or global secondary index to query.
:order - Index scaning order e/o #{:asc :desc}.
:limit - Max num >=1 of items to eval (≠ num of matching items).
Useful to prevent harmful sudden bursts of read activity.
:consistent? - Use strongly (rather than eventually) consistent reads?
(create-table client-opts :my-table [:name :s]
{:range-keydef [:age :n] :block? true})
(do (put-item client-opts :my-table {:name "Steve" :age 24})
(put-item client-opts :my-table {:name "Susan" :age 27}))
(query client-opts :my-table {:name [:eq "Steve"]
:age [:between [10 30]]})
=> [{:age 24, :name "Steve"}]
comparison-operators e/o #{:eq :le :lt :ge :gt :begins-with :between}.
For unindexed item retrievel see `scan`.
Ref. http://goo.gl/XfGKW for query+scan best practices.
remove-empty-attr-vals
Alpha, subject to change.
Util to help remove (or coerce to nil) empty val types prohibited by DDB,
Ref. http://goo.gl/Xg85pO. See also `freeze` as an alternative.
scan
(scan client-opts table & [{:keys [attr-conds last-prim-kvs span-reqs return limit total-segments filter-expr segment return-cc?], :as opts}])
Retrieves items from a table (unindexed) with options:
:attr-conds - {<attr> [<comparison-operator> <val-or-vals>] ...}.
:expr-attr-names - Expression attribute names, as a map of {"#attr_name" "name"}
:expr-attr-vals - Expression attribute names, as a map of {":attr" "value"}
:filter-expr - Filter expression string
:index - Index name to use.
:proj-expr - Projection expression as a string
:limit - Max num >=1 of items to eval (≠ num of matching items).
Useful to prevent harmful sudden bursts of read activity.
:last-prim-kvs - Primary key-val from which to eval, useful for paging.
:span-reqs - {:max _ :throttle-ms _} controls automatic multi-request
stitching.
:return - e/o #{:all-attributes :all-projected-attributes :count
[<attr> ...]}.
:total-segments - Total number of parallel scan segments.
:segment - Calling worker's segment number (>=0, <=total-segments).
:consistent? - Use strongly (rather than eventually) consistent reads?
comparison-operators e/o #{:eq :le :lt :ge :gt :begins-with :between :ne
:not-null :null :contains :not-contains :in}.
(create-table client-opts :my-table [:name :s]
{:range-keydef [:age :n] :block? true})
(do (put-item client-opts :my-table {:name "Steve" :age 24})
(put-item client-opts :my-table {:name "Susan" :age 27}))
(scan client-opts :my-table {:attr-conds {:age [:in [24 27]]}})
=> [{:age 24, :name "Steve"} {:age 27, :name "Susan"}]
For automatic parallelization & segment control see `scan-parallel`.
For indexed item retrievel see `query`.
Ref. http://goo.gl/XfGKW for query+scan best practices.
scan-parallel
(scan-parallel client-opts table total-segments & [opts])
Like `scan` but starts a number of worker threads and automatically handles
parallel scan options (:total-segments and :segment). Returns a vector of
`scan` results.
Ref. http://goo.gl/KLwnn (official parallel scan documentation).
shard-iterator
(shard-iterator client-opts stream-arn shard-id iterator-type & [{:keys [seq-num], :as opts}])
Returns the iterator string that can be used in the get-stream-records call
or nil when the stream or shard doesn't exist.
transact-get-items
(transact-get-items client-opts raw-req)
transact-write-items
(transact-write-items client-opts raw-req)
Executes a transaction comprising puts, updates, deletes and cond-checks; either all succeed or all fail.
Execution order is the order in the items vector.
For example:
(far/transact-write-items client-opts
{:items [[:cond-check {:table-name ttable
:prim-kvs {:id 300}
:cond-expr "attribute_exists(#id)"
:expr-attr-names {"#id" "id"}}]
[:put {:table-name ttable
:item i2
:cond-expr "attribute_not_exists(#id)"
:expr-attr-names {"#id" "id"}}]
[:delete {:table-name ttable
:prim-kvs {:id 303}
:cond-expr "attribute_exists(#id)"
:expr-attr-names {"#id" "id"}}]
[:update {:table-name ttable
:prim-kvs {:id 300}
:update-expr "SET #name = :name"
:cond-expr "attribute_exists(#id) AND #name = :oldname"
:expr-attr-names {"#id" "id", "#name" "name"}
:expr-attr-vals {":oldname" "foo", ":name" "foobar"}}]]})
returns {:cc-units {<table> <consumed-capacity>}}}
update-item
(update-item client-opts table prim-kvs & [{:keys [update-map return expected return-cc? cond-expr update-expr], :as opts}])
Updates an item in a table by its primary key with options:
prim-kvs - {<hash-key> <val>} or {<hash-key> <val> <range-key> <val>}
:update-map - DEPRECATED in favor of `:update-expr`,
{<attr> [<#{:put :add :delete}> <optional value>]}
:cond-expr - "attribute_exists(attr_name) AND|OR ..."
:update-expr - "SET #attr_name = :attr_value"
:expr-attr-names - {"#attr_name" "name"}
:expr-attr-vals - {":attr_value" "value"}
:return - e/o #{:none :all-old :updated-old :all-new :updated-new}
:expected - DEPRECATED in favor of `:cond-expr`,
{<attr> <#{:exists :not-exists [<comparison-operator> <value>] <value>}> ...}
With comparison-operator e/o #{:eq :le :lt :ge :gt :begins-with :between}.
update-table
(update-table client-opts table update-opts & [{:keys [span-reqs]}])
Returns a future which updates table and returns the post-update table
description.
Update opts:
:throughput - {:read <units> :write <units>}
:billing-mode - :provisioned | :pay-per-request ; defaults to provisioned
:gsindexes - {:operation ; e/o #{:create :update :delete}
:name ; Required
:throughput ; Only for :update / :create
:hash-keydef ; Only for :create
:range-keydef ;
:projection ; e/o #{:all :keys-only [<attr> ...]}}
:stream-spec - {:enabled? ;
:view-type ; e/o #{:keys-only :new-image :old-image :old-and-new-images}}
Only one global secondary index operation can take place at a time.
In order to change a stream view-type, you need to disable and re-enable the stream.
update-ttl
(update-ttl client-opts table-name enabled? key-name)
Updates the TTL configuration for a table.
with-attr-multi-vs
macro
(with-attr-multi-vs & body)
with-thaw-opts
macro
(with-thaw-opts opts & body)
without-attr-multi-vs
macro
(without-attr-multi-vs & body)