zio.config.ConfigDescriptorModule
ConfigDescriptor
Related Doc:
package ConfigDescriptorModule
<*> is an alias to function zip.
<*> is an alias to function zip.
This is used to represent retrieving the config as a tuple.
Example:
val config: ConfigDescriptor[(String, Int)] = string("URL") <*> int("PORT")
This is a description that represents the following: Retrieve values of URL and PORT which are String and Int respectively, and return a tuple.
The above description is equivalent to
val config2: ConfigDescriptor[(String, Int)] = (string("URL") |@| int("PORT")).tupled
Using |@| over <> avoids nested tuples.
<+> is an alias to function orElseEither.
<+> is an alias to function orElseEither.
This is used to represent fall-back logic when we describe config retrievals. Unlike orElse, the
the fall-back config parameter can have a different type in orElseEither.
Example:
val config: ConfigDescriptor[Either[Int, String]] = int("MONTH") <+> string("MONTH")
This is a description that represents the following:
Try to retrieve the value of a MONTH as an Int, and if there is a format error, try and retrieve it as a String.
Detail:
We know ConfigDescriptor is a program that describes the retrieval of a set of configuration parameters.
In the below example, we can either depend on a configuration called password
or a token both being of the same type, in this case, a String.
Example:
Given:
final case class BasicAuth(username: String, password: String) final case class OAuth(clientId: String, secret: String) val basicAuth: ConfigDescriptor[BasicAuth] = (string("USERNAME") |@| string("PASSWORD"))(BasicAuth.apply, BasicAuth.unapply) val oAuth: ConfigDescriptor[OAuth] = (string("CLIENT_ID") |@| string("SECRET"))(OAuth.apply, OAuth.unapply) val myConfig: ConfigDescriptor[Either[BasicAuth, OAuth]] = basicAuth <+> oAuth
then,
val source = ConfigSource.fromMap(Map("USERNAME" -> "abc", "PASSWORD" -> "cde") read(myConfig from source)
returns:
Left(BasicAuth("abc", "def")
Similarly,
val source = ConfigSource.fromMap(Map("CLIENT_ID" -> "xyz", "SECRET" -> "afg==") read(myConfig from source)
returns:
Right(OAuth("xyz", "afg==")
<> is an alias to function orElse.
<> is an alias to function orElse.
This is used to represent fall-back logic when we describe config retrievals.
Example:
val config: ConfigDescriptor[String] = string("token") <> string("password")
This is a description that represents the following: Try to retrieve the value of a parameter called "token", or else try to retrieve the value of parameter called "password"
We know ConfigDescriptor is a program that describes the retrieval
of a set of configuration parameters.
In the below example, we can either depend on a configuration called
password or a token both being of the same type, in this case, a String.
Example:
final case class Config(tokenOrPassword: String, port: Int) object Config { val databaseConfig: ConfigDescriptor[Config] = (string("token") <> string("password") |@| int("PORT"))(Config.apply, Config.unapply) }
?? is an alias to describe which allows us to inject additional
documentation to the configuration parameters.
?? is an alias to describe which allows us to inject additional
documentation to the configuration parameters.
Example:
val port = int("PORT") ?? "database port"
A more detailed example:
Here is a program that describes (or a ConfigDescriptor that represents)
reading a USERNAME which is a String and PORT which is an Int,
and load it to a case class Config
final case class Config(userName: String, port: Int) object Config { val dbConfig: ConfigDescriptor[Config] = (string("USERNAME") |@| int("PORT"))(Config.apply, Config.unapply) }
Later on you decided to annotate each one of them with extra documentation,
which is later seen in error messages if config retrieval
is a failure, and it's also used while documenting your configuration
using ConfigDocsModule
val dbConfigWithDoc: ConfigDescriptor[Config] = (string("USERNAME") ?? "db username" |@| int("PORT") ?? "db port" )(Config.apply, Config.unapply)
If you try and read this config from an empty source, it emits an error message with the details you provided.
import zio.config._, ConfigDescriptor._ println( read(Config.databaseConfig from ConfigSource.fromMap(Map.empty)) )
returns:
╥ ╠══╦══╗ ║ ║ ║ ║ ║ ╠─MissingValue ║ ║ ║ path: PORT ║ ║ ║ Details: db port, value of type int ║ ║ ▼ ║ ║ ║ ╠─MissingValue ║ ║ path: USERNAME ║ ║ Details: db username, value of type string ║ ▼ ▼
Or, you can also use a common documentation for an entire set of config parameters.
val detailedConfigDescriptor: ConfigDescriptor[Config] = configDescriptor ?? "Configuration related to database"
apply is a "convenient" version of
transformOrFail, and is mainly used to apply and unapply case-classes from elements.
apply is a "convenient" version of
transformOrFail, and is mainly used to apply and unapply case-classes from elements.
Given A and B the apply can be used to
convert a ConfigDescriptor[A] to ConfigDescriptor[B].
Let's define a simple ConfigDescriptor,
that talks about retrieving a String configuration (example: PORT).
val port: ConfigDescriptor[Int] = int("PORT")
For some reason, if we decide to convert Int to String,
Int => StringA => B becomes and that will always work.
However, the reverse relationship which is String => Int is no more a
total function, unless it is represented as String => Either[error, Int] or String => Option[Int].
That is, not all elements of set String can be converted to Int.
To overcome this, we can do s => Try(s.toInt).toOption to mark the possibility of errors.
This is a function of the type: B => Option[A].
val portString: ConfigDescriptor[Int] = port.apply[String](_.toString, (s: String) => Try(s.toInt).toOption)
With the above information you can read a port from a source and convert to a string, and write the stringified port back to a source representation.
READ:
import zio.config._ val configSource: ConfigSource = ConfigSource.fromMap(Map.empty) // Read val configResult: Either[ReadError[String], String] = read(portString from configSource)
Write back the config:
Now given a stringified port "8888", you can also write it
back to the source safely. It is safe write, and it is guaranteed that we can
read back this config successfully. This is because, during writing back, it
ensures that it is a real port that can be converted to Int.
import zio.config.typesafe._ // NOTE: toJson is available only through zio-config-typesafe module (import zio.config.typesafe._) val writtenBack: Either[String, PropertyTree[String, String]] = write(portString, "8888") val jsonRepr: Either[String, String] = writtenBack.map(_.toJson) // { "port" : "8888" } val mapRepr: Either[String, Map[String, String]] = writtenBack.map(_.flattenString()) // Map("port" -> "8888") // And even this will work import zio.config.typesafe._ val port: Int = 8080 port.toJson(portString) // { "port" : "8080" }
default function allows us to inject default values to existing config
default function allows us to inject default values to existing config
Example:
val port = int("PORT").default(8080)
A more detailed example:
Here is a program that describes (or a ConfigDescriptor that represents) reading a USERNAME which is a String and PORT which is an Int,
and load it to a case class Config
final case class Config(userName: String, port: Int) object Config { val dbConfig: ConfigDescriptor[Config] = (string("USERNAME") |@| int("PORT").default(8080))(Config.apply, Config.unapply) }
In the above case, if username is missing, then it prints out an error, however if PORT is missing, it falls back to 8080.
In fact you can give a default to an entire config
For example:
final case class Config(userName: String, port: Int) object Config { val dbConfig: ConfigDescriptor[Config] = (string("USERNAME") |@| int("PORT"))(Config.apply, Config.unapply).default(Config("jon", 8080)) }
Sometimes this can be used along with automatic derivation supported through zio-config-magnolia.
import zio.config.magnolia._, zio.config._, ConfigDescriptor._ final case class Config(userName: String, port: Int) object Config { val dbConfig: ConfigDescriptor[Config] = descriptor[Config].default(Config("jon", 8080)) } // This is a typical example where we mix auto derivation with manual definitions.
describe function allows us to inject additional documentation to the configuration parameters.
describe function allows us to inject additional documentation to the configuration parameters.
Example:
val port = int("PORT") ?? "database port"
A more detailed example:
Here is a program that describes (or a ConfigDescriptor that represents) reading a USERNAME which is a String and a PORT which is an Int,
and load it to a case class Config
final case class Config(userName: String, port: Int) object Config { val dbConfig: ConfigDescriptor[Config] = (string("USERNAME") |@| int("PORT"))(Config.apply, Config.unapply) }
Later on you decided to annotate each one of them with extra documentation, which is later seen in error messages if config retrieval
is a failure, and it's also used while documenting your configuration using ConfigDocsModule
val dbConfigWithDoc: ConfigDescriptor[Config] = (string("USERNAME") ?? "db username" |@| int("PORT") ?? "db port")(Config.apply, Config.unapply)
If you try and read this config from an empty source, it emits an error message with the details you provided.
import zio.config._, ConfigDescriptor._ println( read(Config.databaseConfig from ConfigSource.fromMap(Map.empty)) )
returns:
╥ ╠══╦══╗ ║ ║ ║ ║ ║ ╠─MissingValue ║ ║ ║ path: PORT ║ ║ ║ Details: db port, value of type int ║ ║ ▼ ║ ║ ║ ╠─MissingValue ║ ║ path: USERNAME ║ ║ Details: db username, value of type string ║ ▼ ▼
Or, you can also use a common documentation for an entire set of config parameters.
val detailedConfigDescriptor: ConfigDescriptor[Config] = configDescriptor ?? "Configuration related to database"
Attach a source to the ConfigDescriptor.
Attach a source to the ConfigDescriptor.
Example:
val config = string("PORT") from ConfigSource.fromMap(Map.empty)
config is a description that says there is a key called PORT in constant map source.
You can use the description to read the config
val either: Either[ReadError[String], String] = read(config)
You can also tag a source per config field, or one global source to an entire config.
final case class Config(userName: String, port: Int) object Config { val dbConfig: ConfigDescriptor[Config] = (string("USERNAME") |@| int("PORT"))(Config.apply, Config.unapply) }
In the above example, dbConfig is not associated with any source. By default the source will be empty.
To attach a config (especially during a read operation) is as easy as:
read(dbConfig from ConfigSource.fromMap(Map("USERNAME" -> "afs", "PORT" -> "8080")) // Right(Config("afs", 8080))
Obviously, source can be attached independently.
val configSource1: ConfigSource = ??? val configSource2: ConfigSource = ??? val dbConfig = (string("USERNAME") from configSource1 |@| int("PORT"))(Config.apply, Config.unapply) from configSource2
In the above case read(dbConfig) implies, zio-config tries to fetch USERNAME from configSource1, and if it
fails (i.e, missing value) it goes and try with the global config which is configSource2.
PORT will be fetched from configSource2.
You can also try various sources for each field.
val configSource1: ConfigSource = ??? // Example: ConfigSource.fromMap(...) val configSource2: ConfigSource = ??? // Example: ConfigSource.fromTypesafeConfig(...) val dbConfig = (string("USERNAME") from configSource1.orElse(configSource2) |@| int("PORT") from configSource2.orElse(configSource1))(Config.apply, Config.unapply) from configSource2
mapKey allows user to convert the keys in a ConfigDescriptor.
mapKey allows user to convert the keys in a ConfigDescriptor.
Example:
Consider you have a config that looks like this
case class Config(url: String, port: Int) object Config { val config = (string("dbUrl") |@| int("dbPort"))(Config.apply, Config.unapply) } val source = Map( "DB_URL" -> "abc.com", "DB_PORT" -> "9090" ) read(Config.config from ConfigSource.fromMap(source)) // will fail since the source doesn't have the keys dbUrl and dbPort, but it has only DB_URL and DB_PORT
The above config retrieval fails since the keys dbUrl and dbPOrt exist, but it has only DB_URL and DB_PORT. In this situation, instead of rewriting the config we can do
import zio.config._, ConfigDescriptor._ read(Config.config.mapKey(key => toSnakeCase(key).toUpperCase) from ConfigSource.fromMap(source)) // Right(Config("abc.com", 9090))
optional function allows us to tag a configuration parameter as optional.
optional function allows us to tag a configuration parameter as optional.
It implies, even if it's missing configuration will be a success.
Example:
val port: ConfigDescriptor[Option[Int]] = int("PORT").optional
A more detailed example:
Here is a program that describes (or a ConfigDescriptor that represents) reading a USERNAME
which is a String and PORT which is an Int,
and load it to a case class Config
final case class Config(userName: String, port: Option[Int]) object Config { val dbConfig: ConfigDescriptor[Config] = (string("USERNAME") |@| int("PORT").optional)(Config.apply, Config.unapply) }
The fact that it is an optional in error messages if config retrieval
is a failure, and it's also used while documenting your configuration using ConfigDocsModule
val dbConfigWithDoc: ConfigDescriptor[Config] = (string("USERNAME") ?? "db username" |@| int("PORT") ?? "db port")(Config.apply, Config.unapply)
import zio.config._, ConfigDescriptor._ val source = ConfigSource.fromMap(Map("USERNAME" -> "af")) println( read(Config.databaseConfig from source) )
returns:
Config("af", None)
Similarly,
val source = ConfigSource.fromMap(Map("USERNAME" -> "af", "PORT" -> "8888")) println( read(Config.databaseConfig from source) )
returns:
Config("af", Some(8888))
However, if you have given PORT, but it's not an integer,
then it fails giving you the error details.
Within the error message, it will also specify the fact that the parameter is an optional parameter, giving you an indication that you can either fix the parameter, or you can completely skip this parameter.
Example:
import zio.config._, ConfigDescriptor._ val source = ConfigSource.fromMap(Map("USERNAME" -> "af", "PORT" -> "abc")) println( read(Config.databaseConfig from source) )
returns:
╥
╠══╗
║ ║
║ ╠─FormatError
║ ║ cause: Provided value is abc, expecting the type int
║ ║ path: PORT
║ ▼
▼Another interesting behaviour, but we often forget about optional parameters is when there is a presence of a part of the set of the config parameters representing a product, where the product itself is optional.
Example:
final case class DbConfig(port: Int, host: String) object DbConfig { val dbConfig: ConfigDescriptor[Option[DbConfig]] = (int("PORT") |@| string("HOST"))(DbConfig.apply, DbConfig.unapply).optional }
In this case if "PORT" is present in the source, but "HOST" is absent,
then config retrieval will be a failure and not None.
Similarly, if "HOST" is present but "PORT" is absent,
the config retrieval will be a failure and not None.
If both of the parameters are absent in the source, then the
config retrieval will be a success and the output will be
None. If both of them is present, then output will be Some(DbConfig(..))
orElse is used to represent fall-back logic when we describe config retrievals.
orElse is used to represent fall-back logic when we describe config retrievals.
Example:
val config: ConfigDescriptor[String] = string("token") <> string("password")
This is a description that represents the following: Try to retrieve the value of a parameter called "token", or else try to retrieve the value of parameter called "password"
We know ConfigDescriptor is a program that describes the retrieval of a set of configuration parameters.
In the below example, we can either depend on a configuration called password or a token both being of the same type, in this case, a String.
Example:
final case class Config(tokenOrPassword: String, port: Int) object Config { val databaseConfig: ConfigDescriptor[Config] = (string("token") <> string("password") |@| int("PORT"))(Config.apply, Config.unapply) }
Note: orElse is different from orElseEither.
While orElse fall back to parameter which is of the same type of the original config parameter,
orElseEither can fall back to a different type giving us Either[A, B].
orElse will be useful in retrieving configuration that are represented as coproducted (sealed trait). However,
it may become fairly verbose, such that usage zio-config-magnolia to derive the config automatically, will become a reasonable
alternative.
orElseEither is used to represent fall-back logic when we describe config retrievals.
orElseEither is used to represent fall-back logic when we describe config retrievals. Unlike orElse,
the fall-back config parameter can have a different type in orElseEither.
Example:
val config: ConfigDescriptor[Either[Int, String]] = int("MONTH") <+> string("MONTH")
This is a description that represents the following:
Try to retrieve the value of a MONTH as an Int, and if there is a format error, try and retrieve it as a String.
Detail:
We know ConfigDescriptor is a program that describes the retrieval of a set of configuration parameters.
In the below example, we can either depend on a configuration called password
or a token both being of the same type, in this case, a String.
Example:
Given:
final case class BasicAuth(username: String, password: String) final case class OAuth(clientId: String, secret: String) val basicAuth: ConfigDescriptor[BasicAuth] = (string("USERNAME") |@| string("PASSWORD"))(BasicAuth.apply, BasicAuth.unapply) val oAuth: ConfigDescriptor[OAuth] = (string("CLIENT_ID") |@| string("SECRET"))(OAuth.apply, OAuth.unapply) val myConfig: ConfigDescriptor[Either[BasicAuth, OAuth]] = basicAuth <+> oAuth
then,
val source = ConfigSource.fromMap(Map("USERNAME" -> "abc", "PASSWORD" -> "cde") read(myConfig from source)
returns:
Left(BasicAuth("abc", "def")
Similarly,
val source = ConfigSource.fromMap(Map("CLIENT_ID" -> "xyz", "SECRET" -> "afg==") read(myConfig from source)
returns:
Right(OAuth("xyz", "afg==")
Fetch all the sources associated with a ConfigDescriptor.
Convert a ConfigDescriptor[A] to a config descriptor of a case class
Convert a ConfigDescriptor[A] to a config descriptor of a case class
This works when A is a single value and B is a single parameter case class with
the same type of parameter, or if A is an tuple and B is a case class with
matching number of parameters and the same types.
See the following example of reading a USERNAME which is a String and PORT which is an Int,
and load it to a case class Config:
final case class Config(userName: String, port: Int) object Config { val dbConfig: ConfigDescriptor[Config] = (string("USERNAME") |@| int("PORT")).to[Config] }
Note that the alternative of passing (Config.apply, Config.unapply) to transform the config descriptor
is not compatible with Scala 3.
Given A and B, f: A => B, and g: B => A, then
transform allows us to transform a ConfigDescriptor[A] to ConfigDescriptor[B].
Given A and B, f: A => B, and g: B => A, then
transform allows us to transform a ConfigDescriptor[A] to ConfigDescriptor[B].
Example :
transform is useful especially when you define newtypes.
final case class Port(port: Int) extends AnyVal val config: ConfigDescriptor[Port] = int("PORT").transform[Port](Port.apply, _.int)
While to: A => B (in this case, Int => Port) is used to read to a Port case class,
from: B => A (which is, Port => Int) is used when we want to write Port directly to a source representation.
Example:
import zio.config.typesafe._ // as toJson is available only through zio-config-typesafe module val writtenBack: Either[String, PropertyTree[String, String]] = write(config, Port(8888)) val jsonRepr: Either[String, String] = writtenBack.map(_.toJson) // { "port" : "8888" } val mapRepr: Either[String, Map[String, String]] = writtenBack.map(_.flattenString()) // Map("port" -> "8888")
Given A and B, transformOrFail function is used to convert a ConfigDescriptor[A] to ConfigDescriptor[B].
Given A and B, transformOrFail function is used to convert a ConfigDescriptor[A] to ConfigDescriptor[B].
It is important to note that both to and fro is fallible, allowing us to represent
almost all possible relationships.
Example:
Let's define a simple ConfigDescriptor, that talks about retrieving a S3Path ( a bucket and prefix in AWS s3).
Given you want to retrieve an S3Path from ConfigSource. Given a string, converting it to S3Path can fail, and even converting
S3Path to a String can fail as well.
import java.time.DateTimeFormatter import java.time.LocalDate final case class S3Path(bucket: String , prefix: String, partition: LocalDate) { def convertToString(partitionPattern: String): Either[String, String] = Try { DateTimeFormatter.ofPattern(partitionPattern).format(partition) }.toEither .map(dateStr => s"${bucket}/${prefix}/${dateStr}").swap.map(_.getMessage).swap } object S3Path { def fromStr(s3Path: String): Either[String, S3Path] = { val splitted = s3Path.split("/").toList if (splitted.size > 3) Left("Invalid s3 path") else for { bucket <- splitted.headOption.toRight("Empty s3 path") prefix <- splitted.lift(1).toRight("Invalid prefix, or empty prefix in s3 path") partition <- splitted.lift(2).toRight("Empty partition").flatMap(dateStr => LocalDate.parse(dateStr)) } yield S3Path(bucket, prefix, partition) } } val s3PathConfig: ConfigDescriptor[S3Path] = string("S3_PATH").transformEither[S3Path](S3Path.fromStr, _.convertToString("yyyy-MM-dd"))
Untag all sources associated with a ConfigDescriptor.
Untag all sources associated with a ConfigDescriptor.
As we know ConfigDescriptor represents a program that describes the retrieval of config parameters.
In fact, the same program can be used to write back the config in various shapes.
Either case, a ConfigDescriptor can exist without a Source attached.
Example:
val stringConfig: ConfigDescriptor[String] = string("USERNAME")
Later on we can read the config by attaching a source.
val result = read(stringConfig from ConfigSource.fromMap(Map.empty))
However, you can attach a source to the configDescriptor at an earlier stage.
For example:
val stringConfig: ConfigDescriptor[String] = string("USERNAME") from ConfigSource.fromMap(Map.empty)
Later on, you can simply read it using:
val result = read(stringConfig) Using unsourced, you can now untag the source from stringConfig.
val stringConfigNoSource: ConfigDescriptor[String] = stringConfig.unsourced
This can be useful in test cases where you want to remove a source and attach a different source.
Example:
val testSource: ConfigSource = ConfigSource.fromMap(Map(..)) val result = stringConfig.unsourced from testSource
updateSource can update the source of an existing ConfigDescriptor
updateSource can update the source of an existing ConfigDescriptor
Example:
val configSource1 = ConfigSource.fromMap(Map.empty) val configSource2 = ConfigSource.fromMap(Map("USERNAME" -> "abc")) val config = string("USERNAME") from configSource1 val updatedConfig = config updateSource (_ orElse configSource2)
In the above example, we update the existing ConfigDescriptor to try another ConfigSource called configSource2, if it fails to retrieve the value of USERNAME from configSource1.
zip is used to represent retrieving the config as a tuple.
zip is used to represent retrieving the config as a tuple.
Example:
val config: ConfigDescriptor[(String, Int)] = string("URL") <*> int("PORT")
This is a description that represents the following: Retrieve values of URL and PORT which are String and Int respectively, and return a tuple.
The above description is equivalent to
val config2: ConfigDescriptor[(String, Int)] = (string("URL") |@| int("PORT")).tupled
Using |@| over <> avoids nested tuples.
zipWith is similar to xmapEither but the function
is mostly used as an internal implementation
in zio-config.
zipWith is similar to xmapEither but the function
is mostly used as an internal implementation
in zio-config. For the same reason, users hardly need zipWith.
Instead take a look at xmapEither (or transformEither).
xmapEither2 deals with retrieving two configurations represented
by ConfigDescriptor[A] and ConfigDescriptor[B],
and corresponding to and from functions converting a tuple (A, B) to C and it's
reverse direction, to finally form a ConfigDescriptor[C].
Those who are familiar with Applicative in Functional programming,
xmapEither2 almost takes the form of Applicative:
F[A] => F[B] => (A, B) => C => F[C].
Implementation detail:
This is used to implement sequence (traverse)
behaviour of ConfigDescriptor[A]
|@| is a ConfigDescriptor builder.
|@| is a ConfigDescriptor builder. We know ConfigDescriptor
is a program that describes the retrieval of a set of configuration parameters.
Below given is a ConfigDescriptor that describes the retrieval of a single config.
val port: ConfigDescriptor[String] = string("PORT")
However, in order to retrieve multiple configuration parameters,
we can make use of |@|.
Example:
final case class Config(userName: String, port: Int) object Config { val dbConfig: ConfigDescriptor[Config] = (string("USERNAME") |@| int("PORT"))(Config.apply, Config.unapply) }
Details:
(string("USERNAME") |@| int("PORT"))(Config.apply, Config.unapply)
is equal to
(string("USERNAME") |@| int("PORT")).apply((a, b) => Config.apply(a, b), Config.unapply)