Difficult upgrades to 1.27.0 (breaking change to config and maps in config)

[EddieWhi]

Hi,

First off, thanks for Kaffe, great library and many thanks for sharing.

I recently updated one of our services using Kaffe to 1.27+, in part to unlock the per-topic configuration feature but found the upgrade process painful.

Initially, it was a bit unexpected that the configuration change was breaking as the version change was technically minor, but most challenging was the fact that we have base config and overrides per environment and a runtime.exs config too.

We're also running in an umbrella application, and I was looking to define individual consumers in each umbrella app config, effectively putting some consumers in one config and other consumers in other configs.

Given the consumer configs are stored in maps, the configs do not merge and we have to write some pretty nasty stuff:

kaffe_consumers =
  Process.get({Config, :config}) |> Keyword.get(:kaffe, []) |> Keyword.get(:consumers, %{})

config :kaffe,
  consumers:
    Map.put(
      kaffe_consumers,
      "consumer_name",
      Map.get(kaffe_consumers, "core", [])
      |> Keyword.merge(
        endpoints: ["some-production-endpoint": 9092],
      )
    )

I started a fork to get this config to something more idiomatic, and also found that it was fairly simple to support the single topic configuration, fully backward compat with pre 1.27 versions with the added advantage that the fallback `:consumer' configuration can be used as a default fallback for multiple consumers.

The fork also still supports the maps/string keys setup as in the current version too.

I'm more than happy to turn this into a PR if you're interested and also very keen in your thoughs and feedback.

https://github.com/spreedly/kaffe/compare/master...EddieWhi:kaffe:feat/keyword-list-consumer-config?expand=1

Using keyword lists can override properties of a single consumer

config :kaffe, consumers: [
  consumer_name: [
    endpoints: ["some-production-endpoint": 9092]
  ]
]

Can define properties for use across multiple consumers:

config :kaffe, consumer: [
    endpoints: ["some-production-endpoint": 9092]
  ]
]

config :kaffe, consumers: [
  consumer_one: [
    topics: ["topic1"]
  ],
  consumer_two: [
    topics: ["topic2"]
  ]
]

Let me know what you think.