Datadog kafka_consumer check fails if consumer_groups value is empty

[kivagant-ba]

I had a conversation with DD support and they explained to me the root cause of the Exception we got in logs:

2019-08-30 13:20:55 UTC | CORE | ERROR | (pkg/collector/runner/runner.go:294 in work) | Error running check kafka_consumer: [{"message": "", "traceback": "Traceback (most recent call last):\n File \"/opt/datadog-agent/embedded/lib/python2.7/site-packages/datadog_checks/base/checks/base.py\", line 503, in run\n self.check(instance)\n File \"/opt/datadog-agent/embedded/lib/python2.7/site-packages/datadog_checks/kafka_consumer/legacy_0_10_2.py\", line 78, in check\n self._validate_explicit_consumer_groups(consumer_groups)\n File \"/opt/datadog-agent/embedded/lib/python2.7/site-packages/datadog_checks/kafka_consumer/legacy_0_10_2.py\", line 550, in _validate_explicit_consumer_groups\n assert isinstance(val, dict)\nAssertionError\n"}] 

It turned out that our cookbook settings caused YAML error:

consumer_groups = {}

// ...dynamic generation of consumer_groups which does not always return something

node.default['datadog']['kafka_consumer']['instances'] = [{ 
'kafka_connect_str' => '127.0.0.1:9092', 
'zk_connect_str' => node['kafka']['broker']['zookeeper.connect'], 
'consumer_groups' => consumer_groups, 
}] 

The resulting YAML looks like this:

instances:
  - kafka_connect_str: 127.0.0.1:9092
    zk_connect_str: my.zookeeper.url:2181
    consumer_groups:

I feel like for some reason this check doesn't work. Am I wrong?

https://github.com/DataDog/chef-datadog/blob/master/templates/default/kafka_consumer.yaml.erb#L11

<% if i['consumer_groups'] -%>

[remeh]

Hi @kivagrant-ba ! I think the check

<% if i['consumer_groups'] -%>

will have the expected behavior if consumer_groups is null but it is an empty object in your example.

[kivagant-ba]

It will and that is actually the problem. An empty object technically means the same.