Kafka 动态配置认证

前言

在 Kafka 中一向都是采用无认证体系去访问 Kafka 应用,这会导致应用不安全,尤其我们计划采用 Kafka 作为企业服务总线的管道.那么我们需要 Kafka 开启认证系统.保证队列与数据的安全.目前 Kafka提供了以下的认证方案.

• SASL/GSSAPI (Kerberos) - starting at version 0.9.0.0

• SASL/PLAIN - starting at version 0.10.0.0

• SASL/SCRAM-SHA-256 and SASL/SCRAM-SHA-512 - starting at version 0.10.2.0

• SASL/OAUTHBEARER - starting at version 2.0

为了 Kafka 在新增用户时不需要重启所以我们将采用 SASL/SCRAM 方案,从我当前学习进度来看,其将用户安全放入了 Zookeeper 中,Kafka 在连接时去 Zookeeper 获取认证数据.那么接下来我们就开始配置该内容

官方文档地址

https://kafka.apache.org/documentation/#security_sasl_scram

环境准备

• Kafka

• ZooKeeper

步骤

1. 创建Broker 进行通信的 admin 管理员账户

2. 创建 普通账户

3. 配置 KafkaBroker

说明

我们将创建

admin 账户,账户为 admin 密码为admin-secret

user 账户,账户为 alice 密码为 alice-secret

创建Admin 账户 admin

用户 Broker 之间通信

$ bin/kafka-configs.sh --zookeeper localhost:2181 --alter --add-config 'SCRAM-SHA-256=[password=admin-secret],SCRAM-SHA-512=[password=admin-secret]' --entity-type users --entity-name admin

Zookeeper TLS

$ bin/kafka-configs.sh --zookeeper localhost:2181 --zk-tls-config-file zk_tls_config.properties --alter --add-config 'SCRAM-SHA-256=[password=admin-secret],SCRAM-SHA-512=[password=admin-secret]' --entity-type users --entity-name admin

创建User账户 alice

$ bin/kafka-configs.sh --zookeeper localhost:2181 --alter --add-config 'SCRAM-SHA-256=[iterations=8192,password=alice-secret],SCRAM-SHA-512=[password=alice-secret]' --entity-type users --entity-name alice

Zookeeper TLS

$ bin/kafka-configs.sh --zookeeper localhost:2182 --zk-tls-config-file zk_tls_config.properties --describe --entity-type users --entity-name alice

配置 Kafka Broker

在项目目录下创建 kafka_server_jaas.conf

KafkaServer {
    org.apache.kafka.common.security.scram.ScramLoginModule required
    username="admin"
    password="admin-secret";
};

修改启动程序kafka-server-start.sh

将以下 JVM 添加之启动服务

-Djava.security.auth.login.config=/etc/kafka/kafka_server_jaas.conf
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

if [ $# -lt 1 ];
then
  echo "USAGE: $0 [-daemon] server.properties [--override property=value]*"
  exit 1
fi
base_dir=$(dirname $0)

if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then
    export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
fi

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G -Djava.security.auth.login.config=$base_dir/../config/kafka_server_jaas.conf"
fi

EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'}

COMMAND=$1
case $COMMAND in
  -daemon)
    EXTRA_ARGS="-daemon "$EXTRA_ARGS
    shift
    ;;
  *)
    ;;
esac

exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

配置 server.properties

listeners=SASL_PLAINTEXT://:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=SCRAM-SHA-256 (or SCRAM-SHA-512)
sasl.enabled.mechanisms=SCRAM-SHA-256 (or SCRAM-SHA-512)

连接验证 Franz

本人将采用 Franz 进行验证应用连通性

附件

展示用户脚本

$ bin/kafka-configs.sh --zookeeper localhost:2181 --describe --entity-type users

删除用户脚本

$ bin/kafka-configs.sh --zookeeper localhost:2181 --alter --delete-config 'SCRAM-SHA-512' --entity-type users --entity-name alice