Confluent/ZooKeeper:修订间差异
跳到导航
跳到搜索
此页面具有访问限制。如果您看见此消息,则说明您没有权限访问此页面。
(创建页面,内容为“== 相關連結 == * Apache ZooKeeper * Confluent”) |
(→首段) |
||
第1行: | 第1行: | ||
== 設定ZooKeeper == | |||
修改<code>/etc/kafka/zookeeper.properties</code>,其中IP address需要填寫對應的位置: | |||
<syntaxhighlight lang="ini"> | |||
# | |||
tickTime=2000 | |||
dataDir=/var/lib/zookeeper/ | |||
clientPort=2181 | |||
initLimit=5 | |||
syncLimit=2 | |||
server.1=1.2.3.4:2888:3888 | |||
server.2=5.6.7.8:2888:3888 | |||
server.3=9.10.11.12:2888:3888 | |||
autopurge.snapRetainCount=3 | |||
autopurge.purgeInterval=24 | |||
</syntaxhighlight> | |||
新增<code>/var/lib/zookeeper/myid</code>,每一台都需要不同,<code>1</code>或<code>2</code>或<code>3</code>: | |||
<syntaxhighlight lang="ini"> | |||
1 | |||
</syntaxhighlight> | |||
然後修改檔案擁有人: | |||
<syntaxhighlight lang="bash"> | |||
sudo chown cp-kafka:confluent /var/lib/zookeeper/myid | |||
</syntaxhighlight> | |||
目前的ZooKeeper(Confluent 2.11版內的ZooKeeper)預設值是使用512 MB的記憶體,但主機有7.5 GB的記憶體,所以會想要讓ZooKeeper可以用7 GB,因此需要修改ZooKeeper的JVM參數。這邊需要新增<code>/lib/systemd/system/confluent-zookeeper.service.d/30-options.conf</code>(目錄可能需要自己建立): | |||
<syntaxhighlight lang="ini"> | |||
[Service] | |||
Environment=KAFKA_HEAP_OPTS="-Xmx7g -Xms7g" | |||
</syntaxhighlight> | |||
理論上就可以啟動了: | |||
<syntaxhighlight lang="bash"> | |||
sudo systemctl daemon-reload | |||
sudo service confluent-zookeeper start | |||
sudo service confluent-zookeeper status | |||
</syntaxhighlight> | |||
可以看輸出的資訊判斷系統狀態: | |||
<syntaxhighlight lang="bash"> | |||
echo stat | nc 127.0.0.1 2181 | |||
</syntaxhighlight> | |||
或是直接透過指令操作測試: | |||
<syntaxhighlight lang="bash"> | |||
zookeeper-shell 127.0.0.1:2181 | |||
</syntaxhighlight> | |||
接下來可以將TCP Port 2181建立對應的TCP Load Balancer(像是用ELB)。 | |||
=== 認證 === | |||
如果需要讓ZooKeeper啟用認證,需要先建立對應的帳號與密碼字串(這個例子裡面是<code>admin</code>與<code>password</code>): | |||
<syntaxhighlight lang="bash"> | |||
java -cp "$(echo /usr/share/java/kafka/* | sed 's/ /:/g')" org.apache.zookeeper.server.auth.DigestAuthenticationProvider admin:password | |||
</syntaxhighlight> | |||
會產生像是這樣的輸出,其中後面的那串值是重點: | |||
<syntaxhighlight lang="text"> | |||
admin:password->admin:bjkZ9W+M82HUZ9xb8/Oy4cmJGfg= | |||
</syntaxhighlight> | |||
然後直接放進設定內: | |||
<syntaxhighlight lang="bash"> | |||
KAKFA_OPTS=-Dzookeeper.DigestAuthenticationProvider.superDigest=admin:bjkZ9W+M82HUZ9xb8/Oy4cmJGfg= | |||
</syntaxhighlight> | |||
這樣就可以在<code>zookeeper-shell</code>裡面認證: | |||
<syntaxhighlight lang="bash"> | |||
addauth digest admin:password | |||
</syntaxhighlight> | |||
== 相關連結 == | == 相關連結 == | ||
* [[Apache ZooKeeper]] | * [[Apache ZooKeeper]] | ||
* [[Confluent]] | * [[Confluent]] |
2019年3月13日 (三) 03:08的版本
设定ZooKeeper
修改/etc/kafka/zookeeper.properties
,其中IP address需要填写对应的位置:
#
tickTime=2000
dataDir=/var/lib/zookeeper/
clientPort=2181
initLimit=5
syncLimit=2
server.1=1.2.3.4:2888:3888
server.2=5.6.7.8:2888:3888
server.3=9.10.11.12:2888:3888
autopurge.snapRetainCount=3
autopurge.purgeInterval=24
新增/var/lib/zookeeper/myid
,每一台都需要不同,1
或2
或3
:
1
然后修改档案拥有人:
sudo chown cp-kafka:confluent /var/lib/zookeeper/myid
目前的ZooKeeper(Confluent 2.11版内的ZooKeeper)预设值是使用512 MB的记忆体,但主机有7.5 GB的记忆体,所以会想要让ZooKeeper可以用7 GB,因此需要修改ZooKeeper的JVM参数。这边需要新增/lib/systemd/system/confluent-zookeeper.service.d/30-options.conf
(目录可能需要自己建立):
[Service]
Environment=KAFKA_HEAP_OPTS="-Xmx7g -Xms7g"
理论上就可以启动了:
sudo systemctl daemon-reload
sudo service confluent-zookeeper start
sudo service confluent-zookeeper status
可以看输出的资讯判断系统状态:
echo stat | nc 127.0.0.1 2181
或是直接透过指令操作测试:
zookeeper-shell 127.0.0.1:2181
接下来可以将TCP Port 2181建立对应的TCP Load Balancer(像是用ELB)。
认证
如果需要让ZooKeeper启用认证,需要先建立对应的帐号与密码字串(这个例子里面是admin
与password
):
java -cp "$(echo /usr/share/java/kafka/* | sed 's/ /:/g')" org.apache.zookeeper.server.auth.DigestAuthenticationProvider admin:password
会产生像是这样的输出,其中后面的那串值是重点:
admin:password->admin:bjkZ9W+M82HUZ9xb8/Oy4cmJGfg=
然后直接放进设定内:
KAKFA_OPTS=-Dzookeeper.DigestAuthenticationProvider.superDigest=admin:bjkZ9W+M82HUZ9xb8/Oy4cmJGfg=
这样就可以在zookeeper-shell
里面认证:
addauth digest admin:password