Trino:修订间差异
跳到导航
跳到搜索
此页面具有访问限制。如果您看见此消息,则说明您没有权限访问此页面。
(→Worker) |
(→常用指令) |
||
(未显示同一用户的30个中间版本) | |||
第4行: | 第4行: | ||
PrestoSQL是fork自[[Presto]]的專案,後來成立公司。在2018年時Facebook出面阻止使用其商標營利,改名為Trino。 | PrestoSQL是fork自[[Presto]]的專案,後來成立公司。在2018年時Facebook出面阻止使用其商標營利,改名為Trino。 | ||
== 硬體需求 == | |||
測試時用<code>t4g.small</code>就跑的動了,正式環境可以考慮先用<code>c6g.medium</code>。 | |||
== 安裝 == | == 安裝 == | ||
第10行: | 第14行: | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
sudo apt install -y openjdk-11-jre; cd /tmp; wget https://repo1.maven.org/maven2/io/trino/trino-server/ | TRINO_VERSION=363 sudo apt install -y openjdk-11-jre python-is-python3; sudo apt clean; cd /tmp; wget https://repo1.maven.org/maven2/io/trino/trino-server/${TRINO_VERSION}/trino-server-${TRINO_VERSION}.tar.gz; cd /opt; sudo tar xvf /tmp/trino-server-${TRINO_VERSION}.tar.gz; sudo mv trino-server-${TRINO_VERSION} trino-server; cd trino-server; sudo mkdir data etc var; sudo chown -R nobody:nogroup /opt/trino-server/ | ||
</syntaxhighlight> | </syntaxhighlight> | ||
== 設定 == | == 設定 == | ||
=== 防火牆 === | |||
{| class="sortable wikitable" | |||
! Port | |||
! Usage | |||
! Comment | |||
|- | |||
| 8080/tcp | |||
| Trino | |||
| | |||
|} | |||
=== 設定檔 === | |||
基本上都是照著官方的安裝文件<ref>{{Cite web |url= https://trino.io/docs/current/installation/deployment.html |title=Deploying Trino |language=en |accessdate=2021-08-10}}</ref>設定,這邊有些設定要注意的: | 基本上都是照著官方的安裝文件<ref>{{Cite web |url= https://trino.io/docs/current/installation/deployment.html |title=Deploying Trino |language=en |accessdate=2021-08-10}}</ref>設定,這邊有些設定要注意的: | ||
第28行: | 第46行: | ||
sudo tee etc/jvm.config > /dev/null <<EOF | sudo tee etc/jvm.config > /dev/null <<EOF | ||
-server | -server | ||
- | -Xmx1536M | ||
-XX:-UseBiasedLocking | -XX:-UseBiasedLocking | ||
-XX:+UseG1GC | -XX:+UseG1GC | ||
第42行: | 第60行: | ||
-Djdk.nio.maxCachedBufferSize=2000000 | -Djdk.nio.maxCachedBufferSize=2000000 | ||
EOF | EOF | ||
sudo chown -R nobody:nogroup /opt/trino-server/etc/ | |||
</syntaxhighlight> | |||
然後是systemd的設定: | |||
<syntaxhighlight lang="bash"> | |||
sudo tee /lib/systemd/system/trino.service > /dev/null <<EOF | |||
# | |||
# Install into /lib/systemd/system | |||
[Unit] | |||
Description=Trino daemon | |||
After=remote-fs.target | |||
[Service] | |||
ExecStart=/opt/trino-server/bin/launcher run | |||
Group=nogroup | |||
Restart=always | |||
RestartSec=60 | |||
Type=simple | |||
User=nobody | |||
[Install] | |||
WantedBy=multi-user.target | |||
EOF | |||
sudo systemctl daemon-reload | |||
</syntaxhighlight> | </syntaxhighlight> | ||
第47行: | 第90行: | ||
=== Coordinator === | === Coordinator === | ||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
第55行: | 第96行: | ||
node-scheduler.include-coordinator=false | node-scheduler.include-coordinator=false | ||
http-server.http.port=8080 | http-server.http.port=8080 | ||
query.max-memory= | query.max-memory=1TB | ||
discovery.uri=http://example.net:8080 | discovery.uri=http://example.net:8080 | ||
EOF | EOF | ||
sudo chown -R nobody:nogroup /opt/trino-server/etc/ | |||
</syntaxhighlight> | </syntaxhighlight> | ||
=== Worker === | === Worker === | ||
這邊<code>query.max-memory-per-node</code>與<code>query.max-total-memory-per-node</code>建議設成 記憶體 一半的大小,預設是JVM heap的10% 大小 ,很明顯太小 。 | |||
<syntaxhighlight lang="bash"> | <syntaxhighlight lang="bash"> | ||
第70行: | 第110行: | ||
coordinator=false | coordinator=false | ||
http-server.http.port=8080 | http-server.http.port=8080 | ||
query.max-memory= | query.max-memory=1TB | ||
query.max-memory-per-node= | query.max-memory-per-node=32GB | ||
query.max-total-memory-per-node= | query.max-total-memory-per-node=32GB | ||
discovery.uri=http://example.net:8080 | discovery.uri=http://example.net:8080 | ||
EOF | EOF | ||
sudo chown -R nobody:nogroup /opt/trino-server/etc/ | |||
</syntaxhighlight> | |||
== 啟動 == | |||
<syntaxhighlight lang="bash"> | |||
sudo systemctl daemon-reload; sudo systemctl enable trino; sudo service trino start | |||
</syntaxhighlight> | |||
== 常用指令 == | |||
先連上Trino: | |||
<syntaxhighlight lang="bash"> | |||
trino --server trino-coordinator-1.example.com:8080 | |||
</syntaxhighlight> | |||
另外可以看系統的各種狀況: | |||
<syntaxhighlight lang="sql"> | |||
SHOW SCHEMAS FROM system; | |||
SHOW TABLES FROM system.runtime; | |||
SELECT * FROM system.runtime.nodes; | |||
</syntaxhighlight> | |||
目前系統裡面的Query: | |||
<syntaxhighlight lang="sql"> | |||
SELECT * FROM system.runtime.queries; | |||
</syntaxhighlight> | </syntaxhighlight> | ||
2022年3月28日 (一) 16:59的最新版本
Trino是一套SQL查询引擎,旧名PrestoSQL,是源自于Presto的引擎。
简介
PrestoSQL是fork自Presto的专案,后来成立公司。在2018年时Facebook出面阻止使用其商标营利,改名为Trino。
硬件需求
测试时用t4g.small
就跑的动了,正式环境可以考虑先用c6g.medium
。
安装
在Ubuntu类的系统上没有套件可以装,这边建议直接下载并且安装到/opt
下(另外需要Java 11):
TRINO_VERSION=363 sudo apt install -y openjdk-11-jre python-is-python3; sudo apt clean; cd /tmp; wget https://repo1.maven.org/maven2/io/trino/trino-server/${TRINO_VERSION}/trino-server-${TRINO_VERSION}.tar.gz; cd /opt; sudo tar xvf /tmp/trino-server-${TRINO_VERSION}.tar.gz; sudo mv trino-server-${TRINO_VERSION} trino-server; cd trino-server; sudo mkdir data etc var; sudo chown -R nobody:nogroup /opt/trino-server/
设定
防火墙
Port | Usage | Comment |
---|---|---|
8080/tcp | Trino |
设定档
基本上都是照着官方的安装文件[1]设定,这边有些设定要注意的:
etc/node.properties
的node.environment
。etc/jvm.config
的-Xmx1G
。
cd /opt/trino-server; sudo mkdir data etc; sudo tee etc/node.properties > /dev/null <<EOF
node.environment=production
node.id=$(uuidgen)
node.data-dir=/opt/trino-server/data
EOF
sudo tee etc/jvm.config > /dev/null <<EOF
-server
-Xmx1536M
-XX:-UseBiasedLocking
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+ExplicitGCInvokesConcurrent
-XX:+ExitOnOutOfMemoryError
-XX:+HeapDumpOnOutOfMemoryError
-XX:-OmitStackTraceInFastThrow
-XX:ReservedCodeCacheSize=512M
-XX:PerMethodRecompilationCutoff=10000
-XX:PerBytecodeRecompilationCutoff=10000
-Djdk.attach.allowAttachSelf=true
-Djdk.nio.maxCachedBufferSize=2000000
EOF
sudo chown -R nobody:nogroup /opt/trino-server/etc/
然后是systemd的设定:
sudo tee /lib/systemd/system/trino.service > /dev/null <<EOF
#
# Install into /lib/systemd/system
[Unit]
Description=Trino daemon
After=remote-fs.target
[Service]
ExecStart=/opt/trino-server/bin/launcher run
Group=nogroup
Restart=always
RestartSec=60
Type=simple
User=nobody
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
接下来会依照不同的机器设定不同的值:
Coordinator
cd /opt/trino-server; sudo tee etc/config.properties > /dev/null <<EOF
coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=8080
query.max-memory=1TB
discovery.uri=http://example.net:8080
EOF
sudo chown -R nobody:nogroup /opt/trino-server/etc/
Worker
这边query.max-memory-per-node
与query.max-total-memory-per-node
建议设成内存一半的大小,预设是JVM heap的10%大小,很明显太小。
cd /opt/trino-server; sudo tee etc/config.properties > /dev/null <<EOF
coordinator=false
http-server.http.port=8080
query.max-memory=1TB
query.max-memory-per-node=32GB
query.max-total-memory-per-node=32GB
discovery.uri=http://example.net:8080
EOF
sudo chown -R nobody:nogroup /opt/trino-server/etc/
启动
sudo systemctl daemon-reload; sudo systemctl enable trino; sudo service trino start
常用指令
先连上Trino:
trino --server trino-coordinator-1.example.com:8080
另外可以看系统的各种状况:
SHOW SCHEMAS FROM system;
SHOW TABLES FROM system.runtime;
SELECT * FROM system.runtime.nodes;
目前系统里面的Query:
SELECT * FROM system.runtime.queries;
相关连结
参考文献
- ↑ Deploying Trino. [2021-08-10] (English).