Trino

来自Gea-Suan Lin's Wiki
跳到导航 跳到搜索

Trino是一套SQL查询引擎,旧名PrestoSQL,是源自于Presto的引擎。

简介

PrestoSQL是fork自Presto的专案,后来成立公司。在2018年时Facebook出面阻止使用其商标营利,改名为Trino。

硬件需求

测试时用t4g.small就跑的动了,正式环境可以考虑先用c6g.medium

安装

Ubuntu类的系统上没有套件可以装,这边建议直接下载并且安装到/opt下(另外需要Java 11):

TRINO_VERSION=363 sudo apt install -y openjdk-11-jre python-is-python3; sudo apt clean; cd /tmp; wget https://repo1.maven.org/maven2/io/trino/trino-server/${TRINO_VERSION}/trino-server-${TRINO_VERSION}.tar.gz; cd /opt; sudo tar xvf /tmp/trino-server-${TRINO_VERSION}.tar.gz; sudo mv trino-server-${TRINO_VERSION} trino-server; cd trino-server; sudo mkdir data etc var; sudo chown -R nobody:nogroup /opt/trino-server/

设定

防火墙

Port Usage Comment
8080/tcp Trino

设定档

基本上都是照着官方的安装文件[1]设定,这边有些设定要注意的:

  • etc/node.propertiesnode.environment
  • etc/jvm.config-Xmx1G
cd /opt/trino-server; sudo mkdir data etc; sudo tee etc/node.properties > /dev/null <<EOF
node.environment=production
node.id=$(uuidgen)
node.data-dir=/opt/trino-server/data
EOF
sudo tee etc/jvm.config > /dev/null <<EOF
-server
-Xmx1536M
-XX:-UseBiasedLocking
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+ExplicitGCInvokesConcurrent
-XX:+ExitOnOutOfMemoryError
-XX:+HeapDumpOnOutOfMemoryError
-XX:-OmitStackTraceInFastThrow
-XX:ReservedCodeCacheSize=512M
-XX:PerMethodRecompilationCutoff=10000
-XX:PerBytecodeRecompilationCutoff=10000
-Djdk.attach.allowAttachSelf=true
-Djdk.nio.maxCachedBufferSize=2000000
EOF
sudo chown -R nobody:nogroup /opt/trino-server/etc/

然后是systemd的设定:

sudo tee /lib/systemd/system/trino.service > /dev/null <<EOF
#
# Install into /lib/systemd/system
[Unit]
Description=Trino daemon
After=remote-fs.target

[Service]
ExecStart=/opt/trino-server/bin/launcher run
Group=nogroup
Restart=always
RestartSec=60
Type=simple
User=nobody

[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload

接下来会依照不同的机器设定不同的值:

Coordinator

cd /opt/trino-server; sudo tee etc/config.properties > /dev/null <<EOF
coordinator=true
node-scheduler.include-coordinator=false
http-server.http.port=8080
query.max-memory=1TB
discovery.uri=http://example.net:8080
EOF
sudo chown -R nobody:nogroup /opt/trino-server/etc/

Worker

这边query.max-memory-per-nodequery.max-total-memory-per-node建议设成内存一半的大小,预设是JVM heap的10%大小,很明显太小。

cd /opt/trino-server; sudo tee etc/config.properties > /dev/null <<EOF
coordinator=false
http-server.http.port=8080
query.max-memory=1TB
query.max-memory-per-node=32GB
query.max-total-memory-per-node=32GB
discovery.uri=http://example.net:8080
EOF
sudo chown -R nobody:nogroup /opt/trino-server/etc/

启动

sudo systemctl daemon-reload; sudo systemctl enable trino; sudo service trino start

常用指令

先连上Trino:

trino --server trino-coordinator-1.example.com:8080

另外可以看系统的各种状况:

SHOW SCHEMAS FROM system;
SHOW TABLES FROM system.runtime;
SELECT * FROM system.runtime.nodes;

目前系统里面的Query:

SELECT * FROM system.runtime.queries;

相关连结

参考文献

  1. Deploying Trino. [2021-08-10] (English). 

外部链接