当前位置： 首页 > news >正文

gravitino0.9 安装部署集成Trino-439查询Hive3.1.3和MySQL8.0.33

news 2025/7/1 12:50:03

目录

1、官网下载Apache Gravitino Downloads 0.9.0-incubating

2、 gravitino-0.9.0-incubating-bin.tar.gz解压安装

2.1、gravitino-env.sh 

2.2、gravitino-iceberg-rest-server.conf 

2.3、gravitino.conf 

2.4、根据实际情况修改catalog内容

​编辑2.5、例如lakehouse-iceberg （可选操作）

3、MySQL脚本初始化

4、Apache Gravitino启动

5、部署gravitino-iceberg-rest-server 

5.1 、gravitino-env.sh

5.2、gravitino-iceberg-rest-server.conf 

5.3、hadoop下的core .xml和hdfs-site.xml复制到/cluster/gravitino-iceberg-rest-server/conf

​编辑5.4、启动执行gravitino-iceberg-rest-server

6、安装部署trio（版本使用439版本）

6.1、创建配置文件，在/cluster/trino/etc目录下 

6.1.1、config.properties

 6.1.2、node.properties

6.1.3、log.properties

6.1.4、hive.properties 

6.1.5、password-authenticator.properties 

6.1.6、/cluster/trino/etc/catalog 

6.1.6.1、catalog_mysql.properties 

6.1.6.2、catalog_hive.properties 

6.1.6.3、gravitino.properties

6.2、下载客户端

6.3、修改启动脚本

6.3.1、launcher 

6.3.2、launcher.py 

6.3.3、launcher.properties

6.4、复制Gravition trino connector驱动包到trino的plugin目录下

 6.5、启动trino

6.6、使用Trino Client连接Trino-Server

7、执行效果 

8、需要Hadoop 3.3.6、Hive3.1.3、Spark环境，省略

1、官网下载Apache Gravitino Downloads 0.9.0-incubating

Apache Gravitino

2、 gravitino-0.9.0-incubating-bin.tar.gz解压安装

解压到/cluster/gravitino/conf目录

2.1、gravitino-env.sh 

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#

# In build.gradle.kts, we would automatic replace GRAVITINO_VERSION to the current version of Gravitino
# when running `./gradlew compileDistribution` or `./gradlew compileIcebergRESTServer` command.
GRAVITINO_VERSION=0.9.0-incubating

# Debug Gravitino server
# export GRAVITINO_DEBUG_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=8000 -Dlog4j2.debug=true"

export JAVA_HOME=/cluster/jdk
export GRAVITINO_HOME=/cluster/gravitino
export GRAVITINO_CONF_DIR=/cluster/gravitino/conf
export GRAVITINO_LOG_DIR=/cluster/gravitino/logs     # Where log files are stored.  PWD by default.
# export GRAVITINO_MEM         # Gravitino jvm mem options Default -Xms1024m -Xmx1024m -XX:MaxMetaspaceSize=512m

2.2、gravitino-iceberg-rest-server.conf 

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied.  See the License for the
# specific language governing permissions and limitations
# under the License.
#
gravitino.auxService.names = iceberg-rest
# Iceberg REST服务的类路径
gravitino.iceberg-rest.classpath = iceberg-rest-server/libs, iceberg-rest-server/conf
# THE CONFIGURATION FOR Iceberg REST SERVER
gravitino.iceberg-rest.shutdown.timeout = 3000

# THE CONFIGURATION FOR Iceberg REST WEB SERVER
# The host name of the built-in web server
gravitino.iceberg-rest.host = 0.0.0.0
# The http port number of the built-in web server
gravitino.iceberg-rest.httpPort = 9001
# The min thread size of the built-in web server
gravitino.iceberg-rest.minThreads = 24
# The max thread size of the built-in web server
gravitino.iceberg-rest.maxThreads = 200
# The stop timeout of the built-in web server
gravitino.iceberg-rest.stopTimeout = 30000
# The timeout of idle connections
gravitino.iceberg-rest.idleTimeout = 30000
# The executor thread pool work queue size of the built-in web server
gravitino.iceberg-rest.threadPoolWorkQueueSize = 100
# The request header size of the built-in web server
gravitino.iceberg-rest.requestHeaderSize = 131072
# The response header size of the built-in web server
gravitino.iceberg-rest.responseHeaderSize = 131072

# THE CONFIGURATION FOR Iceberg catalog backend
# The Iceberg catalog backend, it's recommended to change to hive or jdbc
gravitino.iceberg-rest.catalog-backend = hive
# The warehouse directory of Iceberg catalog
gravitino.iceberg-rest.warehouse = /cluster/gravitino/warehouse

# THE CONFIGURATION EXAMPLE FOR JDBC CATALOG BACKEND WITH S3 SUPPORT

gravitino.iceberg-rest.catalog-backend = hive
gravitino.iceberg-rest.jdbc-driver = org.postgresql.Driver
gravitino.iceberg-rest.uri = jdbc:postgresql://127.0.0.1:5432/postgres
gravitino.iceberg-rest.jdbc-user = postgres
gravitino.iceberg-rest.jdbc-password = postgres
gravitino.iceberg-rest.jdbc-initialize = true

# change to s3a://test/my/key/prefix for Hive catalog backend
gravitino.iceberg-rest.warehouse = s3://cluster/gravitino/warehouse/iceberg
gravitino.iceberg-rest.io-impl= org.apache.iceberg.aws.s3.S3FileIO
gravitino.iceberg-rest.s3-access-key-id = JQAoj5bZIaWHwum9
gravitino.iceberg-rest.s3-secret-access-key = m2HrAR4DU2vpDFoi21F5XhHsbDAcxUGF
gravitino.iceberg-rest.s3-endpoint = http://127.0.0.1:9000
gravitino.iceberg-rest.s3-region = GMT+0800
 

2.3、gravitino.conf 

# Licensed to the Apache Software Foundation (ASF) under one
# 本产品受Apache软件基金会（ASF）许可
# or more contributor license agreements.  See the NOTICE file
# 或更多贡献者许可协议的保护。请参阅随本工作分发的NOTICE文件
# distributed with this work for additional information
# 以获取有关版权所有权的附加信息
# regarding copyright ownership.  The ASF licenses this file
# 关于版权所有权的信息。 ASF根据以下许可证向您许可此文件
# to you under the Apache License, Version 2.0 (the
# 根据Apache License, Version 2.0（以下简称
# "License"); you may not use this file except in compliance
# "许可证"）；除非符合许可证的规定，否则您不得使用此文件
# with the License.  You may obtain a copy of the License at
# 您可以在以下位置获取许可证的副本
#
#  http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# 除非适用法律要求或书面同意
# software distributed under the License is distributed on an
# 根据许可证分发的软件按"按原样"分发
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# 无任何形式的明示或暗示保证或条件
# KIND, either express or implied.  See the License for the
# 有关许可证下的特定语言 governing permissions and limitations
# 请参阅许可证中 governing permissions and limitations
# under the License.
# 在许可证下的特定语言

# THE CONFIGURATION FOR Gravitino SERVER
# Gravitino服务器配置
# 服务器优雅关闭的超时时间（毫秒）
gravitino.server.shutdown.timeout = 3000

# THE CONFIGURATION FOR Gravitino WEB SERVER
# Gravitino Web服务器配置
# 内置Web服务器的主机名，0.0.0.0表示监听所有可用网络接口
gravitino.server.webserver.host = 0.0.0.0
# 内置Web服务器的HTTP端口号
gravitino.server.webserver.httpPort = 8090
# 内置Web服务器的最小工作线程数
gravitino.server.webserver.minThreads = 24
# 内置Web服务器的最大工作线程数
gravitino.server.webserver.maxThreads = 200
# 内置Web服务器停止时的超时时间（毫秒）
gravitino.server.webserver.stopTimeout = 30000
# 连接空闲超时时间（毫秒）
gravitino.server.webserver.idleTimeout = 30000
# 内置Web服务器线程池工作队列的大小
gravitino.server.webserver.threadPoolWorkQueueSize = 100
# 内置Web服务器请求头的最大大小（字节）
gravitino.server.webserver.requestHeaderSize = 131072
# 内置Web服务器响应头的最大大小（字节）
gravitino.server.webserver.responseHeaderSize = 131072

# THE CONFIGURATION FOR Gravitino ENTITY STORE
# Gravitino实体存储配置
# 使用的实体存储类型，目前仅支持relational（关系型）
gravitino.entity.store = relational
# 关系型实体存储的后端实现，目前仅支持JDBC
gravitino.entity.store.relational = JDBCBackend

# 实体存储的JDBC连接URL
gravitino.entity.store.relational.jdbcUrl = jdbc:mysql://127.0.0.1:3306/gravitino?useUnicode=true&characterEncoding=utf-8&useSSL=true&serverTimezone=Asia/Shanghai
# JDBC驱动类名
gravitino.entity.store.relational.jdbcDriver = com.mysql.cj.jdbc.Driver
# JDBC用户名
gravitino.entity.store.relational.jdbcUser = root
# JDBC密码
gravitino.entity.store.relational.jdbcPassword = 123456

# THE CONFIGURATION FOR Gravitino CATALOG
# Gravitino目录缓存配置
# 目录缓存的清理间隔时间（毫秒），默认1小时
gravitino.catalog.cache.evictionIntervalMs = 3600000

# THE CONFIGURATION FOR authorization
# Gravitino授权配置
# 是否启用授权功能
gravitino.authorization.enable = false
# Gravitino服务的管理员列表，多个管理员用逗号分隔
gravitino.authorization.serviceAdmins = anonymous

# THE CONFIGURATION FOR AUXILIARY SERVICE
# 辅助服务配置
# 辅助服务名称列表，用逗号分隔
#gravitino.auxService.names = iceberg-rest
# Iceberg REST服务的类路径
#gravitino.iceberg-rest.classpath = iceberg-rest-server/libs, iceberg-rest-server/conf
# Iceberg REST服务的主机名
#gravitino.iceberg-rest.host = 0.0.0.0
# Iceberg REST服务的HTTP端口号
#gravitino.iceberg-rest.httpPort = 9001
# Iceberg REST服务的后端目录类型，建议更改为hive或jdbc，默认使用内存目录
#gravitino.iceberg-rest.catalog-backend = hive
# Iceberg REST服务的仓库目录路径
#gravitino.iceberg-rest.warehouse = /cluster/gravitino/warehouse
 

2.4、根据实际情况修改catalog内容

2.5、例如lakehouse-iceberg （可选操作）

core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- 
  核心配置文件说明：
  此文件包含 Hadoop 的全局配置参数，用于覆盖默认的 core-default.xml 设置。
  所有配置项需包裹在 <configuration> 标签中。
-->

<configuration>
    
    <!-- 
      NameNode 访问地址配置
      fs.defaultFS: 定义默认文件系统 URI
      格式: hdfs://<主机名或IP>:<端口>
      此处指向运行 NameNode 服务的节点地址（端口 9000）
    -->
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://127.0.0.1:9000</value>
    </property>

    <!-- 
      临时目录配置
      hadoop.tmp.dir: Hadoop 临时文件存储路径
      要求：该目录需有足够磁盘空间且对 Hadoop

http://www.lqws.cn/news/585343.html

相关文章：

  • 88.LMS当幅度和相位同时失配时，为啥最后权值w的相位angle(w(end))收敛到angle(mis)不是-angle(mis)
  • 从零到一通过Web技术开发一个五子棋
  • SpringBoot --项目启动的两种方式
  • js遍历对象的方法
  • 【MySQL】数据库基础
  • .net8导出影像图片按现场及天拆分
  • 51单片机CPU工作原理解析
  • 借助 KubeMQ 简化多 LLM 集成
  • YOLOv12_ultralytics-8.3.145_2025_5_27部分代码阅读笔记-torch_utils.py
  • 后台填坑记——Golang内存泄漏问题排查（一）
  • 设计模式（六）
  • 大模型开源技术解析 4.5 的系列开源技术解析：从模型矩阵到产业赋能的全栈突破
  • 2025年06月30日Github流行趋势
  • 遥控器双频无线模块技术要点概述
  • SegChange-R1：基于大型语言模型增强的遥感变化检测
  • 07-three.js Debug UI
  • Webpack原理剖析与实现
  • QT中QSS样式表的详细介绍
  • 【MySQL基础】MySQL索引全面解析：从原理到实践
  • 汽车轮速测量专用轮速传感器
  • 51c~UWB~合集1
  • SpringBoot项目开发实战销售管理系统——项目框架搭建！
  • 【windows上VScode开发STM32】
  • C#数字格式化全解析：从基础到进阶的实战指南
  • 电铸Socket弹片测试全解析：如何提升5G连接器稳定性？
  • 华为物联网认证：开启万物互联的钥匙
  • uni-app开发app保持登录状态
  • 【C++】简单学——模板初阶
  • 中证500股指期货一手多少钱呢？风险如何？
  • 易语言-登录UI演示