Milvus向量数据库最全部署配置手册来了,值得收藏
一、部署方式选择
1. Docker单机部署(快速上手)
# 拉取最新镜像docker pull milvusdb/milvus:latest# 启动容器docker run -d \ --name milvus-standalone \ -p 19530:19530 \ -p 9091:9091 \ -v ~/milvus/db:/var/lib/milvus \ milvusdb/milvus:latest
2. Helm集群部署(生产环境)
# values.yaml 配置文件cluster: enabled: true replicas: 3 etcd: replicaCount: 3 minio: mode: distributed replicas: 4二、关键配置详解
1. 存储引擎配置
# server_config.yamlstorage: path: /var/lib/milvus auto_flush_interval: 1 file_cleanup_timeout: 10wal: enabled: true buffer_size: 256MB path: /var/lib/milvus/wal
2. 查询节点配置
queryNode: gracefulTime: 5000 segcore: chunk_rows: 32768 nprobe: 16 nlist: 16384三、索引配置实战
1. 创建带索引的集合
from pymilvus import connections, CollectionSchema, FieldSchema, DataType# 连接Milvusconnections.connect(host='localhost', port='19530')# 定义字段fields = [ FieldSchema(name="id", dtype=DataType.INT64, is_primary=True), FieldSchema(name="embedding", dtype=DataType.FLOAT_VECTOR, dim=128)]# 创建集合schema = CollectionSchema(fields)collection = Collection("my_collection", schema)# 创建IVF_FLAT索引index_params = { "metric_type": "L2", "index_type": "IVF_FLAT", "params": {"nlist": 1024}}collection.create_index("embedding", index_params)
2. 混合索引配置
# HNSW + IVF_PQ 组合索引index_params = { "index_type": "IVF_PQ", "params": { "nlist": 4096, "m": 16, "nbits": 8 }, "metric_type": "IP"}# 分区索引partition_params = { "partition_name": "partition_1", "index_params": index_params}四、性能调优配置
1. 内存优化配置
# 调整缓存大小cache: cache_size: 4GB insert_buffer_size: 1GB preload_collection: "my_collection"# 线程池配置queryNode: search_reserve_threads: 4 build_index_threads: 8
2. 查询参数优化
# 精确查询配置search_params = {"metric_type": "L2", "params": {"nprobe": 32}}# 范围搜索range_search_params = { "radius": 0.8, "range_filter": 1.2}# 分批查询batch_search_params = { "anns_field": "embedding", "param": search_params, "limit": 1000, "batch_size": 100}五、监控与维护配置
1. 监控配置
metrics: enabled: true address: 0.0.0.0 port: 9091 log: level: info file: maxSize: 300 maxAge: 10 maxBackups: 5
2. 备份配置
# Python备份脚本from pymilvus import utility# 创建备份backup = utility.create_backup( collection_name="my_collection", async_backup=True)# 恢复数据utility.restore_backup( backup_name="backup_2024", collection_names=["my_collection"])六、安全配置
1. 认证配置
common: security: authorizationEnabled: true clusterRootPassword: "Milvus123" proxy: minio: accessKeyID: "minioadmin" secretAccessKey: "minioadmin"
2. TLS加密
# 启动带TLS的Milvusdocker run -d \ --name milvus-tls \ -p 19531:19531 \ -v /path/to/certs:/certs \ -e MILVUS_TLS_MODE=1 \ -e MILVUS_TLS_CERT=/certs/server.crt \ -e MILVUS_TLS_KEY=/certs/server.key \ milvusdb/milvus:latest七、故障排除配置
1. 连接池配置
# 连接池管理connections.connect( alias="default", host='localhost', port='19530', pool_size=10, timeout=30)# 重试机制retry_params = { "retry_times": 3, "retry_interval": 1.0}
2. 健康检查端点
# Kubernetes探针配置livenessProbe: httpGet: path: /healthz port: 9091 initialDelaySeconds: 30 periodSeconds: 10总结
本配置手册覆盖从单机部署到生产集群的全场景配置。关键要点:
- 根据数据规模选择合适索引类型
- 监控指标实时调整资源配置
- 安全配置必不可少
- 定期备份确保数据安全
配置示例均为生产验证方案,可直接使用。根据实际数据量和查询需求微调参数即可获得最优性能。
声明:本站所有文章资源内容,如无特殊说明或标注,均为采集网络资源。如若本站内容侵犯了原著者的合法权益,可联系本站删除。
