0%

Redis之监控工具

通常来讲,缓存的命中率越高则表示使用缓存的收益越高,应用的性能越好(响应时间越短、吞吐量越高),抗并发的能力越强。

一、常用工具

  1. redis-stat,几年前的老项目
  2. redmon,几年前的老项目
  3. redis-faina,几年前的老项目
  4. redis-broswer,几年前的老项目
  5. RedisLive,几年前的老项目,由python编写的并且开源的图形化监控工具,核心服务部分只包含一个web服务和一个基于redis自带的info及monitor命令的监控服务。
  6. zabbix
  7. promethuse
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
# Server
redis_version:6.0.7
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:cd01c408c5307915
redis_mode:standalone
os:Darwin 19.6.0 x86_64
arch_bits:64
multiplexing_api:kqueue
atomicvar_api:atomic-builtin
gcc_version:4.2.1
process_id:40616
run_id:f0c38cff282eea9724af6dd9a63646150fa8cd90
tcp_port:6379
uptime_in_seconds:85774
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:51027
executable:/usr/local/redis-6.0.7/./redis-server
config_file:/usr/local/redis-6.0.7/6379.conf
io_threads_active:0

# Clients
connected_clients:2
client_recent_max_input_buffer:2
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:1162800
used_memory_human:1.11M
used_memory_rss:1429504
used_memory_rss_human:1.36M
used_memory_peak:1207648
used_memory_peak_human:1.15M
used_memory_peak_perc:96.29%
used_memory_overhead:1114108
used_memory_startup:1079728
used_memory_dataset:48692
used_memory_dataset_perc:58.61%
allocator_allocated:1116192
allocator_active:1391616
allocator_resident:1391616
total_system_memory:8589934592
total_system_memory_human:8.00G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.25
allocator_frag_bytes:275424
allocator_rss_ratio:1.00
allocator_rss_bytes:0
rss_overhead_ratio:1.03
rss_overhead_bytes:37888
mem_fragmentation_ratio:1.28
mem_fragmentation_bytes:313312
mem_not_counted_for_evict:104
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:33972
mem_aof_buffer:104
mem_allocator:libc
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1627357908
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
aof_current_size:232
aof_base_size:91
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0

# Stats
total_connections_received:7
total_commands_processed:16690
instantaneous_ops_per_sec:0
total_net_input_bytes:234900
total_net_output_bytes:65226538
instantaneous_input_kbps:0.01
instantaneous_output_kbps:2.40
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:589
evicted_keys:0
keyspace_hits:97
keyspace_misses:9
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:467
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_reads_processed:16700
total_writes_processed:16694
io_threaded_reads_processed:0
io_threaded_writes_processed:0

# Replication
role:master
connected_slaves:0
master_replid:b1dd50279f85b1339917f1ae030f9dd1aca6db3d
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:107.256007
used_cpu_user:28.073143
used_cpu_sys_children:0.006693
used_cpu_user_children:0.001512

# Modules
module:name=panda,ver=1,api=1,filters=0,usedby=[],using=[],options=[]

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=6,expires=0,avg_ttl=0

二、使用

  1. redis-stat

    • 安装

      • 通过Ruby安装
      • 通过Jar方式安装
    • 使用

      • 通过web页面
      • 通过终端命令行redis-stat --auth=your_password,无法启动运行,看报错信息意思是有的方法已经废弃了,只能通过web端查看
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      /Library/Ruby/Gems/2.6.0/gems/sinatra-1.3.6/lib/sinatra/base.rb:1070: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/sinatra-1.3.6/lib/sinatra/base.rb:1070: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/sinatra-1.3.6/lib/sinatra/base.rb:1070: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/sinatra-1.3.6/lib/sinatra/base.rb:1070: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/sinatra-1.3.6/lib/sinatra/base.rb:1070: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/si-0.1.4/lib/si/patch.rb:5: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/si-0.1.4/lib/si/patch.rb:9: warning: constant ::Bignum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/option_initializer-1.5.1/lib/option_initializer.rb:154: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/option_initializer-1.5.1/lib/option_initializer.rb:198: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/redis-3.0.7/lib/redis/client.rb:386: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/si-0.1.4/lib/si/module.rb:4: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/si-0.1.4/lib/si/module.rb:21: warning: constant ::Fixnum is deprecated
      /Library/Ruby/Gems/2.6.0/gems/si-0.1.4/lib/si/module.rb:10: warning: constant ::Fixnum is deprecated
      undefined method `display_width' for #<String:0x00007fe7c5225678>
      Did you mean? display
  2. redmon

    • 安装sudo gem install redmon
    • 启动redmon
    • 查看http://0.0.0.0:4567/
  3. redis-faina

  4. redis-broswer

  5. RedisLive(较早的工具,后期已不见维护,调试时因为版本问题各种出错,暂时放弃)

    • 安装RedisLive,clone或直接下载zip压缩包

    • 安装依赖

      • tornado pip install tornado
      • redis.py pip install redis
      • python-dateutil pip install python-dateutil
        • 指定目录pip install tornado --target /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
        • 指定版本pip install -v tornado==2.1.1
    • cd RedisLive(zip解压后不叫此名字)

    • cp redis-live.conf.example redis-live.conf

    • 编辑redis-live.conf

      • RedisServers是要监控的实例
      • RedisStatsServer则是监控信息临时写入的实例
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    {
    "RedisServers":
    [
    {
    "server": "154.17.59.99",
    "port" : 6379
    },

    {
    "server": "localhost",
    "port" : 6380,
    "password" : "some-password"
    }
    ],

    "DataStoreType" : "redis",

    "RedisStatsServer":
    {
    "server" : "ec2-184-72-166-144.compute-1.amazonaws.com",
    "port" : 6385
    },

    "SqliteStatsStore" :
    {
    "path": "to your sql lite file"
    }
    }
    • 启动./redis-monitor.py --duration=120

      • 报错,一般是由版本兼容导致的修改内容如下redisprovider.py,改完后重试正常或直接修改文件开头#! /usr/bin/env python3.8
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      15
      16
      17
      18
      19
      20
      #30行
      # self.conn.zadd(server + ":memory", str(timeutils.convert_to_epoch(timestamp)), data)
      self.conn.zadd(server + ":memory", data)

      #60行
      command_count_key = server + ":CommandCount:" + epoch
      # pipeline.zincrby(command_count_key, command, 1)
      pipeline.zincrby(command_count_key, 1, command)

      command_count_key = server + ":DailyCommandCount:" + current_date
      # pipeline.zincrby(command_count_key, command, 1)
      pipeline.zincrby(command_count_key, 1, command)

      key_count_key = server + ":KeyCount:" + epoch
      # pipeline.zincrby(key_count_key, keyname, 1)
      pipeline.zincrby(key_count_key, 1, keyname)

      key_count_key = server + ":DailyKeyCount:" + current_date
      # pipeline.zincrby(key_count_key, keyname, 1)
      pipeline.zincrby(key_count_key, 1, keyname)
      • 扩展
      1
      2
      3
      4
      5
      6
      7
      python与redis旧版本数据库的交互:
      zadd: db.zadd(REDIS_KEY, score, member)
      zincrby: db.zincrby(REDIS_KEY, member, increment)

      python与redis新版本数据库交互:
      zadd:db.zadd(REDIS_KEY, {member:score})
      zincrby:db.zincrby(REDIS_KEY, increment, menber)
    • 启动./redis-live.py

      • 报错
        • 直接修改文件开头#! /usr/bin/env python3.8
      1
      2
      3
      4
      5
      6
      7
      Traceback (most recent call last):
      File "./redis-live.py", line 3, in <module>
      import tornado.ioloop
      File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/tornado/ioloop.py", line 67
      def fileno(self) -> int:
      ^
      SyntaxError: invalid syntax
      • 报错
        • 加入以下代码
          1
          2
          3
          import sys
          sys.path.append("api/controller")
          sys.path.append("dataprovider")
      1
      2
      3
      4
      5
      6
      Traceback (most recent call last):
      File "./redis-live.py", line 15, in <module>
      from api.controller.ServerListController import ServerListController
      File "/usr/local/RedisLive/src/api/controller/ServerListController.py", line 1, in <module>
      from BaseController import BaseController
      ModuleNotFoundError: No module named 'BaseController'
      • 报错TabError: inconsistent use of tabs and spaces in indentation

        • 修改api/util/timeutils.py
        1
        2
        3
        4
        5
        6
        7
        8
        9
        10
        11
        12
        13
        14
        15
        def convert_to_epoch(timestamp):
        if (type(timestamp) is datetime.date):
        timestamp = datetime.datetime.fromordinal(timestamp.toordinal())
        timestamp = timestamp.replace(tzinfo=None)
        diff = (timestamp - datetime.datetime(1970, 1, 1))
        seconds = int(total_seconds(diff))
        return seconds

        def convert_to_epoch1(timestamp):
        if (type(timestamp) is datetime.date):
        timestamp = datetime.datetime.fromordinal(timestamp.toordinal())
        timestamp = timestamp.replace(tzinfo=None)
        diff = (timestamp - datetime.datetime(1970, 1, 1))
        seconds = int(total_seconds(diff))
        return seconds
      • 报错TypeError: '>' not supported between instances of 'NoneType' and 'int'

  6. zabbix

    • 安装MySQ,建议5.7版本

    • 安装zabbix server

      • 下载源码包wget https://cdn.zabbix.com/zabbix/sources/stable/4.0/zabbix-4.0.32.tar.gz
      • 解压tar -zxvf zabbix-4.0.32.tar.gz
      • cd zabbix-4.0.32
  7. promethuse

三、参考

  1. 参考一
  2. 参考二
  3. 参考三
  4. 参考四
  5. 参考五