테스트 환경
Hostname
|
IP
|
Role
|
Version
|
jh-redis
|
192.168.100.86
|
redis server
|
redis-3.2.8
|
mysql
|
192.168.100.85
|
mysql server
|
8.0.28
|
[root@jh-redis ~]# yum -y install gcc-c++
[root@jh-redis ~]# wget http://download.redis.io/releases/redis-3.2.8.tar.gz
[root@jh-redis redis-3.2.8]# tar -zxvf redis-3.2.8.tar.gz
[root@jh-redis redis-3.2.8]# cd redis-3.2.8/
[root@jh-redis redis-3.2.8]# make && make install
[root@jh-redis redis-3.2.8]# cd utils/
[root@jh-redis utils]# ./install_server.sh <enter>
Welcome to the redis service installer
This script will help you easily set up a running redis server
Please select the redis port for this instance: [6379]
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]
Selected default - /var/lib/redis/6379
Please select the redis executable path [/usr/local/bin/redis-server]
Selected config:
Port : 6379
Config file : /etc/redis/6379.conf
Log file : /var/log/redis_6379.log
Data dir : /var/lib/redis/6379
Executable : /usr/local/bin/redis-server
Cli Executable : /usr/local/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server...
Installation successful!
## 패스워드 설정
[root@jh-redis ~]# vi /etc/redis/6379.conf
..
masterauth redis
requirepass redis
## 재기동
[root@jh-redis ~]# /etc/init.d/redis_6379 restart
Stopping ...
Redis stopped
Starting Redis server...
## 접속 확인
[root@jh-redis ~]# redis-cli
127.0.0.1:6379> auth redis
OK
127.0.0.1:6379> set name redis
OK
127.0.0.1:6379> get name
"redis"
[root@jh-redis ~]# wget https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tar.xz
[root@jh-redis ~]# tar -xvf Python-3.11.0.tar.xz
[root@jh-redis ~]# cd Python-3.11.0/
[root@jh-redis Python-3.11.0]# ./configure
...
checking for stdlib extension module _testbuffer... yes
checking for stdlib extension module _testimportmultiple... yes
checking for stdlib extension module _testmultiphase... yes
checking for stdlib extension module _xxtestfuzz... yes
checking for stdlib extension module _ctypes_test... yes
checking for stdlib extension module xxlimited... yes
checking for stdlib extension module xxlimited_35... yes
configure: creating ./config.status
config.status: creating Makefile.pre
config.status: creating Misc/python.pc
config.status: creating Misc/python-embed.pc
config.status: creating Misc/python-config.sh
config.status: creating Modules/Setup.bootstrap
config.status: creating Modules/Setup.stdlib
config.status: creating Modules/ld_so_aix
config.status: creating pyconfig.h
configure: creating Modules/Setup.local
configure: creating Makefile
configure:
[root@prometheus Python-3.11.0]# make && make install
[root@prometheus Python-3.11.0]# python3.11 -V
Python 3.11.0
■ pip redis 모듈 설치(redis server)
[root@jh-redis Python-3.11.0]# pip3 install redis
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting redis
Downloading https://files.pythonhosted.org/packages/76/ad/dd7b6423295394b95e03d961d454e3046b569715dcc2dd4a030bb43a7cff/redis-4.3.5-py3-none-any.whl (248kB)
100% |████████████████████████████████| 256kB 4.4MB/s
Collecting async-timeout>=4.0.2 (from redis)
Downloading https://files.pythonhosted.org/packages/d6/c1/8991e7c5385b897b8c020cdaad718c5b087a6626d1d11a23e1ea87e325a7/async_timeout-4.0.2-py3-none-any.whl
Collecting typing-extensions; python_version < "3.8" (from redis)
Downloading https://files.pythonhosted.org/packages/45/6b/44f7f8f1e110027cf88956b59f2fad776cca7e1704396d043f89effd3a0e/typing_extensions-4.1.1-py3-none-any.whl
Collecting packaging>=20.4 (from redis)
Downloading https://files.pythonhosted.org/packages/05/8e/8de486cbd03baba4deef4142bd643a3e7bbe954a784dc1bb17142572d127/packaging-21.3-py3-none-any.whl (40kB)
100% |████████████████████████████████| 40kB 5.5MB/s
Collecting importlib-metadata>=1.0; python_version < "3.8" (from redis)
Downloading https://files.pythonhosted.org/packages/a0/a1/b153a0a4caf7a7e3f15c2cd56c7702e2cf3d89b1b359d1f1c5e59d68f4ce/importlib_metadata-4.8.3-py3-none-any.whl
Collecting pyparsing!=3.0.5,>=2.0.2 (from packaging>=20.4->redis)
Downloading https://files.pythonhosted.org/packages/6c/10/a7d0fa5baea8fe7b50f448ab742f26f52b80bfca85ac2be9d35cdd9a3246/pyparsing-3.0.9-py3-none-any.whl (98kB)
100% |████████████████████████████████| 102kB 10.5MB/s
Collecting zipp>=0.5 (from importlib-metadata>=1.0; python_version < "3.8"->redis)
Downloading https://files.pythonhosted.org/packages/bd/df/d4a4974a3e3957fd1c1fa3082366d7fff6e428ddb55f074bf64876f8e8ad/zipp-3.6.0-py3-none-any.whl
Installing collected packages: typing-extensions, async-timeout, pyparsing, packaging, zipp, importlib-metadata, redis
Successfully installed async-timeout-4.0.2 importlib-metadata-4.8.3 packaging-21.3 pyparsing-3.0.9 redis-4.3.5 typing-extensions-4.1.1 zipp-3.6.0
[root@jh-redis Python-3.11.0]# vi redis_connect.py
import redis
r = redis.Redis(host="192.168.100.86", port=6379, password="redis",decode_responses=True)
print(r.ping())
print(r.set(name="name", value="redis"))
print(r.get(name="name"))
[root@jh-redis Python-3.11.0]# python3.6 redis_connect.py
True
True
redis
[root@jh-redis ~]# pip3 install pymysql
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting pymysql
Downloading https://files.pythonhosted.org/packages/4f/52/a115fe175028b058df353c5a3d5290b71514a83f67078a6482cff24d6137/PyMySQL-1.0.2-py3-none-any.whl (43kB)
100% |████████████████████████████████| 51kB 8.5MB/s
Installing collected packages: pymysql
Successfully installed pymysql-1.0.2
[root@jh-redis ~]# pip3 install pandas
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting pandas
Downloading https://files.pythonhosted.org/packages/c3/e2/00cacecafbab071c787019f00ad84ca3185952f6bb9bca9550ed83870d4d/pandas-1.1.5-cp36-cp36m-manylinux1_x86_64.whl (9.5MB)
100% |████████████████████████████████| 9.5MB 132kB/s
Collecting python-dateutil>=2.7.3 (from pandas)
Downloading https://files.pythonhosted.org/packages/36/7a/87837f39d0296e723bb9b62bbb257d0355c7f6128853c78955f57342a56d/python_dateutil-2.8.2-py2.py3-none-any.whl (247kB)
100% |████████████████████████████████| 256kB 4.9MB/s
Collecting numpy>=1.15.4 (from pandas)
Downloading https://files.pythonhosted.org/packages/45/b2/6c7545bb7a38754d63048c7696804a0d947328125d81bf12beaa692c3ae3/numpy-1.19.5-cp36-cp36m-manylinux1_x86_64.whl (13.4MB)
100% |████████████████████████████████| 13.4MB 94kB/s
Collecting pytz>=2017.2 (from pandas)
Downloading https://files.pythonhosted.org/packages/85/ac/92f998fc52a70afd7f6b788142632afb27cd60c8c782d1452b7466603332/pytz-2022.6-py2.py3-none-any.whl (498kB)
100% |████████████████████████████████| 501kB 2.8MB/s
Collecting six>=1.5 (from python-dateutil>=2.7.3->pandas)
Downloading https://files.pythonhosted.org/packages/d9/5a/e7c31adbe875f2abbb91bd84cf2dc52d792b5a01506781dbcf25c91daf11/six-1.16.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, numpy, pytz, pandas
Successfully installed numpy-1.19.5 pandas-1.1.5 python-dateutil-2.8.2 pytz-2022.6 six-1.16.0
[root@jh-redis ~]# pip3 install openpyxl
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting openpyxl
Downloading https://files.pythonhosted.org/packages/7b/60/9afac4fd6feee0ac09339de4101ee452ea643d26e9ce44c7708a0023f503/openpyxl-3.0.10-py2.py3-none-any.whl (242kB)
100% |████████████████████████████████| 245kB 4.8MB/s
Collecting et-xmlfile (from openpyxl)
Downloading https://files.pythonhosted.org/packages/96/c2/3dd434b0108730014f1b96fd286040dc3bcb70066346f7e01ec2ac95865f/et_xmlfile-1.1.0-py3-none-any.whl
Installing collected packages: et-xmlfile, openpyxl
Successfully installed et-xmlfile-1.1.0 openpyxl-3.0.10
※ 사전에 DB 서버 유저 및 database, test테이블 생성
mysql> select user, host from mysql.user;
+------------------+-----------+
| user | host |
+------------------+-----------+
| kim | % |
| prouser | % |
| mysql.infoschema | localhost |
| mysql.session | localhost |
| mysql.sys | localhost |
| root | localhost |
+------------------+-----------+
6 rows in set (0.00 sec)
## DB 연동 python
[root@jh-redis ~]# vi mysql.py
import pymysql
mysql_db = pymysql.connect(
user="kim",
passwd="kim",
host="192.168.100.85",
port=3306,
db="kim",
)
cursor = mysql_db.cursor()
sql = "select * from test;"
cursor.execute(sql)
result = cursor.fetchall()
import pandas as pd
result = pd.DataFrame(result)
result
result.to_excel('test.xlsx')
## python 실행
[root@jh-redis ~]# python3.6 mysql.py
■ Cache test
[root@jh-redis ~]# vi test.py
import redis
import pymysql
import time
# 레디스 클라이언트를 만든다
# Redis<ConnectionPool<Connection>>
r = redis.Redis(host="localhost", port=6379, password="redis",decode_responses=True)
#pymysql로 mysql 연결
mysql_db = pymysql.connect(
user="kim",
passwd="kim",
host="192.168.100.85",
port=3306,
db="sakila",
)
#여기부터 실제 로직
result_list = []
#id=int(input("emp_no 입력 (6자리)")) #직접입력귀찮아서;
for address_id in range(500,600):
id=int(address_id)
first_time = time.time()
result = r.get(name=id)
print(id)
if not result:
print("no cache")
cursor = mysql_db.cursor()
sql = '''select address_id,address, district, city_id, location from address where address_id=%s'''
cursor.execute(sql, (int(address_id)))
result = cursor.fetchone()
r.set(name=result[0], value=result[1])
else:
print("cache")
result = r.get(name=id) #if문에 넣을수있지만 동일한 조건을 주기위함
last_time = time.time()
timetotime = last_time-first_time
print(timetotime, "Second")
print(result)
result_list.append(timetotime)
print("Finish")
print(sum(result_list)/len(result_list))
※ cache 없을 때
## redis 키 값 확인
[root@jh-redis ~]# redis-cli
127.0.0.1:6379> auth redis
OK
127.0.0.1:6379> keys *
(empty list or set)
[root@jh-redis ~]# python3.6 test.py
500
no cache
0.0018432140350341797 Second
362 Rajkot Lane
501
no cache
0.0009305477142333984 Second
1060 Tandil Lane
502
no cache
0.0009379386901855469 Second
1515 Korla Way
503
no cache
0.0008525848388671875 Second
1416 San Juan Bautista Tuxtepec Avenue
504
no cache
0.0008611679077148438 Second
1 Valle de Santiago Avenue
505
no cache
0.0008366107940673828 Second
519 Brescia Parkway
506
no cache
0.0008485317230224609 Second
414 Mandaluyong Street
507
no cache
0.0008752346038818359 Second
1197 Sokoto Boulevard
508
no cache
0.0008466243743896484 Second
496 Celaya Drive
509
no cache
0.0007996559143066406 Second
786 Matsue Way
Finish
0.0009632110595703125
※ cache 있을 때
## redis에 저장된 값 확인
127.0.0.1:6379> keys *
1) "505"
2) "507"
3) "500"
4) "504"
5) "503"
6) "502"
7) "501"
8) "509"
9) "506"
10) "508"
[root@jh-redis ~]# python3.6 test.py
500
cache
0.001024007797241211 Second
362 Rajkot Lane
501
cache
0.000213623046875 Second
1060 Tandil Lane
502
cache
0.00019288063049316406 Second
1515 Korla Way
503
cache
0.0001881122589111328 Second
1416 San Juan Bautista Tuxtepec Avenue
504
cache
0.00018835067749023438 Second
1 Valle de Santiago Avenue
505
cache
0.0002117156982421875 Second
519 Brescia Parkway
506
cache
0.00018715858459472656 Second
414 Mandaluyong Street
507
cache
0.00018739700317382812 Second
1197 Sokoto Boulevard
508
cache
0.000186920166015625 Second
496 Celaya Drive
509
cache
0.00018715858459472656 Second
786 Matsue Way
Finish
0.0002767324447631836
■ 속도 비교
약 4배정도 속도가 빠르다.
## cache 없을때 0.0009632110595703125
## cache에 있을때 0.0002767324447631836
참고
https://www.lostcatbox.com/2020/10/23/DB-cache-server/
DB-cache-server에 대해서 · lostcatbox
Word count: 1.2kReading time: 7 min 2020/10/23 Share
www.lostcatbox.com
https://yunhyeonglee.tistory.com/25
[Redis] Redis Cache Server 만들기!
아는 선배에게 많은 기업들이 현재 Redis를 이용해서 Cache Server를 사용하고 있다는 말을 듣게 되었다. 해서 이번에는 Redis를 이용한 Cache Server 구축에 관하여 포스팅을 해볼 생각이다. ■1.Cache 란? -
yunhyeonglee.tistory.com
[REDIS] 📚 레디스 소개 & 사용처 (캐시 / 세션) - 한눈에 쏙 정리
Redis (Remote Dictionary Server) Redis는 Remote(원격)에 위치하고 프로세스로 존재하는 In-Memory 기반의 Dictionary(key-value) 구조 데이터 관리 Server 시스템이다. 여기서 key-value 구조 데이터란, mysql 같은 관계형
inpa.tistory.com
https://daakludens.github.io/project/redis-cache/
DB 부하 분산을 위한 캐시 적용
redis, cache
daakludens.github.io
'Others > Opensource Tool' 카테고리의 다른 글
[MySQL - Online DDL] use. Gh-ost (4) | 2023.02.05 |
---|---|
[Apache Jmeter - 설치 및 설정 for 부하 테스트] (0) | 2022.12.01 |
[Terraform - NaverCloud vm생성] part 2 (0) | 2022.06.19 |
[Terraform - NaverCloud vm생성] part 1 (0) | 2022.06.19 |
[MySQL - Slow query 메일 전송] use. pt-qeury-digest (0) | 2022.05.14 |