Redis basis

Quick reason：存储内存、io多路复用、单线程;

1.基本数据结构

五种：string、list、hash、set、zset(5.0加入stream)

string

底层数据结构：动态字符串(sds),redis基于c开发,字符数组为char[],redis没有采用这种,自己构建了一个结构体struct{int len;int free;char[] buf},有点类似go的slice底层,go存的是一个指针;

sdsGetting the length is faster(直接len字段o(1),c为o(n)),Buffer overflow can be prevented,有free字段,There is enough space for direct splicing,不够再扩容;Expansion is less than1M翻倍,大于1M,每次加1M;The memory reallocation caused by modification can be reduced(Freeing up space is not really freeing,而是设置free值);

应用：计数、分布式锁(写lua脚本,保证setnx,setex原子操作);

list

底层：小于512压缩列表,Otherwise doubly linked list;

压缩列表：连续的内存,包括多个entry;

struct entry {

int prelen; 前一个元素的字节长度

int encoding;

byte[] content; content of the element

}

struct ziplist {

int bytes; The number of bytes occupied by the entire compressed list

int tail_offset; The number of bytes the last element is from the start of the packed list, Easy to traverse in reverse

Int length; 元素总数

T[] entires; 元素列表

int end; 压缩列表结束标志

}

双端链表：The pointer will compress the linked list connection;

struct quickListNode {

        quickListNode* prev; Points to the previous compressed list

        quickListNode* next; Points to the latter compressed list

        ziplist* p; Points to the current compressed list header

        int count; The total number of elements in the current packed list

        int size; The total number of bytes occupied by the current compressed list

}

struct quickList {

        quickListNode* head; Quick list header

        quickListNode* tail; The end of the quick linked list

        int count; 元素总数

        int nodes; The number of compressed lists
}

参考：https://blog.csdn.net/jinixin/article/details/110004723?spm=1001.2014.3001.5502

应用：消息队列

hash

底层：小于512压缩列表,Otherwise a dictionary(哈希表实现,A dictionary is implemented by two hash tables,One for progressiverehash);

typedef struct dict {
    dictType *type;
    void *privdata;
    // 哈希表
    dictht ht[2];
    //rehash 索引,当 rehash 不在进行时,值为 -1
    int rehashidx; /* rehashing not in progress if rehashidx == -1 */
    // 目前正在运行的安全迭代器的数量
    int iterators; /* number of iterators currently running */
} dict;

应用：Mainly used to store objects,注意大key问题;

set

底层：小于512,intset(整数集合,All consist of integers),Otherwise a dictionary(val为空);

typedef struct intset {   
    // 编码方式
    uint32_t encoding;
    // 集合包含的元素数量
    uint32_t length;
    // 保存元素的数组 按照从小到大的顺序,且不重复
    int8_t contents[];
} intset;

应用：公共好友(setCan be handed over and operated)

zset

底层：Data compaction list,Use dictionaries+跳表,查找o(1),排序,范围获取o(logn);

Dictionary storage format：key score member(value),skip tablescore并以此排序

struct zset {

        dict A; 所有val=>score对

        skiplist A; 跳跃列表

}

struct skiplistnode {

        string value;

        double score;

        skiplistnode*[] forwards; Multiple layers of pointers

}

struct skiplist {

        skiplistnode* header; 首元素指针

        int maxLevel; The current highest level

}

Range lookup can directly find the first element,Afterwards, the subsequent elements can be returned directly according to the level;

应用：排行榜(游戏top排名,字段+时间戳排序,You can combine fields to be sorted and timestamps intoscore);

分布式一致性问题：

延迟双删：先删缓存,再更新数据库,Delete the cache after a while(Business reading time decision);

redis持久化：

rdb：快照(n分钟m次记录),Binary writes to disk,Rebooting will lose data,但相比于aof快,fork子进程,共享内存,Execute business while synchronizing,A record will be lost;

aof：追加到file尾部,Reboot to replay,耗时,不会丢数据;

rdb+aof：两者结合,aofFull logs are not stored,Saved after the last persistence,重启时,Load the data from the snapshot first,然后重放增量;

redis主从

Divided into full and incremental synchronization,为非阻塞,The first time is full synchronization;

Full steps：

1.slave连接master,发送sync(fsync)命令;

2.master收到sync命令后,执行bgsave命令保存rdb快照文件,At the same time, the buffer area is used to save all write commands executed afterward;

3.master执行bgsave后,向所有slave发送rdb文件,Write commands are also logged during transmission;

4.slave收到快照后,丢弃旧数据,Load a new snapshot;

5.masterAfter sending the snapshot, continue to send the write command to the buffer area;

6.slaveExecute the write command to the buffer area;

增量步骤：

1.slave发slave_repl_offset,Sync with offset position,为增量;

2.master执行bgsave,发送offsetafter the write command;

redis异常

击穿：redis没有,数据库有,单keyHotspot keeps requesting,Hit the database directly when it expires,造成数据库压力过大,请求阻塞;

解决：1.热点数据永不过期;2.加互斥锁(Distributed systems use distributed locks),key过期时,请求打到数据库,Open a mutex,It is guaranteed that only one process can access the current one at a timekey;

穿透：redis没有,数据库也没有,Attackers keep requesting data that doesn't exist,This results in a large number of requests hitting the database in a short period of time,dbwill be too stressful;

解决：1.布隆过滤器(Based on bitmaps and hash functions,marray of bits+k个哈希函数,To reach the judgment that it does not exist must not exist,The presence of data does not necessarily have effects);There is a problem that the Bloom filter cannot be deleted,The cuckoo filter can play a certain optimization effect;2.redis存储空对象,设置过期时间;

雪崩：大批量的key过期,Reaching the database causes downtime;

解决：设置随机过期时间,均匀过期;

redis key过期

1.惰性过期：一个keyThe expiration time is judged when it is used,如果过期则删除;内存不友好,if one expireskeyNot used for a long time,会占用内存;cpu友好,只有cpuDeleted only by operation;

2.主动过期：Regular random scan part of the set expiration timekey,进行删除;

3.定期过期(两者结合,redis采用)：The expiration time will be setkey放入字典,每100ms扫描一次,Random scan draw20个,Maximum time per scan25ms,防止卡顿;

从库过期策略：Handle expiration from the librarykeyfor passive,Does not expire scans;主库的key过期,会生成del指令到aof文件,然后同步给从库,Execute from library passdel进行删除;Instructions are synchronous to asynchronous,如果不及时,会造成主从不一致,The main library does not have itkeyAlso exists in the slave library;

redis内存淘汰

1.noeviction：Only read and delete operations are supported,Write service error;

2.allkeys-xxx：ttl：Expiration time is small to delete;random：随机删除;lru：Least recently used delete;lfu：Delete is used least often;

3.volatile-xxx：ttl：Expiration time is small to delete;random：随机删除;lru：Least recently used delete;lfu：Delete is used least often;

redis集群模式

主从

哨兵

分片