matters needing attention

1. About Spring Integrate Redis Fragmentation
Fragmentation ---- Capacity expansion
Pay attention to the file writing method node node , After traversal, get .
2. Redis Sentinel mechanism — High availability
Pay attention to the premise of high availability ---- Master slave mount

Sentinel fissure Cluster cleft brain
3. Redis Cluster building ---- Introductory cases

1. About Spring Integrate Redis Fragmentation

1.1 About slice description

Downtime is a computer term , In oral English, we simply call stopping the machine down machine , Converting to Chinese characters is “ Downtime ”, But many people call it “ Crash ”/“ crash ”, Although not standardized, it is popular .

Downtime , The inability of the operating system to recover from a serious system error , Or there is something wrong with the system hardware , So that the system does not respond for a long time , And had to restart the computer . It belongs to a normal phenomenon of computer operation , This happens to any computer .

Redis The main function of sharding is to realize the expansion of memory data ,Redis If fragmentation fails to achieve high availability !!!
Redis The calculation of fragmentation takes place in the business server , That is to say tomact The server . Store the data to be saved in redis in .
Redis The execution efficiency of fragmentation is the highest . Because he put a lot of calculation rules in someone else's house , and redis Only responsible for deposit and withdrawal . So it's efficient .

1.2 Spring Integrate Redis Fragmentation

1.2.1 edit pro The configuration file (common)

explain :
1). For the convenience of modifying the port number in the future , Need to put ip+port Write to the configuration file , Then dynamically inject it into the configuration class .
2). If it's like a single redis It's too troublesome to configure one by one , Directly through nodes Write in the way of nodes .

# add to redis Configuration of
# Add a single configuration
#redis.host=192.168.126.129
#redis.port=6379
# To configure redis Fragmentation mechanism Nodes and nodes are distinguished by identifiers "," Used to distinguish nodes: node
redis.nodes=192.168.126.129:6379,192.168.126.129:6380,192.168.126.129:6381

1.2.2 edit JedisConfig Configuration class

explain : The original implementation of a single machine in the configuration class redis Object to spring Container management comments out . Now implement the fragment object and give it to spring Containers to manage .

package com.jt.config;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import redis.clients.jedis.JedisShardInfo;
import redis.clients.jedis.ShardedJedis;
import java.util.ArrayList;
import java.util.List;
@Configuration // I am a configuration class 
@PropertySource("classpath:/properties/redis.properties")
public class JedisConfig {

@Value("${redis.nodes}")
private String nodes; //node,node,node
/** * add to Redis Partition configuration * demand 1: Dynamic acquisition IP Address /PORT port * Dynamically obtain the information of multiple nodes . Easy to expand later ( That is, in the form of configuration file ) */
@Bean
public ShardedJedis shardedJedis(){

nodes = nodes.trim(); // Remove extra space on both sides 
List<JedisShardInfo> list = new ArrayList<>();
// according to ',' How to split The split result is an array 
String[] strArray = nodes.split(","); //[node,node,node]
for (String node : strArray){
 //ip:port
String host = node.split(":")[0];
// After break up to string Type digital , Out-of-service int Type to accept , hold String The number of type is converted to int Type is receiving .
int port = Integer.parseInt(node.split(":")[1]);
list.add(new JedisShardInfo(host,port ));
}
return new ShardedJedis(list);
}
/* @Value("${redis.host}") private String host; @Value("${redis.port}") private Integer port; *//**
* take jedis Object to spring Container management
*//*
@Bean
public Jedis jedis(){

// Because writing code is not conducive to expansion , So add the fixed configuration to the configuration file 
return new Jedis(host,port);
}*/
}

1.2.3 modify CacheAOP The injection term in

explain : take AOP The injection item in the cache is changed from a single machine to a fragment object , The same name, no need to change the code .

1.2.4 test

2. Redis Sentinel mechanism

2.1 About Redis It's divided into sections

advantage : Realize the expansion of memory data .
shortcoming : If redis There is a problem with a node in the shard ., Then the whole redis There must be some problems with the fragmentation mechanism Directly affect the use of users .
Solution : Realization redis High availability .

2.2 To configure Redis The master-slave structure of

Problem analysis : If you need to implement Redis High availability of servers , The precondition should be to realize the master-slave configuration .

reason : Only the master-slave configuration can be realized , The host has data synchronization to the slave Library , Host downtime in the future , Use slave ( The slave and host data are consistent ), To achieve high availability .

Build rules :

1. 6379 As a host
2. 6380/6381 As a slave

2.2.1 Prepare the sentry Directory

1). Shut down first Redis Shard server .、
Be careful : It can be closed directly through the process , You can also close one by one through the command .

2). Copy shards File for sentinel Sentinel Directory
explain : Now use 3 Servers , and shards The directory structure is the same. There are already 3 platform redis 了 , So just copy this directory .

3). Delete persistent files
explain : If you don't delete the persistent file that is restarted at this time, it is still the original fragmented persistent file , I can't test it later .

4). function 4 platform Redis The server

5). Check whether to start

2.2.2 Realize the master-slave mount

1). Check the master-slave status : info replication , It was found by inspection that 3 platform redis The default is the host .
eg:

2). Realize the mount of master-slave slaveof host IP Host port
explain : Respectively in 80 and 81 Mount on the host .

3). About the characteristics of master-slave mount
Check the status of the slave , master： host slave： Slave

Check the status of the host :, online: On-line
explain : It can be seen that redis The master and slave in the can recognize each other .

4). explain : operation redis The cache is still the main library for reading and writing redis,2 A slave library is just for backup , Unlike the database, it has read-write separation operation , There is no middleware .

2.2.3 Description of Mount error

explain : Misoperation may lead to abnormal mounting of the master-slave structure . How to mount it again ???
answer ： Can be redis All servers shut down , After the restart Under the default condition, the master-slave attachment will fail , Then mount it again .( Because these data are stored in memory , As soon as the memory is closed and the data information is released, it is gone .)

Additional explanation : because slaveof The instruction takes effect in memory . If memory resources are released , Then the master-slave relationship will fail . In order to achieve permanent effectiveness , The master-slave relationship should be written in the configuration file .
The emergence of new problems : If the host goes down unexpectedly , Who will complete the modification of the configuration file ? ( The configuration file cannot be written dead )
Such as :

2.3 How the sentry works

explain :
1). The user is directly connected to the sentry , The sentinel is connected to the host .
2). Host down , from 1 As a host , So from 2 As a slave of the new master .
3). If the host goes down , Slave 1 Become a host , Even if the host is repaired, it can only be used as a slave of the new host .
4). Sharding solves the problem of capacity expansion , Sentinels solve the problem of high availability , Slices store different data , The sentinel stores the same data .

How the sentry works :
1. To configure redis The structure of master-slave .
2. When sentinel service starts , Will monitor the current host . At the same time, get the details of the host ( The structure of master-slave )
3. When the sentinel uses the heartbeat detection mechanism (PING-PONG), Check whether the host is normal . If it's continuous 3 It is found that the host has no response information for the first time . Then elections begin .
4. When the sentry finds out that the host is down , The election mechanism will be opened , Choose one of the current slaves Redis As a mainframe .
5. Put the others redis The node is set as the slave of the new host .

2.4 The sentinel mechanism is realized

2.4.1 Copy sentry profile

Copy the sentry configuration file to the sentry Directory .

2.4.2 Modify the configuration file

1). Change protection mode

2). Turn on background operation

3). Modify the Sentinel's monitoring to the host's ip, Among them 1 Indicates the number of votes in effect .
A sentry wrote 1, More than half , Such as :3 A sentinel wrote 2,5 A sentinel wrote 3.

4). Election time after the shutdown of the sentry
If the host goes down 10 The selection starts in seconds .

5… When the election failed
explain : If the election does not end beyond the specified time , Re election .

2.4.3 Sentinel high availability test

1). Activate the sentry

[root@localhost sentinel]# redis-sentinel sentinel.conf

2). testing procedure :
1. Check the status of the host

2. take redis Main server down wait for 10 second Then check whether the slave is selected as the new host

3. restart 6379 The server ., Check whether it becomes the slave of the new host .

2.5 Description of the flat vote in the sentinel election

If there are multiple sentinels for election , If continuity 3 Time Vote failed , May trigger Split brain The occurrence of phenomena .
problem : What is the probability of the occurrence of brain fissure ?? 1/8 = 12.5%
Solutions : Just increase the number of elected nodes , It can effectively reduce the occurrence of brain fissure . probability theory

Split brain :
redis In the master-slave mode, cerebral fissure refers to network problems , Lead to redis master The node follows redis slave Nodes and sentinel Clusters are in different network partitions , At this time because sentinel The cluster cannot perceive master The existence of , Will put one slave The node is promoted to master node . At this point, there are two different master node , It's like one brain split into two .

In the cluster cleft problem , If the client is still based on the original master The node continues to write data , So new master Nodes will not be able to synchronize this data , When the network problem is solved ,sentinel The cluster will be the original master The node is reduced to slave node , Now it's time to start again master Data synchronization in , It will cause a lot of data loss .

2 A sentinel is prone to brain splitting , Because each vote is the same , Electoral defeat . 3 A non-stop ticket .

2.6 An introductory case of a linked sentry

Parameter description :
The first parameter of sentinel connection pool object , Host variable name . In the sentry's configuration file . The second parameter is String Type of set aggregate .

 /** * The Sentinel's test * Parameter description : masterName: Host variable name * sentinels: Set of linked sentinels . * understand : Sentinel although link 3 platform redis. however 3 platform redis The data stored in is the same . For users * It's just one . */
@Test
public void testSentinel(){

Set<String> sets = new HashSet<>();
//1. Pass the sentinel configuration information 
sets.add("192.168.126.129:26379");//26379 The default port number of the sentry 
// Create sentinel connection pool object 
JedisSentinelPool sentinelPool = new JedisSentinelPool("mymaster",sets);
// Get the resource file in the pool , Because the sentinel will be connected to a certain station in the future redis Process operation , Specific work redis It's single .
Jedis jedis = sentinelPool.getResource();
jedis.set("aaa", " sentinel test ");
System.out.println(jedis.get("aaa"));
jedis.close(); // Shut down after use 
}

retrieval , And close the sentry kill

2.7 SpringBoot Integrate Redis sentry (10 minute )

2.7.1 edit pro The configuration file

# To configure redis Single server
redis.host=192.168.126.129
redis.port=6379
# To configure redis Fragmentation mechanism
redis.nodes=192.168.126.129:6379,192.168.126.129:6380,192.168.126.129:6381
# Configure sentinel nodes If there are multiple sentinels, write them in pieces
redis.sentinel=192.168.126.129:26379

2.7.2 edit redis Configuration class

 package com.jt.config;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import redis.clients.jedis.JedisPoolConfig;
import redis.clients.jedis.JedisSentinelPool;
import java.util.HashSet;
import java.util.Set;
@Configuration // I am a configuration class 
@PropertySource("classpath:/properties/redis.properties")
public class JedisConfig {

@Value("${redis.sentinel}")
private String sentinel;//192.168.126.129:26379
@Bean
//@Scope("prototype")
public JedisSentinelPool jedisSentinelPool(){

Set<String> sentinels = new HashSet<>();
sentinels.add(sentinel);
// Number of connections in the control pool Do not write according to the default 
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setMaxTotal(100);// Connection pool Maximum number of links 
jedisPoolConfig.setMaxIdle(40);// The maximum amount of free time 
jedisPoolConfig.setMinIdle(20);// Minimum amount of free time 
/** * reflection : Why return the connection pool object instead of being a redis object ? * Because if you return a single object , It is equivalent to sharing a connection object in the future Not good. . * solve :1, Directly return the connection pool object . * 2, return redis Single object , add @Scope("prototype") Create multiple objects , Although this can solve , * But in this way, each time you create a connection pool object , The speed will decrease , Therefore, the optimal writing method is to directly return the pool object . */
return new JedisSentinelPool("mymaster",sentinels,jedisPoolConfig);
}
}

2.7.3 modify CacheAOP The injection term in

test :

2.8 A summary of the fragment sentinel

Fragmentation :
1. The main function is to realize the expansion of memory data .
2. Because the operation takes place in the business server , So the execution is more efficient .
3.Redis There is no highly available effect of fragmentation . If there is a problem with one of the nodes, the program will run incorrectly .

Sentinel mechanism :
1. Realization Redis High availability , When redis When the server goes down , Sentinels can flexibly monitor . Realize automatic election Fault migration .
2. Monitored by the sentry redis The data in the node is the same . Unable to realize massive data storage .
3. Although sentinels can achieve redis High availability , However, the sentinel itself did not achieve high availability . So there are risks .
If you want to minimize loss , It is recommended not to introduce third-party monitoring

upgrade :
need Redis Content expansion also requires Redis High availability, so you should use Redis colony .

3. Redis Cluster building

3.1 Steps to build

explain :Redis Cluster building steps See the pre class documentation .
Cluster is the combination of high availability and capacity expansion .
redis Clusters can monitor each other .

Be careful : The premise of building a cluster redis There's no data , because redis The data in the cluster needs to be stored according to a specific algorithm , If there is no calculation, the algorithm and redis The algorithms stored in the cluster are inconsistent , So wrong reporting .

3.2 About Redis Description of cluster setup error

Prerequisite : redis.conf Your configuration file should first be configured correctly There is... In the code cloud redis To configure .

Steps to build :

1. Close all Redis Services
   
2. Delete nodes.conf The configuration file
   Because after building the cluster , All cluster building information will be written to nodes.conf In file , If you restart next time, you will read the configuration information to realize redis Build the master-slave cluster . So if you need to rebuild the cluster , You must delete the file and regenerate .
   
3. restart Redis After the server, rebuild the cluster

# start-up redis
sh start.sh
# Cluster mount
redis-cli --cluster create --cluster-replicas 1 192.168.126.129:7000 192.168.126.129:7001 192.168.126.129:7002 192.168.126.129:7003 192.168.126.129:7004 192.168.126.129:7005

3.3 Description of the working principle of cluster

3.3.1 redis Cluster high availability test

1). Check redis Host state

2). Turn off the host
redis-cli -p 7000 shutdown

3). Check whether the host is switched

4). restart 7000 The server . Check if it is 7003 From the
exit sign out .

3.4 SpringBoot Integrate Redis colony

3.4.1 Entry case testing

 /** * redis An introductory case of cluster : * jedisCluster Operate the whole redis colony , link redis All the nodes of * reflection :key How to save to redis Medium 3 Console host ??? By algorithm . * redis Partition algorithm ??? */
@Test
public void testCluster(){

Set<HostAndPort> sets = new HashSet<>();
sets.add(new HostAndPort("192.168.126.129", 7000));
sets.add(new HostAndPort("192.168.126.129", 7001));
sets.add(new HostAndPort("192.168.126.129", 7002));
sets.add(new HostAndPort("192.168.126.129", 7003));
sets.add(new HostAndPort("192.168.126.129", 7004));
sets.add(new HostAndPort("192.168.126.129", 7005));
// Cluster object , Request value transfer HostAndPort Type of set aggregate 
JedisCluster jedisCluster = new JedisCluster(sets);
jedisCluster.set("cluster", " Cluster testing ");// Store value 
System.out.println(jedisCluster.get("cluster"));// Value 
}

3.5 About Redis Cluster interview questions （day15am）

principle : Redis Of Missing memory Then the cluster crashes
Downtime conditions ： If the number of primary nodes cannot be maintained, the cluster will crash . namely : If the host in the cluster is down , Then the slave can continue to provide services , When there is no slave in the host , It borrows redundant slaves from other hosts . Continued provision of services . If there is no slave available when the host goes down , Then the cluster crashes .

3.5.1 If 3 Lord 3 from (1 Lord 1 from ) redis Minimum nodes / Down a few , Cluster crash ? B

 A. 1 platform B.2 platform C.3 platform D.4 platform

3.5.2 If 3 Lord 6 from (1 Lord 2 from ) reida At least a few nodes are down , Cluster crash ? C

 A. 3 platform B.4 platform C.5 platform D.6 platform
** explain : If there are no child nodes It will borrow the redundant slave data from other hosts .**

3.6 About the electoral mechanism - Cleft brain

explain : When the cluster elects , If it's continuous 3 If the result of the even vote appears every time, the phenomenon of cerebral fissure may appear .
problem : What is the probability of cleft brain ??? 1/8
mathematical modeling :
Throw silver coins in succession 3 How much is the concept of flat ticket ? 1/8=12.5%
for the first time : Zhengzheng Pros and cons Anyway Against 1/2
The second time : Zhengzheng Pros and cons Anyway Against 1/2
third time : Zhengzheng Pros and cons Anyway Against 1/2
The prevention of : Increasing the number of master nodes can effectively reduce the occurrence of cleft brain .