Redis [4] case introduction + cache breakdown, cache avalanche and cache penetration

You and other students 2022-02-13 08:09:00 阅读数:34

redis case introduction cache breakdown

redis4 Case introduction + Cache breakdown 、 Cache avalanche 、 Cache penetration

This article has been arranged since childhood d Class notes and java Advanced warehouse , any similarity , Most of them are written by others

java Advanced warehouse :

Cache breakdown of distributed cache + Solution

  • Cache breakdown ( Some hot spot key Cache failed )

    • There's no data in the cache, but there's data in the database , If it's hot data , that key At the moment the cache expires , There are a lot of requests at the same time , All of these requests go to DB, Cause instantaneous DB A lot of requests 、 Pressure builds up .
    • The difference between cache avalanche and cache avalanche is that this is for a certain key cache , The latter are many key.
  • The prevention of

    • ** If the cached data is basically not updated :** You can set the hotspot data not to expire
    • ** If the cached data is not updated frequently :** Set the mutex

    When multiple requests arrive at the server , A request found the cache expired , So add a lock , Then go to the database to request data and update it to the cache , The next other requests go to the cache to find

    • If the cached data is updated frequently or the cache refresh process takes a long time : Scheduled tasks regularly update the cache , Ensure that the request can access the corresponding cache all the way
  • SpringCache Solution

    • Cache synchronization sync
    • sync You can instruct the underlying layer to lock the cache , So that only one thread can enter the calculation , Other threads are blocked , Until the returned result is updated to the cache

Cache avalanche of distributed cache + Solution

  • Cache avalanche ( Multiple hotspots key It's all overdue )

    • a large number of key Set the same expiration time , This causes all caches to fail at the same time , Cause instantaneous DB A lot of requests 、 The pressure surged , Cause an avalanche

    • The prevention of

      • The expiration time of stored data is set randomly , Prevent a large number of data expiration at the same time
      • Set hotspot data never to expire , Scheduled tasks are updated regularly
    • SpringCache Solution

      • Set different expiration times
      • such as CacheManager Configure multiple expiration time dimensions
      • The configuration file time-to-live To configure
       time-to-live: 3600000
      # Whether to cache empty results , Prevent cache penetration , Tacit thinking true
      cache-null-values: true

The solution of the avalanche is as follows :

  • In advance :Redis High availability , Master-slave + sentry ,Redis cluster, Avoid total collapse .
  • In the matter : Local ehcache cache + hystrix Current limiting & Downgrade , avoid MySQL Killed .
  • After the event :Redis Persistence , Once the restart , Automatically load data from disk , Quick recovery of cached data .

Current limiting components , Can set requests per second , How many can pass through the components , Remaining failed requests , What do I do ? Take the demotion ! You can return some default values , Or friendship tips , Or null .

benefits :

  • Database will never die , The current limiting component ensures how many requests can pass each second .
  • As long as the database doesn't die , That is to say , For users ,2/5 All requests of can be processed .
  • As long as there is 2/5 Can be processed , That means your system is not dead , For users , Maybe it's just that you can't swipe the page after clicking several times , But a few more times , You can brush it out

Cache penetration of distributed cache + Solution

  • Cache penetration ( Query data does not exist )
    • Query a nonexistent data , Because the cache is not hit , And for the sake of fault tolerance , If initiated as id by “-1” Nonexistent data
    • If the data cannot be found from the storage layer, it will not be written to the cache, which will cause the nonexistent data to be queried from the storage layer every request , It loses the meaning of caching . There are a lot of data that does not exist for query , Probably DB It's gone , This is also a non-existent way for hackers to take advantage of key A way to attack applications frequently .
    • The prevention of
      • Verification is added to the interface layer , Data rationality verification
      • Cache unreachable data , In the database, there is no access to , At this time, you can also key-value Write as key-null, Set a shorter expiration time , Prevent the same key Being attacked all the time
    • SpringCache Solution
      • Empty results are also cached , Default is not configured condition perhaps unless Just go

Common methods

Add bloom filter before caching , Map all possible data hashes in the database to the bloom filter . Then make the following judgment for each request :

  • Requesting data key Does not exist in the bloom filter , You can be sure that the data will not exist in the database , The system can immediately return non-existent .
  • Requesting data key In the bloom filter , Then continue to query the cache .

Using Bloom filter can play a certain role in screening access requests , Avoid the query pressure caused by the absence of data .

Online education - Day hot video list actual combat -List Data structure design

  • demand
    • Need a video learning list , Update once a day
    • Need to support manual operation to replace the position on the list
  • Processes in the enterprise
    • The timed task calculates the video that most people learned yesterday
    • evening 12 Point to 1 Click to update to the list
    • Reserve an interface , Support manual operation
  • Similar scenes
    • JD.COM : Top selling mobile phone list 、 Computer lists, etc
    • Baidu : Search hot list
  • Really high and send the project , Are pre calculated results , Then store it directly into the cache , If necessary, return directly to , And the storage structure is the simplest

[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-E21I85Ii-1636980700068)(/Users/dk/Library/Application Support/typora-user-images/image-20211108134042136.png)]

Self operated e-commerce platform - Shopping cart implementation case -Hash Data structure best practices

  • background

    • E-commerce shopping cart realizes , Support to buy multiple items , Each item can be bought in different quantities
    • Support high-performance processing
  • Common implementation methods of shopping cart

    • Implementation mode I : Store in database
      • There is a performance bottleneck
    • Implementation mode II : Front end local storage -localstorage-sessionstorage
      • localstorage Store... In a browser key/value Yes , No expiration time .
      • sessionstorage Store... In a browser key/value Yes , This data will be deleted after closing the session window .
    • Implementation mode 3 : The back end is stored in the cache, such as redis
      • Can be opened AOF Persistence prevents restart loss ( recommend )
  • Introduction to shopping cart data structure

    • In a shopping cart , There are multiple shopping items
    • therefore The shopping cart structure is a double-layer Map:
      • Map<String,Map<String,String>>
      • first floor Map,Key Is the user id
      • The second floor Map,Key It's the goods in the shopping cart id, The value is the shopping cart data
  • Corresponding redis The storage inside

    • redis There are many kinds of data structures , What kind of ?
    • The answer is hash structure

De duplication of user portrait labels under big data required by case practice

  • Introduce
    • User portrait English is User Profile, It is based on the user's basic attributes 、 Social attributes 、 Behavior properties 、 Psychological attributes and other real information 、 Virtual user model .“ User portrait ” The essence of is to “ people ” Digitization of .
    • There are many scenarios , For example, personalized recommendation 、 Precision marketing 、 Financial risk control 、 Fine operation and so on , Take an example to understand the practical application value of user portrait , We often use mobile phone online shopping , Thousands of people and faces in Taobao
    • adopt “ label tag” To refine and identify the multi-dimensional characteristics of users , Then everyone's user portrait needs to be stored ,set The collection is suitable for de duplication
    • User portraits are not just for someone , It can also be a portrait of a certain group or industry
    • utilize redis Can be a good weight removal

Case practice SortedSet User points real-time list best practices

  • background
    • Users play games - Point real-time list
    • IT Video hot selling real-time list
    • Real time list of hot sales of e-commerce products
    • The general ranking list reads more and writes less , It can be done to master Write operation , And then many slave Read operation .
    • If it's an object, remember to rewrite HashCode And Equals Method

copyright:author[You and other students],Please bring the original link to reprint, thank you.