Cache breakdown ( Some hot spot key Cache failed )
There's no data in the cache, but there's data in the database , If it's hot data , that key At the moment the cache expires , There are a lot of requests at the same time , All of these requests go to DB, Cause instantaneous DB A lot of requests 、 Pressure builds up .
The difference between cache avalanche and cache avalanche is that this is for a certain key cache , The latter are many key.
The prevention of
** If the cached data is basically not updated ：** You can set the hotspot data not to expire
** If the cached data is not updated frequently ：** Set the mutex
When multiple requests arrive at the server , A request found the cache expired , So add a lock , Then go to the database to request data and update it to the cache , The next other requests go to the cache to find
If the cached data is updated frequently or the cache refresh process takes a long time ： Scheduled tasks regularly update the cache , Ensure that the request can access the corresponding cache all the way
Cache synchronization sync
sync You can instruct the underlying layer to lock the cache , So that only one thread can enter the calculation , Other threads are blocked , Until the returned result is updated to the cache
Cache avalanche of distributed cache + Solution
Cache avalanche ( Multiple hotspots key It's all overdue )
a large number of key Set the same expiration time , This causes all caches to fail at the same time , Cause instantaneous DB A lot of requests 、 The pressure surged , Cause an avalanche
The prevention of
The expiration time of stored data is set randomly , Prevent a large number of data expiration at the same time
Set hotspot data never to expire , Scheduled tasks are updated regularly
Set different expiration times
such as CacheManager Configure multiple expiration time dimensions
In advance ：Redis High availability , Master-slave + sentry ,Redis cluster, Avoid total collapse .
In the matter ： Local ehcache cache + hystrix Current limiting & Downgrade , avoid MySQL Killed .
After the event ：Redis Persistence , Once the restart , Automatically load data from disk , Quick recovery of cached data .
Current limiting components , Can set requests per second , How many can pass through the components , Remaining failed requests , What do I do ？ Take the demotion ！ You can return some default values , Or friendship tips , Or null .
Database will never die , The current limiting component ensures how many requests can pass each second .
As long as the database doesn't die , That is to say , For users ,2/5 All requests of can be processed .
As long as there is 2/5 Can be processed , That means your system is not dead , For users , Maybe it's just that you can't swipe the page after clicking several times , But a few more times , You can brush it out
Cache penetration of distributed cache + Solution
Cache penetration （ Query data does not exist ）
Query a nonexistent data , Because the cache is not hit , And for the sake of fault tolerance , If initiated as id by “-1” Nonexistent data
If the data cannot be found from the storage layer, it will not be written to the cache, which will cause the nonexistent data to be queried from the storage layer every request , It loses the meaning of caching . There are a lot of data that does not exist for query , Probably DB It's gone , This is also a non-existent way for hackers to take advantage of key A way to attack applications frequently .
The prevention of
Verification is added to the interface layer , Data rationality verification
Cache unreachable data , In the database, there is no access to , At this time, you can also key-value Write as key-null, Set a shorter expiration time , Prevent the same key Being attacked all the time
Empty results are also cached , Default is not configured condition perhaps unless Just go
Add bloom filter before caching , Map all possible data hashes in the database to the bloom filter . Then make the following judgment for each request ：
Requesting data key Does not exist in the bloom filter , You can be sure that the data will not exist in the database , The system can immediately return non-existent .
Requesting data key In the bloom filter , Then continue to query the cache .
Using Bloom filter can play a certain role in screening access requests , Avoid the query pressure caused by the absence of data .
Online education - Day hot video list actual combat -List Data structure design
Need a video learning list , Update once a day
Need to support manual operation to replace the position on the list
Processes in the enterprise
The timed task calculates the video that most people learned yesterday
evening 12 Point to 1 Click to update to the list
Reserve an interface , Support manual operation
JD.COM ： Top selling mobile phone list 、 Computer lists, etc
Baidu ： Search hot list
Really high and send the project , Are pre calculated results , Then store it directly into the cache , If necessary, return directly to , And the storage structure is the simplest
[ Failed to transfer the external chain picture , The origin station may have anti-theft chain mechanism , It is suggested to save the pictures and upload them directly (img-E21I85Ii-1636980700068)(/Users/dk/Library/Application Support/typora-user-images/image-20211108134042136.png)]
Self operated e-commerce platform - Shopping cart implementation case -Hash Data structure best practices
E-commerce shopping cart realizes , Support to buy multiple items , Each item can be bought in different quantities
Support high-performance processing
Common implementation methods of shopping cart
Implementation mode I : Store in database
There is a performance bottleneck
Implementation mode II : Front end local storage -localstorage-sessionstorage
localstorage Store... In a browser key/value Yes , No expiration time .
sessionstorage Store... In a browser key/value Yes , This data will be deleted after closing the session window .
Implementation mode 3 : The back end is stored in the cache, such as redis
Can be opened AOF Persistence prevents restart loss ( recommend )
Introduction to shopping cart data structure
In a shopping cart , There are multiple shopping items
therefore The shopping cart structure is a double-layer Map：
first floor Map,Key Is the user id
The second floor Map,Key It's the goods in the shopping cart id, The value is the shopping cart data
Corresponding redis The storage inside
redis There are many kinds of data structures , What kind of ？
The answer is hash structure
De duplication of user portrait labels under big data required by case practice
User portrait English is User Profile, It is based on the user's basic attributes 、 Social attributes 、 Behavior properties 、 Psychological attributes and other real information 、 Virtual user model .“ User portrait ” The essence of is to “ people ” Digitization of .
There are many scenarios , For example, personalized recommendation 、 Precision marketing 、 Financial risk control 、 Fine operation and so on , Take an example to understand the practical application value of user portrait , We often use mobile phone online shopping , Thousands of people and faces in Taobao
adopt “ label tag” To refine and identify the multi-dimensional characteristics of users , Then everyone's user portrait needs to be stored ,set The collection is suitable for de duplication
User portraits are not just for someone , It can also be a portrait of a certain group or industry
utilize redis Can be a good weight removal
Case practice SortedSet User points real-time list best practices
Users play games - Point real-time list
IT Video hot selling real-time list
Real time list of hot sales of e-commerce products
The general ranking list reads more and writes less , It can be done to master Write operation , And then many slave Read operation .
If it's an object, remember to rewrite HashCode And Equals Method
copyright：author[You and other students]，Please bring the original link to reprint, thank you.