Over the past month, Coinbase has continued to prioritize scaling our backend systems so that we can provide a reliable customer experience during periods of high traffic. Despite our improvements, on December 12th, all time high levels of traffic lead to periods of slowness and elevated error rates.
On December 11th at 22:00 PST Coinbase underwent scheduled maintenance to improve database performance. One of the techniques we utilized during this maintenance window was to distribute database load by splitting existing datasets to new clusters.
On December 12th, starting at 04:00 PST, traffic began to climb as a result of large Litecoin price movements. At 04:15 PST, two of the new clusters from the maintenance window began to experience slowness as a result of the slower network attached disks used by these instances to complete the migration. This database slowness resulted in elevated response times and high error rates. We were able to resolve these issues by 05:42 PST by failing over to high performance nodes which we had begun provisioning the night before.
Following these improvements, site traffic continued to climb, reaching all time high levels by 06:14 PST. Starting at 05:58 PST, one of our primary clusters began to experience degraded performance as a result of overwhelming query volume. We worked throughout the day to reduce pressure on this cluster by deploying application level improvements to more efficiently cache database queries. By 14:00 PST services were fully restored.
Scaling is hard, but we’re working hard to support the millions of people who are discovering cryptocurrency for the first time. We are looking to hire senior backend engineers in San Francisco, London and New York. If working on this sort of challenge excites you please see our careers page.
Author:
Source link
I am definitely bookmarking this website and sharing it with my acquaintances. You will be getting plenty of visitors to your website from me!
You posted quite a few nice points there. I did a search on the subject and found most people will have the same opinion with your blog.
… [Trackback]
[…] Read More to that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Information to that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Find More on that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Information on that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Information on that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Here you will find 40490 additional Information to that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Information to that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Here you can find 17324 more Info to that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Read More on on that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Find More here to that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] There you will find 96823 more Info on that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]
… [Trackback]
[…] Info on that Topic: tradingbtc.com/2018/12/02/december-12th-post-mortem-the-coinbase-blog/ […]