Through the majority of 2020 the infrastructure powering Otaku Square has mainly consisted of 3 servers: a webserver, a "system" server and a database server.

While this architecture worked, it was very hard to keep stuff up-to-date. In fact, no critical updates could be applied without taking down the majority of the website.

To improve upon this in 2021, and ensure high availability I have spent the better part of last month migrating all services running on these servers to single-responsibility VPS servers at a new provider: Hetzner.

The shop itself

As you may be aware, earlier this year I did a complete rewrite of the webshop. This rewrite was really aimed at ensuring that the infrastructure powering it is maintainable and scaleable.

As an example, if I was to take down the product API which crunches data from suppliers and determines a product's taxonomy, the website will currently still function, and you'd still be able to place an order as if nothing was wrong.

This is achieved by introducing a layer between the shop and the actual data. This layer currently consists of two MongoDB servers. Currently, the set-up looks a bit like this:

Power in numbers

Just as a bit of a brag, here's the total power of the set-up above:

32 VCPU
86GB RAM
940GB SSD disk space

In addition to those numbers, there is also the CDN. The CDN set-up is fairly basic. It mainly consists of an S3 bucket which is hooked up to a CloudFront distribution (all edges.)

Again, a bit of a brag, here's the latest numbers for the CDN:

~1.2TB of bandwidth over HTTPS every month
~2 million requests served every month

Administering this infrastructure

Maintaining this amount of servers is not easy, and if it weren't for Ansible it'd likely be a hell to ensure that everything is monitored and up-to-date.

Next to all applications listed in the diagram, there is also a Zabbix and Cachet (https://status.otakusquare.net) server.

To be continued

While this little write-up may be interesting, I'm pretty sure most of you will be more interested in the way these actual systems work.

I hope to write up a blog article for every major API in the near future, so be sure to add my RSS feed to whatever fancypants application you're using to stay up to date!