Minio Distributed Object Storage Architecture, Performance — XenonStack

4 min readOct 26, 2018

What is Minio?

Minio is an open source distributed object storage server written in Go, designed for Private Cloud infrastructure providing S3 storage functionality. Minio is the best server which is suited for storing unstructured data such as photos, videos, log files, backups, and container. Size of an object can be range from a KBs to a maximum of 5TB. Minio server is light enough with application stacks like Redis, MySql, and Gitlab.

How Minio Works?

Minio supports multiple pluggable storage backend including local disk, Kubernetes PVC, NAS and object storage provided by Public Cloud like Azure and GCP. Minio supports eraser coding which replicates & divides the data and spread it across multiple drives providing high availability and reliability.

It has a single standalone deployment with a single disk or multiple disks, distributed cluster where Minio runs on various hosts. Disks can be EBS, Google Disk, Ceph RBD, ISCSI disk, direct attached hard disk, or it can be a folder and folder backed by NFS. Minio uses erasure coding which divides data into smaller objects and spread across multiple disks protecting from disk failure, redundancy, and scalability.

Minio can also leverage existing object storage solutions like Azure blob storage and object storage on Google Cloud Platform. It acts as an S3 object layer transparently on top of existing object storage providing a single interface for accessing multiple Cloud storage.

Also, leverage Minio on existing Docker and Kubernetes environment providing object storage to Microservices. Minio can consume the disks in the form of PVC and store data on them.

Benefits of Minio

S3 API compatibility.
Data Redundancy.
High Availability.
Horizontal and vertical scaling.
Supports multiple Pluggable storage backend.
Data security using encryption on both server and client side

How to Adopt Minio?

Pre-requisites for Implementing Minio

One Ubuntu 16.04 server set up by following initial server setup.
A fully registered domain name.
Following DNS records set up MINIO server. Follow this hostname for details on how to add them.
A record with your server name (minio_server.com) pointing to your object server’s IPv4 address.
Install Go on Ubuntu 16.04.
Installing and Configuring MINIO server.
Installing the Systemd Startup Script.
Starting a MINIO Server.
Securing Access to MINIO Server With an SSL/TLS Certificate.
Securely connecting to MINIO with web Interface using HTTPS.

Why Minio Matters?

Minio in distributed mode to set up a highly — available storage system with a single object storage deployment. With distributed Minio, optimally use storage devices, irrespective of location in a network.

Data Protection

In this, Distributed Minio protects multiple nodes and drives failures and bit rot using erasure code. The minimum disks required for this distributed Minio is 4, this erasure code is automatically hit as distributed Minio launched.

High Availability

A stand-alone Minio server goes down if the server hosting the disk goes offline.There is also Distributed Minio setup with n number of the disk with safe data as long as more disks are online. Therefore, need minimum of disks to create new objects.

Limits

In Minio there are the stand-alone mode, the distributed mode has per usage required minimum limit 2 and maximum 32 servers. But there is no limit of disks shared across the Minio server. If we need multiple setups, quickly spin up multiple Minio Instances managed by tool like Kubernetes.

Best Practices of Minio

Distributed Minio is a Private Cloud deployed via Docker Compose and Swarm mode. The main difference between these two being, Docker Compose creates a single host, multi-container deployment, while Swarm mode creates a multi-host, multi-container deployment.

This means Docker compose get started with Distributed Minio on computer-ideal for deployment, testing staging environment. While deploying Distributed Minio on Swarm offers a more robust, production level deployment.

Minio Custom Access and Secret Keys.
Minio Custom Access and Secret Keys using Docker secrets.
Create a Minio service using Docker service to read from Docker secrets.
Minio Custom Access and Secret Key files.
Retrieving Container ID.
Starting and Stopping Containers.
Minio container logs.
Monitor Minio Docker Container.

Concluding Minio

Minio is the best Private Cloud Storage. It is a high performance distributed object storage server, designed for large-scale Private Cloud infrastructure. Minio widely deployed across the world with over 164.1M + Docker pulls. Minio is distributed object storage server written in Go and Open source under Apache version 2.0. To know more about minio in deep, you are advised to explore our below content:

Learn more about “Software-Defined Storage“
For more information, “Get in Touch” with us.
Explore more about “Hadoop Storage Format“

Originally published at https://www.xenonstack.com on October 26, 2018.