Creating a Cloud Storage Bucket based on Open Source with Typescript

This blog provides you with a step-by-step code example how to create a bucket to store your files in the cloud with Eyevinn Open Source Cloud and MinIO.

Open Source Cloud offers a refreshing alternative that is free from vendor lock-in as the services are based on open source.

– Free tier with 50 GB of storage
– Completely compatible with popular tools like AWS S3
– No complex setup or massive technical knowledge required
– Supports media, images, documents, and more

Follow this step-by-step guide to get started.

Step 1: Sign-up and Setup

Create an account for free at https://app.osaas.io and create your tenant. If you already have access to Eyevinn Open Source Cloud you can skip this step.

Obtain your personal access token by navigating to Settings / API in the web console. Store this value in an environment variable called OSC_ACCESS_TOKEN :

% export OSC_ACCESS_TOKEN=<personal-access-token>

Install Eyevinn Open Source Cloud SDK for Typescript.

% npm install --save @osaas/client-core
% npm install --save @osaas/client-services

Install the AWS S3 client (this does not require an AWS account).

% npm install --save @aws-sdk/client-s3

Install the MinIO client

% npm install --save minio

Step 2: Write example main function

Now we are ready to start writing a function to create a storage bucket in Eyevinn Open Source Cloud. Let us start with a main function including creation of storage and as an example use the AWS S3 client to list buckets on our storage.

import { Context } from '@osaas/client-core';
import { S3Client, ListBucketsCommand } from '@aws-sdk/client-s3';

async function main() {
  const ctx = new Context();
  const storage = await createStorageBucketInEyevinnOSC('mybucket', ctx);
  const client = new S3Client({
    endpoint: storage.endpoint,
    credentials: {
      accessKeyId: storage.accessKeyId,
      secretAccessKey: storage.secretAccessKey
    }
  });
  const command = new ListBucketsCommand({});
  const data = await client.send(command);
  console.log(data.Buckets?.map((b: any) => b.Name));
}

Step 3: Write function to create storage bucket

Now let us develop the function that creates the storage bucket in Eyevinn Open Source Cloud.

import * as Minio from 'minio';
import { Context, getInstance, waitForInstanceReady } from '@osaas/client-core';
import {
  MinioMinio,
  MinioMinioConfig,
  createMinioMinioInstance
} from '@osaas/client-services';

async function createStorageBucketInEyevinnOSC(name: string, ctx: Context) {
  // Generate a service access token for MinIO service
  // and check whether we already have an instance running
  const sat = await ctx.getServiceAccessToken('minio-minio');
  let instance: MinioMinio = await getInstance(ctx, 'minio-minio', name, sat);
  if (!instance) {
    // If no instance found let us create one.

    // Generate a random root password for accessing the storage
    // and create an instance
    const rootPassword = randomBytes(16).toString('hex');
    const config: MinioMinioConfig = {
      name,
      RootUser: 'root',
      RootPassword: rootPassword
    };
    const newInstance = await createMinioMinioInstance(ctx, config);
    instance = newInstance;
    // Wait for instance to be ready and give it an extra 2 seconds
    // to be sure.
    await waitForInstanceReady('minio-minio', name, ctx);
    await delay(2000);

    // Use the MinIO client SDK to create a bucket and set a public
    // bucket policy
    const minioClient = new Minio.Client({
      endPoint: new URL(instance.url).hostname,
      accessKey: 'root',
      secretKey: instance.RootPassword || ''
    });
    await minioClient.makeBucket(name);
    await minioClient.setBucketPolicy(
      name,
      JSON.stringify(createPublicBucketPolicy(name))
    );
  }

  return {
    name: instance.name,
    endpoint: instance.url,
    accessKeyId: 'root',
    secretAccessKey: instance.RootPassword || ''
  };
}

The bucket policy we will be using in this example allows public and anonymous access to this bucket.

function createPublicBucketPolicy(name: string) {
  return {
    Statement: [
      {
        Action: ['s3:GetBucketLocation', 's3:ListBucket'],
        Effect: 'Allow',
        Principal: {
          AWS: ['*']
        },
        Resource: [`arn:aws:s3:::${name}`]
      },
      {
        Action: ['s3:GetObject'],
        Effect: 'Allow',
        Principal: {
          AWS: ['*']
        },
        Resource: [`arn:aws:s3:::${name}/*`]
      }
    ],
    Version: '2012-10-17'
  };
}

Step 4: Upload a file to bucket

To upload a file to this bucket we can use the AWS S3 client.

% export AWS_ACCESS_KEY_ID=root AWS_SECRET_ACCESS_KEY=<RootPassword>
% aws s3 \
  --endpoint https://eyevinnlab-mybucket.minio-minio.auto.prod.osaas.io \
  cp pod-sprint24.m4a s3://mybucket/
upload: ./pod-sprint24.m4a to s3://mybucket/pod-sprint24.m4a 

The endpoint is the storage.endpoint i.e. the URL to the MinIO instance created.

As this was created with public access we can access this files directly from a web browser using the URL: https://eyevinnlab-mybucket.minio-minio.auto.prod.osaas.io/mybucket/pod-sprint24.m4a

Summary

We have in this blog post shown an example on how you can easily enable storage for your application using a open web service available in Eyevinn Open Source Cloud. The key benefit of using a service based on open source is that you are not locked in with a specific vendor and you have the option to run the very same code in your own infrastructure or cloud.