FileCloud
Child pages
  • Setting up FileCloud Managed S3 Storage
Skip to end of metadata
Go to start of metadata

As an administrator, you can integrate FileCloud Server to store user data on an Amazon S3 storage server.

  • Amazon Simple Storage Service (Amazon S3) is storage for the Internet.
  • You can use Amazon S3 to store and retrieve any amount of data at any time, from anywhere on the web.
  • You can accomplish these tasks using the AWS Management Console.

Getting Started with Amazon Simple Storage Service


WARNINGS:

  • Only change the FileCloud storage type to S3 for new installations.
  • Do not change the FileCloud storage type to S3 if FileCloud has been in use and data is already stored.
  • Be very careful when changing the storage path, If done improperly it could lead to data loss. 
  • When changing the storage type from local to Amazon S3, the files and folders that have already been saved to local storage will not automatically be moved to S3 storage.
    • For existing files and folders, the administrator must manually export them from local storage before changing the storage type.
    • After changing the storage type to S3, the administrator must manually import pre-existing files and folders.
  • The S3 Bucket should NEVER be modified outside of FileCloud subsystem.
  • Do not add/edit/modify files directly using S3 tools. Doing so will destabilize your FileCloud installation.


Integrate Amazon S3 Storage


 1. Change the Storage Type to S3

NOTE:

In this step you will need to access WWWROOT. It is typically located at: 

Windows

Linux

(later than Ubuntu 14.04)

Linux

(earlier than Ubuntu 14.04)

c:\xampp\htdocs/var/www/html/var/www

To enable Amazon s3 storage as the backend:

  1.  To make sure that your server does not have any time variations, set up the time on your server to be synchronized.
    1. Configure an authoritative time server in Windows Server
    2. Synchronize Time with NTP in Linux

  2. Open the following file for editing:

    WWWROOT/config/cloudconfig.php

  3. Find the following line:

    define("TONIDOCLOUD_STORAGE_IMPLEMENTATION", "local");

  4. Change it to this line:

    define("TONIDOCLOUD_STORAGE_IMPLEMENTATION", "amazons3");
  5. Save and close the file.

  6. Find the following file: 

    WWWROOT/config/amazons3storageconfig-sample.php

  7. Rename it to:

    WWWROOT/config/amazons3storageconfig.php

(lightbulb) Nothing needs to be added or edited in amazons3storageconfig.php

 2. Configure Credentials

After you have setup the storage implementation key in step 1, you can configure the following credentials:

Field

Description

S3 KeyThis is your amazon authentication key (To get your access key, visit Amazon security portal) . For IAM user, it requires at least the following permissions.
S3 SecretThis is your amazon authentication secret (To get your access key, visit Amazon security portal). For IAM user, it requires at least the following permissions .
S3 Bucket Name

Provide a bucket name.

The bucket should be new (in some circumstances, previously used buckets in FileCloud could be used).

It is very important that the S3 bucket is never modified outside of the FileCloud subsystem.

S3 Storage Folder

Optional: All files will be stored inside this root storage folder.

  • This folder will be created automatically.
S3 Region

Optional: Provide the region string. If the region is not provided, then US Standard region will be used.

  • If your bucket is in a different region, (Europe, Asia) provide the correct region string. The strings should match the region string published by amazon.
  • Note: For govcloud installs, you must use region string: us-gov-west-1
S3 End Point URL

Optional: This is the S3 endpoint.

  • Use this to specify your own S3 endpoint (typically S3 compatible storage)
  • Use this if it is a unpublished region.

To use an AWS end point, it must be one of the values published AWS S3 endpoints

To configure Digital Ocean S3 Credentials

  1. Open a browser and log into Admin Portal.
  2. In the left navigation panel, under SETTINGS, select Settings. 
  3. On the Manage Settings screen, select the Storage tab.
  4. Type in or select the settings for your environment.
  5. Click Save.
 3. Enable Encryption

To protect data at rest in Filecloud Server, you can use S3 Managed Storage Encryption.

  • The communication from FileCloud to AWS will use SSL encryption resulting in complete protection for data in transit.
  • Once the S3 is setup correctly, a new field called S3 Encryption will be available under Amazon S3 Storage Settings.

FileCloud supports the following Server Side Encryption:

Encryption TypeNotes
Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3)All data is encrypted at rest using AES256 bit encryption. The data can only be accessed using the supplied key/secret credentials. The data will be accessible via S3 Console (which should NOT done for FileCloud Managed storage data)
Server-Side Encryption with AWS KMS-Managed Keys (SSE-KMS)Similar to SSE-S3 but the key itself is managed using Amazon's KMS service. This allows management of specific keys and their permissions for encrypting the data. The data is still encrypted at rest and is accessible via S3 Console with appropriate credentials.
Server-Side Encryption with Customer-Provided Keys (SSE-C) This is a new support available from FileCloud v15 on-wards. The data will be encrypted using customer supplied 32 bit encryption key. This option will have SLOWER performance due to restriction on how this data can be decrypted (Amazon server will NOT be able to decrypt the data and the data has be first downloaded to FileCloud server and decrypted). The data will NOT be accessible via S3 console as well.

WARNINGS:

  • Enabling encryption will start a process that attempts to encrypt all available data in the bucket as well as all new data.
  • This process can take some time depending on the amount of existing data in the bucket.
  • It is recommended that you modify the encryption setting when there is minimal activity on the FileCloud Server.

Although changing the Encryption setting can be done at any time, we recommend using off-peak hours to avoid any unexpected access issues.


To enable S3 encryption:

 (warning) If you are not running the current version of FileCloud Server:

You must enable an additional extension in the php.ini file

 

  1. On the FileCloud server, open the following file for editing:

    WEBROOT\php\php.ini

  2. Add the following line to the file:

    extension=php_com_dotnet.dll
  3. Save your changes and close the file.
  4. Restart the Apache server.

(warning)  If you are running FielCloud Server on Windows

AND

      Your xampp folder is installed in a location other than c:\xampp

You must add a key to the cloudconfig.php file

  • For example, if your xampp folder is in D:\xampp\htdocs\config\cloudconfig.php
  • Then you would add the following line: define("PHPBIN_PATH","D:\\xampp\\php\\php.exe");


  1.  On the FileCloud Server, open the following file for editing:

    <your xampp folder>\htdocs\config\cloudconfig.php
  2. Add the following line anywhere:

    Replacing <location> with your path to the xampp folder

    define("PHPBIN_PATH","<location>:\\xampp\\php\\php.exe");
  3. Save your changes and close the file.

  1. Open a browser and log into Admin Portal.
  2. In the left navigation panel, under SETTINGS, select Settings. 
  3. On the Manage Settings screen, select the Storage tab.
  4. On the Storage tab, click Manage.
  5. On the Manage S3 Encryption dialog, in Encryption Type, select Amazon S3.
  6. Click Enable encryption.



Troubleshooting


 Using Override Configuration Keys

The following keys are not typically used, however they may be needed in specific circumstances. 

KEYVALUEDescription
TONIDOCLOUD_NODE_COMMON_TEMP_FOLDER"/somepath/location"In HA installs, temp folder must be a commonly accessible location. This key must be set in each of the HA nodes
TONIDOCLOUD_S3_PROXY"http://proxyaddress" or "http://ip"If a proxy is set in the env, then this key must be set to allow FileCloud service to use the proxy to access S3 servers
TONIDOCLOUD_S3_REDUCED_REDUNDANCY"1"This will store the objects with "reduced redundancy"
TONIDOCLOUD_DISABLE_S3_REDIRECT"1"(NOT RECOMMENDED) This will force filecloud server to download the file from S3 to the filecloud server system and then send it to client on file downloads (Can be slow)
 How to Correct Issues with Image Previews

If you are having problems in previewing images, you should add a line to the .htaccess file.

To add a line to the .htaccess file:

  1. Open the following file:
    1. Windows:  C:\xampp\htdocs\.htaccess
    2. Linux: /var/www/html/.htaccess

  2. Add the following line:

    Header set Content-Security-Policy: "default-src 'self' *.live.com; style-src 'unsafe-inline' 'self';script-src 'unsafe-inline' 'unsafe-eval' 'self';font-src 'self' data:;img-src 'self' *.amazonaws.com *.live.com  data: *.duosecurity.com"
 How to Correct Issues with Text Editors

If you encounter issues where documents stored in AmazonS3 share object storage cannot be edited using a text editor, you can use a workaround to correct this.

Workaround:

  1.  Change the Header set in the Content-Security-Policy
  2. Use the Amazon S3 console to add a cross-origin resource sharing (CORS) configuration to an S3 bucket.

Change the Content-Security-Policy

Content Security Policy (CSP) is an HTTP header that allows site operators control over where resources can be loaded from on their site.

  • The use of this header is the best method to prevent cross-site scripting (XSS) vulnerabilities. 

To change the Header set in CSP:

  1. Open a command-line prompt.

  2. Type in the following code (or copy and paste):

    Content-Security-Policy: "default-src 'self' *.live.com *.amazonaws.com; style-src 'unsafe-inline' 'self';script-src 'unsafe-inline' 'unsafe-eval' 'self';font-src 'self' data:;img-src 'self' data

Add a CORS Policy

To configure your bucket to allow cross-origin requests, you add CORS configuration to the bucket. A CORS configuration is an XML document that defines rules that identify the origins that you will allow to access your bucket, the operations (HTTP methods) supported for each origin, and other operation-specific information.

For more information about CORS, see Cross-Origin Resource Sharing (CORS) in the Amazon Simple Storage Service Developer Guide.

To allow the use of a text editor:

(lightbulb)  The CORS configuration is an XML file. The text that you type in the editor must be valid XML.

  1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
  2. In the Bucket name list, choose the name of the bucket that you want to create a bucket policy for.
  3. Choose Permissions, and then choose CORS configuration.

  4. In the CORS configuration editor text box, type or copy and paste the following CORS configuration:

    <CORSConfiguration>          
    <CORSRule>            
    <AllowedOrigin>*</AllowedOrigin>            
    <AllowedMethod>GET</AllowedMethod>            
    <AllowedHeader>*</AllowedHeader>        
    <MaxAgeSeconds>3000</MaxAgeSeconds>          
    </CORSRule>
</CORSConfiguration>

  5. Click Save.

 How to Correct Issues with playing mp4 videos

If you are having problems in playing  mp4 videos, you should add a line to the .htaccess file.

To add a line to the .htaccess file:

  1. Open the following file:
    1. Windows:  C:\xampp\htdocs\.htaccess
    2. Linux: /var/www/html/.htaccess

  2. Add the following line:

    Header set Content-Security-Policy: "default-src 'self' *.live.com *.amazonaws.com; style-src 'unsafe-inline' 'self';script-src 'unsafe-inline' 'unsafe-eval' 'self';font-src 'self' data:;img-src 'self' *.live.com  data: *.duosecurity.com *.amazonaws.com"

Add a CORS Policy

To configure your bucket to allow cross-origin requests, you add CORS configuration to the bucket. A CORS configuration is an XML document that defines rules that identify the origins that you will allow to access your bucket, the operations (HTTP methods) supported for each origin, and other operation-specific information.

For more information about CORS, see Cross-Origin Resource Sharing (CORS) in the Amazon Simple Storage Service Developer Guide.

To allow the use of a text editor:

(lightbulb)  The CORS configuration is an XML file. The text that you type in the editor must be valid XML.

  1. Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
  2. In the Bucket name list, choose the name of the bucket that you want to create a bucket policy for.
  3. Choose Permissions, and then choose CORS configuration.

  4. In the CORS configuration editor text box, type or copy and paste the following CORS configuration:

    <CORSConfiguration>          
    <CORSRule>            
    <AllowedOrigin>*</AllowedOrigin>            
    <AllowedMethod>GET</AllowedMethod>            
    <AllowedHeader>*</AllowedHeader>        
    <MaxAgeSeconds>3000</MaxAgeSeconds>          
    </CORSRule>
</CORSConfiguration>

  5. Click Save.


  • No labels