HubStor Inc.

How To Create Connectors In HubStor


This article describes the process of how to create connectors in the HubStor Connector Service (HCS). 


Add Connector 

In the HubStor Connector Service (HCS) app, go to  'Connectors' and click 'Add Connector '. 

 

 

In the drop-down, select the type of connector and click ' OK '. 

 

 

This will display the Connector Properties windows. 

Configure Connector's General Settings 

 

 

In the Connector Properties window, complete the following settings: 

  1. Connector Name -- This is a display name. It is generally a good idea to name this something you will easily recognize as this name will display in the HubStor Export Utility and throughout the Admin Portal. 
  2. Stor -- The Stor where this connector will write data to. Below are the common Stor names:
  1. For capturing file data:  Files
  2. SharePoint/OneDrive:  SharePoint-ODB
  3. O365 Audit Log:   O365 Audit Log
  4. O365 Mailboxes:  O365 Mailboxes
  5. For capturing Box.com data:  Box
  1. #Threads -- By default, this is set to 10. For faster performance writing to the cloud, increase this number. 
  2. Get Location ACLs -- This setting will differ based on connector type. Reference the article specific to the connector type for guidance with this option. 
  3. Get Item ACLs -- This setting will differ based on connector type. Reference the article specific to the connector type for guidance with this option. 
  4. Pre-ingest Encrypt -- This option is only available when the Stor selected has the 'Support Pre-ingest Encryption' option enabled. See article: ENCRYPTION AT REST WITH YOUR ENCRYPTION KEY 

  5. To continue configuring Polices (Write, Stub, etc), skip to Configure Connector's Write Policy section.  Otherwise, click OK to save the changes thus far, and in the next section, choose the link for the connector type required.  You may then come back to this article to configure polices. 
     

Configure Connector's Type Specific Settings

Each type of connector has different settings.  Reference the links below for specifics on the type of connector to be configured.    

  1. Amazon S3 -- Provides the ability for HubStor to crawl the folders and files from an S3 bucket (supporting article pending)
  1. Azure Blob Storage -- For capturing data directly from Blob Storage. (supporting article pending)
  1. Box -- For capturing data from a Box.com account(s).  How To Configure A Box.com Connector
  1. Centera -- Used for migrating data from a Centera device directly to HubStor.  How To Create A Centera Connector
  1. Eml/MsgFile -- For archiving of .EML or .MSG files residing on a file share with smart message processing. This connector type is intended for email archive migrations and/or ingestion of SMTP Journaling feeds. How to create EML/MSG connectors
  1. Exchange/Exchange Online -- For archiving or backup via Exchange Web Services (EWS) of mailboxes hosted in Office 365 or on premise Exchange Server 2010, 2013, or 2016. HOW TO CREATE CONNECTORS FOR OFFICE 365 / EXCHANGE MAILBOXES.
  1. Files -- For archiving or backup of Windows/CIFS/NFS file storage. HOW TO CREATE CONNECTORS FOR FILES
  1. O365 Audit Log -- For harvesting the event history data from Office 365's unified audit log.  How To Create an O365 Audit Log Connector
  1. PST Files -- For deep processing of PST files whereby the contents are extracted as folders and individual message items with smart processing of the message recipients and metadata. HCS processes any PSTs in the configured path with the smart message extraction.  (supporting article pending)
  1. SharePoint/SPO/Groups/OneDrive for Business/Teams -- For harvesting various types of Site Collections from SharePoint on-premises or in Office 365. How To Create SharePoint Connectors
  1. SFTP -- For capturing data directly from an SFTP server.  How To Create FTP/SFTP Connectors
  1. FTP -- For capturing data directly from an FTP server. How To Create FTP/SFTP Connectors 
  1. Slack -- For capturing different data sources from a Slack environment.   How to Create a Slack Connector


Configure Connector's Write Policy 

The Write Policy is what tells the connector which data to capture.  There are optional inclusion and exclusion criteria that can be configured.

If you wish to include ALL data (I.e. 'get everything' policy), click the 'Add All Grouping' button.  




If there is a need to be more specific, reference the article for configuring specific metadata clauses within the Write Policy.  

 

Configure Connector's Stub Policy (Optional)

Stub policies also have the inclusion and exclusion options as criteria.  The type of stub the policy created is determined by the connector's stub setting.  Stubbing is only supported for File Systems and SharePoint Document Libraries.  

For information on Stubbing, see:  Overview of Stubbing 

Configure Connector's Delete Policy (Optional)

A Delete Policy allows for removing original content from the source after it has been successfully uploaded to HubStor in the cloud.  The policy allows for optionally creating inclusion and exclusion criteria.  

For information on this policy, see: How To Configure A Delete Policy 

Configure Connector's Schedule for Crawling (Optional)

A connector can be configured to run on a specific schedule. Click the ' Add Task ' button to configure the schedule settings. 

Configure Connector's Notifications (Optional)

In this section, configure an email address (or multiple) to get notifications on status of the connector.  When a connector completes a crawl, it will generate an email with details on that particular crawl and a log file attached.  It is important to note the attachment is a .zip file.  If these are blocked within the mail system, the email will not be received.  

Save Changes

Once all policies have been applied, Click 'OK' to save the connector's configuration. 

Run the Connector 

After all settings have been applied for the connector, it is now time to run the connector crawl.  

To manually run the connector's crawl process, select it and click the 'Start Crawl' button. 


 

While the connector is running, click the 'Crawl Tasks ' tab to see the progress and crawl history. 


For more information on how to monitor a crawl, see:  How to Monitor Connector Crawls

 

 

 

 



    • Related Articles

    • How to Create Connectors for Files

      This article describes how to configure settings specific to File connectors within the General and Advanced tabs of the connector properties.  The connector must be created before these settings can be applied. For information on creating a ...
    • How to Install and Configure the HubStor Connector Service (HCS)

      If you have access to the HubStor Admin Portal, you can download the HubStor Connector Service installer by going to 'Administration' and then 'Tools & Utilities'. Step 1 -- Run the Installer Open the HCS installer file ...
    • Overview of the HubStor Connector Service (HCS)

      The HCS is software that is deployed behind your firewall to provide policy-based archiving to your HubStor cloud tenant. The HCS runs on a Windows VM, uses a service account, and can be installed and configured in minutes. With the HCS installed, ...
    • How To Create SharePoint Connectors

      This article describes how to configure settings specific for the following connectors: SharePoint Online SharePoint Server on premises (versions 2013 and higher) SharePoint Groups/Teams  Each connector type listed above requires its own connector.  ...
    • How to Create a Slack Connector

      This article will walk through the steps on how to configure a Slack connector within the HubStor Connector Service (HCS).    The connector must be created before these settings can be applied. For information on creating a connector, refer to the ...