HubStor Inc.

How To Create SharePoint Connectors


This article describes how to configure settings specific for the following connectors:

  1. SharePoint Online
  2. SharePoint Server on premises (versions 2013 and higher)
  3. SharePoint Groups/Teams 

Each connector type listed above requires its own connector.  However, the settings are all very similar with the exception of a few which have been called out specifically where needed.
   

Pre-requisites: 

  1. The connector must be created before these settings can be applied. For information on creating a connector, refer to the article:  How To Create Connectors In HubStor
  1. Steps outlined in the Preparing for O365 article must be completed.  

Configure Connector's General Settings



  1. Get Location ACLs -- This option will synchronize unique ACLs at the folder level. Generally, this option should be enabled.
  2. Get Item ACLs -- This option will synchronize unique ACLs at the item level and is disabled by default. Note that syncing item-level ACLs introduces an additional call per item as part of the crawl process thereby slowing down the performance of the crawl. 

Configure Connector Type Specific Settings

General Tab: 



  1. SharePoint Online -- Enable this option if this is for SharePoint O365.  If SharePoint on premise, see section below called 'SharePoint On Premises Configuration'
  2. Mode -- See below for the specifics:
    1. For SharePoint Online, choose either of the following modes: 
      1. All Site Collections -- This will capture data from all available site collections in SharePoint.
      2. Specific Site Collections -- This option allows specifying which Site Collections to capture from. When this option is chosen, a new tab will be displayed labeled:  'Specific Site Collections'. Within here, a list of site collections are added individually or imported from a file (I.e. .csv).
    2. For OneDrive connectors, choose the following mode:
      1. All My Site/OneDrive for Business Site Collections  -- See article: How To Create A OneDrive Connector
    3. For SharePoint Groups and Team Site connectors, choose the following mode:
      1. All Group and Team Site Collections
      2. Specific Site Collections -- This option allows specifying which Site Collections to capture from. When this option is chosen, a new tab will be displayed labeled:  'Specific Site Collections'. Within here, a list of site collections are added individually or imported from a file (I.e. .csv).
  3. Auto-add SCOM Account as Site Collection Admins -- This option can automatically add user accounts as Site Collection Admins.  These user accounts are specified within the 'CSOM Auth' tab which will be discussed in more detail.  It is recommended to enable this option.  
  4. Admin Site URL -- Enter the URL for the Admin Site.  Example:  https://xyzcompany-admin.sharepoint.com

SharePoint On Premises Configuration

The following are options to configure when capturing data from a SharePoint server on premise.  



  1. SharePoint Online -- This should not be enabled
  2. Mode -- Choose 'All Site Collections for Web Application'
  3. Auto-add SCOM Account as Site Collection Admins -- This option can automatically add user accounts as Site Collection Admins.  These user accounts are specified within the 'CSOM Auth' tab which will be discussed in more detail.  It is recommended to enable this option.  
  4. Web Application ID -- This will auto-populate once the Base URL is entered and the ellipse button is selected. Note: Settings in the 'Config DB Connection' tab must first be populated. 
  5. Web Application Base URL -- Enter the URL 

Options Tab: 



  1. Continuous Data Protection -- CDP provides near-real time capture of new or changed items in SharePoint. For complete details regarding CDP and specific options, refer to article: How To Configure Continuous Data Protection
  2. Process Document Libraries Only -- When enabled, this option will exclude capturing of lists, blogs, wikis, etc.  
  3. Exclude System Lists -- These include the master page gallery for example.  Usually recommended enabling this option.
  4. Large List Threshold -- This setting should not be modified unless directed by a HubStor representative. 
  5. Diagnostic Logging -- This is only used for troubleshooting purposes as directed by a HubStor representative. 

CSOM Auth Tab: 




There are two modes to choose from: 
  1. Basic -- This is the default setting where it utilizes service accounts that have been created as part of the Preparing for O365 process document of which will be automatically added as site collection admins if the option 'Auto-add SCOM Account as Site Collection Admins' is enabled.  
  2. Modern/OAuth -- This is the preferred method.  To use this setting, the fields are populated with information after creating the SharePoint Application. To configure this application, refer to the article: How To Create A SharePoint App For OAuth/Modern Authentication
  3. Choose either 'Use Autodiscovery' or 'Manually Set EWS URL' based on the O365 configuration.  

PowerShell Auth Tab:
Depending on which option was chosen in the previous step (Basic or Modern/OAuth), depends on how the below options are displayed.  



For Basic CSOM Auth, the settings are: 
  1. Enter the SMTP address and password for a user with the SharePoint administrator Role. 
For Modern/OAuth, the Client ID and Client Secret are the same as CSOM Auth settings above.  



User Filter Tab: 
      

      
  1. This tab is only available when the Mode 'All My Site/OneDrive for Business Site Collections' is selected on the General tab.  
  2. This tab allows filtering of OneDrive site collections based on an extended AD attribute.  This filtering can take the form of an exact match, a wildcard match, or even a regular expression.  In the example above, the connector will match those users whose ‘department’ is either Engineering, or something like ‘Executive’, which would match the regex specified. Additionally, a very powerful filter that can be used is the value of:  [null]  - when used as an extended AD attribute filter value, it will match exactly those users who have a null or missing value for the given attribute.

Site Collection Scope Tab: 


No Site Collection Scope (all site collections) -- This option will process all site collections returned from SharePoint.
Rolling Site Collection Scope -- This rolling multi-instance mode will result in a given SharePoint connector only processing a certain number of site collections, and leaving the rest to another connector.  The goal is to allow really large O365 tenants to have the work distributed amongst multiple connectors and even multiple Stors. This option is available for the following 3 modes within the general tab: 
 
  1. All Site Collections
  2. All OneDrive Site Collections
  3. All Groups/Teams Site Collections

Restrict to Users With Specific Mail Domains -- This option is only available when selecting the mode 'All My Site/OneDrive for Business Site Collections on the general tab. It allows for restricting which sites to collect based on user(s) who have an email address in a specific domain. 


Groups/Teams Credentials Tab:


This is feature is optional.  If group awareness is not enabled, it will simply use the raw list names, whereas enabling group awareness allows the list name to be resolved to the user-friendly name. Screenshot below shows an example of a raw List name:



In order for this feature to work, it is required to create a new AAD application in the Azure portal.  For steps related to this process, refer to article: How To Configure A Group/Teams Awareness Azure AAD Application.  Once the application is created, populate the fields within this setting that were obtained from the application creation process. 



    • Related Articles

    • How To Create Connectors In HubStor

      This article describes the process of how to create connectors in the HubStor Connector Service (HCS).  Add Connector  In the HubStor Connector Service (HCS) app, go to  'Connectors' and click 'Add Connector '.      In the drop-down, select the type ...
    • How To Create A OneDrive Connector

      This article will walk through the process of configuring a connector that will capture OneDrive Sites.   Pre-requisites:  The connector must be created before these settings can be applied. For information on creating a connector, refer to the ...
    • How to Create Connectors for Files

      This article describes how to configure settings specific to File connectors within the General and Advanced tabs of the connector properties.  The connector must be created before these settings can be applied. For information on creating a ...
    • How To Configure A Group/Teams Awareness Azure AAD Application

      This article will walk through the steps on how to create an Azure AAD Application that will allow the use of the Group/Teams Awareness option within a SharePoint connector. This process requires access to the Azure Admin Portal with a user in the ...
    • How To Create A SharePoint App For OAuth/Modern Authentication

      This article will discuss the steps on how to create a SharePoint App that will be used for a connector that is configured for OAuth.  This includes connectors for SharePoint sites and OneDrive sites each of which can use the same app details.   ...